|
I found at least part of the problem. A string variable cannot contain Unicode characters. So Dim lpFileName_str As String = "\\?\UNC\" & path & "\*" will not work and comes back with, for example, "\\?\UNC\d:\?????????\*". Now I think I see why the string parameter needs to be changed to a pointer to the path, rather than a string representing the path.
Also, when I put the path into memory, and then retrieve it, everything is correct except the "\*" at the end...It gets interpreted as "[foreign character]*". So I think I'm finally on to at least part of the solution. I'll take a look at how to handle backslashes in Unicode strings and see what happens. That also explains why the return value from the function was "-1", file not found.
modified 4-Jun-13 18:28pm.
|
|
|
|
|
|
Bummer...It's in some form of C. I'm not a C guy. I can struggle through it and eventually figure it out, but the hardest part is converting to vb and getting the function declarations correct. One oddity in his code, is I did not see any references to FindFirstFileEx()...Only FindFirstFile() which cannot deal with wide path names.
|
|
|
|
|
treddie wrote: string variable cannot contain Unicode characters
Considering every single string in .NET is stored as Unicode, that statement is not true. The encoding is UTF-16.
treddie wrote: for example, "\\?\UNC\d:\?????????\*
The problem with what your looking at is that not every font has glyphs for every Unicode character, so you get boxes for those. In addition, if the string is being displayed by a control or other rendering device that doesn't support the encoding, characters that are not recognized are replaced with question marks. Read this[^].
treddie wrote: Now I think I see why the string parameter needs to be changed to a pointer to
the path, rather than a string representing the path
Hehe. You never pass a "string". Strings are always passed by pointer. You can only put one value, per parameter, on the stack when function call is made. In the case of strings, or other object that is not an intrinsic type, it's always a pointer.
|
|
|
|
|
Dave Kreskowiak wrote: string variable cannot contain Unicode characters
Yah, that was sloppy reasoning on my part. Although, I also can't explain why I was getting inconsistent behavior in what was being reported in the string. Sometimes, when I would pass the pathname to the string, it would come back with ?'s. So without having a stable situation that I can depend on, I can't come to any conclusion on what that was all about. Especially since I can't duplicate the problem.
But now, what I have found is that the string holds the foreign characters just fine, but something odd happens when I send the string to an RTB and a plain TextBox. The RTB displays "\*" as "<foreign char="">*", but the TextBox had no problem at all and correctly shows, "\*". But if I hover my mouse over the "rtb1.text =" statement in the code, it reads fine! Weird.
Dave Kreskowiak wrote: You never pass a "string".
But why do they declare strings in any function that accepts them? I've used ANSI strings before with no problem, just as MSDN suggests. The headaches happen when I get to Unicode strings.
|
|
|
|
|
treddie wrote: But why do they declare strings in any function that accepts them?
Because you're definfing the TYPE of the pointer. What kind of object does it point to? What are the rules for incrementing that pointer if needed??
If you go back to C, a "string" in it's most basic form is just a pointer to an array of type char (character). Under Windows, you can have ASCII strings and wide-character strings. Incrementing the pointer for an ASCII character just adding 1 to the pointer, but, for multi-byte characters, such as Unicode, you increment the pointer by 2 or more.
|
|
|
|
|
|
treddie wrote: I also found through tests that the TRUE limit for Windows, is 240 characters
max.
Not quite. A fully qualifed filePATH to a file (including filename) is 248 characters max. A fileNAME (with no path information at all) is 260 characters max. Think of setting the "current directory" first before trying to access a file with a name like that. That's one way to get a filepath longer than the 248 character limit.
treddie wrote: This may be a bug in Windows
Considering that code has been around for, literally, decades, its not a bug. These are hardcoded limits in NTFS.
modified 4-Jun-13 18:58pm.
|
|
|
|
|
Dave Kreskowiak wrote: I also found through tests that the TRUE limit for Windows, is 240 characters
max.
I'm not so sure about that, Dave. I ran some extensive tests many months ago when working on this program and found that there are 4 Windows limits. Their numbers are as follows, according to MS:
Folder name = 260 characters
File name (w/ extension) = 260 characters
Folder path w/o file name = 260 characters
Full path w/ filename = 260 characters
Obviously you can't have 260 in all categories so it's a matter of whatever the actual amounts are for each category, that they don't exceed any of the 4 limits.
At any rate, I tested all 4 cases and found that when you subtract out the twelve 8.3-filename chars, the 3 root characters, and the null character at the end of any path (all the characters the user has no control over), you are left with at that point only 244 characters, including backslashes, that you can play with. But, you have to actually limit yourself to 240 maximum, user-controlled characters to prevent the Windows crash on folder/file name right-click -> properties. I think that last case classifies as a bug in that your Windows session gets trashed if you go over 240 characters when getting a file's properties.
This was true in my tests for both WinXP (latest SP), and Win7 (6.1.7600 Build 7600), although the manner of the crash was different for each system. The WinXP crash was really bad and would hang the computer.
|
|
|
|
|
Actually, NTFS is not at all that limited. It's just that the kernel32.dll API interface is limited to those path limits.
From MSDN, CreateFile page:
In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path. For more information, see Naming Files, Paths, and Namespaces.
By adding \\?\ you tell the kernel32 API to NOT do any parsing or processing on the path, and instead pass it directly on to the filesystem driver. From MSDN page on Naming Files, Paths, and Namespaces (Windows):
For file I/O, the "\\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system.
But not all functions can work with the extended paths, because some functions inherently need to do processing on the path.
|
|
|
|
|
Yeah, I knew that. Slip of the brain when I typed it.
|
|
|
|
|
OK, I'm finally getting this to work. I took the suggestion more seriously that GetLastError() might provide some insight into the problem. It wasn't anything that explicitly told me what was wrong, but it did cause me to take a much closer look at all of my declarations, and realizing that using "\\?\UNC\" was not correct...It needs to be just "\\?\".
But also took to heart your comments on strings and pointers. In that regard, I found some typing problems here and there, and missed some lines that were still not rewritten for pointers.
I'll post the resulting code here, shortly, after I copy my test code over to my final demo program and fix some remaining issues.
|
|
|
|
|
One hickup...I need to deal with a C type, LPWSTR. When I research how to get this in vb.Net, I do not find any consistent explanation of how to deal with marshaling unmanaged types. There are no real-world examples and what MSDN offers is cryptic at best. Here is my best guess that does not show any syntax errors:
Dim LPWSTR As UnmanagedType
Dim MyUnicodePointer As New MarshalAsAttribute(LPWSTR)
|
|
|
|
|
An LPWSTR is just a Long Pointer to a Wide character String. You're just marshalling a normal string from .NET to the function your calling. Usually, you don't have to do anything special in your function Declaration other than tell it that the paramter is a String.
|
|
|
|
|
I'm confused. The function requires a pointer to a wide string, so you can't use 'As String' as a passed argument to the function FindFirstFileEx(). You have to use 'As LWPSTR' or, if I understand you correctly, 'As Long'. If that is true, then I have to wrap the string in a 'long' pointer, which for a 64bit platform would translate to Int32 (or rather, IntPtr), since a win32 function can only accept 32bits arguments and parameters.
If that is true, then my existing marshal should be correct:
Dim MyStringPtr As Long = Marshal.StringToHGlobalAuto(MyString)
But when I pass MyStringPtr to FindFileEx(), I only get back the first character in a Unicode folder/file name, and then, only when the first character in the string happens to be in the ANSI range.
|
|
|
|
|
I haven't read all of this thread but I don't see either the Auto or Unicode character set modifiers in your external procedure declarations, which means that the default of Ansi is being applied. This could be the root cause of your string problems as UTF16 encoded strings treated as Ansi will get truncated at the first null, i.e. after the first character.
See Declare statement[^] and also Calling Windows APIs[^] for examples using both Declare and DllImport.
Alan.
|
|
|
|
|
That basically has been my conclusion, though I don't know how to fix it yet. I figured SOMETHING had to be cutting the returned string short, and I know that the input string is getting to memory correctly, because I can retrieve it just fine. So that leaves the WIN32_FIND_DATA structure as the culprit. Documentation says the following declaration is required:
<VBFixedString(MAX_PATH), MarshalAs(UnmanagedType.ByValTStr, SizeConst:=MAX_PATH)> Public cFileName As String
And for my string pointer, I used:
Dim lpFileName_IntPtr As Long = Marshal.StringToHGlobalAuto(lpFileName_str)
I experimented with changing MAX_PATH to 32767 rather than 260, but alone by itself it does nothing. And anyway, that is a max character length issue, not an ANSI vs. wide character issue. So I feel that the problem is with declaring cFileName, correctly. That was where I was at when I saw your post. I do not know how to squeeze some martialing in there to account for a wide string.
modified 6-Jun-13 20:22pm.
|
|
|
|
|
Hi treddie,
Based on my understanding of your code, I see no reason to use FindFirstFileEx versus FindFirstFile. Also, the directory filter flag option can fail silently if it is not supported and you need to check the attributes anyways if it was your intent to just recover directories. But it appears as your code is also doing these checks.
Just a word of caution about interop examples you find on the web; many of these were originally done in VB6 and may not have been properly modified for DotNet types.
Here are links to a reference and a tool that you may find useful in working with interop.
Miscellaneous Marshaling (look at the links under "See Also") Samples[^]
PInvoke Interop Assistant[^]
I was bored so I tried putting together a simple TreeView file explorer that bypasses the 260 character limit. It may serve as an example for what you are trying to do. Just place a TreeView control on new WinForm project and paste this code.
I used the command prompt "Subst" command to create a drive reference to a path that reached the 260 character barier and then added some more directories under that substituted drive to get to a file that had 400+ characters in it's path to test this. It appears to work.
Edit: There appears to be a length limit on codeblock code.
Imports System.Runtime.InteropServices
Public Class Form1
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
TreeView1.Nodes.Add(DriveToTreeNode("c"c))
End Sub
Private Sub TreeView1_AfterCollapse(ByVal sender As Object, ByVal e As System.Windows.Forms.TreeViewEventArgs) Handles TreeView1.AfterCollapse
e.Node.Nodes.Clear()
e.Node.Nodes.Add(New TreeNode With {.Tag = New NodeData With {.IsDirectory = True}}) ' add dummy node
End Sub
Private Sub TreeView1_BeforeExpand(ByVal sender As Object, ByVal e As System.Windows.Forms.TreeViewCancelEventArgs) Handles TreeView1.BeforeExpand
' only directories can have nodes
e.Node.Nodes.Clear()
EnumerateDirectory(e.Node)
End Sub
Private Sub TreeView1_MouseDoubleClick(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles TreeView1.MouseDoubleClick
If e.Button = Windows.Forms.MouseButtons.Left Then
Dim hti As TreeViewHitTestInfo = TreeView1.HitTest(TreeView1.PointToClient(MousePosition))
If CType(hti.Node.Tag, NodeData).IsFile Then
Dim fullpath As String = CType(hti.Node.Parent.Tag, NodeData).DirectoryPath & hti.Node.Text
MessageBox.Show("Long Path: " & fullpath & vbCrLf & "Path Length: " & fullpath.Length.ToString() & _
vbCrLf & "Short Name: " & GetShortPathName(fullpath), _
hti.Node.Text, MessageBoxButtons.OK)
End If
End If
End Sub
#Region "Drive Enumeration"
Private Function DriveToTreeNode(ByVal drive As Char) As TreeNode
Dim node As New TreeNode
node.Text = drive.ToString().ToUpper()
node.Tag = New NodeData With {.IsDirectory = True, .DirectoryPath = node.Text & ":\"}
' recursing all filedirectories would take too long for a drive
' so will handle it on each directory node expansion
EnumerateDirectory(node, recursedirectories:=False)
Return node
End Function
' see: Naming Files, Paths, and Namespaces
' Maximum Path Length Limitation
' http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx#maxpath
Private Const LongPathPrefix As String = "\\?\"
Private Sub EnumerateDirectory(ByVal node As TreeNode, Optional ByVal recursedirectories As Boolean = False)
Dim path As String = CType(node.Tag, NodeData).DirectoryPath
Dim findData As New WIN32_FIND_DATA
Dim hFile As Microsoft.Win32.SafeHandles.SafeFileHandle
hFile = FindFirstFile(LongPathPrefix & path & "*", findData)
If hFile.DangerousGetHandle.ToInt32 = -1 Then
' just ignoring errors for this demo, most common is 5 - Access Denied
' System Error Codes
' http://msdn.microsoft.com/en-us/library/windows/desktop/ms681381%28v=vs.85%29.aspx
'Dim [error] As Int32 = Marshal.GetLastWin32Error()
'Stop
Return
Else
Do
If IsFlagSet(findData.dwFileAttributes, IO.FileAttributes.Directory) Then
If findData.cFileName = "." Then Continue Do
If findData.cFileName = ".." Then Continue Do
Dim subdir As New TreeNode With {.Text = findData.cFileName, _
.Tag = New NodeData With {.IsDirectory = True, _
.DirectoryPath = path & findData.cFileName & "\" _
} _
}
If recursedirectories Then
EnumerateDirectory(subdir, recursedirectories)
Else
' need to add a dummy node to make it expandable
subdir.Nodes.Add(New TreeNode With {.Tag = New NodeData With {.IsDirectory = True}})
End If
node.Nodes.Add(subdir)
Else ' Filename
node.Nodes.Add(New TreeNode With {.Text = findData.cFileName, .Tag = New NodeData With {.IsDirectory = False}})
End If
Loop While FindNextFile(hFile, findData)
FindClose(hFile) ' make sure to close the handle
End If
End Sub
Private Function IsFlagSet(ByVal value As System.ValueType, ByVal testflag As System.ValueType) As Boolean
Dim val As UInt32 = CType(value, UInt32)
Dim flag As UInt32 = CType(testflag, UInt32)
Return (val And flag) = flag
End Function
<DllImport("kernel32.dll", CharSet:=CharSet.Unicode, SetLastError:=True, EntryPoint:="FindFirstFile")> _
Private Shared Function FindFirstFile(<System.Runtime.InteropServices.In(), MarshalAs(UnmanagedType.LPWStr)> ByVal fileName As String, _
<[In](), Out(), MarshalAs(UnmanagedType.LPStruct)> ByVal data As WIN32_FIND_DATA) As Microsoft.Win32.SafeHandles.SafeFileHandle
End Function
<DllImport("kernel32.dll", CharSet:=CharSet.Unicode, SetLastError:=True, EntryPoint:="FindNextFileW")> _
Private Shared Function FindNextFile(ByVal hFindFile As Microsoft.Win32.SafeHandles.SafeFileHandle, _
<[In](), Out(), MarshalAs(UnmanagedType.LPStruct)> ByVal ByvallpFindFileData As WIN32_FIND_DATA) As Boolean
End Function
<DllImport("kernel32.dll")> _
Private Shared Function FindClose(ByVal hFindFile As Microsoft.Win32.SafeHandles.SafeFileHandle) As Boolean
End Function
<DllImport("kernel32.dll", SetLastError:=True, CharSet:=CharSet.Auto)> _
Private Shared Function GetShortPathName(ByVal longPath As String, _
<MarshalAs(UnmanagedType.LPTStr)> ByVal ShortPath As System.Text.StringBuilder, _
<MarshalAs(Runtime.InteropServices.UnmanagedType.U4)> ByVal bufferSize As Integer) As Integer
End Function
2nd part as text (sorry about format)
Private Shared Function GetShortPathName(ByVal path As String) As String
Dim sb As New System.Text.StringBuilder(1000)
Dim len As Int32 = GetShortPathName(LongPathPrefix & path, sb, sb.Capacity)
If len > sb.Capacity Then
sb.Capacity = len
GetShortPathName(LongPathPrefix & path, sb, sb.Capacity)
End If
Return sb.ToString().Replace(LongPathPrefix, "")
End Function
<structlayout(layoutkind.sequential, charset:="CharSet.Unicode)," bestfitmapping(false)=""> _
Private Class WIN32_FIND_DATA
<marshalas(unmanagedtype.u4)> Public dwFileAttributes As IO.FileAttributes
<marshalas(unmanagedtype.u4)> Private ftCreationTime_dwLowDateTime As UInt32
<marshalas(unmanagedtype.u4)> Private ftCreationTime_dwHighDateTime As UInt32
<marshalas(unmanagedtype.u4)> Private ftLastAccessTime_dwLowDateTime As UInt32
<marshalas(unmanagedtype.u4)> Private ftLastAccessTime_dwHighDateTime As UInt32
<marshalas(unmanagedtype.u4)> Private ftLastWriteTime_dwLowDateTime As UInt32
<marshalas(unmanagedtype.u4)> Private ftLastWriteTime_dwHighDateTime As UInt32
<marshalas(unmanagedtype.u4)> Public nFileSizeHigh As UInt32
<marshalas(unmanagedtype.u4)> Public nFileSizeLow As UInt32
<marshalas(unmanagedtype.u4)> Public dwReserved0 As UInt32
<marshalas(unmanagedtype.u4)> Public dwReserved1 As UInt32
' Note: by changing the cFileName size constant from 260 to 32767 to handle long
' path names, it appears that cAlternateFileName always returns blank
' if the ShortPathName is needed, then use the GetShortPathName function
' http://msdn.microsoft.com/en-us/library/windows/desktop/aa364989%28v=vs.85%29.aspx
<marshalas(unmanagedtype.byvaltstr, sizeconst:="32767)"> Public cFileName As String
<marshalas(unmanagedtype.byvaltstr, sizeconst:="14)"> Public cAlternateFileName As String
Public Function CreationTime() As DateTime
Return DateTime.FromFileTime(MergeToInt64(ftCreationTime_dwLowDateTime, ftCreationTime_dwHighDateTime))
End Function
Public Function LastAccessTime() As DateTime
Return DateTime.FromFileTime(MergeToInt64(ftLastAccessTime_dwLowDateTime, ftLastAccessTime_dwHighDateTime))
End Function
Public Function LastWriteTime() As DateTime
Return DateTime.FromFileTime(MergeToInt64(ftLastWriteTime_dwLowDateTime, ftLastWriteTime_dwHighDateTime))
End Function
Public Function FileSizeInBytes() As UInt64
Return (CULng(nFileSizeHigh) << 32) Or CULng(nFileSizeLow)
End Function
Private Function MergeToInt64(ByVal low As UInt32, ByVal high As UInt32) As Int64
Dim bighigh As UInt64 = high
bighigh <<= 32
bighigh = bighigh Or CULng(low)
Return CLng(bighigh)
End Function
End Class ' WIN32_FIND_DATA
#End Region
Private Class NodeData
Private _IsDirectory As Boolean
Public Property IsDirectory() As Boolean
Get
Return _IsDirectory
End Get
Set(ByVal value As Boolean)
_IsDirectory = value
End Set
End Property
Public ReadOnly Property IsFile() As Boolean
Get
Return Not IsDirectory
End Get
End Property
Public DirectoryPath As String
End Class 'NodeData
End Class
modified 6-Jun-13 0:59am.
|
|
|
|
|
But FindFirstFile() cannot handle Unicode, as far as MSDN seems to suggest.
|
|
|
|
|
Where in the documentation does it suggest that?
Quite the opposite is suggested in http://msdn.microsoft.com/en-us/library/windows/desktop/aa364418%28v=vs.85%29.aspx[^]
Quote: In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path. For more information, see Naming a File.
|
|
|
|
|
Damn. I swear to gawd I saw it say you needed to use the Ex version to make it happen. Somewhere along the line I got messed up there. Too many questions all at once, I guess.
But now that I know that, my overly-verbose usage should still work...I just have to keep track of extra arguments. Also, it is a good exercise in understanding both, since I may in the future need to use the Ex function.
I haven't had a chance yet, today, to run your example.
|
|
|
|
|
Quote: Damn. I swear to gawd I saw it say you needed to use the Ex version to make it happen.
I blame it all on micro-blackhole that make one jump from one parallel universe to another just to screw with your mind when the minor differences bite you in the behind. 
All I can say is that you are making an effort to learn and that is a rare and beautiful thing to see in a forum these days. 
|
|
|
|
|
Thanks, man!
It can be frustrating, but it is more frustrating to stay confused and not see the frikkin light!
|
|
|
|
|
I built your project, but when run, the treeview stays blank.
|
|
|
|
|
Edit: I deleted the upload, forgot I was in the middle of trying an ASCII version. Will repost a link once I get it back to Unicode.
That's strange. I tried running it on a low privilege account and it run fine on Vista/32 bit.
In case anything got messed up while I was trying to post the code(I had some issues) I've uploaded the project (VS2008) to: http://sdrv.ms/18SnASR
This version is a bit different because I've been playing with it bit this evening. You will also find the syntax for the FindFirstFileEx function in the scraps.txt file.
Try un-commenting the code in this block:
If hFile.DangerousGetHandle.ToInt32 = -1 Then
Dim [error] As Int32 = Marshal.GetLastWin32Error()
Stop
Return
Else
and see what error code, if any, you are getting.
modified 6-Jun-13 21:43pm.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
