The Lounge - CodeProject

Re: Win10 FileSystem: File / Path too long

Mark_Wallace7-Mar-17 21:13

7-Mar-17 21:13

Randor wrote:
You do realize that you are ~~on a forum with old MSFT engineers from the operating systems group right~~ in the CP Lounge?

Chill a little, Bud.

I wanna be a eunuchs developer! Pass me a bread knife!

Re: Win10 FileSystem: File / Path too long

raddevus8-Mar-17 2:45

raddevus

8-Mar-17 2:45

Randor wrote:
The CreateFileW[^] function can absolutely create files with paths greater than MAX_PATH.

I don't dispute that it can. I am saying there may be some problem (or trick that must be applied) on these OSes. I would love to discuss this further, but I'm afraid it'll be quite a long post. I hope you might be interested.
I hope you'll really try this and post back.

Okay, so I'm actually pinvoking the API method from a .NET framework. I will provide the entire code for you and you can easily run it in LINQPad (free utility at LINQPad - The .NET Programmer's Playground[^]

Running the Source Code
If you get that tool you can copy paste the entire listing directly into a new linqpad "query" and it'll run normally as any other .NET program. Of course, you can easily create a console app in VStudio 2015 or whatever. The code will still run.

MessageBoxW
If you run the code you will see that it will successfully call MessageBoxW and display some text in the message box. That was a test to insure that the text of the path was being marshalled properly.
You can see the pinvoke has the charset set to Auto and it does indeed convert and display the chars properly. That's important since we are going to pass the string for the text file.

If you set the Charset to Charset.Ansi, you'll see nonsense kanji characters or something since this is a W method.

Why MessageBoxW?
I'm simply setting up the idea that my pinvoke does seem to work properly, the string marshalling is correct and the error I get when attempting to create the file is more likely related to the file system or API than it is to a bad path string being passed.
Note: You'll need to change the path to a location that exists on your c:\ drive
So, if you run the code
*the MessageBox will pop up
*the code will attempt to create a file with a long name and fail
*the code will attempt to create a file with a shorter name and succeed -- proves the call to CreateFileW does indeed work.

The output (after the messagebox displays )looks like the following:

Output
3/8/2017 8:31:59 AM
Failed!
HRESULT: -2147024773 Error: 123
Seems to work..

The Error 123 is the error from the OS (GetLastError) which we've talked about.

The "Seems to work.." is the successful creation of the 2nd file (which is not a long file name) to prove that the call to CreateFileW() does indeed work as I've defined it and the pinvoke seems to be set up correctly.

What Do You Think?
Can you explain this? Can you provide a very small example of working code (written in C++ or something) which will actually create a long file name?
I put a lot into this so I hope it has caught your interest.
Great discussion. Thumbs Up | :thumbsup:

[DllImport("kernel32.dll", SetLastError=true, CharSet=CharSet.Auto)]
public static extern SafeFileHandle CreateFileW(
   String lpFileName,
   EFileAccess dwDesiredAccess,
   EFileShare dwShareMode,
   IntPtr lpSecurityAttributes,
   ECreationDisposition dwCreationDisposition,
   EFileAttributes dwFlagsAndAttributes,
   IntPtr hTemplateFile);
   
    
   [DllImport("user32.dll", CharSet=CharSet.Auto)]
    static extern int MessageBoxW(IntPtr hWnd, String text, String caption, int options);

   
void Main()
{
	
	 MessageBoxW(IntPtr.Zero, @"\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt", "Caption", 0);
	
	Console.WriteLine(DateTime.Now);
	
	SafeFileHandle sfh = CreateFileW(@"\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt", 
		EFileAccess.GenericWrite, EFileShare.Read, 
		IntPtr.Zero, ECreationDisposition.CreateAlways,
		EFileAttributes.Archive, IntPtr.Zero);
	if (sfh.IsInvalid ){
		Console.WriteLine("Failed!");
		int HRESULT = Marshal.GetHRForLastWin32Error();
		int ERRORVAL = Marshal.GetLastWin32Error();
		Console.WriteLine(String.Format("HRESULT: {0} Error: {1}",HRESULT, ERRORVAL));
	}
	else{
		Console.WriteLine("Seems to work..");
	}
	sfh = CreateFileW(@"\\?\C:\Users\<username>\temp\notlongName_2017_03_11_2.txt", 
	EFileAccess.GenericWrite, EFileShare.Read, 
		IntPtr.Zero, ECreationDisposition.CreateAlways,
		EFileAttributes.Archive, IntPtr.Zero);
	if (sfh.IsInvalid ){
		Console.WriteLine("Failed!");
		int HRESULT = Marshal.GetHRForLastWin32Error();
		int ERRORVAL = Marshal.GetLastWin32Error();
		Console.WriteLine(String.Format("HRESULT: {0} Error: {1}",HRESULT, ERRORVAL));
	}
	else{
		Console.WriteLine("Seems to work..");
	}
}

public enum EFileAccess : uint
    {
        //
        // Standart Section
        //

        AccessSystemSecurity = 0x1000000,   // AccessSystemAcl access type
        MaximumAllowed = 0x2000000,     // MaximumAllowed access type

        Delete = 0x10000,
        ReadControl = 0x20000,
        WriteDAC = 0x40000,
        WriteOwner = 0x80000,
        Synchronize = 0x100000,

        StandardRightsRequired = 0xF0000,
        StandardRightsRead = ReadControl,
        StandardRightsWrite = ReadControl,
        StandardRightsExecute = ReadControl,
        StandardRightsAll = 0x1F0000,
        SpecificRightsAll = 0xFFFF,

        FILE_READ_DATA = 0x0001,        // file & pipe
        FILE_LIST_DIRECTORY = 0x0001,       // directory
        FILE_WRITE_DATA = 0x0002,       // file & pipe
        FILE_ADD_FILE = 0x0002,         // directory
        FILE_APPEND_DATA = 0x0004,      // file
        FILE_ADD_SUBDIRECTORY = 0x0004,     // directory
        FILE_CREATE_PIPE_INSTANCE = 0x0004, // named pipe
        FILE_READ_EA = 0x0008,          // file & directory
        FILE_WRITE_EA = 0x0010,         // file & directory
        FILE_EXECUTE = 0x0020,          // file
        FILE_TRAVERSE = 0x0020,         // directory
        FILE_DELETE_CHILD = 0x0040,     // directory
        FILE_READ_ATTRIBUTES = 0x0080,      // all
        FILE_WRITE_ATTRIBUTES = 0x0100,     // all

        //
        // Generic Section
        //

        GenericRead = 0x80000000,
        GenericWrite = 0x40000000,
        GenericExecute = 0x20000000,
        GenericAll = 0x10000000,

        SPECIFIC_RIGHTS_ALL = 0x00FFFF,
        FILE_ALL_ACCESS =
        StandardRightsRequired |
        Synchronize | 
        0x1FF,

        FILE_GENERIC_READ =
        StandardRightsRead |
        FILE_READ_DATA |
        FILE_READ_ATTRIBUTES |
        FILE_READ_EA |
        Synchronize,

        FILE_GENERIC_WRITE =
        StandardRightsWrite |
        FILE_WRITE_DATA |
        FILE_WRITE_ATTRIBUTES |
        FILE_WRITE_EA |
        FILE_APPEND_DATA |
        Synchronize,

        FILE_GENERIC_EXECUTE = 
        StandardRightsExecute |
          FILE_READ_ATTRIBUTES |
          FILE_EXECUTE |
          Synchronize
    }
	
	[Flags]
public enum EFileAttributes : uint
{
   Readonly         = 0x00000001,
   Hidden           = 0x00000002,
   System           = 0x00000004,
   Directory        = 0x00000010,
   Archive          = 0x00000020,
   Device           = 0x00000040,
   Normal           = 0x00000080,
   Temporary        = 0x00000100,
   SparseFile       = 0x00000200,
   ReparsePoint     = 0x00000400,
   Compressed       = 0x00000800,
   Offline          = 0x00001000,
   NotContentIndexed= 0x00002000,
   Encrypted        = 0x00004000,
   Write_Through    = 0x80000000,
   Overlapped       = 0x40000000,
   NoBuffering      = 0x20000000,
   RandomAccess     = 0x10000000,
   SequentialScan   = 0x08000000,
   DeleteOnClose    = 0x04000000,
   BackupSemantics  = 0x02000000,
   PosixSemantics   = 0x01000000,
   OpenReparsePoint = 0x00200000,
   OpenNoRecall     = 0x00100000,
   FirstPipeInstance= 0x00080000
}

[Flags]
public enum EFileShare : uint
{
   /// <summary>
   /// 
   /// </summary>
   None = 0x00000000,
   /// <summary>
   /// Enables subsequent open operations on an object to request read access. 
   /// Otherwise, other processes cannot open the object if they request read access. 
   /// If this flag is not specified, but the object has been opened for read access, the function fails.
   /// </summary>
   Read = 0x00000001,
   /// <summary>
   /// Enables subsequent open operations on an object to request write access. 
   /// Otherwise, other processes cannot open the object if they request write access. 
   /// If this flag is not specified, but the object has been opened for write access, the function fails.
   /// </summary>
   Write = 0x00000002,
   /// <summary>
   /// Enables subsequent open operations on an object to request delete access. 
   /// Otherwise, other processes cannot open the object if they request delete access.
   /// If this flag is not specified, but the object has been opened for delete access, the function fails.
   /// </summary>
   Delete = 0x00000004
}

public enum ECreationDisposition : uint
{
   /// <summary>
   /// Creates a new file. The function fails if a specified file exists.
   /// </summary>
   New = 1,
   /// <summary>
   /// Creates a new file, always. 
   /// If a file exists, the function overwrites the file, clears the existing attributes, combines the specified file attributes, 
   /// and flags with FILE_ATTRIBUTE_ARCHIVE, but does not set the security descriptor that the SECURITY_ATTRIBUTES structure specifies.
   /// </summary>
   CreateAlways = 2,
   /// <summary>
   /// Opens a file. The function fails if the file does not exist. 
   /// </summary>
   OpenExisting = 3,
   /// <summary>
   /// Opens a file, always. 
   /// If a file does not exist, the function creates a file as if dwCreationDisposition is CREATE_NEW.
   /// </summary>
   OpenAlways = 4,
   /// <summary>
   /// Opens a file and truncates it so that its size is 0 (zero) bytes. The function fails if the file does not exist.
   /// The calling process must open the file with the GENERIC_WRITE access right. 
   /// </summary>
   TruncateExisting = 5
}

Re: Win10 FileSystem: File / Path too long

Randor 8-Mar-17 8:23

Randor

8-Mar-17 8:23

raddevus wrote:
What Do You Think?Can you explain this?

I am getting old and don't really See Sharp.

However I do see several problems with the code you have written.

1.) You are using some invalid/reserved characters in your path. The < and > are reserved characters.
2.) You are incorrectly assuming that you can make a really long directory entry of 32,767 characters.
3.) You are mistakenly attempting to create directories with the CreateFile Function[^]

Follow these rules:
1.) Individual path components such as file names and directory names should be <= 255 characters. (This is probably why your code fails.)
2.) The entire path can be up to 32,767 wide characters.
3.) Prepend your path with //?/ and use an absolute path.
4.) Use CreateDirectoryW for creating directories and CreateFileW to create files.

Best Wishes,
-David Delaune

Re: Win10 FileSystem: File / Path too long

raddevus8-Mar-17 8:45

raddevus

8-Mar-17 8:45

Well, I appreciate you replying.
But...

Randor wrote:
1.) You are using some invalid/reserved characters in your path. The < and > are reserved characters.

1. My path doesn't actually contain any < or > That was just for you to replace. You know, Microsoft docs use < > to denote variable values, that's what I was doing.
I was saying add your <username> directory into the path so it will run on your machine.

\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt

Note: that's really all one line but this editor makes it look odd.

Randor wrote:
2.) You are incorrectly assuming that you can make a really long directory entry of 32,767 characters.

2. No, the path already exists on my computer. That's a temp path that was created long ago. That's why I use that same path to create the other file (shorter name) which successfully calls CreateFileW.

Randor wrote:
3.) You are mistakenly attempting to create directories with the CreateFile Function[^]

3. again, no the path is already created.
If I do not provide any path in the string then it will create the file in the EXE path of my executable so I'm just pointing it to a path that already exists.

Finally, I actually am following those rules. Do you have any other ideas?

Thanks

Re: Win10 FileSystem: File / Path too long

Randor 8-Mar-17 9:56

Randor

8-Mar-17 9:56

Hi,

I gave you absolutely all the information needed to create a path greater than MAX_PATH (260). Everything in my prior analysis is correct.

You seem extremely argumentative and/or having trouble understanding me.

Let me be specific:

raddevus wrote:
Finally, I actually am following those rules. Do you have any other ideas?

No, you are not. Look again:

1.) Individual path components such as file names and directory names should be <= 255 characters. (This is probably why your code fails.)
2.) The entire path can be up to 32,767 wide characters.

The filename: superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt

This filename 312 characters length which is greater than 255 characters. Refer to what I said in rule 1.

Best Wishes,
-David Delaune

Re: Win10 FileSystem: File / Path too long

raddevus8-Mar-17 10:15

raddevus

8-Mar-17 10:15

Sheesh.
I'm an engineer. You wanted me to be exact and I was exact in my language and so I was.
I honestly wasn't being argumentative. I was being precise. I was enjoying our discussion.

Keep in mind that you started this entire conversation and your introduction with the following:

Randor wrote:
Wow, you wrote a disk monitoring application but did not know about NTFS filename limits? How is that even possible? Are you a .NET coder or something?

That was argumentative and not a fantastic introduction.

You also set up the original problem like the following:

Randor wrote:
You could always create files longer than MAX_PATH ... even on Windows NT 4 all you do is prefix them with the "\\?\"

Plus read the exact wording on lpFileName (not path) at the Microsoft docs:

Quote:
lpFileName [in]
The name of the file or device to be created or opened. You may use either forward slashes (/) or backslashes (\) in this name.
In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path.

The Focus Seemed to Be On FileName -- not filename + path
This all set my thinking-context so I was thought, "Hey, I can create files that are 32,767 chars long. Let's do it!!"

That's why I've been focusing on CreateFileW. I will add CreateDirectoryW() now and work with that.

Alas, this is path + filename. Confused | :confused:

Surely, you can see how confusing this has been.

Re: Win10 FileSystem: File / Path too long

Randor 8-Mar-17 11:12

Randor

8-Mar-17 11:12

raddevus wrote:
Plus read the exact wording on lpFileName (not path) at the Microsoft docs:
Quote:
lpFileName [in]The name of the file or device to be created or opened. You may use either forward slashes (/) or backslashes (\) in this name.In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path.

I can see why this is unclear. Keep in mind that you can use CreateFile to open COM ports and various other devices. It can also be used to open files on storage devices. This is why it is important to train yourself to never pay any attention to any variable names.

Looks like the engineer decided to name that variable lpFileName when the variable is actually [drive][path][filename] or [IO device].

Even in Windows... Everything is a File[^].

Best Wishes,
-David Delaune

Re: Win10 FileSystem: File / Path too long

raddevus8-Mar-17 11:28

raddevus

8-Mar-17 11:28

Randor wrote:
I can see why this is unclear. Keep in mind that you can use CreateFile to open COM ports and various other devices.

Thanks and yes I got stuck in a thinking-track and couldn't move out of it.
I have successfully used CreateDirectoryW and CreateFileW now to create extremely long file paths / names.

It all works great.

\\?\C:\Users\<username>\temp\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\abcdefghijklmnopqrstuvwxzy1234abcdefghijklmnopqrstuv\X_abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234_X\Y_abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234_Y/superlongfilenamean12henilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink1234567890123456789012superlongfilenamean12henilookatthefiletest1234567890ThisIsGoingToGoOnForALongLon345678901234567890123456789012345_2017_03_10_3.txt

It's still odd though because Explorer and the console can barely handle these files.
I was surprised to see that I could double-click the file from Explorer and windows notepad came up and let me edit it. Explorer won't let me copy the path out though -- ends up using the tilde names like:

C:\Users\ROGER~1.DEU\temp\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\ABCDEF~1\X_ABCD~1\Y_ABCD~1

Thanks for all your help. I think this was a good discussion.

Re: Win10 FileSystem: File / Path too long

Randor 8-Mar-17 13:10

Randor

8-Mar-17 13:10

raddevus wrote:
Explorer won't let me copy the path out though -- ends up using the tilde names like:
C:\Users\ROGER~1.DEU\temp\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\ABCDEF~1\X_ABCD~1\Y_ABCD~1

That is for backwards compatibility with DOS and FAT partitions... 8.3 short names[^]. That feature can be disabled:
How to disable 8.3 file name creation on NTFS partitions.[^]

You can also pass that 8.3 short name to the GetLongPathName function[^] and get the full path.

Best Wishes,
-David Delaune

For My Birthday...

Bootking7-Mar-17 3:23

Bootking

7-Mar-17 3:23

I want someone to suggest an air supply I can use to automate a slide whistle.

Fish tank air pumps do not produce enough air flow. I'm trying to stay away from anything that requires an air tank and anything that's louder than a slide whistle. I tried a 5v 120mm ducted blower fan with a 3d printed nozzle on it and I was suprised that didn't do it.

Suggestions?

Welcome to the Lounge