|
Sander Rossel wrote: Ugh, horrible! We have an automated Jenkins build
Right! Thanks for posting about this. That's the kinds of things we are seeing.
I was wondering if others have struggled with it too.
It's a real bother. 
|
|
|
|
|
Hi,
Wow, you wrote a disk monitoring application but did not know about NTFS filename limits? How is that even possible? Are you a .NET coder or something?
raddevus wrote: So I'm just curious. Does anyone know how this can even work? How can Visual Studio and/or Android Studio continue to work when the files they are attempting to create are too long?
You could always create files longer than MAX_PATH ... even on Windows NT 4 all you do is prefix them with the "\\?\" prefix and use the wide version of CreateFile or directly call NtCreateFile. The MAX_PATH limitation only applies to a few Windows API such as shell32 and of course the explorer shell. Actually most of the operating system supports paths > 260.
Where do you think all those rootkits and viri hide? 
The actual NTFS limit is 32,767 characters. Better look into updating your disk monitoring tool.
Best Wishes,
-David Delaune
P.S.
It gets really interesting when working near the NTFS limits... try passing some long arguments to executables at the true ntfs limit.
|
|
|
|
|
Seems like a troll.
Actually, I wrote an app today that calls CreateFileW() API method.
So I'm not sure if you've actually written an app that can create a file that has that many chars either. I ran the app and you get a windows error (on Win10 and Server 2012 R2) of 123.
Do you know what that error is?
Well, that's probably because you have no experience developing apps. 
So that error is
Quote: ERROR_INVALID_NAME
123 (0x7B)
The filename, directory name, or volume label syntax is incorrect.
from:
System Error Codes (0-499) (Windows)[^]
Here's the interesting part from the CreateFile docs
Quote: In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend //?/ to the path. For more information, see Naming Files, Paths, and Namespaces.
The program I wrote creates a file using the syntax above and CreateFileW() just fine as long as it is under 260. Then when I pass a long name 350 chars or so, you get the 123 error from GetLastError()
Here's the additionally interesting part from docs about this (just as I mentioned in my original post):
Naming Files, Paths, and Namespaces[^]
Quote: The shell and the file system have different requirements. It is possible to create a path with the Windows API that the shell user interface is not able to interpret properly.
It was interesting, right? That's what this was all about. Have you written a file monitoring tool? It would have to be better than mine. I wrote mine in a few minutes a long time ago.
Whatch'a got? Write one up and post as an article for the community. Seriously it would be great.
modified 7-Mar-17 20:37pm.
|
|
|
|
|
Hmmm,
raddevus wrote: Well, that's probably because you have no experience developing apps.
You do realize that you are on a forum with old MSFT engineers from the operating systems group right?
The CreateFileW[^] function can absolutely create files with paths greater than MAX_PATH.
Best Wishes,
-David Delaune
|
|
|
|
|
Randor wrote: You do realize that you are on a forum with old MSFT engineers from the operating systems group right in the CP Lounge?
Chill a little, Bud.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
Randor wrote: The CreateFileW[^] function can absolutely create files with paths greater than MAX_PATH.
I don't dispute that it can. I am saying there may be some problem (or trick that must be applied) on these OSes. I would love to discuss this further, but I'm afraid it'll be quite a long post. I hope you might be interested.
I hope you'll really try this and post back.
Okay, so I'm actually pinvoking the API method from a .NET framework. I will provide the entire code for you and you can easily run it in LINQPad (free utility at LINQPad - The .NET Programmer's Playground[^]
Running the Source Code
If you get that tool you can copy paste the entire listing directly into a new linqpad "query" and it'll run normally as any other .NET program. Of course, you can easily create a console app in VStudio 2015 or whatever. The code will still run.
MessageBoxW
If you run the code you will see that it will successfully call MessageBoxW and display some text in the message box. That was a test to insure that the text of the path was being marshalled properly.
You can see the pinvoke has the charset set to Auto and it does indeed convert and display the chars properly. That's important since we are going to pass the string for the text file.
If you set the Charset to Charset.Ansi, you'll see nonsense kanji characters or something since this is a W method.
Why MessageBoxW?
I'm simply setting up the idea that my pinvoke does seem to work properly, the string marshalling is correct and the error I get when attempting to create the file is more likely related to the file system or API than it is to a bad path string being passed.
Note: You'll need to change the path to a location that exists on your c:\ drive
So, if you run the code
*the MessageBox will pop up
*the code will attempt to create a file with a long name and fail
*the code will attempt to create a file with a shorter name and succeed -- proves the call to CreateFileW does indeed work.
The output (after the messagebox displays )looks like the following:
Output 3/8/2017 8:31:59 AM
Failed!
HRESULT: -2147024773 Error: 123
Seems to work..
The Error 123 is the error from the OS (GetLastError) which we've talked about.
The "Seems to work.." is the successful creation of the 2nd file (which is not a long file name) to prove that the call to CreateFileW() does indeed work as I've defined it and the pinvoke seems to be set up correctly.
What Do You Think?
Can you explain this? Can you provide a very small example of working code (written in C++ or something) which will actually create a long file name?
I put a lot into this so I hope it has caught your interest.
Great discussion.
[DllImport("kernel32.dll", SetLastError=true, CharSet=CharSet.Auto)]
public static extern SafeFileHandle CreateFileW(
String lpFileName,
EFileAccess dwDesiredAccess,
EFileShare dwShareMode,
IntPtr lpSecurityAttributes,
ECreationDisposition dwCreationDisposition,
EFileAttributes dwFlagsAndAttributes,
IntPtr hTemplateFile);
[DllImport("user32.dll", CharSet=CharSet.Auto)]
static extern int MessageBoxW(IntPtr hWnd, String text, String caption, int options);
void Main()
{
MessageBoxW(IntPtr.Zero, @"\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt", "Caption", 0);
Console.WriteLine(DateTime.Now);
SafeFileHandle sfh = CreateFileW(@"\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt",
EFileAccess.GenericWrite, EFileShare.Read,
IntPtr.Zero, ECreationDisposition.CreateAlways,
EFileAttributes.Archive, IntPtr.Zero);
if (sfh.IsInvalid ){
Console.WriteLine("Failed!");
int HRESULT = Marshal.GetHRForLastWin32Error();
int ERRORVAL = Marshal.GetLastWin32Error();
Console.WriteLine(String.Format("HRESULT: {0} Error: {1}",HRESULT, ERRORVAL));
}
else{
Console.WriteLine("Seems to work..");
}
sfh = CreateFileW(@"\\?\C:\Users\<username>\temp\notlongName_2017_03_11_2.txt",
EFileAccess.GenericWrite, EFileShare.Read,
IntPtr.Zero, ECreationDisposition.CreateAlways,
EFileAttributes.Archive, IntPtr.Zero);
if (sfh.IsInvalid ){
Console.WriteLine("Failed!");
int HRESULT = Marshal.GetHRForLastWin32Error();
int ERRORVAL = Marshal.GetLastWin32Error();
Console.WriteLine(String.Format("HRESULT: {0} Error: {1}",HRESULT, ERRORVAL));
}
else{
Console.WriteLine("Seems to work..");
}
}
public enum EFileAccess : uint
{
AccessSystemSecurity = 0x1000000,
MaximumAllowed = 0x2000000,
Delete = 0x10000,
ReadControl = 0x20000,
WriteDAC = 0x40000,
WriteOwner = 0x80000,
Synchronize = 0x100000,
StandardRightsRequired = 0xF0000,
StandardRightsRead = ReadControl,
StandardRightsWrite = ReadControl,
StandardRightsExecute = ReadControl,
StandardRightsAll = 0x1F0000,
SpecificRightsAll = 0xFFFF,
FILE_READ_DATA = 0x0001,
FILE_LIST_DIRECTORY = 0x0001,
FILE_WRITE_DATA = 0x0002,
FILE_ADD_FILE = 0x0002,
FILE_APPEND_DATA = 0x0004,
FILE_ADD_SUBDIRECTORY = 0x0004,
FILE_CREATE_PIPE_INSTANCE = 0x0004,
FILE_READ_EA = 0x0008,
FILE_WRITE_EA = 0x0010,
FILE_EXECUTE = 0x0020,
FILE_TRAVERSE = 0x0020,
FILE_DELETE_CHILD = 0x0040,
FILE_READ_ATTRIBUTES = 0x0080,
FILE_WRITE_ATTRIBUTES = 0x0100,
GenericRead = 0x80000000,
GenericWrite = 0x40000000,
GenericExecute = 0x20000000,
GenericAll = 0x10000000,
SPECIFIC_RIGHTS_ALL = 0x00FFFF,
FILE_ALL_ACCESS =
StandardRightsRequired |
Synchronize |
0x1FF,
FILE_GENERIC_READ =
StandardRightsRead |
FILE_READ_DATA |
FILE_READ_ATTRIBUTES |
FILE_READ_EA |
Synchronize,
FILE_GENERIC_WRITE =
StandardRightsWrite |
FILE_WRITE_DATA |
FILE_WRITE_ATTRIBUTES |
FILE_WRITE_EA |
FILE_APPEND_DATA |
Synchronize,
FILE_GENERIC_EXECUTE =
StandardRightsExecute |
FILE_READ_ATTRIBUTES |
FILE_EXECUTE |
Synchronize
}
[Flags]
public enum EFileAttributes : uint
{
Readonly = 0x00000001,
Hidden = 0x00000002,
System = 0x00000004,
Directory = 0x00000010,
Archive = 0x00000020,
Device = 0x00000040,
Normal = 0x00000080,
Temporary = 0x00000100,
SparseFile = 0x00000200,
ReparsePoint = 0x00000400,
Compressed = 0x00000800,
Offline = 0x00001000,
NotContentIndexed= 0x00002000,
Encrypted = 0x00004000,
Write_Through = 0x80000000,
Overlapped = 0x40000000,
NoBuffering = 0x20000000,
RandomAccess = 0x10000000,
SequentialScan = 0x08000000,
DeleteOnClose = 0x04000000,
BackupSemantics = 0x02000000,
PosixSemantics = 0x01000000,
OpenReparsePoint = 0x00200000,
OpenNoRecall = 0x00100000,
FirstPipeInstance= 0x00080000
}
[Flags]
public enum EFileShare : uint
{
None = 0x00000000,
Read = 0x00000001,
Write = 0x00000002,
Delete = 0x00000004
}
public enum ECreationDisposition : uint
{
New = 1,
CreateAlways = 2,
OpenExisting = 3,
OpenAlways = 4,
TruncateExisting = 5
}
|
|
|
|
|
raddevus wrote: What Do You Think?Can you explain this?
I am getting old and don't really See Sharp.
However I do see several problems with the code you have written.
1.) You are using some invalid/reserved characters in your path. The < and > are reserved characters.
2.) You are incorrectly assuming that you can make a really long directory entry of 32,767 characters.
3.) You are mistakenly attempting to create directories with the CreateFile Function[^]
Follow these rules:
1.) Individual path components such as file names and directory names should be <= 255 characters. (This is probably why your code fails.)
2.) The entire path can be up to 32,767 wide characters.
3.) Prepend your path with //?/ and use an absolute path.
4.) Use CreateDirectoryW for creating directories and CreateFileW to create files.
Best Wishes,
-David Delaune
|
|
|
|
|
Well, I appreciate you replying.
But...
Randor wrote: 1.) You are using some invalid/reserved characters in your path. The < and > are reserved characters.
1. My path doesn't actually contain any < or > That was just for you to replace. You know, Microsoft docs use < > to denote variable values, that's what I was doing.
I was saying add your <username> directory into the path so it will run on your machine.
\\?\c:\users\<username>\temp\superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt Note: that's really all one line but this editor makes it look odd.
Randor wrote: 2.) You are incorrectly assuming that you can make a really long directory entry of 32,767 characters.
2. No, the path already exists on my computer. That's a temp path that was created long ago. That's why I use that same path to create the other file (shorter name) which successfully calls CreateFileW.
Randor wrote: 3.) You are mistakenly attempting to create directories with the CreateFile Function[^]
3. again, no the path is already created.
If I do not provide any path in the string then it will create the file in the EXE path of my executable so I'm just pointing it to a path that already exists.
Finally, I actually am following those rules. Do you have any other ideas?
Thanks
|
|
|
|
|
Hi,
I gave you absolutely all the information needed to create a path greater than MAX_PATH (260). Everything in my prior analysis is correct.
You seem extremely argumentative and/or having trouble understanding me.
Let me be specific:
raddevus wrote: Finally, I actually am following those rules. Do you have any other ideas?
No, you are not. Look again:
1.) Individual path components such as file names and directory names should be <= 255 characters. (This is probably why your code fails.)
2.) The entire path can be up to 32,767 wide characters.
The filename: superlongfilenameand1234567890thisoneisgoingintherecordbooks1234567890andwillprobablytoastthisentire1234567890thingandiwonderwhatitisgoing1234567890todowhenilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink123456789012345678901234567890123456789012345678901234567890_2017_03_10_2.txt
This filename 312 characters length which is greater than 255 characters. Refer to what I said in rule 1.
Best Wishes,
-David Delaune
|
|
|
|
|
Sheesh.
I'm an engineer. You wanted me to be exact and I was exact in my language and so I was.
I honestly wasn't being argumentative. I was being precise. I was enjoying our discussion.
Keep in mind that you started this entire conversation and your introduction with the following:
Randor wrote: Wow, you wrote a disk monitoring application but did not know about NTFS filename limits? How is that even possible? Are you a .NET coder or something?
That was argumentative and not a fantastic introduction.
You also set up the original problem like the following:
Randor wrote: You could always create files longer than MAX_PATH ... even on Windows NT 4 all you do is prefix them with the "\\?\"
Plus read the exact wording on lpFileName (not path) at the Microsoft docs:
Quote: lpFileName [in]
The name of the file or device to be created or opened. You may use either forward slashes (/) or backslashes (\) in this name.
In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path.
The Focus Seemed to Be On FileName -- not filename + path
This all set my thinking-context so I was thought, "Hey, I can create files that are 32,767 chars long. Let's do it!!"
That's why I've been focusing on CreateFileW. I will add CreateDirectoryW() now and work with that.
Alas, this is path + filename. 
Surely, you can see how confusing this has been.
|
|
|
|
|
raddevus wrote: Plus read the exact wording on lpFileName (not path) at the Microsoft docs:Quote: lpFileName [in]The name of the file or device to be created or opened. You may use either forward slashes (/) or backslashes (\) in this name.In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend "\\?\" to the path.
I can see why this is unclear. Keep in mind that you can use CreateFile to open COM ports and various other devices. It can also be used to open files on storage devices. This is why it is important to train yourself to never pay any attention to any variable names.
Looks like the engineer decided to name that variable lpFileName when the variable is actually [drive][path][filename] or [IO device].
Even in Windows... Everything is a File[^].
Best Wishes,
-David Delaune
|
|
|
|
|
Randor wrote: I can see why this is unclear. Keep in mind that you can use CreateFile to open COM ports and various other devices.
Thanks and yes I got stuck in a thinking-track and couldn't move out of it.
I have successfully used CreateDirectoryW and CreateFileW now to create extremely long file paths / names.
It all works great.
\\?\C:\Users\<username>\temp\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\12345678901234567890\abcdefghijklmnopqrstuvwxzy1234abcdefghijklmnopqrstuv\X_abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234_X\Y_abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234abcdefghijklmnopqrstuvwxyz1234_Y/superlongfilenamean12henilookatthefiletest1234567890ThisIsGoingToGoOnForALongLongLongLongLongTimeIthink1234567890123456789012superlongfilenamean12henilookatthefiletest1234567890ThisIsGoingToGoOnForALongLon345678901234567890123456789012345_2017_03_10_3.txt
It's still odd though because Explorer and the console can barely handle these files.
I was surprised to see that I could double-click the file from Explorer and windows notepad came up and let me edit it. Explorer won't let me copy the path out though -- ends up using the tilde names like:
C:\Users\ROGER~1.DEU\temp\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\ABCDEF~1\X_ABCD~1\Y_ABCD~1
Thanks for all your help. I think this was a good discussion.
|
|
|
|
|
raddevus wrote: Explorer won't let me copy the path out though -- ends up using the tilde names like:
C:\Users\ROGER~1.DEU\temp\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\123456~1\ABCDEF~1\X_ABCD~1\Y_ABCD~1
That is for backwards compatibility with DOS and FAT partitions... 8.3 short names[^]. That feature can be disabled:
How to disable 8.3 file name creation on NTFS partitions.[^]
You can also pass that 8.3 short name to the GetLongPathName function[^] and get the full path.
Best Wishes,
-David Delaune
|
|
|
|
|
I want someone to suggest an air supply I can use to automate a slide whistle.
Fish tank air pumps do not produce enough air flow. I'm trying to stay away from anything that requires an air tank and anything that's louder than a slide whistle. I tried a 5v 120mm ducted blower fan with a 3d printed nozzle on it and I was suprised that didn't do it.
Suggestions?
|
|
|
|
|
Happy birthday!
I'd start by measuring the airflow and pressure you need - until you know that, you're just whistling in the dark...
(But I'd probably use a tanked solution with a pressure sensitive top-up motor and a regulator. If noise is a problem, you can move the tank and pump well away from the whistle if you use big enough bore tubing.)
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
Thanks Griff.
Best I can figure from my research, musical instruments with a fipple (mouthpiece with a block or channel in them) require 1 PSI or less. A trumpet operates at up to 3 PSI, and the most that a stupendous human being can produce is 5 or 6 PSI.
The problem is mostly in volume.
|
|
|
|
|
How about mechanised bellows? A pair of small bellows arranged in an one-up-one-down, couple of return springs and a cammed motor should do it.
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
|
|
|
|
|
Interesting concept. I'll have to see what I can do with a single manual bellow and see if this is feasible. Also I like that using a bellow gives me a way to put numbers on what sort of air flow I require. Thanks again.
|
|
|
|
|
They're not cheap, but I once used my CPAP machine to inflate a queen-sized air mattress in less than 2 minutes - and they are very quiet.
Cheers,
Mick
------------------------------------------------
It doesn't matter how often or hard you fall on your arse, eventually you'll roll over and land on your feet.
|
|
|
|
|
That's actually pretty funny. But wow, expensive.
|
|
|
|
|
I've just had to replace my old one - nothing wrong with the mechanism, just one of the silicone seals where the humidifier attaches split, and Fisher & Paykel no longer carry spare parts (model was discontinued about 5 years ago). If you were local, I could let you try it and/or have it cheap. Cost me over $1000 just for some split silicone, which was bloody annoying.
Cheers,
Mick
------------------------------------------------
It doesn't matter how often or hard you fall on your arse, eventually you'll roll over and land on your feet.
|
|
|
|
|
Compressed air can? (Of the sort used for airbrush painting). It would probably need a restricting valve to reduce air flow, and of course when it's gone it's gone. But a simple press on the nozzle starts airflow and there's no mechanical noise.
|
|
|
|
|
Interesting. I didn't even know these existed. I would still prefer something with limitless on-demand air supply but this could be worth trying if there's a reasonable way to switch the supply on and off with an Arduino.
|
|
|
|
|
Just don't throw your birthday balloons away.
I wanna be a eunuchs developer! Pass me a bread knife!
|
|
|
|
|
How about your local MP/Congresscritter? 
I see only three problems:
- They tend to leak
- Turning them off is a bit of a problem
- You can't control them from an Arduino
If you have an important point to make, don't try to be subtle or clever. Use a pile driver. Hit the point once. Then come back and hit it again. Then hit it a third time - a tremendous whack.
--Winston Churchill
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
