|
A common beginner's mistake I see when doing file I/O is writing or reading one byte at a time (or a very small number of bytes). You can greatly increase I/O time by writing/reading good sized chunks at a time. Perhaps this is your issue?
How big to make the buffers is pretty much application dependant, but even going to say several K at a time can make a huge difference.
|
|
|
|
|
Actually a have been considering that fact for a reason:
As an algorithm restriction, I must deal with data byte by byte in order to accomplish the compression. So I begun wondering which was faster:
to read byte by byte or
to have to apply shift rights, bit wise orings and andings to extract the data (into a single byte) and then apply the rest of the algorithm on that byte which also includes those bit wise operations.
I would guess that the 2nd option is faster because it works only with registers and main memory but there quite a bit of those operations on the algorithm.
So my question is: does it really payoff to read memory chunks instead of byte by byte in this particular case?
Thx!
hint_54
|
|
|
|
|
You can read the file in large chunks and process that chunk byte by byte. Even when cached, doing disk I/O one byte at a time is painfully slow.
"Take only what you need and leave the land as you found it." - Native American Proverb
|
|
|
|
|
hmmm... I see! Thx a lot 
hint_54
|
|
|
|
|
Actually file I/O is never done on a byte level. The least bit of information that can be read from ,or written to a file is one sector. A sector is always an even multiple of 128 bytes, the most common sector size is 512 bytes.
So the runtime does buffer (at least) one sector (or more likely, a cluster).
I think the main reason for slowing down file reading a byte at the time, is the function call overhead and all the checks that has to be done, in the runtime, before the runtime can return the byte in question.
|
|
|
|
|
kakan [[]], a few things on that
kakan wrote: So the runtime does buffer (at least) one sector (or more likely, a cluster).
Can't you be more precise on which of those does the runtime buffers at a single shot? A sector or a cluster?
and
What if I read more than just a sector/cluster, will it buffer the necessary sectors/clusters with a single operation or will it take the same amount of time it would if I read the 2, 3, or more sectors/clusters on different operations?
thx!
|
|
|
|
|
Hello.
I'm a bit on thin ice here. For CreateFile, you can set how the file will be acessed. I think MS calls it a "hint" for the file system.
See the docs for CreateFile and all of the FILE_FLAG_-flags.
It's quite informative.
Kakan
|
|
|
|
|
Hello.
I would suggest you to:
1.
Get the cluster size of the disk you are using. Then create a buffer of that size. Do all reads and writes (if possible) with the cluster size.
2.
Turn of stack checking, at least for the functions you use most frequently.
3.
Do not, repeat NOT, use time (and cpu) consuming functions in your code. Especially avoid using the (x)printf functions at all times. It's incredibly time and cpu consuming!
Another question: You say there shouldn't be any limitations to the file size. You are aware that the f-funcs has a file size limit of about 4 GB? If you want to avoid that limitaion, the you got to use the real Win32-functions CreateFile, ReadFile etc.
If you decide to use them instead, then you have the possibility to use overlapped I/O, which might speed up the file I/O.
Else, if you stay with the f-funcs, then considder to use the open(), read(), write(), close() e.t.c. They are closer to the file system than the f-funcs (not much, but its worth trying).
Kakan
|
|
|
|
|
Hi there!
kakan wrote: I would suggest you to:
1.
Get the cluster size of the disk you are using. Then create a buffer of that size. Do all reads and writes (if possible) with the cluster size.
2.
Turn of stack checking, at least for the functions you use most frequently.
3.
Do not, repeat NOT, use time (and cpu) consuming functions in your code. Especially avoid using the (x)printf functions at all times. It's incredibly time and cpu consuming!
By the same order 
1. Ok with that.
2. How do i disable the stack checking?
3. Also ok with that, i'm not using them. 
I have also noted that you advise the use of CreateFile, ReadFile, etc instead of f-functions because of theyr limitation. Does that limitation also apply for the open(), read(), write (and so on) functions? And which are faster: Win32-functions or the DOS open/read/open.. ones?
Thx!
hint_54
|
|
|
|
|
Hello and good morning.
About the stack check, here is a snippet:
#pragma check_stack(off)
/* Funcs that are called often... */
char * _fastcall CWrTapeTh::w32fgets(char *string, int n)
{
....
}
#pragma check_stack(on)
The 4 GB limitation applies to all the old file handling funcs, bot the f-funcs (fopen, fwrite, ...) and open, write.
The reason for this limitation is a 32-bit value (unsigned long, I think), that holds the actual position in the file. And that counter wraps at (approx) 4 GB.
The Win32-funcs doesn't have that limit of file size.
Which one is fastest? To be honest, I don't know, really. But the Win32-funcs are the only way to go if you want to be able to handle files of any size.
My guess is that the Win32-funcs can be really fast.
Besides, (as I said in my earlier post), the Win32-funcs can use overlapped I/O, which means that you can have several read/writes going on at the same time.
Just try to write a file to a diskette with the f-funcs. Get the time for it.
Then write the same file to the hard drive. Now, copy that file to the diskette. Get the time for the copy. Compare the times. You will see a remarkable difference. Why?
I'm not 100% sure, but my guess is that Windows copy uses overlapped I/O.
I know there is samples of overlapped file I/O at MSDN.
Maybe I should dig deeper in this and post an article at CP?
Kakan
|
|
|
|
|
Hi,
Are there any Gantt chart components on the market,
free or commercial ?
Thanks
|
|
|
|
|
Hi,
Are there any Gantt chart components on the market,
free or commercial ?
Thanks
|
|
|
|
|
Yeap... Try Project2000, its a microsoft office application.
|
|
|
|
|
See:
http://www.ganttchart.com/
http://associate.com/gantt/
http://office.microsoft.com/en-us/assistance/HA010346051033.aspx
"Take only what you need and leave the land as you found it." - Native American Proverb
|
|
|
|
|
Hi
Im actually coding using VC++ with Visual Studio .NET 2003
When i was using VC6, i could do
CString a = "abc";
CString b = "123";
a += b;
and then a would be "abc123".
But now, i dont know what happens, but a ends up being "123"
this is driving me crazy.
The solution i found is doing
strcat(a.GetBuffer(), b.GetBuffer());
But that sucks.
Any suggestions??
|
|
|
|
|
it works well on my version of VS.net 2003 ...
maybe there is something else that you overlook.
Maximilien Lincourt
Your Head A Splode - Strong Bad
|
|
|
|
|
I can't see anything wrong with the sample code you provided. In fact, I just did a simple test and it works as it should
The only difference I know of between VS6 and VS7.1 is that the MFC CString class in MFC7 is templated. But the functionalities are the same.
I Dream of Absolute Zero
|
|
|
|
|
I deleted the whole method and did it again, and now seems to be working fine...
Is strange because isn't the first time something like this happens to me.
Thanks for the suggestions
|
|
|
|
|
Kharfax wrote: strcat(a.GetBuffer(), b.GetBuffer());
Never do that!
What you are doing is appending to a string's internal buffer when you have no idea what that buffer's valid (usable) length is. For example, if a's internal buffer was 6 characters long, your call to strcat(...) would exceed the length of the buffer and would write the terminating NUL character off of the end of the allocated memory.
While your problem sounds odd, I would make sure no one accidently #defined "+=" to be "=" (I have seen it done before, do not ask why). You could also test by adding up to ints in the same way and see if they total correctly.
Peace!
-=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!
DeleteFXPFiles & CheckFavorites (Please rate this post!)
|
|
|
|
|
This one drives me up the walls...
I frequently use the "Called by"-feature, available from the drop down occuring when right-clicking a function symbol in the Class View.
Now, for no apparent reason, this has stopped working!
I have tried deleting all of the files in the Debug-folder, deleting the ncb file, checking that the Build Browse Info box is checked in the project settings and then re-compiling. All to no avail.
Selecting "Called by" for any function renders... nothing. Not even an error message!
Any clues, people! Thanx in advance for any tips on this one!
------------------------------------------------------------
"Pooollyyy!! ...Polly Parrot! Wakey-wakey!"
|
|
|
|
|
A coworker has experienced an interesting problem that involves Java applets when used in a hosted IE situation. I have been able to recreate the issue by using an AppWizard-generated CHTMLView application (with no other code changes made).
The applets seem to load and function correctly; they draw and respond to mouse actions as expected. However, one of the applets contains an action button that submits a form or otherwise transmits data back to the server (the data fields are part of the Java applet's UI), that causes a popup to be generated that confirms the details specified on the form.
When using this series of applets in IE, everything works fine. However, when using them in the hosted browser, some data is not making it back to the server and the generated popup is corrupted (invalid/missing data).
The applets are on a 3-rd party site and we do not have direct access to them for debugging purposes -- so I cannot determine if the Applet is generating the popup directly, or it comes from the server, or a combination of both.
I expect that the Java applet itself is trying to send data back to the server but something is blocking this data when it is hosted.
I have Googled (both the web and Usenet) for information but I have come up empty.
Has anyone seen a problem like this before? Or do they have an idea of what is going on? I am at a loss here...
Peace!
-=- James
If you think it costs a lot to do it right, just wait until you find out how much it costs to do it wrong!
Tip for new SUV drivers: Professional Driver on Closed Course does not mean your Dumb Ass on a Public Road!
DeleteFXPFiles & CheckFavorites (Please rate this post!)
|
|
|
|
|
I have both Visual Studio 6.0 and .NET (2003) versions installed on my PC. I am running an application (compiled for debug but not running under the debugger) and it throws an exception. The OS picks .NET as the debugger but I would like it to pick VC++6.0 as the debugger. Where is this setting?
|
|
|
|
|
I am not sure of the setting but I would say that I have installed .net 2003 just to debug my vc6 applications with the .net debugger. As is a lot better than the one with vc6 especially if you want to do remote debugging. One way to use the .net debugger is to load your project with it .net and let it upgrade to .net then exit. It will create a .sln and .vcproj for .net to load, don't worry it will not trash your .dsp or .dsw files so you still can compile and develop with vc6. I do this all the time.
John
|
|
|
|
|
I'm in case that function returned for me JPEG file data in CString.
so I write the CString data to IStream Object and then save the file but I'm
getting InvalidParameter , i do not know why? here is code snippet
Write(CString image, int len)
{
CLSID encoderClsid;
GetEncoderClsid(L"image/jpeg", &encoderClsid);
LPBYTE lpBits = (LPBYTE) GlobalAlloc(GMEM_MOVEABLE, len);
IStream *pStream;
HRESULT res = ::CreateStreamOnHGlobal(lpBits,FALSE,&pStream);
LPTSTR sImage = image.GetBuffer(len);
pStream->Write(sImage,sizeof(sImage),NULL);
LARGE_INTEGER li;
li.QuadPart = 0;
res = pStream->Seek(li,STREAM_SEEK_SET,NULL);
if(SUCCEEDED(res))
{
Image *jpgImage = Image::FromStream(pStream);
jpgImage->Save(L"C:\\myImage.jpg",&encoderClsid,NULL);
//here Save return InvalidParameter , WHY? any suggestions?
}
}
|
|
|
|
|
All items are grayed out, why?
void CMyFrameWnd::OnRButtonUp(UINT nFlags, CPoint p)
{
CMenu menu;
menu.CreatePopupMenu();
menu.AppendMenu( MF_STRING | MF_ENABLED , 700 , "Zoom All" );
menu.AppendMenu( MF_STRING | MF_ENABLED , 701 , "Properties" );
ClientToScreen(&p);
menu.TrackPopupMenu(TPM_CENTERALIGN,p.x,p.y,this,NULL);
CFrameWnd::OnRButtonUp(nFlags, point);
}
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
