Click here to Skip to main content
15,892,161 members
Please Sign up or sign in to vote.
3.50/5 (2 votes)
See more:
I'm trying to write a program that moves doubles out of memory onto disk for later recovery and processing. A typical file size is around 30GBytes. I'm using ofstream for binary output and ifstream for binary retrieval.

The problem I've encountered is with the maximum size of the types "steamsize" and "ifstream::pos_type" both of which, being variants of int, are too small to manage file transfers of the size needed. Outputting to disk doesn't seem to be a problem but reading blocks of the binary file from disk means that the read byte position in the file is too large for the default "pos_type" and the size of the block read back is too large for "streamsize".

Can anyone tell me if it's possible to alter the type def of "pos_type" and "streamsize" within ifstream and ofstream to __int64 and if so how it can be done? Are there any other methods of moving blocks of say 30Mb from within 30GB of binary stuff on disk to memory fast?

Any help would be very much appreciated.
David G
Posted
Comments
Dark Darkling 22-Oct-13 6:56am    
Is it really needed to use STL? You could use standard C functions such as fopen(), fread(), fclose() and etc. In this case just only one thing you need for proper work that is to define '#define _FILE_OFFSET_BITS 64'.
Dark Darkling 22-Oct-13 7:02am    
I could be wrong but it seems that this problem has been solved in VC11.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
Top Experts
Last 24hrsThis month


CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900