Click here to Skip to main content
15,896,727 members
Home / Discussions / C / C++ / MFC
   

C / C++ / MFC

 
GeneralRe: Need help with MDI Pin
britonleap26-Jun-04 9:29
britonleap26-Jun-04 9:29 
Questionstl's list.. how to delete an item of the list? Pin
kfaday28-Apr-04 14:19
kfaday28-Apr-04 14:19 
AnswerRe: stl's list.. how to delete an item of the list? Pin
f6428-Apr-04 14:43
f6428-Apr-04 14:43 
GeneralChanging source files location Pin
luddet28-Apr-04 14:08
luddet28-Apr-04 14:08 
GeneralRe: Changing source files location Pin
luddet29-Apr-04 11:47
luddet29-Apr-04 11:47 
GeneralPlease help Pin
Stefan Troschuetz28-Apr-04 10:10
Stefan Troschuetz28-Apr-04 10:10 
GeneralRe: Please help Pin
Christian Graus28-Apr-04 10:57
protectorChristian Graus28-Apr-04 10:57 
GeneralUsing ifstream to read huge binary files Pin
Ron Hinthorn28-Apr-04 9:51
Ron Hinthorn28-Apr-04 9:51 
Does anyone know how to use ifstream to read binary files over 4 GB in size?

Environment: Microsoft Visual C++ 6.0.

We have a simulator that can generate huge amounts of time-series data. We store this data in binary files. We then read that data as needed for charts, map views, etc. To read a particular item, we use the known layout of the binary files to compute the offset from the beginning of the file:

m_iFstream.open(strFName, ios::in | ios::binary);
int totalOffset = timeStepOffset + ObjectOffset + attributeOffset;
char* buf = NULL;
buf = new char[nLength+1];
m_iFstream.seekg(totalOffset, ios::beg);
m_iFstream.read((char*)buf,(sizeof(char)) * nLength);
buf[nLength] = '\0';

We then copy the read buffer into float.

This works great until the offset exceeds the MAX_INT value. To overcome that, I tried a series of relative offsets as needed, using 64-bit integers to provide a little "head room":

m_iFstream.open(strFName, ios::in | ios::binary);
__int64 llTotalOffset = llTimeStepOffset + objectOffset + attributeOffset;
char* buf = NULL;
buf = new char[nLength+1];
int nRemainingOffset;
if (llTotalOffset < INT_MAX)
{
// Single, direct seek is possible.
//m_iFstream.seekg(0);//seek to begining of file
nRemainingOffset = llTotalOffset;
m_iFstream.seekg(nRemainingOffset,ios::beg);//seek from the beginning of the file
}
else
{
// The offset exceeds what an int can represent.
// Must iteratively seek to the location.
int nMaxSeek = INT_MAX;
m_iFstream.seekg(nMaxSeek, ios::beg); // first seek is from the beginning
llTotalOffset -= nMaxSeek;
while(llTotalOffset >= nMaxSeek)
{
m_iFstream.seekg(nMaxSeek, ios::cur);
llTotalOffset -= nMaxSeek;
}
nRemainingOffset = llTotalOffset;
if (nRemainingOffset > 0)
m_iFstream.seekg(nRemainingOffset, ios::cur);
}
m_iFstream.read((char*)buf,(sizeof(char)) * nLength);
buf[nLength] = '\0';

In theory, this should work. In practice, it doesn’t work consistently. I think the failure must be due to the ifstream class not itself being able to store current file offset (ie same issue we had with variable ‘totalOffset’).

Does anyone know how I might be able to use ifstream in this context?

Note that I did look into declaring a specialized char_traits for the stream. I’m no expert with STL file I/O, but it appears to me that the ‘off_type’ and ‘pos_type’ have to correspond to the ‘char_type’ – so I’m hesitant to try this since we are using the file with standard ‘char’ character type.

Thanks in advance for your advice


Ron Hinthorn
GeneralHi,MFC and big data files Pin
Jamie Kenyon28-Apr-04 8:58
Jamie Kenyon28-Apr-04 8:58 
GeneralRe: Hi,MFC and big data files Pin
David Crow28-Apr-04 9:02
David Crow28-Apr-04 9:02 
GeneralRe: Hi,MFC and big data files Pin
Jamie Kenyon28-Apr-04 12:21
Jamie Kenyon28-Apr-04 12:21 
GeneralRe: Hi,MFC and big data files Pin
l a u r e n28-Apr-04 14:29
l a u r e n28-Apr-04 14:29 
GeneralRe: Hi,MFC and big data files Pin
Jamie Kenyon28-Apr-04 23:01
Jamie Kenyon28-Apr-04 23:01 
GeneralRe: Hi,MFC and big data files Pin
David Crow29-Apr-04 2:40
David Crow29-Apr-04 2:40 
GeneralPut the suo and ncb files in another directory Pin
Anonymous28-Apr-04 8:03
Anonymous28-Apr-04 8:03 
GeneralRe: Put the suo and ncb files in another directory Pin
Anonymous28-Apr-04 8:18
Anonymous28-Apr-04 8:18 
GeneralRe: Put the suo and ncb files in another directory Pin
David Crow28-Apr-04 8:55
David Crow28-Apr-04 8:55 
GeneralRe: Put the suo and ncb files in another directory Pin
Anonymous28-Apr-04 9:15
Anonymous28-Apr-04 9:15 
QuestionThread and UpdateData?? Pin
bryanbryan28-Apr-04 7:12
bryanbryan28-Apr-04 7:12 
AnswerRe: Thread and UpdateData?? Pin
David Crow28-Apr-04 7:25
David Crow28-Apr-04 7:25 
GeneralToolbox and VC++.NET Pin
pmdanger28-Apr-04 7:03
pmdanger28-Apr-04 7:03 
GeneralPassing classes into threads Pin
roadragedave28-Apr-04 6:58
roadragedave28-Apr-04 6:58 
GeneralRe: Passing classes into threads Pin
David Crow28-Apr-04 7:19
David Crow28-Apr-04 7:19 
GeneralRe: Passing classes into threads Pin
roadragedave28-Apr-04 8:31
roadragedave28-Apr-04 8:31 
GeneralRe: Passing classes into threads Pin
David Crow28-Apr-04 8:52
David Crow28-Apr-04 8:52 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.