|
My task is as follows...
I have 2 data files a Source (A) and a Destination (B) both located in the shared network drive.
A is a data file (plain text) between 2GB and 3GB made up of records of fixed size.
A is always being written into by other processes (new records being appended at the end).
I would like to read through the whole file A, read each record, leave what I still need and archive the rest to B.
There is a 10 character string I can use within each record as a filter.
Any tips how to go about this? Is MMF the path to take?
Thanks for any help.
----------------------------------------------------------
Lorem ipsum dolor sit amet.
|
|
|
|
|
Read it in in chunks. fopen() does that for you, but you could optimize the processing by doing it yourself. Since you are using fixed size records, this is even easier to code.
MMF is definitely NOT the way you want to do this for several reasons include the size of the file and that it's on a network resource.
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
actually, i am using MMF for this reason (the size of the file). With MMF, I can specify in MapViewOfFile which part of the file to load, instead of opening the file the usual way which loads the whole file in memory. Also, I can just write to the file via the mapped region of memory directly. This would be tricky in the usual way as I have to remove records in place (meaning if a record does not fit the filter, i have to write in its place the next record that does.)
----------------------------------------------------------
Lorem ipsum dolor sit amet.
|
|
|
|
|
Since you are reading sections of the file sequentially, memory mapped files offer zero benefit. Since shifting the window of the memory mapping requires operations, you are making the program slower for no reason. I fail to see how this is tricky using multiple buffers and straight I/O.
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
maybe i'm missing something but it is mainly for the operations i am performing to file A which is large.
say, i partition file A in blocks of 64kb. i read block1 and filter those records i am not interested in. i write back to this same block1 those i want to keep. when i am finished with this block, i move to the next block2. i may still be not done writing to block1 because i will always be interested in less than or equal records to the original.
therefore i have 2 mapviews of fileA, view1 is just reading which is always the same or advanced in the file as view2 which writes back.
the other point i think MMF is useful is the fact the file is too large as to load entirely in memory via the usual file open operations. i hope i make sense to you? do you still think i could do this same as or better with usual fopen?
----------------------------------------------------------
Lorem ipsum dolor sit amet.
|
|
|
|
|
I understand what you're doing and still wonder why you don't just use CreateFile() and two buffers?
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
I've finished my small program as I described. Out of curiosity, how do you suggest I do it using CreateFile and 2 buffers? Thanks.
----------------------------------------------------------
Lorem ipsum dolor sit amet.
|
|
|
|
|
Use CreateFile() to open the file. Allocate a read and write buffer that are multiples of the record size. Each buffer would have an associated offset. Start reading records. Once you find records to be extracted, set the second buffer to that offset and start copying records into it as needed and flushing it when required.
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
You can use ReadDirectoryChanges WIN 32 API, which registers for file changes in a directory, and use GetQueuedCompletionStatus function to wait for the changes. So when ever new records get added to A you will get notification of the changes, and read the added record and archive it to B.
You can store a pointer on where you are in the file (like the number of bytes read) and continue from it when the next change arrive so that you don't have to read the whole file every time a change occurs.
If you want to delete contents from the file though you will need some kind of process synchronization to avoid simultaneous write.
|
|
|
|
|
I have a linux application, that i want to use to find and manually read/edit the windows Vista/7 BCD file. I cannot run any existing win32 applications, I need to hard code one in C/C++
Please help
Thank You
~Salil
|
|
|
|
|
So what's your question?
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
My problem is that the windows Boot Configuration Data is a binary file, which cannot be accesses directly, and since my working environment is Linux, I cannot use any Microsoft released code or any third party windows application
So where can I find information on how to read and modify the windows BCD using linux (like how is this file structured etc.)
Thanks a lot
~Salil
|
|
|
|
|
You could try using CRT code to read from the c:\boot\bcd file. If Windows is up and running, however, it my have that file opened/locked exclusively.
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
I'm sorry, I'm not familiar with "CRT code" could you also please suggest some resource where I can find out about this?
|
|
|
|
|
SalilJain wrote: could you also please suggest some resource where I can find out about this?
See here.
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
Hello,
I'm a developer and one of my faved languages is C++. Now I've been coding a lot with Visual Studio and I can do lot's of things with C++. Time to time I get interested in Linux projects, Opensource libraries and other stuff. So I tried to use Eclipse in various ways to work out my C++ stuff, but I get so much in trouble that I give up before even writing a few lines of code. The problem is that I get confused with using Eclipse CDT, the Linux handles libraries and dependency nightmares. For example, I install a library, say Libxml++ (which is a c++ wrapper for libxml). Now apt (using ubuntu) will install it in /usr/include/libxml++-2.6/libxml++ . The include files then get nervous as they reference simply <libxml++ stuff.h="">. So GCC, and it will complain with errors. Now this is the usual kind of problem I get in when I try to do such things. And I end up posting to forums, getting a reply days later, solving the problem in an impossible to know way, while in the meantime I lost any interest in what I wanted to do and I just don't go on with it anymore. Now as it is the nth time I get into this, I ask: is it so complicated or is it just that I miss some reading, some special reference???? Is it possible that on visual studio you can code thinking about code, while on linux you have to spend so much time in setting up the environment??? I'd be really interested in using OSS software and libs, or even partecipating to projects, but it is so difficult to get in. So, what are your suggestions? What should I read, do, experience?
|
|
|
|
|
When you install Visual Studio, you are installing a lot of libraries for working with the OS and a lot of associated libraries. But if you install any other third-party libraries (like libxml++, for example), then you'll go through similar pain as with Linux development.
You either need to set environment variables to tell GCC where to find header and library files[^], or specify those directories as suitable GCC options in your makefile (or project file, if that's what Eclipse CDT uses).
For example - I program on Mac OS X (which is Unix underneath the GUI). I have the following environment variables defined to find Boost and the libraries I've installed with MacPorts (equivalent to Ubuntu's apt-get):
CPLUS_INCLUDE_PATH=/usr/local/include/boost-1_38/boost/tr1/tr1/:/usr/local/include/boost-1_38/:/opt/local/include:
LIBRARY_PATH=/usr/local/lib/:/opt/local/lib:
So, I've just added libxml++ (or libxmlxx as MacPorts calls it):
sudo port install libxmlxx
---> Fetching libxmlxx
---> Attempting to fetch libxml++-1.0.5.tar.bz2 from ftp:
---> Attempting to fetch libxml++-1.0.5.tar.bz2 from ftp:
---> Attempting to fetch libxml++-1.0.5.tar.bz2 from http:
---> Verifying checksum(s) for libxmlxx
---> Extracting libxmlxx
---> Configuring libxmlxx
---> Building libxmlxx
---> Staging libxmlxx into destroot
---> Installing libxmlxx @1.0.5_0
---> Activating libxmlxx @1.0.5_0
---> Cleaning libxmlxx
I know port installs into /opt/local. Sure enough, in /opt/local/include, I see a directory for libxml++-1.0. I now know that to use libxml++, I can add /opt/local/include/libxml++-1.0/ to CPLUS_INCLUDE_PATH or (and this is more likely) I can add the option -I/opt/local/include/libxml++-1.0/ to the gcc command-line.
g++ -c -O2 -I/opt/local/include/libxml++-1.0/ program-that-uses-libxml++.CC
If you use a makefile, then add -I/opt/local/include/libxml++-1.0/ to the CXXFLAGS make variable:
CXXFLAGS += -I/opt/local/include/libxml++-1.0/
That will add the include folder option to the gcc command line only when compiling C++ files.
HTH!
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
can any body help me out what is this error.or does we have to include any lib or header file to include.
|
|
|
|
|
It's contained in the Exchange Server 5.5 SDK samples. Microsoft no longer supports Exchnage 5.5 and this link[^] seems to be the only link that contains any useful Exchange 5.5 stuff - including the function you want. Look in the LibSamples/AddrLkup (think that's what it's called) directory - the function source code is in there.
BTW - if yu don't already have the Exchange 5.5 SDK, you're out of luck. That function requires that SDK and Microsoft have pulled it.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
Hi everyone
I have one question about the perfect way that can help me find what I need (functions,classes..or libraries) for example if I want to find all functions to run any program from my program,what is the point should I start my search from it?
could anyone help me and explain her/his way to find any function from a hundred functions exist in VC++.net development environment
|
|
|
|
|
On first reading this sounds like a bit of a daft question, but on thinking about it I can see what you mean. If you're not very familiar with .net and all it contains, then trying to work out which classes, methods or events are relevant to your problem can be quite difficult, and a lot of work.
Even those of us that do know it reasonably well (at least I think I do) can still sometimes get confused trying to find the best way to do things.
I don't know what other people do, but I generally start with Google, and use a search string that narrows down the results as much as possible. Be very specific, and think about what it is you're trying to achieve, what your problem is, then add extra search terms to look for specific information areas.
For example, someone was asking very recently about drag and drop in a ListView control, so I would search on Google using something like :-
"drag and drop" .net C++ ListView
which gives a lot of very good results.
Or, for the question below, on detecting screen resolution changes, I used :-
.net c++ windows detect screen "resolution change"
The Code Project articles are also a very good source of information, and using the advanced search mode there can also narrow down the results to the language or platform you're interested in. And there are a few other forums which have good articles and knowledge bases.
Finally, make sure you have Intellisense turned on, and have a browse through it - you can learn a lot about what a particular control, type, object etc. can do just by looking at its properties, events and methods.
There are three kinds of people in the world - those who can count and those who can't...
|
|
|
|
|
Hi!
I need to detect screen resolution change cause i am working in a toolbar that docks in the borders of the screen.
Thanks
|
|
|
|
|
Easy peasy.
Handle the WM_DISPLAYCHANGE message.
Nnnnext!
|
|
|
|
|
Thats what I get for answering a post before my first morning cup of coffee. I didnt even realize that was what he was asking. 
Best Wishes,
-David Delaune
|
|
|
|
|
If you us the GetSystemMetrics Function[^]
int x = GetSystemMetrics(SM_CXSCREEN);
int y = GetSystemMetrics(SM_CYSCREEN);
On a multi-monitor system you can use SM_CXVIRTUALSCREEN and SM_CYVIRTUALSCREEN.
You could also use the GetDeviceCaps Function[^]
int x = GetDeviceCaps(hDC, HORZRES);
int y = GetDeviceCaps(hDC, VERTRES);
Best Wishes,
-David Delaune
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
