|
Hi
Can you look at the codes i made?
in the project?
|
|
|
|
|
I am reading a huge .txt file which has data arranged in form of rows and columns.
There are 60 columns and 10000 rows and i have to read each data(which is integer or float type) from the file to use it for some purpose.
I am using scanf() to read. But this is taking long time and the program slows down.
how can i improve the speed of reading ??????????
Regards
raghav
|
|
|
|
|
|
You can get more speed if you're ready to use up more memory.
Read the entire file into a CString variable.
And then use CString::Tokenize to read each field.
The CString variable will use up enough memory to load the entire file.
Or you can logically divide the file into fixed size chunks and read one chunk at a time and do the same processing.
«_Superman_»
I love work. It gives me something to do between weekends.
|
|
|
|
|
Split each line into fields and use atof[^] (to read floats) and atol[^] (to read integers). That should be quicker than scanf.
Java, Basic, who cares - it's all a bunch of tree-hugging hippy cr*p
|
|
|
|
|
Raghav0710 wrote: how can i improve the speed of reading ??????????
By not reading one line at a time from disk.
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
Read the whole thing into memory and parse it. If it's too big, read it in in chunks.
You can make it go even faster by converting the data while parsing.
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
Hello All,
Sorry in advance for the lengthy explanation...
I have two applications (a server and a client) that talk to each other via TCP/IP. (WinXP to WinXP) I've noticed that if the server runs on a relatively poor performing machine and the TCP/IP traffic is high, that TCP/IP will be smart and combined messages to save on network traffic. My server/client are able to handle that just fine; however, I've recently noticed that in certain circumstances, like over a VPN connection where the network performance is degraded, TCP/IP will not only combined packets, but will also split the packets. That is to say where one packet was sent, two are received. In reality what happens is that four are sent and two are received; the first 3 are combined with 1/2 of the fourth and the other half of the fourth is in the second packet. This wouldn't be a problem except I don't know how to know which packets need to be reconnected and in what order. This seems to only occur when a large number of packets are being sent in rapid succession. TCP/IP does not guarantee that the order sent is the order received. The data that is being sent must pieced together in the correct order for it to be useable.
Is this normal behavior and if so, how do you make the connection between packets?
Thanks in advance.
|
|
|
|
|
|
His description sounds more like UDP to me.
|
|
|
|
|
First a book recomendation: Effective TCP/IP Programming: 44 Tips to Improve Your Network Programs[^].
TCP is stream oriented. The order of the bytes going in one side always remains the same when it comes out the other side. At the application level there are no packets. Packets come into play after TCP gets your data and transmits it. It is free to do whatever is needed to get the bytes from one place to another -- split, combine, distribute amongst several homing pigeons, whatever. The bytes can arrive on the other side in any order but they are always reassembled in the sending order prior to presentation to the application.
I'm guessing you are sending messages of specific lengths over the socket and most of the time when you do a write on one side, your read on the other side gets the full message in one chunk. This is not always the case. When the receiver does a read(), all of the bytes in the stream may not have arrived yet and you will come up short or bytes from a following message may have arrived and you will get part of the next message. This is normal. You can't prevent it from happening. Assuming there is a one to one coorespondence between reads and writes is a very common mistake. It usually works when you are on the test bench but when things get busy, it all falls apart.
Typically what is done is to add some structure to your messages. Start them with some sort of Start of Message marker followed by the message length. The receiver reads this header portion and then reads again for the indicated length. The read may come up short in which you have to dip in again waiting for additional bytes. Once length bytes have been read, the next read should be for a message header again and the process repeats.
|
|
|
|
|
Thanks for the very helpful reply. This confirms what I had surmised. I'll check out the book too; I've run into many problems along the way that has been quite frustrating. I'm considering a re-write of the server side in Java for it's cross platform compatibility and so I'd like as much information up front during the design phase.
Thanks again.
|
|
|
|
|
Asking this in regard to Having CAsyncSocket run as a seprate thread from My CDialog Thread
If I need a Window/Control/CWnd Class to process notfications
by I just create COntrol/Cdialog Class and not display the Window
thankx
|
|
|
|
|
The base class for the message map architecture is CCmdTarget.
And CWnd is derived from CCmdTarget.
«_Superman_»
I love work. It gives me something to do between weekends.
|
|
|
|
|
So.......... When Include one these macros DECLARE_DYNCREATE and IMPLEMENT_DYNCREATE macros
Is creates that Heirarchy
|
|
|
|
|
ForNow wrote: Is creates that Heirarchy
No, those macros have nothing to do with a class' hierarchy. They are used for serialization, runtime information, and dynamic creation.
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
As David said, it has nothing to do with the message map.
These macros are the functionality implemented by the CObject class.
«_Superman_»
I love work. It gives me something to do between weekends.
|
|
|
|
|
Sorry about responding so late as I am Sabbath Observant ??
I read the article that David pointed me to
When I saw the ON_THREAD_MESSAGE and the I noticed PostThreadMessage
I got the IDEA that an app does not need GUI objects to have a message
pump
BTW cann't seemd to find the TCP/IP notification message
for e.g. onSend... would it be WM_SEND I'll look around...
|
|
|
|
|
ForNow wrote: Asking this in regard to Having CAsyncSocket run as a seprate thread from My CDialog Thread
If I need a Window/Control/CWnd Class to process notfications
Have you seen this and this?
"Old age is like a bank account. You withdraw later in life what you have deposited along the way." - Unknown
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
|
|
|
|
|
Thankx for your patience I'll try
Spend more time reading posts before I hit keyboard
|
|
|
|
|
Hi i am to this forum..
I have doubt on Visual C++.How to include the english dictionary .My task is to check whether the entered text is meaningful or not?
Could some one help me out ..
Thanks in advance
|
|
|
|
|
If you find a way to do it, try running your post through the application......
|
|
|
|
|
You're a bad boy. 
If the Lord God Almighty had consulted me before embarking upon the Creation, I would have recommended something simpler.
-- Alfonso the Wise, 13th Century King of Castile.
This is going on my arrogant assumptions. You may have a superb reason why I'm completely wrong.
-- Iain Clarke
[My articles]
|
|
|
|
|
The original post is better than most of the posts here.
This signature was proudly tested on animals.
|
|
|
|
|
plz sendz codez......
Here have a ![Beer | [beer]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/beer.gif) and lighten up  ....
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
