|
It does not make any sense at all:
1.
you showed the following content of file1:
C0000000001 C0001000010
C0000000002 C0001000011
C0000000003 C0001000012
which is some 25 characters per line, times 2000 lines, equals 50KB (or 100KB when all in wide characters), not 289KB. With 289KB the average line length would be 140 bytes. Check its content!
2.
file2 holding 247KB for 80,000 lines results in some 30 characters per line, that sounds fine.
3.
the result file growing beyond 800MB is utter nonsense. For it to grow that large you would have to:
- either have 80,000 lines of 10,000 characters each (where are these coming from?)
- or have many more lines. How would you explain that?
Did you bother checking what is inside that file? is anything in there correct?
Now start looking at what you have, and stop making up stories. 
Luc Pattyn [Forum Guidelines] [My Articles]
The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.
|
|
|
|
|
The way I had the code written was that for each instance of ID in file1 found in file2 it wrote the whole of file2 as I had file2 completely read into memory and for some stupidness of myself I had it coded that way. Now, I have ammended this code to the below which seems to work just fine but the performance is still "pish" as it itterates 2000* through 80000 records to find a match. There are instances where an ID in FILE1 could exist more then once in FILE2. So, on a file of 80000 records it does a 160 000 000 reads on file2 which could mean a major hit on performance.
Luc Pattyn wrote: Now start looking at what you have, and stop making up stories
I swear I'm not making up stories. 
THis is what I have refined the code to now but I am sure it CAN be sped up much more.
My actual files have these amounts of records on it.
FILE1 = 10948 Lines of 25 characters per line. (289KB)
FILE2 = 360 Lines of 650 characters per line. (247KB)
FILE3 = 79341 Lines of 650 characters per line. (54,392KB)
FILE4 = 76283 Lines of 700 characters per line. (50,508KB)
To process this it takes all of 34.61 seconds to complete which is fine and the resultant file is 100% spot on.
Now, processing FILE1 with 10948 lines of data against another file, say FILE3 with 79341 lines with 650 characters per line it takes forever.
The code implimented is as follows:
string fileNot;
string fileHid;
string outPath = @"c:\Documents and Settings\Mumbleb\Desktop\Ess\";
fileNot = textBox1.Text;
fileHid = textBox2.Text;
StreamWriter sw = new StreamWriter(outPath + "ER_AU_TBSZ_SEQMHN.TXT");
string[] LineHID = File.ReadAllLines(fileHid);
string[] LineNot = File.ReadAllLines(fileNot);
foreach (string hid in LineHID)
{
string OldHid = hid.Substring(0, 11);
string NewHid = hid.Substring(14, 11);
foreach (string content in LineNot)
{
string matchid = content.Substring(0, 11);
string noting = content.Substring(11, 605).Trim();
if (OldHid == matchid)
{
string contentnew = Regex.Replace(content, matchid, NewHid);
sw.WriteLine(NewHid + noting);
}
}
}
sw.Close();
I am about to try and process the bigger files on my Dualcore Laptop and see what the performance is like. I will post if I have any timings on the performance.
Excellence is doing ordinary things extraordinarily well.
|
|
|
|
|
Your new data does not fit your earlier statements.[^]
Kwagga wrote: I swear I'm not making up stories
You're just stupid then?
Kwagga wrote: it takes forever.
No, with decent code it does not. I told you all there is to it. The subject is closed. 
Luc Pattyn [Forum Guidelines] [My Articles]
The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.
|
|
|
|
|
Luc Pattyn wrote: You're just stupid then?
Yeah, I recon I probably am just stupid but could be slightly smarter than you!!!!...........
Luc Pattyn wrote: No, with decent code it does not.
If you knew the decent code I am sure you would have suggested it, but you just as clueless as I am. I will resolve this and share my findings.
Harsh statements like the above is appreciated but if a matter is not resolved then I recon it is uncalled for.
Thanks for the help mate.
Excellence is doing ordinary things extraordinarily well.
|
|
|
|
|
One way this might be speeded up without sorting etc., although that would undoubtedly help a great deal, is to do things backwards. At least relative to the way you are currently doing things.
Currently you load the first file into the array, then you iterate the array and test, possibly every line of the second file, for a match to each member.
Why not load first file, as now. Then read file2 line by line and search the array for a match. Personally I'd change the array for a List<string>, or even a SortedList<string, string="">, to make searching easier.
Henry Minute
Do not read medical books! You could die of a misprint. - Mark Twain
Girl: (staring) "Why do you need an icy cucumber?"
“I want to report a fraud. The government is lying to us all.”
|
|
|
|
|
I am trying to install a vs2005 web setup project in window vista with iis7 but give me error of "setup has been intrupted"
is there any problem bcoz i got stuck in this problem any help would be appreciated\
thanks sohaib.
|
|
|
|
|
Hi friends,
I've a WinForm consisting of a TextBox and a button. Now, I want that, whenever user enters any text in the text box and clicks the button, the entered text be saved in a precreated text file. What code should I implement in the Button_Click event handler in order to do so?
Help appreciated,
Rajdeep.NET 
|
|
|
|
|
Rajdeep.NET wrote: What code should I implement in the Button_Click event handler in order to do so?
Code to write the file out. Now, buy a book and find out what that code is.
I can't believe that you've posted questions about how to sell your software when this limping pile of scrodspittle is the level of competence you are currently at. Do yourself a favour - buy some books; then read them.
"WPF has many lovers. It's a veritable porn star!" - Josh Smith As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
My blog | My articles | MoXAML PowerToys | Onyx
|
|
|
|
|
Pete O'Hanlon wrote: Code to write the file out. Now, buy a book and find out what that code is.
Hi Pete,
Why do you think like that? I searched for the above matter in Google and MSDN. I indeed found million results, but all based upon console application. I ain't a dumb. I remember what I said, "You should have searched google before posting here", and I did and it resulted in futile. Now thats the reason for asking you guys.
Hope you understand me,
Rajdeep.NET 
|
|
|
|
|
Rajdeep.NET wrote: Why do you think like that?
Because writing to a text file is the most basic programming you can do. Almost all books / classes cover this in the first chapter / lesson.
Rajdeep.NET wrote: all based upon console application
The code for writing to a text file remains the same for console applications or for winform applications.
|
|
|
|
|
Tom Deketelaere wrote: The code for writing to a text file remains the same for console applications or for winform applications
Is that so? I wonder!!!  Can you post me the code for doing this in a Win application? Dont worry about Console, I'll make that out 
|
|
|
|
|
Rajdeep.NET wrote: Is that so?
YES
Rajdeep.NET wrote: Can you post me the code for doing this in a Win application? Dont worry about Console, I'll make that out
NO
How about you actually trying the code you found, and do some work of your own.
|
|
|
|
|
Tom Deketelaere wrote: How about you actually trying the code you found, and do some work of your own.
Ok, thanks.
|
|
|
|
|
It's time to start working dude
|
|
|
|
|
Rajdeep.NET wrote: Why do you think like that?
Because you asked a question that is so basic. Adding text to a testfile can be done in a single line of code in VB.NET. If you don't know this, you can't possibly know other things that you would need to know to write an decent software anyone would want to buy!
Rajdeep.NET wrote: I indeed found million results, but all based upon console application.
There is no difference between writing to a textfile in a Console app and a Windows Forms app.
Rajdeep.NET wrote: and I did and it resulted in futile
No, it didn't. What was futile was you didn't even TRY what you saw in your own application. This simple concept "Try it and see what happens" is what is going to hold you back as a developer.
|
|
|
|
|
Rajdeep.NET wrote: Why do you think like that? I
I think like this because there are times when Google is sufficient, but if you haven't got the basics down then it ain't going to help. That's where a book will help - it will guide you through from the simplest tasks (assuming it's a good book).
"WPF has many lovers. It's a veritable porn star!" - Josh Smith As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
My blog | My articles | MoXAML PowerToys | Onyx
|
|
|
|
|
Pete O'Hanlon wrote: it will guide you through from the simplest tasks (assuming you can read).
fixed it. 
Panic, Chaos, Destruction.
My work here is done.
|
|
|
|
|
Pete O'Hanlon wrote: this limping pile of scrodspittle is ...
it is getting all to high-brow for me now, even Google[^] doesn't help any more.
Luc Pattyn [Forum Guidelines] [My Articles]
The quality and detail of your question reflects on the effectiveness of the help you are likely to get.
Show formatted code inside PRE tags, and give clear symptoms when describing a problem.
|
|
|
|
|
It's 100% Pete. I doubt you'll find it anywhere else.
"WPF has many lovers. It's a veritable porn star!" - Josh Smith As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
My blog | My articles | MoXAML PowerToys | Onyx
|
|
|
|
|
Pete O'Hanlon wrote: It's 100% Pete.
Patented by Pete. ™
 (I voted a 5 for it)
Vasudevan Deepak Kumar
Personal Homepage
Tech Gossips
The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep!
|
|
|
|
|
The Urban Dictionary[^] is beckoning.
My failometer is detecting vast quantities of FAIL!
"Its SQL - hardly programming..." (Caslen)
|
|
|
|
|
Pete O'Hanlon wrote: find out what that code is
His next q would be 'Where is the code in the book?' and he might write that to the author too. Poor author!
Vasudevan Deepak Kumar
Personal Homepage
Tech Gossips
The woods are lovely, dark and deep,
But I have promises to keep,
And miles to go before I sleep,
And miles to go before I sleep!
|
|
|
|
|
|
|
Rajdeep.NET wrote: You should have googled for this, before posting it here.
My failometer is detecting vast quantities of FAIL!
"Its SQL - hardly programming..." (Caslen)
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
