|
Hi folks,
I am currently working on a side project for entertainment/ minor work related use. I am very much an entry level programmer.
The issue I am having, is I am attempting to output a series of strings with line breaks into a read only text box (user read only; I can of course access it in the code). I am storing them in a temporary string (although I also tried line by line printing), and then printing them to the labelbox. It is taking a huge ammount of time; 9 items takes .5 seconds. 200 takes several minutes. This is on an i7 2600k etc. etc (although the app is on one thread but still).
txtOutput.Clear();
StringBuilder builder = new StringBuilder();
foreach (string s in sorted)
{
builder.Append(s);
builder.Append("\r\n");
}
txtOutput.Text = builder.ToString();
}
I have pinpointed the problem to the code in bold. I have tried both stringbuilder and + concatenate. Can anyone help me improve efficiency and explain why its so slow?
Thank you,
Travis
modified 2-Dec-11 8:45am.
|
|
|
|
|
Good question! Should be like lightning. What is 'sorted'?
Regards,
Rob Philpott.
|
|
|
|
|
string[] strOutput = alFormOutput.ToArray(typeof(string)) as string[];
Func<string, object=""> convert = str =>
{
try { return int.Parse(str); }
catch { return str; }
};
var sorted = strOutput.OrderBy(
str => Regex.Split(str.Replace(" ", ""), "([0-9]+)").Select(convert),
new sortAlg.EnumerableComparer<object>());
Part of this I borrowed from another document, but to my understanding sorted is an array of strings. Before I send it to the 'sort' (where all of the above code and previous code posted is located) function, it is an arrayList. Do note that the sorting itself takes probably only a second on several hundred items. I tested that with timers and it was not the issue. Wish I could answer better, but I only partially understand natural sort.
Here is the article the code above is based on.
http://www.interact-sw.co.uk/iangblog/2007/12/13/natural-sorting[^]
scroll to the bottom.
appreciate the quick reply
|
|
|
|
|
Ah, well 'sorted' isn't a simple collection, its an IEnumerable which means in you foreach loop it will actually go away for each iteration and do all that sorting stuff. The Regex in the middle of it all is probably responsible.
Make sense? The thing you're iterating over isn't sorted, it gets sorted with each iteration.
Regards,
Rob Philpott.
|
|
|
|
|
I get the general sense of what your saying. Will have to research this ienumerator business. So is 'sorted' somehow calling class related sorting functions every time it is concatened?
Additionally, is there a better way to do a natural sort? Also Cint is off limits, as mixed letters and numeric characters is quite possible. And finally, is this a memory transfer issue? Processor is about dead while all of this is going on.
Really appreciate the good info.
~Travis
|
|
|
|
|
Pretty much. sorted is not an array, its an IEnumerable class which has basic 'get next' functionality. The foreach calls the get next which calls the RegEx stuff.
Without intense study, I can't see what the sorting algorithm is trying to do. Call me old fashioned, but put everything in an array, create an IComparer and sort the array using Array.Sort with that.
Sorry, I can see no reason why the CPU would idle while you wait for this.
Regards,
Rob Philpott.
|
|
|
|
|
That isn't true unless OrderBy is really, really dumb. It should create an order vector which the enumerator will then read sequentially.
|
|
|
|
|
Fair dos. So why does it take so long to execute then?
Regards,
Rob Philpott.
|
|
|
|
|
If I knew that I'd have posted a reply directly under the OP
|
|
|
|
|
First off please use the correct formatting when posting code snippets to make it easier to read. Use the "code" toolbar item above the textbox when entering your question/response.
If all you are doing is iterating through the string array to insert a separator you could use the String.Join[^] method
string withSeperators = String.Join("/r/n", sorted);
However, with very few items in the array the StringBuilder method should be faster than you have indicated.
No comment
|
|
|
|
|
Tried the join method. It actually took quite a bit longer than the builder one. I think the processing time has to do with the class I attached in the link in my 3rd post. Apparently the methods in that class are called upon when concatenating 'sorted' with "\r\n" (didn't even know that was possible to tell the truth).
Anyone know of a better natural sort, or am I maybe implementing it wrong?
Can I convert 'sorted' (ienumerator type) to a string[] somehow, and then append based on that copy?
If anyone can help demystify the best approach to a natural sort that would be great.
Appreciate all the help guys. I'm off to school, so I won't reply for some time most likely.
~Travis
|
|
|
|
|
|
Try changing your Func:
Func<string, object> convert = str =>
{
int value = 0;
return int.TryParse(str, out value) ? (object)value : str;
};
That could fix your problem.
|
|
|
|
|
Emardini,
Tried your code. Worked like a dream. That said, this part of the program was borrowed from the link above, so I never really understood it in the first place. I have gone from several minute sorts to < 1 second.
Func<string, object> convert = str =>
{
try { return int.Parse(str); }
catch { return str; }
};
Func<string, object> convert = str =>
{
int value = 0;
return int.TryParse(str, out value) ? (object)value : str;
};
Emardini, or anyone else. Can you break this down and explain what the difference/ meaning of both functions is? I know that sounds silly since it's from my program, but this project is merely to polish up on what I learned in previous semesters, complete a mundane task, and learn as much as possible along the way.
Edit:Having looked at this a bit more, if I understand right 'out' vs 'return' is the key difference. Still not entirely sure on what that means in the context of the above code.
modified 3-Dec-11 3:15am.
|
|
|
|
|
The simple fact is that "exception handling" (the "try...catch" block) is very "expensive" in terms of overhead; what with "stack unwinding", etc.
"try ... catch" blocks are for "exceptional" conditions; not for performing "data validation" (i.e. instead of "TryParse").
|
|
|
|
|
Great answer. Thanks a lot to everyone that replied to this thread. Really great info!! Learned a lot, and it works great too.
~Travis
|
|
|
|
|
Other minor points:
Try AppendFormat rather than two Appends.
And try pre-alloctaing the StringBuilder --
StringBuilder builder = new StringBuilder(txtOutput.Text.Length);
(before clearing the TextBox)
|
|
|
|
|
If "sorted" is as you say an array of strings, why faff with it at all?
txtOutput.Lines = sorted;
I just tried your code and mine:
string[] data = File.ReadAllLines(@"D:\Temp\MyLargeTextFile.txt");
Stopwatch s1 = new Stopwatch();
Stopwatch s2 = new Stopwatch();
s1.Start();
textBox1.Lines = data;
s1.Stop();
s2.Start();
StringBuilder sb = new StringBuilder();
foreach (string s in data)
{
sb.Append(s);
sb.Append("\r\n");
}
textBox2.Text = sb.ToString();
s2.Stop();
Console.WriteLine("{0} lines: {1}/{2}", data.Length,s1.Elapsed, s2.Elapsed);
The output I get is nothing like yours:
22777 lines: 00:00:01.5580246/00:00:01.5555734
How did you time your operations?
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
|
|
|
|
|
You're using two distinct stopwatches. When was the last time you had them calibrated?
|
|
|
|
|
I make sure to have my .NET stopwatches calibrated every time I re-wind them...
Ideological Purity is no substitute for being able to stick your thumb down a pipe to stop the water
|
|
|
|
|
Can you show where you put your timing code?
There is no reason why adding to a StringBuilder should take any noticeable amount of time. I wrote a simple test function in my toy language
x:
dt←$DateTime:Now;
sb←#new $StringBuilder ⍬;
{sb:AppendLine (⍵:ToString)}¨⍳x;
($DateTime:Now)-dt
... and it can add 10,000 lines in under a second. (Most of that is to do with pushing the function in my language onto the stack 10,000 times, I think.)
Your times look quadratic. Are you calling this function (sorted.Count) times?
|
|
|
|
|
You could always use the StrCmpLogicalW function that Windows Explorer uses to sort your items.
How to use in C#: StrCmpLogicalW
Use this in an IComparer like here.
Then your StringBuilder loop should run really fast.
|
|
|
|
|
You could combne the two lines:
builder.AppendFormat("{0}\r\n", s);
".45 ACP - because shooting twice is just silly" - JSOP, 2010 ----- You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010 ----- "Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass." - Dale Earnhardt, 1997
|
|
|
|
|
builder.AppendLine(s); would do the trick also
No memory stick has been harmed during establishment of this signature.
|
|
|
|
|
Since you're using readonly single-line items, I'd suggest using a ListBox . You might want to temporarily disable painting when adding multiple items.
Bastard Programmer from Hell
|
|
|
|