.NET (Core and Framework)

25-Apr-07 15:35

Hi,

I have a form that takes a number from the user and loops through this number to do some calculations and database access:

for (int i=0; i <NUM; ++i){<br />
//do calculation<br />
// access database<br />
}<br />

my problem here is that:
for example when NUM=1000, it takes 20 seconds to complete, when I tried 1500 it took more than 60 seconds which is unrealistic because each loop takes the same time. I tried to stop the loop when i=1000 but NUM is still = 1500 to see when it does finish, I was shocked to see that 60 seconds was required to do the 1000!! how could this possibly be?

P.S: I did NUM=1200 and 1400 and they all took 20 seconds to complete the 1000!! why does 1500 is different?

another thing, is multithreading useful to speed up long operations?
I have read about it and know that it is useful when I want more than one process to be executed at the same time, but what if I only have one process, does it speed it up or something?

thanks,

-- modified at 21:46 Wednesday 25th April, 2007

Brady Kelly25-Apr-07 19:26

Brady Kelly

25-Apr-07 19:26

The only thing that speeds up long operations is more powerful machine. Multithreading can only prevent long operations blocking other operations, and to be picky, it will actually slow the long operation down by a minuscule factor. The whole point there is being able to do other things while the long operation is churning away in the background.

Luc Pattyn26-Apr-07 1:28

Luc Pattyn

26-Apr-07 1:28

Brady Kelly wrote:
The only thing that speeds up long operations is more powerful machine

I strongly disagree. You can apply a better algorithm, choose a better implementation,
or improve parallellism, which is able to reduce the overall time provided there
were spare CPU cycles to begin with (indicating part of the code is either waiting
on some resource, or just sleeping for some reason).

For some resources, behavior is non-linear; AFAIK sockets use some resources that
are available to some extent, and include a delay before getting freed automatically.

Smile | :)

Luc Pattyn

[My Articles]

Brady Kelly26-Apr-07 1:38

Brady Kelly

26-Apr-07 1:38

OK, maybe my answer was grossly over simplified, and for that it should also been much shorter. Blush | :O

sarah_malik26-Apr-07 3:09

26-Apr-07 3:09

thanks for the info guys. Smile | :)

pbraun27-Apr-07 5:13

pbraun

27-Apr-07 5:13

I'm betting this loop can be made faster if the access to the DB is optimized. What I mean by that is: Does the DB really need to be accessed each time through the loop? If the answer is no, then create a conditional that will only access the DB when it is necessary.

Another way to look at this is to do all of the DB access at once. In other words, split your processing up. Do the calculations and store them in a manner that is efficient for later DB access once the calculations have been finished (after the calculation loop is completed).

Without knowing much about what the calculation is, why the DB access needs to be done, and how the data in the DB is affected, I can't give a good answer on how to split the processing up.

Phil

sarah_malik27-Apr-07 9:11

27-Apr-07 9:11

pbraun wrote:
Does the DB really need to be accessed each time through the loop?

Yes, it does. It accesses the DB seeking for some different info each time.
About doing all the DB access at once, I can see what do u mean, but does it really make a difference? I can split the loop into 2 loops(with the same lenght). the first one is to access the DB and the other to do the calculations.

pbraun wrote:
Do the calculations and store them in a manner that is efficient for later DB access once the calculations have been finished

Actually, my calculations depends on the values retrieved from the DB, that is, I have to access the DB first, get some values, do calculations, have result and then next loop.
Also, the data in DB is never affected. This process only retrieves records.

I can simplfy the process as following:

<br />
for (int i=0; i<NUM ; ++i){<br />
<br />
"SELECT * FROM TABLE WHERE CONDITION"; // condition differs at each loop[i]<br />
<br />
while (reader->Read()){<br />
<br />
//reader->GetValue;<br />
<br />
//do calculation depending on (GetValue)<br />
<br />
}<br />
<br />
reader->>Close();<br />
<br />
}<br />
<br />

pbraun27-Apr-07 9:28

pbraun

27-Apr-07 9:28

Hmmm. From that description, it seems that it would be better to send one query to the DB that retrieves all of the necessary data from the DB, then do the processing (in your case, calculations).

It is my opinion, that the constant queries to the DB is costing you the time. Of course, you will then have the problem of memory management if your number of data becomes sizable. In that case, you may want to complicate your algorithm by reading in up to a predetermined number of records, doing some of the calculations, then reading in the next batch of records, and so on until finished. This will minimize the DB access.

Phil

sarah_malik27-Apr-07 10:28

27-Apr-07 10:28

pbraun wrote:
it seems that it would be better to send one query to the DB that retrieves all of the necessary data from the DB, then do the processing (in your case, calculations).

I'm afraid that it is not possible in my case, my condition is already complex enough at each loop Cry | :((

, also it depends on some data outside the loop ...

pbraun wrote:
the constant queries to the DB is costing you the time

Yes, the queries are costing much of the time!

I would like to ask also does the .NET framework do anything behind the scene when a long operation is in run?
I have encountered a warning when "another" long operation was running:

The CLR has been unable to transition from COM context 0x1a2008 to COM context 0x1a2178 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations.

Thanks for your help Phil, I really appreciate it Big Grin | :-D

About multi-threading

Diego F.25-Apr-07 4:51

Diego F.

25-Apr-07 4:51

Hello. I have a doubt about multi-threading, if it is or not the best way to achieve the following:

I have a server application that listens a port through a socket.
A client will send messages to that application (many messages).
Because the amount of messages is huge, the server application should only send each received message to another process (a thread?) that manages the message, and then go with the next message.

I want a limited number of threads, so the server app just looks for one that is free and send the message to it. Once the threads finish the work, they wait to be called by the server app.

Are threads the solution for that?

Regards,

Diego F.

kubben25-Apr-07 5:55

kubben

25-Apr-07 5:55

This sounds like a good example of when to use MSMQ. The microsoft message queue allows you to put messages on the MSMQ and then you can have something like a window service that is listening to the queue. It will read the item from the queue process it and then get the next item on the queue. I would write it first without threading. If the performace is not good enough then you could look at using threads to help with the processing.

If you do use threads, you should use the thread pool and just keep track of how many threads in the pool you are using. There is a limit to the number of threads in the pool, but it would probably be a good idea to keep track of how many you are using and only use a few. Still try the solution without threads first.

Ben

Diego F.26-Apr-07 23:12

Diego F.

26-Apr-07 23:12

Thank you. I did a first try with thread pool. The system will receive many messages through the socket (may be 500 per second), so I want the best performance.

Regards,

Diego F.

pbraun27-Apr-07 5:04

pbraun

27-Apr-07 5:04

In that case the thread pool will limit your performance as many of the messages will not be processed due to the thread pool queue being full. But that also depends on how long it takes for the messages to be processed. Still, don't risk dropping messages because the thread pool queue is full. Your best bet is to use the MSMQ or write your own thread pool code. MSMQ is most likely the best bet.

Phil

Install time JIT tutorial

Diego F.30-Apr-07 2:12

Diego F.

30-Apr-07 2:12

Thank you for your advise. I'll try a second approach with MSMQ.

Regards,

Diego F.

SE_GEEK25-Apr-07 4:39

SE_GEEK

25-Apr-07 4:39

Can anybody point me to a tutorial, or a page explaining how to ngen your assembly during install time?

I have an installer project and I would like to avoid an ugly command window popping up when I do an install that runs ngen on the asseblies that I have just installed.

This seems like it should be a common task so I am surprised I cant find any good tutorials on how to do it (all I can find are pages telling you to do it, but not explaining how).

Any guidance/suggestions would be much appreciated Smile | :)

Truly wise men don't play leap frog with Unicorns.

Re: Install time JIT tutorial

Pete O'Hanlon25-Apr-07 5:14

Pete O'Hanlon

25-Apr-07 5:14

http://msdn.microsoft.com/msdnmag/issues/05/04/NGen/#S4[^]

Deja View - the feeling that you've seen this post before.

Re: Install time JIT tutorial

SE_GEEK27-Apr-07 4:46

SE_GEEK

27-Apr-07 4:46

So are the steps listed in that article done from a batch file as a custom install step?

I tried playing with that on my development computer, and I don't notice any speed boost at all when I ngen the assembly ... is the native code stored in some other location that I need to launch instead of the old assembly?

Truly wise men don't play leap frog with Unicorns.

Transparency in Txxtbox

sianatia25-Apr-07 2:48

sianatia

25-Apr-07 2:48

Hi,
when we set the backcolor of a panel to Transparent,it works but when we hover or put the panle on a TextBox the backcolor shows as Control BackColor and no more transparency is supported.
How can we enable the trasparency of the textbox?
thanks

Dave Kreskowiak25-Apr-07 4:41

Dave Kreskowiak

25-Apr-07 4:41

Transparent isn't transparent. Setting the BackColor of a control to Transparent tells it to take on the background properties of its parent container. Controls sitting behind your Panel will NOT show through the panel.

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

sianatia27-Apr-07 19:20

sianatia

27-Apr-07 19:20

Right,but id there any way to do what i want?

Dave Kreskowiak28-Apr-07 7:37

Dave Kreskowiak

28-Apr-07 7:37

WPF (Windows Presentation Foundation).

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

PIEBALDconsult26-Apr-07 12:20

PIEBALDconsult

26-Apr-07 12:20

Not sure what you mean. What are you trying to achieve?

Maybe you want to send the Panel to the back? Or bring the TextBox to the front?