|
Break it up into chunks using LIMIT keyword, that way your application has less to retrieve per each view. You can put a simple button to go to next page etc.
|
|
|
|
|
Don't use a dataset. They are crippling memory hogs. For large amounts of data you need to use a data reader. If the data is sufficiently large you need to break the return up into chucks.
Need software developed? Offering C# development all over the United States, ERL GLOBAL, Inc is the only call you will have to make.
If you don't ask questions the answers won't stand in your way.
Most of this sig is for Google, not ego.
|
|
|
|
|
Hear hear!
DataSet is evil!
|
|
|
|
|
Hi,
some controls don't need all the data all the time, they can run in virtual mode, needing only the data that is actually visible at any point in time. They fire an event when they need more data to show it. DataGrid is one of those controls.
Luc Pattyn [Forum Guidelines] [My Articles]
- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets
|
|
|
|
|
Hello everyone,
Here is my sample program for web service server side and client side. I met with a strnage performance problem, which is, even if I increase the number of threads to call web services, the performance is not improved. At the same time, the CPU/memory/network consumption from performance panel of task manager is low. I am wondering what is the bottleneck and how to improve it?
(My test experience, double the number of threads will almost double the total response time)
Client side:
class Program
{
static Service1[] clients = null;
static Thread[] threads = null;
static void ThreadJob (object index)
{
for (int i = 0; i < 100; i++)
{
clients[(int)index].HelloWorld();
}
}
static void Main(string[] args)
{
Console.WriteLine("Specify number of threads: ");
int number = Int32.Parse(Console.ReadLine());
clients = new Service1[number];
threads = new Thread[number];
for (int i = 0; i < number; i++)
{
clients [i] = new Service1();
ParameterizedThreadStart starter = new ParameterizedThreadStart(ThreadJob);
threads[i] = new Thread(starter);
}
DateTime begin = DateTime.Now;
for (int i = 0; i < number; i++)
{
threads[i].Start(i);
}
for (int i = 0; i < number; i++)
{
threads[i].Join();
}
Console.WriteLine("Total elapsed time (s): " + (DateTime.Now - begin).TotalSeconds);
return;
}
}
Server side:
[WebMethod]
public double HelloWorld()
{
return new Random().NextDouble();
}
thanks in advance,
George
|
|
|
|
|
Threads do not run concurrently (which means at the same time) unless you have hyperthreading technology (which I'm not even sure about that). So the more threads you have, the more checks it has to do, starting and stopping. I am not that great with threading, you'll probably get a much more detailed better response from someone else around here, but that's my understanding of it.
|
|
|
|
|
Thanks EliottA,
I have tried to use thread pool to reduce thread context switch time, here is my code. But the performance never improves, and both the server side and client side CPU/Memory/Network are only used a little from performance tab of task manager. So, I think there are much rooms to optimize?
I am using Windows Server 2003 x64 on both client side and server side.
Any ideas about how to improve performance?
class Program
{
static Service1[] clients = null;
static Thread[] threads = null;
static ManualResetEvent[] doneEvents = null;
static void ThreadJob (object index)
{
for (int i = 0; i < 100; i++)
{
clients[(int)index].HelloWorld();
}
doneEvents[(int)index].Set();
}
static void Main(string[] args)
{
Console.WriteLine("Specify number of threads: ");
int number = Int32.Parse(Console.ReadLine());
clients = new Service1[number];
threads = new Thread[number];
doneEvents = new ManualResetEvent[number];
for (int i = 0; i < number; i++)
{
doneEvents[i] = new ManualResetEvent(false);
clients [i] = new Service1();
clients[i].EnableDecompression = true;
ThreadPool.QueueUserWorkItem(ThreadJob, i);
}
DateTime begin = DateTime.Now;
WaitHandle.WaitAll(doneEvents);
Console.WriteLine("Total elapsed time (s): " + (DateTime.Now - begin).TotalSeconds);
Console.ReadLine();
return;
}
}
regards,
George
|
|
|
|
|
as i know thread.join means calling thread waits in indefinitely till the other thread terminated.so it might be the bottle neck. the other thing is usually we use threads for concurrent processing so if we don't ask the threads to wait till other thread comes they will work independently
yasith
|
|
|
|
|
Thanks yasith,
"so if we don't ask the threads to wait till other thread comes they will work independently" -- I am interested, could you show me your code please?
regards,
George
|
|
|
|
|
Add another NIC. Only one request can go through one network card at a time.
Also, in looking at your code you are locking up each thread with your call to hello world. If you had a stack of methods to call say:
Stack<Service1> clients ...;
You can then use thread safe code inside of your ThreadJob method to pop a service, call its method and then return. Because each thread is now no longer depended on a given service a slow running server will not tie up the entire operation.
Need software developed? Offering C# development all over the United States, ERL GLOBAL, Inc is the only call you will have to make.
If you don't ask questions the answers won't stand in your way.
Most of this sig is for Google, not ego.
|
|
|
|
|
Thanks Ennis,
Looks like you are an expert of this topic. Cool!
I read your reply 3 times but still confused.
"You can then use thread safe code inside of your ThreadJob method to pop a service, call its method and then return. " -- it is appreciated if you could show me your code please? Honestly, I never wrote such code before.
regards,
George
|
|
|
|
|
Stack<Items> stack ...;
public void MyThread(object whocares){
while(go){
Item item = null;
lock(stack){
item = stack.Pop();
}
if(item != null){
}
}
}
Then you need somewhere to run start the loop
System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(MyThread));
Now just push items on the stack when they need to be called in another thread. There are many modifications to this basic theme.
Need software developed? Offering C# development all over the United States, ERL GLOBAL, Inc is the only call you will have to make.
If you don't ask questions the answers won't stand in your way.
Most of this sig is for Google, not ego.
|
|
|
|
|
Thanks Ennis,
I have some confusion about your soluton. I think the reason of why you use stack is because there are some shared object instances between the threads, correct? But in my situation, there is no shared items between my thread jobs. You can read my code again for my thread job in method "ThreadJob".
Any comments? Please feel free to correct me if I am wrong.
regards,
George
|
|
|
|
|
Maybe a single task is not taking enough time to be worth creating a thread for? If that's the case, try to use the ThreadPool and see if that works better. You don't have to use it "explicitly", creating a delegate and calling BeginInvoke on it should be fine. If you want to wait for all tasks to finish, you could store the IAsyncResults in an array and later call EndInvoke on all of them.
edit: lol @ my stupid typo Last modified: 42mins after originally posted --
|
|
|
|
|
|
Ok, this may not make much of a difference (if any), but you could use some elements like:
(this is not code - just some 'useful' elements)
delegate void ThreadJobDelegate(int index);
IAsyncResult[] results = new IAsyncResult[number];
ThreadJobDelegate th = new ThreadJobDelegate(ThreadJob);
results[index] = th.BeginInvoke(index, null, null);
foreach(IAsyncResult res) th.EndInvoke(res);
I would also time each call to ThreadJob, it doesn't have to be fancy, just to get an idea of how long each call is taking - and time the total (from before the first BeginInvoke till after the last EndInvoke), to see whether threading improved it at all (and if so, by how much).
|
|
|
|
|
Thanks harold!
I have applied your ideas but performance never improves. Here is my code modified according to your idea. Any ideas? Or anything wrong with my code?
class Program
{
static Service1[] clients = null;
static IAsyncResult[] results = null;
static AsyncMethodCaller[] callers = null;
public delegate void AsyncMethodCaller(object index);
static void ThreadJob (object index)
{
for (int i = 0; i < 100; i++)
{
clients[(int)index].HelloWorld();
}
}
static void Main(string[] args)
{
Console.WriteLine("Specify number of threads: ");
int number = Int32.Parse(Console.ReadLine());
clients = new Service1[number];
results = new IAsyncResult[number];
callers = new AsyncMethodCaller[number];
for (int i = 0; i < number; i++)
{
clients[i] = new Service1();
clients[i].EnableDecompression = true;
callers[i] = new AsyncMethodCaller(ThreadJob);
results[i] = callers[i].BeginInvoke(i, null, null);
}
DateTime begin = DateTime.Now;
int j = 0;
foreach (IAsyncResult res in results)
{
callers[j].EndInvoke(res);
j++;
}
Console.WriteLine("Total elapsed time (s): " + (DateTime.Now - begin).TotalSeconds);
Console.ReadLine();
return;
}
}
regards,
George
|
|
|
|
|
Ok that would explain it then I suppose..
It should be possible to use on 1 new AsyncMethodCaller - but I seriously doubt it would have a impact on the performance.
If performance can not be improved with threading, then it will just have to be improved in some other way..
|
|
|
|
|
Thanks harold!
"It should be possible to use on 1 new AsyncMethodCaller - but I seriously doubt it would have a impact on the performance." -- how could we use only one AsyncMethodCaller instance? Could you show me some simple pseudo code based on my code please?
regards,
George
|
|
|
|
|
Like so:
class Program
{
static Service1[] clients = null;
public delegate void AsyncMethodCaller(int index);
static void ThreadJob (int index)
{
for (int i = 0; i < 100; i++)
{
clients[index].HelloWorld();
}
}
static void Main(string[] args)
{
Console.WriteLine("Specify number of threads: ");
int number = Int32.Parse(Console.ReadLine());
clients = new Service1[number];
IAsyncResult[] results = new IAsyncResult[number];
AsyncMethodCaller TJ = new AsyncMethodCaller(ThreadJob)
for (int i = 0; i < number; i++)
{
clients[i] = new Service1();
clients[i].EnableDecompression = true;
results[i] = TJ.BeginInvoke(i, null, null);
}
DateTime begin = DateTime.Now;
foreach (IAsyncResult res in results)
TJ.EndInvoke(res);
Console.WriteLine("Total elapsed time (s): " + (DateTime.Now - begin).TotalSeconds);
Console.ReadLine();
return;
}
}
|
|
|
|
|
Thanks harold,
I have tried but no performance improvements. Do you have any other ideas to improve performance? I find CPU/memory/network are used a little part, so I think there is room to improve performance.
regards,
George
|
|
|
|
|
Very strange.. I'm really out of ideas now..
There is just 1 explanation I can think of - the web server is handling all requests sequentially and non-pipelined.
|
|
|
|
|
Thanks harold,
Two more questions,
1.
My code is posted and simple. Could you reproduce my issue? I am not suspecting it is my environment issue.
2.
"the web server is handling all requests sequentially and non-pipelined" -- do you have any ways or documents to prove it?
regards,
George
|
|
|
|
|
1. no, I would have to know what HelloWorld() does, and how it does it, and probably a lot more
2. well, if it weren't, threading would cause a speed-up. And it didn't.
|
|
|
|
|
Thanks harold,
For 1,
Here is my code at server side. Could you reproduce my problem?
[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[System.ComponentModel.ToolboxItem(false)]
public class Service1 : System.Web.Services.WebService
{
[WebMethod]
public double HelloWorld()
{
return new Random().NextDouble();
}
}
For 2,
I could understand your points. But do you have any documents to prove? I think if web service could only serve requests sequentially other than simualtenaously, it is so bad and degrade performance design. I can not believe it is designed in this way.
regards,
George
|
|
|
|