|
is this [echotracker] ?
'g'
|
|
|
|
|
Yes it was before it was deleted.
|
|
|
|
|
I am in the process of coding an application wherein there is a server application running as a Windows Service on a "big, powerful" box. Scattered around the country are computers that communicate with this "Host" in real-time. There will be a few thousand "clients" connected (10K or less) at any given time.
That's the background. I've already written the Sockets logic for both server and client, and the connections are speedy (using the Send* calls), so that's not an issue.
Here is my situation and question: at a pre-determined time every day, each of these clients' data needs to be tallied/processed/whatever-you-want-to-call-it and then the client needs to be notified this processing has completed. Every client could have a different time-of-day at which this needs to be done. Is there a more efficient way than using a timer (with the appropriate Interval calculated) for each client? I was thinking that perhaps using a single timer and some sort of hashtable that associates clients with time-of-day may work. But I'm soliciting all ideas. Don't assume your idea is not good enough...I want to hear as many possibilities as I can.
Thanks.
Tim
|
|
|
|
|
I'm assumeing the tallying is being done on the "big, powerful" box. Couldn't you simply send a message back to the client when it happens?
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
Thanks for your reply, but you missed my point. I'm trying to design the mechanism that determines the time has been reached to process the subset of clients whose "closeout time" (a time of day) has been reached.
|
|
|
|
|
TimWallace wrote: Thanks for your reply, but you missed my point. I'm trying to design the mechanism that determines the time has been reached to process the subset of clients whose "closeout time" (a time of day) has been reached.
I can't reach your point even now.
I just don't get what you want. I mean you need the time that has beeen reached ... whose "closeout time" has been reached
|
|
|
|
|
Remote machines ("clients") may all have a unique "closeout" time, which is a time of day at which the host process ("server") needs to do processing specific to that client. When the processing is done, the server sends a message to the client. Does this clear things up for you?
|
|
|
|
|
yep but there is nothing I could add to the asnwers offered by Mark, Anticast and others.
|
|
|
|
|
Since you are dealing with machines located all over the world the only constant would be the server, correct? In which case I would consider having it send the closeout signal rather than worry about coordianting thousands of machines; 24 hours from now is still 24 hours regardless of where you are in the world. It would certainly make it easier to update the schedules when necessary also.
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
The remote machines, the "clients", don't need to do anything when their "closeout" time is reached. The host process, the "server", needs to do some stuff and then send a message to the client. And the closeout times can be unique for every client (inasmuch as the number of seconds in a day permit).
|
|
|
|
|
Ok. When the client initial connects have it send a refresh interval that is stored at the server, like in a hashtable as you mention.
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
Maybe you could have a table/list of all of the client's update times which is sorted in an acending manor. Then you have a single, one-time use timer that fires when the next client needs to be updated. The first thing you do in your timer callback is create a new timer that fires at the time needed for the next client in the table/list?
|
|
|
|
|
Thanks for your reply. That is along the lines of what I meant by using a hashtable in my original post. Glad to see it wasn't a far-fetched idea.
|
|
|
|
|
Anticast wrote: create a new timer that fires at the time needed for the next client in the table/list?
and what happens when the times are the same for multiple clients or close enough that the timer can't start and trigger in time? A situation that is very likely to occur with thousands of clients involved.
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
I'll implement a stack or queue and add all clients to it when it is determined that their closeouts need to be processed. Processing will be done in a separate thread and locks will be used to control access to the stack/queue/whatever.
|
|
|
|
|
Mark Nischalke wrote: what happens when the times are the same for multiple clients
I would be using a linked list kind of format and at the start of the Timer callback set the new Timer to fire at _nextClient.ClientPingTime, and ...
Mark Nischalke wrote: close enough that the timer can't start and trigger in time
... if its in the future, great, if its already passed then the callback should be executed almost immedatley. If he needs more precision than that he probably shouldn't be using timers anyway.
|
|
|
|
|
Interesting puzzle... Ok, how about this...
A) Dictionary/HashTable linking client ID to a record that includes ID and next scheduled time.
B) Doubly-Linked List, arranged by time, of the same records
C) Variable (Let's call it the "Insert Point") that stores either the last element of the list (If it's too short) or a reference to the entry nearest to 24 hours from now.
So it would work like this:
1) New client connects... Check the dictionary to see if he's already scheduled (If so, nothing needs to be done). If not, go to the Insert Point, which should be set to a record -about- 24 hours from now. Shift it forward if needed, insert a new record into the Linked List with that client's scheduled time, and put the record in the hash table too.
The Linked List is used so we can efficiently insert a new item at any point, without having to reallocate the underlying array (A List in .NET is just a dynamic array).
The Insert Point is needed because it would take O(n) time to iterate through the list to find the right location. The main disadvantage to using a Linked List.
2) Instead of a timer, work directly with a continuous background thread... It runs in an endless loop... Look at the Linked List, see if the first element's scheduled time has come... If so, process it, remove it, and tack it onto the end of the list. If not, Sleep() the thread until the next one is due.
This isn't a perfect solution, but it might give you a starting point... There's one main issue with this approach. There won't be even distribution...
With this simplified process, they would be allocated as they first connected... If you find a way to distribute them more evenly, you'll need a more efficient insertion mechanism. One that comes to mind would be a 96-member sorted dictionary, with its members pointing to locations in the list that are approximately 15 minutes apart (24 * 4). You could use that to estimate the proper location, then search the list itself to get the exact spot. If you use a method like this, you'll also need to give the background thread a maximum sleep time (Don't want it sleeping for an hour, when you just inserted a new record for 2 minutes from now).
|
|
|
|
|
Server maintains a table, if a client is connected and the current time is past the due time for the message ask the client for it. Client of course waits for this message. Then when a client connects from an outage the message will still be sent as appropriate. Or if the client is always on the server requests it from the client. This would allow the message to be deffered during a period of extreme load.
|
|
|
|
|
Hi,
Maybe I still did not fully understand it, however I see no major problem.
I assume the server has all necessary data available, so it can compute when it decides to do so; the client involvement is limited to receiving a "done" signal.
Here is my attempt to keep it simple:
1. the server maintains a database table "tasks" with fields "clientID", "startTime", "state", and more if necessary.
2. the server has one or a few threads that:
- execute a transaction containing a task fetch ("SELECT TOP 1 FROM tasks WHERE state='idle' ORDER BY startTime") and an assignment setting state to "assigned".
- wait until the start time is reached;
- execute the task;
- update the state to "done"
3. the client has a thread polling the same database (say once a minute), looking for its task with a "done" state.
Pros:
1. Simple.
Cons:
1. An urgent task added at the last moment, will not execute right away, as each server thread is either busy, or has already decided which task to do next. So the latency is one full task.
2. The client isn't event driven, as servers are not supposed to send something to their clients without an explicit request.
|
|
|
|
|
Sounds like you have two questions here.
- How do you process the data on the server at the correct time?
- How do you notify the client of the finished processing?
For #1, I'd say you can do as another person said and store the times in a database table with an index on the date to process. That way, the lookup is very quick. You could use SQL Server or SQL Server CE for this simple task. If you want it to be in-memory, you could store it in a binary tree or some other sorted structure. If the list is not modified very often (sounds like it isn't), the recommendation somebody else had of using a linked list would be a good one. You just get the next time that will be processed and start a timer that will go off at that time (or you have a background thread and use sleep commands to do the same thing). You then either maintain a pointer (linked list) to the next item to be processed or use the date (binary tree, indexed database table) to find the next item to schedule.
For #2, that depends on if the server can send notifications to the clients. If not, then I'm sure the clients know when they are expecting the data to be processed (if not, they can ask the server when they're supposed to be processed). You can have the clients request, from the server, the data at that time of the day... you can do this using a timer. If the data hasn't been generated yet, you can either have the server block until it is generated, then return the data back to the client. Or, you can have the server send a response saying "sorry, no data yet". The client can wait a second, then send the request again. Then 2 seconds, then 4, then 8, then 16, and so on (doubles each time) until it finally gets a valid response.
One thing you'll need to consider is where you will store the data until the client requests it. You could store that in a dictionary in memory, or you could serialize it and store it to a database until the client requests it.
|
|
|
|
|
No, I am only looking for suggestions toanswer your #1. I know how I will code for your #2. Thanks for the suggestion.
|
|
|
|
|
This is similar to how I've done scheduled reporting and such. As others have said, I'd use a database table with the clients and their due times. A Windows Service periodically queries the table for any jobs past due, performs them, and updates the next run time.
|
|
|
|
|
Is there a way to save an object to my exe rather than a different file? The MSDN talks about user settings, but it says that they're saved to a user.config file, and it says that application settings can only be modified at design-time.
I'm looking for a way to move a dynamic list around to different computers while only needing to move one file (the exe).
Any ideas?
Thanks!
|
|
|
|
|
Use a database
I know the language. I've read a book. - _Madmatt
|
|
|
|
|
Anticast wrote: The MSDN talks about user settings, but it says that they're saved to a user.config file, and it says that application settings can only be modified at design-time.
If they(MSDN) say you can only do it at design time they are wright. Here's why. Because altering the
data in a *.exe is dangerous. It is a compiled file so you can only add stuff at compile/design time.
So no I don't think there is.
As for a workaround I would create a binary file and store the byte[] for the object be it picture or whaterver in there. Or in a light weight DB such as SQLite or even a *.mdb file and store it in a byte or OLE Object for the *.mdb.
Just an ideea. Sure others have better ones. You just have to keep looking.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
