|
I have been working on something in a project for the past few days, and I have been getting a little frustrated, and I have to decided to start from scratch. I am looking for some input on what I want to do, so any suggestions are certainly welcome.
I have a form which unzips data files, then calls an Import method from a class in a seperate project. When importing it uses SQLBulkCopy to copy all data in these files to a sql database. There is a large amount of data, so I need to keep the user informed of the progress during importation. So I want to display on the Form the data file name, count, and index while also displaying on a progress bar the percentage of data copied of the total amount from that file. This information will be updated on said form everytime a Batchsize is completed.
Input on how you would tackle this would be great.
|
|
|
|
|
Well you definitely need to use multithreading to make you GUI responsive during the time-consuming unzipping operation. You need to use raise events from your worker thread to notify the GUI of the progress. There needs to be proper marshalling between the threads.
Maybe this link:
http://msdn2.microsoft.com/en-us/library/ms998490.aspx[^]
will give you some ideas.. Especially the section "Using Tasks to Handle Interaction Between the UI Thread and Other Threads" will be relevant to you..
Hope this helps,
Bijesh
|
|
|
|
|
I would use a third party tool to take care of the SQL stuff. I use Red-Gate tools and have never had a problem with them. (No, I don't work for them) Red-Gate SQL Packager will package objects and data with an executable and supports command line.
Just my thoughts.
Hogan
|
|
|
|
|
I've been asked to help solve a performance issue if I can. A person requires an application which can distribute, virtually simultaneously, to as many as tens of thousands of client applications, a small amount of data. The data will usually be a single integer, but it is critical that all of the client applications receive the data with the very least possible delay, and, if possible, at virtually the exact same time. It cannot be known beforehand exactly when the server application will be able to determine and broadcast the data. The client applications must link to the server application; wait; and all be able to react to the data broadcast immediately.
Has anyone here written such a set of applications, or is anyone familiar with what kind of architecture or approach is necessary to achieve this kind of performance?
All reasonable approaches are legitimate candidates for solution.
TIA
m
|
|
|
|
|
I had to deal with a situation like this, but the data was coming from PLC's and we had an OPC server running on the server, the software running on the client side using an OPC client would subscribe to the item and receive data. All the clients would simultaneously get the data, the updates were pretty fast also(considering each client subscribed to 100's of items). You could probably try to google about OPC and find out what kind of architecture they are using to do it.
|
|
|
|
|
OPC is described to be an open standard similar to OLE:
http://www.opcfoundation.org/Default.aspx/01_about/01_whatis.asp
OPC's Alarms and Events specification addresses issues most similar to those of my question, but does not appear to be a method for actual simultaneous broadcasting of data (in fact, it appears to be a pull technology, versus a push technology); and I am wondering if a "simultaneous" broadcasting technology exists.
I'm trying to get my brain around a low level concept that will provide the best achievable performance. It seems to me that any possible implementation (including OPC?) will still be subject to having to serially broadcast (or, in the case of OPC, request) the data.
I am assuming 1) that at best a server process is going to have to iterate an array of clients; 2) that consequently the number of supported clients can only inherently degrade process speed; and 3) that optimal speed can only be achieved by minimizing interaction.
If these assumptions are correct, then there is no practical way to "simultaneously" broadcast/expose the data, and a limitation for maximum possible performance exists. The performance ceiling therefore would be something like NumberOfClients x SpeedOfSingleBroadcast, whereas the speed of the desirable "simultaneous" broadcast is 1 x SpeedOfSingleBroadcast.
Does anyone have any further thoughts on this?
|
|
|
|
|
This is a pretty good article on OPC:
http://blog.matrikonopc.com/index.php/thoughts-on-the-opc-ua-information-model/[^]
A few exerpts:
"A good starting point to understanding what the Information Model might represent, is to think in terms of the OPC Item address space, and Browsing interfaces in OPC DA. The specification described interfaces for viewing and navigating the address space (browse space). It did not dictate how the browse space was built or persisted in the underlying OPC DA COM server."
"In some cases the OPC Server did not persist this information, it simply made calls into the underlying native system to return the Items, like the OPC Server for Performance Monitor. In other cases the OPC Server would persist the information as a memory map, relational database or XML schema, such as the OPC Server for Microsoft Excel. In both cases it was transparent to OPC Client where the information came from."
This leads me to think we should be able to achieve OPC performance with standard .NET techniques.
?
|
|
|
|
|
I would say that, your option of SingleBroadcast to all connected clients would give you the simultaneous broadcast to all the clients. OPC actually checks for all the clients connected and the items subscribed by the client and then send the required data, this would mean some lag of 1/1000 second or even lesser.
Does you software need to be that accurate in simultaneously sending data? We are taking about time in milliseconds to process a complete array of clients and sending data to them.
|
|
|
|
|
Tarakeshwar Reddy wrote: I would say that, your option of SingleBroadcast to all connected clients would give you the simultaneous broadcast to all the clients. OPC actually checks for all the clients connected and the items subscribed by the client and then send the required data, this would mean some lag of 1/1000 second or even lesser.
Thanks for pointing this out, because it was not evident from the documentation I saw.
As to the "virtually simultaneous" requirement, these people are far from meeting their goal. Achieving the best practical performance subject to the limitations of such a process is as good as it's going to get for them, so that's what I'm interested in establishing. I haven't seen their code yet, so I don't know what they're doing. I assume their poor performance is a consequence of a fundamental problem such as mis-use of TCP/IP. They're having great trouble supporting a handful of clients.
I did find some interesting OPC components, and may write the company to determine if they can solve the problem if it looks like we can't solve the problem ourselves. Except perhaps for the number of clients, it seems this should be relatively simple, and that we should be able to avoid layering such substantial technology over such a simple application.
|
|
|
|
|
Well - this could be a candidate for Sql Notification Services. Basically, you would have your applications register their interest in the data and then the update would be triggered at point A. Mind you, you are going to have a fun job updating that many clients "simultaneously".
Deja View - the feeling that you've seen this post before.
|
|
|
|
|
This was one of my initial suggestions to these people -- the reservation being whether the underlying process can do the job better than a minimalized process we can write ourselves. I would think that to achieve best possible speed, we need to be able to spin off separate threads of execution to handle each callback. Each succeding broadcast then would not have to wait for conclusion of the preceding. Because the few threads available to the .NET thread pool would seem to preclude much advantage from multi-threading in a .NET server implementation, I was hoping that some "easy" integration of an RDBMS might circumvent the minimal available threads of the the thread pool. Is there an RDBMS you might recommend for being particularly strong in this area -- a good candidate for this job?
|
|
|
|
|
Well, we've had some success in this area with SQL Server 2005 and Notification Services. To be honest, MS has invested a lot of money in this technology and done a lot of work on it to ensure its robustness. To get this level of robustness/concurrency in custom code you are going to have to do a lot of work that has already been done for you, and you will have to do an amazing amount of testing.
Deja View - the feeling that you've seen this post before.
|
|
|
|
|
I'll take your advice quite seriously.
Thanks a bunch, Pete. I appreciate it.
|
|
|
|
|
No problem. I'm glad to help.
Deja View - the feeling that you've seen this post before.
|
|
|
|
|
Hi friends,
I have seen something like below in a resource file. Can somebody tell me what is (.)dot notation for.
Name: Value
myButton.Label SubmitButon
Thanks
|
|
|
|
|
The dot signifies an object's member (property, field, method, etc.). The statement would imply the Label property of the myButton object is "SubmitButton".
/ravi
|
|
|
|
|
Thanks for replying. But then how comes in the code when i type name of the resource file i do see this myResource.myButton_Label
|
|
|
|
|
netJP12L wrote: Thanks for replying. But then how comes in the code when i type name of the resource file i do see this myResource.myButton_Label
The code you are seeing (myResource.myButton_Label) is accessing the string resource through the resource class that Visual Studio auto-generates for you. The auto-generation translates the "." in the resource name to an "_" to make it a legal C# property name.
-----------------------------
In just two days, tomorrow will be yesterday.
|
|
|
|
|
I know how to send a service a command using the ExecuteCommand but i'm wanting to recieve a response back to the client. Any Ideas?
Thanks
Don't be overcome by evil, but overcome evil with good
|
|
|
|
|
I want to write in a Xml file some data, but I have an error because I have the same attribute "ClusterRoute" who appears many times.
XmlTextWriter myXmlTextWriter = new XmlTextWriter(@"c:\\Results.xml", Encoding.UTF8);
myXmlTextWriter.WriteStartDocument();
for (int i = 0; i < clustersList.Count; i++)
{
myXmlTextWriter.WriteStartElement("Clusters");
for (int j = 0; j < clustersList[i].Count; j++)
{
myXmlTextWriter.WriteAttributeString("ClusterRoute", clustersList[i][j].CityID.ToString());
}
myXmlTextWriter.WriteEndElement();
}
myXmlTextWriter.WriteEndDocument();
|
|
|
|
|
you can only have on instance of an attribute per element. That is just how XML works.
led mike
|
|
|
|
|
Hello everyone,
I am using a DataGridView control to display data which is stored in a remote database server. Using filter, I am trying to get the rows of information that I am looking for in the dataset. I was wondering if it is possible to get the data in a specific column/cell rather than the entire data in the row?
The following is the code that I am using for filtering the data in the DataGridView control.
string strText;<br />
string strExpr;<br />
string strSort;<br />
DataRow[] foundRows;<br />
<br />
strExpr = "shop_id >= '4' AND shop_id <= '7'";<br />
strSort = "shop_id ASC";<br />
<br />
foundRows = myTable.Select(strExpr, strSort);<br />
<br />
strText = null;<br />
for (int i = 0; i <= foundRows.GetUpperBound(0); i++)<br />
{<br />
for (int j = 0; j <= foundRows[i].ItemArray.GetUpperBound(0); j++)<br />
{<br />
strText = strText + foundRows[i][j].ToString() + "\t";<br />
}<br />
strText = strText + "\r\n";<br />
textBox.Text = strText;<br />
}
Thank you very much and have a great day.
Khoramdin
-- modified at 11:03 Thursday 10th May, 2007
|
|
|
|
|
You should go look at the C# Articles to get a handle on what you are doing, you are way off base.
led mike
|
|
|
|
|
Hello mike,
Thank you very much for your respond. You could be absolutely correct regarding my approach. But could you be kind enough to tell me where I need to start and what is the right approach? Then I will be able to narrow down my search and find the right approach.
Believe it or not, but I found that approach after reading number of C# articles. I am sure you agree with me when I say there are all sort of articles online. The problem is to find the right one and in order to find the right one I need to have the knowledge to assess the articles. But the point is, if I have the knowledge to make such a distinction then I have reached the level of knowledge that I won't be needing such articles!
Thank you very much and have a great day.
Khoramdin
|
|
|
|
|
Well I meant the C# Database Articles here on CodeProject, try this one[^]
led mike
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
