|
I have a form that gets a list of files for processing I am storing the path in an sting array then passing the array to a class that does the work on the files. the single thread approach takes about 30 mins to process 100 files and update an access DB (will be ported to SQL soon). Currently the single thread makes the form not responsive. I want to at a minimum move the processing to another thread, possibly setup a thread pool. from what I can tell the time consuming part is updating the DB. I have a few questions before I jump to conclusion of adding 1 thread or a thread pool.
1. will the thread pool solution run into locked file problems when writing to the DB from multiple threads? currently using Jet OLEDB to connect to the DB
2. What is the best way to handle passing the files to the class with threads?
Currently I am reading the directory and storing the files in a string array in the form
creating the object for the class
passing the array to the StoreArray method
call the Process method
the process method calls another method that handles the interaction with the array
I think the answer to my 2nd question with the least amount of rewriting code is to create the object of the class as a form variable, but I'm not sure that that is really the best solution.
I'm new to threads and not sure how I can get the data from the Form to the class.
below is two of the methods that should show how i am currently doing this (could have errors I didn't cut and paste)
## form1 ###
private void btnStart_Click(object sender, EventArgs e)
{
int count = 0;
string dirpath = this.txtxmldir.Text.ToString();
if (dirpath != "")
{
//new Dir Info object
DirectoryInfo di = new DirectoryInfo(dirpath);
//load files into array
FileInfo[] rgFiles = di.GetFiles("*.xml");
foreach (FileInfo fi in rgFiles)
{
file[count] = dirpath + "\\" + fi.Name;
count++;
}
}
RRRxml ProcessXml = new RRRxml();
ProcessXml.SetFiles(file, count);
ProcessXml.Process();
}
## RRRxml Class ###
public void Process()
{
while(Arraycounter > 0)
{
string FiletoProcess = ReadFiles(); //Read files is in charge of controlling the array counter and returns the file to load
xml.Load(FiletoProcess);
//Do work....
}
}
|
|
|
|
|
Hi Planker,
i will try to answer your question.
1. I don't know it exactly but it seems that there is only one connection to the db at one time (I think because it is file based). This issue will be solved if you move to a database supporting multiple connections at once. To use multiple threads with the access database you can think about using the producer-consumer pattern. The threads processing your files will produce data. This data will be stored in a single datastructure (attention: must be thread-safe!). If data is passed into this structure the thread containing the db connection will "consume" this data and store it within the db. (search for the producer-consumer-pattern to get an idea of it).
2. Here I would nearly do the same using the producer-consumer-pattern. Store the files in a thread-safe datastructure (single static instance, so that it can be accessed from everywhere in your application), the threads working with the files will "consume" the filenames stored within the datastructure.
The perfect datastructure for this is a Queue. For using the producer-consumer-pattern take a look here:
http://msdn.microsoft.com/en-us/library/yy12yx1f(VS.80).aspx[^][^].
Let me know if this helps you.
Regards
Sebastian
|
|
|
|
|
I think, writing docu on SW with MS-Word and so on is obsolete - too inflexible.
XML, XSLT and ... are todays standard.
But, what I don't know: How are these tools applied by "normal users" (non-xslt-experts)? What is the professional way to write documentation? I also mean more than just what is generated by MsVisualStudio.
Best regards
|
|
|
|
|
|
Frygreen wrote: writing docu on SW with MS-Word and so on is obsolete
Really?
Frygreen wrote: XML, XSLT and ... are todays standard.
As used by obsolete software like Microsoft Word!
Frygreen wrote: How are these tools applied by "normal users" (non-xslt-experts)?
With Microsoft Word.
Frygreen wrote: What is the professional way to write documentation?
With a word processor... Like Microsoft Word.
Seriously, take a Word 2007 document replace the .docx extension with .zip and take a peek inside. Lots of luvvvelee XML!
And, also serious, I will guarantee most professional documentation is written in Microsof Word that it converted to a PDF when ready for "publication".
|
|
|
|
|
|
..hmmmm...
thank you for your answers.
I'm puzzled. I know about MS-Word, Sandcastle, Ndoc - I use all of them.
Integrating (this is transforming it via xslt to html/pdf) Sandcastle, Ndoc in an automatic build-environment is just "plain easy", because it is based on XML.
On the other hand, MS-Word makes also sense, because it's so easy to use for "non-programmers" and e.g. attaching images and the like is no problem.
But unfortunately in my environment (many users/machines/humans updating asynchronously docu) MS-Word is not to maintain in practice; and how to maintan automatically generated docu (let's say generated by NDoc) with MS-Word?
What I really liked in the past (and still today) was/is latex, because there is really no maintenance-cost (independent of the number of users). But LateX is not so well upto date concerning newest technology
|
|
|
|
|
Function read bytes from file and save in new file, but after this operation size of the new file is different
source file: Size 26,6 KB (B: 27300)
after
Size: 26,8KB (B: 27468)
bufferSize= 252
enter= 27300
enter / bufferSize = 108,(3) so 109 blocks
109* bufferSize = 27468
How can I do this correct?
Please for Your help
C# code
private void button1_Click(object sender, EventArgs e)
{
readMyStream("D:\\files\\myFile.txt");
}
Stream inputStream;
Stream outStream;
int bufferSize = 252;
byte[] buffer;
private void readMyStream(string fileName)
{
buffer = new byte[bufferSize];
inputStream = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize, true);
outStream = new FileStream("D:\\plikiTxt\\az.txt", FileMode.Create, FileAccess.Write, FileShare.Write, bufferSize, false);
inputStream.BeginRead(buffer, 0, buffer.Length, new AsyncCallback(OnCompletedRead), null);
}
private void OnCompletedRead(IAsyncResult asyncResult)
{
int bytesRead = inputStream.EndRead(asyncResult);
if (bytesRead > 0)
{
outStream.Write(buffer, 0, buffer.Length);
inputStream.BeginRead(buffer, 0, buffer.Length, new AsyncCallback(OnCompletedRead), null);
}
else
{
inputStream.Close();
outStream.Close();
}
}
|
|
|
|
|
In this snippet
if (bytesRead > 0)
{
outStream.Write(buffer, 0, buffer.Length);
inputStream.BeginRead(buffer, 0, buffer.Length, new AsyncCallback(OnCompletedRead), null);
}
what if bytesRead is less than buffer.Length? Wouldn't you be writing more bytes to the file than you read?
P.S. use PRE tags around code to preserve formatting
|
|
|
|
|
Yes , I know now
That is ok.
Do You know better solution?
FileStream fstFile;
FileStream fstWrite;
private void sendData()
{
string fileName = "d:\\az.txt";
// The file stream will read bytes from the file that the user has chosen
fstFile = new FileStream(fileName, FileMode.Open, FileAccess.Read);
fstWrite = new FileStream("d:\\zapis.txt", FileMode.Create, FileAccess.Write);
int bytesSize = 0;
byte[] downBuffer = new byte[252];
string tekscik;
while ((bytesSize = fstFile.Read(downBuffer, 0, downBuffer.Length)) > 0)
{
fstWrite.Write(downBuffer, 0, bytesSize);
}
fstFile.Close();
fstWrite.Close();
}
|
|
|
|
|
|
I want get byte[] from file, manipulate them(add detection code), send by UDP ...
|
|
|
|
|
|
in C # I designed a program including WebBrowser, 1 combobox called cbo1.I want to control a checkbox combobox's Web site on a WebBrowser cbo1 by the user control.I programming will be like?
|
|
|
|
|
Stop spamming over and over, this is the third time you asked. Do you think that the people who regularly post here won't recognize it? You got all the help you're going to get..
|
|
|
|
|
...
"Well, we're getting "F"'d at work. WPF, WCF, and WWF... WTF?" --John Simmons
|
|
|
|
|
hi, i want to detect whenever the printer starts printing and so in this way i can count total numbers of printed pages and shows to the user, i think that there should be an event that i have to handle(sth like PrintPageStart) but i didn't find the solution after long search  how i can detect and count numbers of printed pages?
thanks
|
|
|
|
|
Hii
I'd like to know the steps and get help for building a DBMS software for handling biological data using C#.
I don't know where to start from and how I could build this software but perhaps a guide from you will help me.
Thanx alot.
|
|
|
|
|
lockily right now i have found this  :
<br />
Microsoft.Reporting.WinForms.ReportViewer p = new ReportViewer();<br />
p.Print += new CancelEventHandler(p_Print);<br />
void p_Print(object sender, CancelEventArgs e)<br />
{<br />
Console.Beep();<br />
}<br />
|
|
|
|
|
how stupid am i!!!!  sorry that mistake was a shame :P
that code won't work
|
|
|
|
|
Does that mean that you want to build your own DBMS or that yo uwant to use for example SQL Server to store and manipulate biological data using C#? If it is the latter and if you have no previous experience about this, I think you should for example get some good books and get to know the basics of database usage in applications (database creation, tables, relations, SQL syntax, ADO.NET etc.)
|
|
|
|
|
Is it possible to bring back several query data and place them in an array?
I want to use that array to generate pages of information
Ferron
|
|
|
|
|
ferronrsmith wrote: several query data and place them in an array
I'm not sure you are vague enough. Perhaps you could abstract it a little more. 
only two letters away from being an asset
|
|
|
|
|
I am grabbing several user information from a database and because i am cheap and don't wanna use crystal, i want to post the result in my form and some how generate multiple pages (if at all possible)
Ferron
|
|
|
|
|
You really need to work on accurately describing what you looking for.
From the Crystal remark I assume you are wanting to generate some sort of report, the "generate multiple pages" comment though leaves several questions. What do you mean by pages? ASP.NET pages, report pages? Stop trying to be so vague.
only two letters away from being an asset
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
