|
In a production app, I would log the exception and/or take a different course of action.
|
|
|
|
|
Hello,
I am attempting to create a benchmarking software that reads the contents of files.
I would like to report, every five seconds, the average speed per MB read, the speed in MB/sec.
I don't think that calculating a TimeSpan will be the best method, as it will definitely be inaccurate overtime (increasing in inaccuracy).
If I use a timer in the BackgroundWorker thread, how do I pass a variable to the ElapsedEventHandler that contains the amount of data read?
private void timer_Elapsed(object sender, ElapsedEventArgs e)
{
MessageBox.Show(AmountOfBytesRead);
}
public void ReadFiles(string TargetDirectory, bool RecursivePlz, string LogFilename)
{
DateTime StartOfReading = DateTime.Now;
System.Timers.Timer timer = new System.Timers.Timer();
timer = new System.Timers.Timer();
timer.Elapsed += new ElapsedEventHandler(timer_Elapsed);
timer.Interval = 5000;
timer.Start();
...
}
Thanks,
Matt
|
|
|
|
|
A BackgroundWorker can report progress... Turn on the WorkerSupportsProgress flag, cast the "sender" to a BackgroundWorker and use that to send a progress report. You're supposed to send the completion percentage as an integer, but you can also supply a "state" object along with that, which could contain the bytes read.
Now, if you want to get the amount of data read in the past X seconds, you need a rolling buffer... Maybe it could go something like this:
1) Every N bytes (Maybe every 500KB or so), the worker reports the bytes read as a progress event
2) The handler dumps that into a rolling buffer with a time stamp (DateTime), and maintains a pointer to the oldest record within the desired time span.
3) The speed is calculated as (BytesRead[current] - BytesRead[oldest]) / (TimeStamp[current] - TimeStamp[oldest]) and put into a class-level variable.
4) Every so often, your timer can just read that variable to report the speed (Or whatever you want to do with it)
|
|
|
|
|
That's a great idea. I was thinking backward.
The concern is cross-thread communication, and I think you addressed this.
BackgroundWorker_ProgressChange() has access to the main thread, which includes all objects in the inclusive/root class, such as a class-wide variable to hold a value, and the Timer object and events.
I still have the problem of accuracy, "left over" bytes when polling occurs.
I'll get testing!
Thanks very much,
Matt
modified on Wednesday, August 10, 2011 11:33 AM
|
|
|
|
|
bbranded wrote: I still have the problem of accuracy, "left over" bytes when polling occurs.
Not really, with the rolling buffer... You would keep the actual time of each report (You could even pass a DateTime+Integer struct from the worker to eliminate cross-thread delays), so you wouldn't have to make any assumptions... Your buffer would look something like:
11:37:50 1500KB
11:37:52 2000KB
...
11:42:30 58000KB
11:42:33 58500KB
So you would get:
(58500KB - 1500KB) / (11:42:33 - 11:37:50)
= 57000KB / 4:43
And I'm too lazy to calculate that out, but that'll give you a KB/minute for the past ~5 minutes. And of course you can adjust the numbers to get whatever level of precision you want. The timer wouldn't actually be used to do any measurements, but rather to just report the results.
|
|
|
|
|
This is very far from perfect, but will give others a good starting point to address this issue.
I'll open source the final project over at mbrownnyc.wordpress.com shortly.
private void _bw_DoWork(object sender, DoWorkEventArgs e)
{
string[] CommandArguments = e.Argument.ToString().Split('#');
string TargetDirectory = CommandArguments[0];
bool RecursivePlz = Boolean.Parse(CommandArguments[1]);
string LogFilename = CommandArguments[2];
ReadFiles(TargetDirectory, RecursivePlz, LogFilename);
}
public void ReadFiles(string TargetDirectory, bool RecursivePlz, string LogFilename)
{
int readsize = 65536;
double ReadOpsPerMB = readsize / 1048576;
TargetDirectory = TargetDirectory.ToLower();
if (!TargetDirectory.Contains("dfsrprivate"))
{
if (Directory.Exists(TargetDirectory))
{
string[] FileList = Directory.GetFiles(TargetDirectory);
int BytesReadInRound = 0;
foreach (string ffile in FileList)
{
FileInfo finfo = new FileInfo(ffile);
if (finfo.Exists)
{
int SizeOfFile = Int32.Parse(finfo.Length.ToString());
FileStream readStream = new FileStream(ffile, FileMode.Open, FileAccess.Read, FileShare.None, readsize + 1);
BinaryReader binReader = new BinaryReader(readStream);
string Status = "";
bool Readthefile = true;
try
{
if (binReader.PeekChar() <= -1)
{
Status = "EMPTY;" + ffile;
_bw.ReportProgress(0, Status);
Readthefile = false;
}
}
catch (Exception e)
{
Readthefile = true;
_bw.ReportProgress(0, ffile + " caused an exception, but was added to the total count: " + e.Message);
}
if (Readthefile)
{
byte[] valuebytes;
bool donereading = false;
_bw.ReportProgress(0, "reading " + Decimal.Divide(SizeOfFile, 1024).ToString() + " file: " + ffile);
while (donereading == false)
{
DateTime StartOperationTime = DateTime.Now;
valuebytes = binReader.ReadBytes(readsize);
BytesReadInRound = BytesReadInRound + valuebytes.Length;
donereading = (valuebytes.Length < readsize);
if (BytesReadInRound >= 1048576)
{
_bw.ReportProgress(2, "read|" + BytesReadInRound + "|" + LogFilename + "|" + DateTime.Now);
BytesReadInRound = 0;
}
DateTime EndOperationTime = DateTime.Now;
TimeSpan OperationTime = EndOperationTime.Subtract(StartOperationTime);
}
}
readStream.Close();
readStream.Dispose();
binReader.Close();
}
}
string[] SubDirectorylist = Directory.GetDirectories(TargetDirectory);
Array.Reverse(SubDirectorylist);
if (RecursivePlz == true)
{
if (Directory.Exists(TargetDirectory))
{
foreach (string ddir in SubDirectorylist)
{
ReadFiles(ddir, RecursivePlz, LogFilename);
}
}
}
}
}
}
struct SnapshotData
{
public string What;
public int HowManyBytes;
public string LogFilename;
public DateTime When;
}
List<SnapshotData> SnapShots = new List<SnapshotData>();
private void _bw_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
string UserState = e.UserState.ToString();
if (UserState.Contains("|"))
{
string[] UserStateCommands = UserState.Split('|');
SnapshotData Snapshot = new SnapshotData();
Snapshot.What = UserStateCommands[0];
Snapshot.HowManyBytes = Convert.ToInt32(UserStateCommands[1]);
Snapshot.LogFilename = UserStateCommands[2];
Snapshot.When = Convert.ToDateTime(UserStateCommands[3]);
SnapShots.Add(Snapshot);
TimeSpan Snapshotdifference = Snapshot.When.Subtract(SnapShots[0].When);
textBox2.Text = Snapshotdifference.ToString();
if (Snapshotdifference.TotalSeconds >= 10)
{
decimal ReadAfterLastFlushToLog = new decimal();
foreach (SnapshotData member in SnapShots)
{
ReadAfterLastFlushToLog = (ReadAfterLastFlushToLog + member.HowManyBytes);
}
ReadAfterLastFlushToLog = Decimal.Divide(Decimal.Divide(Decimal.Divide(ReadAfterLastFlushToLog,60),1024),1024);
StreamWriter LogFileWriter = new StreamWriter(Snapshot.LogFilename, true);
textBox5.Text = DateTime.Now.ToString() + ";" + ReadAfterLastFlushToLog.ToString();
LogFileWriter.WriteLine(DateTime.Now.ToString() + ";" + ReadAfterLastFlushToLog.ToString());
LogFileWriter.Close();
textBox2.Text = "";
SnapShots.Clear();
}
}
modified on Monday, August 22, 2011 1:09 PM
|
|
|
|
|
 my 5!
|
|
|
|
|
bbranded wrote: I don't think that calculating a TimeSpan will be the best method, as
it will definitely be inaccurate overtime (increasing in inaccuracy).
Only if you do it wrong.
|
|
|
|
|
Oh please please, share your wisdom. It's torture that you're keeping it all to yourself.
|
|
|
|
|
What? Oh.
Well I suspect you're considering calculating TimeSpans and adding them together rather than keeping the beginning time and calculating the whole elapsed time.
I'd likely use a StopWatch anyway, but that's just me.
|
|
|
|
|
Thanks. I'll look into that direction to solve the problem as well.
|
|
|
|
|
Hi All !
I use this Syntax for Filtering LastName field :
string strFilter = "LastName like '" + txtFamily.Text + "%'";
dvShowTable = new DataView(dataconn.dSet.Tables["PersonTable"], "", "", DataViewRowState.OriginalRows);
dvShowTable.RowFilter = strFilter;
dgShowTable.DataSource = dvShowTable;
if (dvShowTable.Count == 0)
{
MessageBox.Show("No find any Record !");
txtFamily.Text = "";
return;
}
It's works , but another filtering is about Account Field and I use this Syntax :
string strFilterAccount = "AccountNo like '" + Convert.ToDouble(txtAccount.Text)+"%'";
I have this Error:
Cannot perform 'Like' operation on System.Double and System.String
And If i use this syntax :
string strFilterAccount = "AccountNo="+ Convert.ToDouble(txtAccount.Text);
mmediately i put any Number , you get the message in the "If" conditional .
what is suitable instead of Like for number Field !
thanks
|
|
|
|
|
The LIKE operator is used only for string filtering, it cannot be used with numeric datatypes. You can try other operators like >= and <= etc.
|
|
|
|
|
Check out this MSDN page on the filter: http://msdn.microsoft.com/en-us/library/system.data.datacolumn.expression.aspx[^]
One thing it says is that the % sign should be between brackets [ ].
Like doesn't work for the double, so that's why you are getting that error. You need to use the compare operators, such as =. You say that is not working? Try putting a space between AccountNo and =, so: string strFilterAccount = "AccountNo = " + Convert.ToDouble(txtAccount.Text);
I wonder though, is Double really the proper type for an AccountNo? AccountNo sounds more like an integer to me.
|
|
|
|
|
MicroVirus wrote:
string strFilterAccount = "AccountNo = " + <br />
Convert.ToDouble(txtAccount.Text);
That shouldn't even compile, let alone work. Why convert to a Double whe you need a String for the filter...
|
|
|
|
|
Course it will, double has a ToString method which will be called at the + 
Converting to a number and back is elementary validation, throwing an exception at that point rather than in the database query.
|
|
|
|
|
Way cool. 
You mean I've been wasting keystrokes all this time actually putting in the .ToString() business?
|
|
|
|
|
I believe conversion from double to string is a widening conversion and thus can be done implicitly.
Anyway, your point about the there and back again conversion is right. I was more focused on solving the stated problem than watching what I copy-pasted.
The input should be validated beforehand, but this there-and-back-again method does avoid some SQL injection-like vulnerabilities.
|
|
|
|
|
Your code has SQL injection written all over it, though I'm not sure how protective the DataView is (i.e. whether it passes that filter through to any database level processor where it could do real damage). Even if it protects you in this instance, get into the habit of escaping anything you put in a query string (and that's what that filter string is).
I agree with MicroVirus that you probably want the account to be an int and use =. With double you can use int or you can use (AccountNo>=value-delta) AND (AccountNo<=value+delta) where delta is a comparison tolerance value which you should pick.
|
|
|
|
|
thanks for Your Replies ,
AccountNo is 13 digit , so i prefer to use to double ,
As i said , I want to Filter AccountNo , when i write "1" in the TextBox , and in the Result all of Records display that begin by "1" , when I write "2" after "1", display all Records begin "12" and etc.
Thanks !
|
|
|
|
|
Okay, to do that kind of string filtering you will need to store the ID as a string (or write a query with a conversion, but that's probably slow). This will prevent you from doing arithmetic on that column and, depending on your database, might be slow as a primary key, so you might want to have a separate column with a string version in it.
For large integers you should use long, not double.
|
|
|
|
|
As BobJanova said, this kind of code leaves you wide open to SQL injection attacks...
You might want to take a look at http://bobby-tables.com/[^] (Yes, it's funny, but it's also important)
|
|
|
|
|
The AccountNo field has a type of double in your database, which cannot be used with the 'like' operator. There are some solutions to this problem.
1. Change the type of AccountNo to text.
-or-
2. If you really want to use double for AccountNo, convert it to text in the query so that you can use the 'like' operator. The function to do the conversion may be different from one database engine to another. For example, in MySql, the function is 'cast', so something like this can be used:
string strFilterAccount = "cast(Accountno as char) like '" + Convert.ToDouble(txtAccount.Text) + "%'";
Considering that in many occasions an account number can really have letters and other non-digit characters in it, I think solution 1 may be better for you. (My water bill account number has letters and dashes embedded in it, for example)
SQL injection is a real issue here.
|
|
|
|
|
Thanks ,
apparently , There No solution for AccountNo , Unless I Change the DataType to Text .
But , if I define AccountNo to a Numeric DataType , Really , There is no solution for it.
By the Way wat is the equel of the "Cast" (in MySql ) for Sql . if we Use Cast , we get this Error !
The expression contains undefined function call cast().
Thanks a Lot !
|
|
|
|
|
If you are using Microsoft Access, the function to use is str(), like so:
string strFilterAccount = "str(Accountno) like '" + Convert.ToDouble(txtAccount.Text) + "%'";
SQL Server recognizes the 'cast' function. Please tell us what system you are using.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
