|
I have to write an application theat reads big files performs some processing on there data like XOR on a some section in it each section is a few maga bytes and then save them to A new file
waht should i do in order to achive max performance?
shoul i read a few sections eaach time and process and append them to the new file or one section each time?
there are several threads that do this proces on several different files at the same time?
sample code will be great help
Thnks
|
|
|
|
|
poqeqw wrote: waht should i do in order to achive max performance?
The limiting factor here is probably the drive speed. Getting the data into RAM is what is going to take the longest. simply XORing the data is insignificant in comparison to how long it will take to load/save the data, so don't mess around with separate threads for processing the data, you'll be wasting your time.
you can use System.IO.File.ReadAllBytes(@"C:\myfile"); to get the whole file. and System.IO.File.WriteAllBytes(@"c;\myfile", thedata); to save the processed data. I'd start by doing it like this. Stick some timers in a time how long the load takes, how long the save take and how long the processing takes. There is nothing you can do to speed up the load/save (apart from buy faster drives).
However, if the processing time is significant, you could start the processing while still loading the file. You could do this by doing the read on a seperate thread like this:
System.IO.FileStream myStream = new System.IO.FileStream(@"c;\myfile", System.IO.FileAccess.Read);
myStream.BeginRead(...);
and allowing the main thread to start processing while the read is still taking place. You'll have to put checks in place to make sure the main thread doesn't get ahead of the data being read.
(Don't bother using threads to do several files at the same time, the drive will only thrash about and read time will actually be slower. focus on doing them sequentially. Remember the limiting factor here is almost certainly the drive access. Unless of course they are on separate drives, or you are doing some really heavy cpu intensive processing)
Simon
|
|
|
|
|
Hi,
I am develping a web project in c#.net 1.1. The application reads an xml file which is stored in the application directory. This xml file is a thirdparty xml from internet. They will update this xml in every point of time (ie: currency xml). In this case, I need a scheduled download of the file from the internet to my application directory. Please guide me on how to achieve this. Thanks.
I would like to appreciate all the techies providing wonderful solutions to all the queries in this website.
Thanks
rijz
|
|
|
|
|
You want a program that downloads a file at specific intervals automatically? Go for a Windows service.
Cheers,
Vikram.
The hands that help are holier than the lips that pray.
|
|
|
|
|
|
Hello everyone,
I am writing XQuery based on an XML file. I am using instance type of XPathDocument, XPathNavigator, XPathExpression and XPathNodeIterator.
My question is does any of them need to be released to free resource? I have not found any Close/Dispose methods for them. It is so strange to me since at least the XML file to be queried needs to be closed.
thanks in advance,
George
|
|
|
|
|
When you load XML file, it loads the contents to memory and free the file handles. So no need to close it externally.
|
|
|
|
|
Thanks N a v a n e e t h,
You mean when we create an XPathDocument, the load file content, parse and close work is done internally? So, no need to care about further manualy work to close or release any resource?
regards,
George
|
|
|
|
|
|
Thanks N a v a n e e t h,
Question answered.
regards,
George
|
|
|
|
|
Hi friends
I am working in windows applications (C#)...In my project i am using UltraGrid..
In that grid there are 3 bands(Country ,State and city...State is below the country and the city is below the state)...Each band consists of 12 columns...Also Each column of all the bands contains its own column headers...Then in my ultragrid i have enabled column moving and column swapping options with the following code:
SFOMainScreen_grpBoxMainDetails_ugDetails.DisplayLayout.Bands[2].Override.AllowColMoving =
AllowColMoving.Default;
SFOMainScreen_grpBoxMainDetails_ugDetails.DisplayLayout.Bands[2].Override.AllowColSwapping =
AllowColSwapping.Default;
so in my ultragrid all the 3 bands with its column headers are displayed when running the application... I can move and swap the columns of all the bands when the column headers of each bands are made visible..According to my requirement, i Just make the country column headers(All column headers of first Band (ie) Country Band) as the visible one and
the column headers of rest of the bands( state and City) as the invisible one...
For the above case,
My requirement is When the column headers of first band is moved then along with the column of first band(country Band) corresponding columns of the state and city bands which are in the band 2 and band 3 respectively can also be moved..I need the C# Source code for the above requirement...Any known friends help me..... Thanks in advance kawin...
r
|
|
|
|
|
When a question relates to a 3rd party product, you will generally be better off posting your question in the forums for that product. You will probably get an answer much quicker than hoping that somebody here uses that product as well.
|
|
|
|
|
I always suspect that when people do this sort of thing it is because they have not actually paid for the product and they don't want to risk exposing themselves on the vendor's website. It would seem absurd to have paid for a product like this without checking out the support options in advance.
|
|
|
|
|
Good point (and one I hadn't considered). I just put it down to laziness.
|
|
|
|
|
In my current application I am getting an error "Cannot insert duplicate key row in object <tablename> with unique index <indexname>.\r\n51062\r\nThe statement has been terminated." while trying to run the command sqlCommand.ExecuteNonQuery();
Please advice me if anyone has any idea on this.
Thanks & Regards
Mishra
|
|
|
|
|
<blockquote class="FQ"><div class="FQA">Ballita wrote:</div>"Cannot insert duplicate key row in object with unique index .\r\n51062\r\nThe statement has been terminated."</blockquote>
was the key field inserted with the same value?
I think the best way for you is to copy the sql statement here, because the error occured in sqlCommand.ExecuteNonQuery();
If we dream, every thing is possible!
|
|
|
|
|
Please do not post the same question to different forums.
Ballita wrote: error "Cannot insert duplicate key row in object with unique index
if you bother to read what its says it blatently obvious you Cannot insert duplicate key row in object with unique index
Bob
Ashfield Consultants Ltd
|
|
|
|
|
Hello everyone,
Any guidelines about when to use Hashtable and when to use Dictionary? I always use Dictionary. 
thanks in advance,
George
|
|
|
|
|
Use Hashtable in V1, otherwise use Dictionary.
|
|
|
|
|
V1 means C# 1.0 or?
regards,
George
|
|
|
|
|
V1 should mean Version1.0 or Version1.1
V2 use dictionary,me too
|
|
|
|
|
Cool, Chao.Fu!
Question answered.
regards,
George
|
|
|
|
|
|
Thanks PIEBALDconsult,
Question answered.
regards,
George
|
|
|
|
|
Well MSDN states that Dictionary(Of TKey, of tValue) is a new class in v2.0. but further explains that the Objects used as Key are required to have an implementation of Equality Comparer, IEqualityComparer whereas in HashTable the objects used as Keys are required to override the IHashCode Provider interface or Object.GetHashCode method.
Internally HashTable uses the Dictionary object as its Key/Value pair but hashtable is faster in retrieval.
Rest of the details are well explained in MSDN.
Muhammad Talha
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
