Click here to Skip to main content
15,888,330 members
Home / Discussions / C#
   

C#

 
AnswerRe: hi!!! Pin
12Code29-Apr-09 17:42
12Code29-Apr-09 17:42 
AnswerRe: hi!!! Pin
12Code29-Apr-09 17:58
12Code29-Apr-09 17:58 
AnswerRe: hi!!! Pin
SayreCC29-Apr-09 19:20
SayreCC29-Apr-09 19:20 
QuestionHow to stop VS to load user controls into toolbox? Pin
Morven Huang29-Apr-09 16:56
Morven Huang29-Apr-09 16:56 
AnswerRe: How to stop VS to load user controls into toolbox? Pin
Luc Pattyn29-Apr-09 17:01
sitebuilderLuc Pattyn29-Apr-09 17:01 
GeneralRe: How to stop VS to load user controls into toolbox? Pin
Morven Huang29-Apr-09 17:08
Morven Huang29-Apr-09 17:08 
GeneralRe: How to stop VS to load user controls into toolbox? Pin
Luc Pattyn29-Apr-09 17:10
sitebuilderLuc Pattyn29-Apr-09 17:10 
QuestionRemoving duplicates from large text files (Performance needed) Pin
Ehsan Baghaki29-Apr-09 14:12
Ehsan Baghaki29-Apr-09 14:12 
Dear all,
I need a way to read a 500MB large TEXT file containing a url on each line and remove duplicates and then write data back on the text file. I know it sounds simple but here i need my algorithm to be as fast as possible. I don't want to design a special data structure for this purpose i just want to use C# pure language to do this.

Here i have some question in mind that i never found a good answer before:

1.When you use
string s = File.ReadAllText(filename);
the exception 'out of memory' will thrown (Because the amount of data is too big). So we should read the file line by line. dose reading line by line affect the speed? isn't it better to read the whole file and then do the processing?

2.What is the fastest data structure in c# to access data on it. i mean between hashtable list String[] array...

3.Because it's the matter of speed, should i think of writing data back to a database after reading from file? is it faster than writing directly back to file? (like inserting data into MySQL server with an index on my url field with UNIQUE type, so by inserting the data it will take care of duplicates)

I know my question is a bit big.. Wink | ;) but if you have any idea in any part i will be glad to know your precious opinions.
Thank you.
AnswerRe: Removing duplicates from large text files (Performance needed) Pin
Luc Pattyn29-Apr-09 15:04
sitebuilderLuc Pattyn29-Apr-09 15:04 
GeneralRe: Removing duplicates from large text files (Performance needed) [modified] Pin
Ehsan Baghaki29-Apr-09 22:28
Ehsan Baghaki29-Apr-09 22:28 
GeneralRe: Removing duplicates from large text files (Performance needed) Pin
Luc Pattyn30-Apr-09 1:14
sitebuilderLuc Pattyn30-Apr-09 1:14 
GeneralRe: Removing duplicates from large text files (Performance needed) Pin
Ehsan Baghaki30-Apr-09 2:58
Ehsan Baghaki30-Apr-09 2:58 
AnswerRe: Removing duplicates from large text files (Performance needed) Pin
Doug Goulden29-Apr-09 15:17
Doug Goulden29-Apr-09 15:17 
QuestionMessage Removed Pin
29-Apr-09 11:39
professionalN_tro_P29-Apr-09 11:39 
AnswerRe: How to Load data into user settings Pin
DaveyM6929-Apr-09 12:07
professionalDaveyM6929-Apr-09 12:07 
AnswerRe: How to Load data into user settings Pin
Henry Minute29-Apr-09 13:10
Henry Minute29-Apr-09 13:10 
QuestionUm... it works when I F5 but not when I CTRL - F5...? Pin
Edmundisme29-Apr-09 11:27
Edmundisme29-Apr-09 11:27 
AnswerRe: Um... it works when I F5 but not when I CTRL - F5...? Pin
Eddy Vluggen29-Apr-09 12:45
professionalEddy Vluggen29-Apr-09 12:45 
GeneralRe: Um... it works when I F5 but not when I CTRL - F5...? Pin
Edmundisme29-Apr-09 12:47
Edmundisme29-Apr-09 12:47 
QuestionC# - File Permission Pin
malharone29-Apr-09 10:44
malharone29-Apr-09 10:44 
AnswerRe: C# - File Permission Pin
Jimmanuel29-Apr-09 11:19
Jimmanuel29-Apr-09 11:19 
AnswerRe: C# - File Permission Pin
Luc Pattyn29-Apr-09 11:33
sitebuilderLuc Pattyn29-Apr-09 11:33 
GeneralRe: C# - File Permission Pin
malharone29-Apr-09 13:17
malharone29-Apr-09 13:17 
QuestionHelp with Web Service Upload Pin
charles Frank29-Apr-09 10:16
charles Frank29-Apr-09 10:16 
QuestionHow can I use cookies stored by Internet Explorer 7(in Vista) in my WinForms application? Pin
rebulanyum29-Apr-09 10:08
rebulanyum29-Apr-09 10:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.