Click here to Skip to main content
15,896,153 members
Home / Discussions / Database
   

Database

 
AnswerRe: Is a monitor tool for MS Access Database? Pin
RichardGrimmer3-Jun-04 5:11
RichardGrimmer3-Jun-04 5:11 
GeneralRe: Is a monitor tool for MS Access Database? Pin
spi_cn3-Jun-04 14:47
spi_cn3-Jun-04 14:47 
Generalconfigure publishing and distribution Pin
guan hwang2-Jun-04 15:27
guan hwang2-Jun-04 15:27 
GeneralSQL Server Memory Process Not Shutting Down Pin
Member 6666072-Jun-04 11:56
Member 6666072-Jun-04 11:56 
GeneralRe: SQL Server Memory Process Not Shutting Down Pin
RichardGrimmer3-Jun-04 5:14
RichardGrimmer3-Jun-04 5:14 
GeneralQuestion involving SQL/Queries/Crystal Reports Pin
frank212-Jun-04 10:15
frank212-Jun-04 10:15 
GeneralRe: Question involving SQL/Queries/Crystal Reports Pin
Kaleem8-Jun-04 22:24
Kaleem8-Jun-04 22:24 
GeneralData processing on large tables using ADO.NET Pin
Anonymous2-Jun-04 9:37
Anonymous2-Jun-04 9:37 
Hello all

I'm working on a data processing application which processes records in multiple (unrelated) tables containing about 2 million records.

Basically it runs through each record, checks various fields in each row according to certain business rules, and potentially updates other fields in the same row with updated values.

Here's my dilemma. I believe I have only 2 options available to me -

1. Using DataSet - I havent tried this yet but with 2 million records, it seems like its a very bad idea to try and pull a dataset of the whole table into memory while processing. Is there a way to page the dataset such that I can pull in only x number of rows into memory at a time?

2. Using DataReader - since I have to update the row after processing, using a data reader by itself won't work. I am currently using a datareader for iterating through the table and SqlCommands to update fields (though this seems to be prone to timeouts - I suspect that the writes to the database are locking out the datareader or vice versa)

Is there a better way to go about doing this sort of processing? Any suggestions would be greatly appreciated.
GeneralRe: Data processing on large tables using ADO.NET Pin
Mazdak2-Jun-04 10:04
Mazdak2-Jun-04 10:04 
GeneralRe: Data processing on large tables using ADO.NET Pin
Michael Potter2-Jun-04 10:36
Michael Potter2-Jun-04 10:36 
GeneralRe: Data processing on large tables using ADO.NET Pin
Edbert P3-Jun-04 12:51
Edbert P3-Jun-04 12:51 
GeneralRe: Data processing on large tables using ADO.NET Pin
Anonymous10-Jun-04 12:24
Anonymous10-Jun-04 12:24 
GeneralSQL Permissions keep reseting (web) Pin
David Flores2-Jun-04 4:23
David Flores2-Jun-04 4:23 
GeneralFixed Pin
David Flores3-Jun-04 3:57
David Flores3-Jun-04 3:57 
Generaltext file Pin
Xaneb1-Jun-04 22:21
Xaneb1-Jun-04 22:21 
GeneralRe: text file Pin
MDem3-Jun-04 3:14
MDem3-Jun-04 3:14 
GeneralRe: text file Pin
RichardGrimmer3-Jun-04 5:16
RichardGrimmer3-Jun-04 5:16 
GeneralSQL server retruning DB error 10025 Pin
Raghunandan S31-May-04 16:40
Raghunandan S31-May-04 16:40 
Generaldynamic connection in sql server reporting services Pin
raf_se31-May-04 0:18
raf_se31-May-04 0:18 
GeneralRe: dynamic connection in sql server reporting services Pin
Mazdak31-May-04 19:07
Mazdak31-May-04 19:07 
GeneralTriggers in SQL Pin
HowRU30-May-04 19:46
HowRU30-May-04 19:46 
GeneralRe: Triggers in SQL Pin
Hesham Amin30-May-04 22:01
Hesham Amin30-May-04 22:01 
GeneralRe: Triggers in SQL Pin
VenkatFor.NET10-Jun-04 11:46
VenkatFor.NET10-Jun-04 11:46 
GeneralADO.NET updates caching Pin
ajenny30-May-04 3:49
ajenny30-May-04 3:49 
GeneralRe: ADO.NET updates caching Pin
sukesh.g30-May-04 19:27
sukesh.g30-May-04 19:27 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.