Click here to Skip to main content
15,887,376 members
Home / Discussions / C#
   

C#

 
GeneralRe: run application c# Pin
Ibrahim.elh24-Oct-14 3:52
Ibrahim.elh24-Oct-14 3:52 
GeneralRe: run application c# Pin
Dave Kreskowiak24-Oct-14 3:53
mveDave Kreskowiak24-Oct-14 3:53 
GeneralRe: run application c# Pin
Pete O'Hanlon24-Oct-14 3:55
mvePete O'Hanlon24-Oct-14 3:55 
GeneralRe: run application c# Pin
Ibrahim.elh24-Oct-14 3:59
Ibrahim.elh24-Oct-14 3:59 
GeneralRe: run application c# Pin
Pete O'Hanlon24-Oct-14 4:04
mvePete O'Hanlon24-Oct-14 4:04 
AnswerRe: run application c# Pin
V.26-Oct-14 21:33
professionalV.26-Oct-14 21:33 
GeneralRe: run application c# Pin
Ibrahim.elh26-Oct-14 21:39
Ibrahim.elh26-Oct-14 21:39 
GeneralRe: run application c# Pin
V.26-Oct-14 21:53
professionalV.26-Oct-14 21:53 
all righty, that's already more clear.

First of all, 1 000 000 records in one go is just a big NO.
As an exercise calculate the size (in bytes) of one record and multiply that by 10^6. That amount of data is transferred through the wire... In addition your objects in code will also be too large memory wise.

Your best bet is to fetch this stuff in bits and parts. The first idea that comes to mind is to use paging, although for a million records this still a 1000 pages of a 1000 records.

In addition to that paging you'll need to filter the data to what users would like to see. You start with an empty grid and allow the user to filter on several columns before fetching anything.

3 hours is a long time. Make sure the query is optimized and the indexes on the database are properly built.


You'll probably need to fiddle with both the code part and the database in order to come to a good solution.
As rule of thumb any collection larger then let's say 10 000 is a no go. (I'm talking normal records, not blobs with huge amounts of data). (This number is not a rule, 100 can be already too much and in some cases 50 000 can still work ok)

This is not a simple problem, take your time to handle it right.
Hope this helps.
V.

(MQOTD rules and previous solutions)

Questionproblem when launching application Pin
Ibrahim.elh23-Oct-14 21:06
Ibrahim.elh23-Oct-14 21:06 
AnswerRe: problem when launching application Pin
OriginalGriff23-Oct-14 21:36
mveOriginalGriff23-Oct-14 21:36 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 21:45
Ibrahim.elh23-Oct-14 21:45 
AnswerRe: problem when launching application Pin
OriginalGriff23-Oct-14 21:59
mveOriginalGriff23-Oct-14 21:59 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 22:05
Ibrahim.elh23-Oct-14 22:05 
GeneralRe: problem when launching application Pin
OriginalGriff23-Oct-14 22:24
mveOriginalGriff23-Oct-14 22:24 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 22:32
Ibrahim.elh23-Oct-14 22:32 
GeneralRe: problem when launching application Pin
OriginalGriff23-Oct-14 22:38
mveOriginalGriff23-Oct-14 22:38 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 22:45
Ibrahim.elh23-Oct-14 22:45 
GeneralRe: problem when launching application Pin
Pete O'Hanlon23-Oct-14 22:42
mvePete O'Hanlon23-Oct-14 22:42 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 23:01
Ibrahim.elh23-Oct-14 23:01 
GeneralRe: problem when launching application Pin
Pete O'Hanlon23-Oct-14 23:08
mvePete O'Hanlon23-Oct-14 23:08 
GeneralRe: problem when launching application Pin
Ibrahim.elh23-Oct-14 23:15
Ibrahim.elh23-Oct-14 23:15 
AnswerRe: problem when launching application Pin
eljainc24-Oct-14 10:59
eljainc24-Oct-14 10:59 
QuestionGenerate xls file Windows Service Pin
Member 841450123-Oct-14 11:05
Member 841450123-Oct-14 11:05 
AnswerRe: Generate xls file Windows Service Pin
Eddy Vluggen24-Oct-14 1:49
professionalEddy Vluggen24-Oct-14 1:49 
QuestionHow to retain the drop down (which is loaded from database) values when page posted to server in MVC (c#)..? Pin
Kranthikumar.polasa23-Oct-14 8:06
Kranthikumar.polasa23-Oct-14 8:06 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.