Click here to Skip to main content
15,886,676 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: Six months of work released in production... Pin
Sander Rossel3-Apr-21 9:45
professionalSander Rossel3-Apr-21 9:45 
GeneralRe: Six months of work released in production... Pin
BillWoodruff3-Apr-21 8:04
professionalBillWoodruff3-Apr-21 8:04 
GeneralRe: Six months of work released in production... Pin
Sander Rossel3-Apr-21 9:44
professionalSander Rossel3-Apr-21 9:44 
GeneralRe: Six months of work released in production... Pin
Slacker0073-Apr-21 8:14
professionalSlacker0073-Apr-21 8:14 
GeneralRe: Six months of work released in production... Pin
Sander Rossel3-Apr-21 9:44
professionalSander Rossel3-Apr-21 9:44 
GeneralRe: Six months of work released in production... Pin
kmoorevs3-Apr-21 9:19
kmoorevs3-Apr-21 9:19 
GeneralRe: Six months of work released in production... Pin
Sander Rossel3-Apr-21 9:44
professionalSander Rossel3-Apr-21 9:44 
GeneralRe: Six months of work released in production... Pin
Dan Neely5-Apr-21 2:55
Dan Neely5-Apr-21 2:55 
Sander Rossel wrote:
We had about two records of test data for this process (and despite me asking several times it was impossible to get more), but in production we have 5000+ records and apparently, the Excel (Open XML) library that I'm using can't handle that much data very well.


FYI Research I did a year or two back had EPPlus (also in nuget) as the C# full featured Excel output library most capable of generating large files. It's still an order of magnitude more of a ramhog than Excel itself; which matters when you're creating spreadsheets with 10s of millions of cells (hundreds of MB of ram, vs several GB; then OOM at around 100m) maybe 0.1% of which can have paragraph size freeform text.

Prior to the customer finally admitting that they didn't actually want everything in every single export of the dataset, the dump everything report was running up within about half of that limit. One of my even more bloated test sets did go over it, triggering my searching for alternatives. The only maybe more capable one I found was from someone else whose client needs were even more immense who ended up writing his own file writer that only supported the minimum subset of .xlsx that he needed. (This seemed too risky to take into production to me.)
Did you ever see history portrayed as an old man with a wise brow and pulseless heart, weighing all things in the balance of reason?
Is not rather the genius of history like an eternal, imploring maiden, full of fire, with a burning heart and flaming soul, humanly warm and humanly beautiful?
--Zachris Topelius

Training a telescope on one’s own belly button will only reveal lint. You like that? You go right on staring at it. I prefer looking at galaxies.
-- Sarah Hoyt

GeneralRe: Six months of work released in production... Pin
Sander Rossel5-Apr-21 3:34
professionalSander Rossel5-Apr-21 3:34 
GeneralBingeable series for the long weekend (probably in lockdown) Pin
OriginalGriff2-Apr-21 10:16
mveOriginalGriff2-Apr-21 10:16 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
David O'Neil2-Apr-21 10:20
professionalDavid O'Neil2-Apr-21 10:20 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Cp-Coder2-Apr-21 10:51
Cp-Coder2-Apr-21 10:51 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Greg Utas2-Apr-21 11:44
professionalGreg Utas2-Apr-21 11:44 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Cp-Coder3-Apr-21 1:31
Cp-Coder3-Apr-21 1:31 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Gary R. Wheeler2-Apr-21 10:55
Gary R. Wheeler2-Apr-21 10:55 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Jon McKee2-Apr-21 12:39
professionalJon McKee2-Apr-21 12:39 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Storm-blade2-Apr-21 11:36
professionalStorm-blade2-Apr-21 11:36 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Storm-blade2-Apr-21 11:39
professionalStorm-blade2-Apr-21 11:39 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Richard MacCutchan2-Apr-21 21:37
mveRichard MacCutchan2-Apr-21 21:37 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
OriginalGriff2-Apr-21 22:19
mveOriginalGriff2-Apr-21 22:19 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Richard MacCutchan2-Apr-21 22:30
mveRichard MacCutchan2-Apr-21 22:30 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Cp-Coder3-Apr-21 0:50
Cp-Coder3-Apr-21 0:50 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Slacker0072-Apr-21 23:12
professionalSlacker0072-Apr-21 23:12 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
Richard MacCutchan2-Apr-21 23:16
mveRichard MacCutchan2-Apr-21 23:16 
GeneralRe: Bingeable series for the long weekend (probably in lockdown) Pin
CodeWraith2-Apr-21 23:09
CodeWraith2-Apr-21 23:09 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.