|
Sharp Ninja wrote: This might be a defensive technique I have developed to maintain the technical integrity of an article in the face of (sometimes badly) broken English.
Or amusement 
|
|
|
|
|
I had a natural tendency to mimic the accent of someone I was talking to, face to face. It began when I started traveling lots, and meeting people with significantly different accents.
Fortunately, as I repeatedly caught myself doing it, and became more aware of it, I was able to suppress it.
|
|
|
|
|
Sharp Ninja wrote: Lately, I have found myself reading articles in the voice and accent that I assume the author to have. This might be a defensive technique I have developed to maintain the technical integrity of an article in the face of (sometimes badly) broken English.
I have done the same with this post but no it is still incomprehensible.
Peter Wasser
"The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts." - Bertrand Russell
modified 18-Feb-19 5:20am.
|
|
|
|
|
35. Ocean and lake swimmer (4)
modified 17-Feb-19 1:58am.
|
|
|
|
|
I really hope the answer isn't "FISH", but that's all I came up with...
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Maybe it is "FROG"
It does not solve my Problem, but it answers my question
modified 19-Jan-21 21:04pm.
|
|
|
|
|
Most frogs can't live in sea water: there is a saltwater frog - The Crab Eating Frog of south Asia - but it's habitat is mangrove swamps and "brackish water", and can only survive brief excursions into the sea.
Crab-eating frog - Wikipedia[^]
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
SEA-L
Cheers,
Peter
Software rusts. Simon Stephenson, ca 1994. So does this signature. me, 2012
|
|
|
|
|
We can’t stop here, this is bat country - Hunter S Thompson RIP
|
|
|
|
|
|
VS(2017) has a maximum supported file size of 10mb? I just found out myself while trying to load a 925mb xml file.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
Is 925mb not quite one bit?
|
|
|
|
|
have you tried Notepad then?
same thing, just with a little less of that annoying intellisense. (after all xml is "human readable", no help should be required.)
Message Signature
(Click to edit ->)
|
|
|
|
|
It's probably to encourage smaller source files: 10MB of code in one file is probably a little too big ...
Why on earth do you want to load a 1GB XML anyway? That's far too big for me to want to read!
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
That's a problem with XML; it has to be read in its entirety before you can do anything with it.
At work I receive a 6GB XML file every stinking day and I have to use SSIS to get it into a database.
I'm beginning to prefer JSON, which I can read one object at a time (provided the outer-most value is a array of objects).
However, I have written a fairly simple XML file splitter so I can make smaller files from one big one when I need to find out where a problem (e.g. non-well-formed XML) exists.
|
|
|
|
|
That's not entirely true. You can use XmlReader, and it sequentially reads a node at a time (it's slower than XDocument, and you can't go reverse read direction, but it solves my issue).
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
Combining XmlReader and LinqToXML, the memory consumption never goes above 350mb, and it takes about 45 minutes to run though the sample files (this includes adding the data to the database, one record at a time (426,000 records).
When I add a dash of TPL, it only takes about 9 minutes to process the same three files.
I think I could get it even faster if I inserted multiple records per query, but I'm tired of dickin' with it.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
Well, for the most part I'm limited to built-in SSIS components.
Potentially I could write something custom, as I have for JSON and CSV files (ones which aren't stable enough for the flat-file components).
|
|
|
|
|
I just wanted to see what was in it. 
They range from 1mb, up to about 3.5gb. I have no control over how large the files to be processed are (they're generated by nessus security scans). The idiots that generate the files are completely unwilling to accommodate us, so it's essentially a "it is what is is" situation.
I have to parse these files and store the results in our database. Using just XDocument, I was running out of memory (the server in question only has 8gb, of which most is already used by other processes), so I have to resort to using a combination of XmlReader and LinqToXml.
Notepad, IE, Firefox, WordPad, and MS Word all load the file, but it takes more than five MINUTES for them, and wordpad/word become completely unusable.
<rant>
I wish people here (not you but some others) would stop f*ckin assuming I'm a rookie programmer. I have more years in the industry than most people on CP have even been alive.
</rant>
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
John Simmons / outlaw programmer wrote: <rant>
I wish people here (not you but some others) would stop f*ckin assuming I'm a rookie programmer. I have more years in the industry than most people on CP have even been alive.
</rant>
It's been a while I've seen you assert yourself on CP. I kinda miss the ol' smackdowns.
|
|
|
|
|
Ouch! That's a stupid amount of data, particularly for a text-based transfer mechanism. Have these people never heard of databases?
On the bright side, at least it's not XLSX? 
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
OriginalGriff wrote: On the bright side, at least it's not XLSX?
dunno, xlsx isn't so bad and easier (ok lazier) to debug if there's bad data elements jus load into excel and scroll down to the line with the issue.
if you're suggesting interop (i.e. slower than molasses) that's a complete other issue, and there are way way faster [read & write] alternatives.
worst comes to worst can unpack the xlsx and viola, it's xml (pretty much exactly the same).
(not criticizing, just unsure why you think it's any worse.)
Message Signature
(Click to edit ->)
|
|
|
|
|
Have you ever tried to load 1GB into Excel? 
(And bear in mind that XLSX is packaged, zipped, XML - and thus slower and more memory hungry than "naked" XML)
Sent from my Amstrad PC 1640
Never throw anything away, Griff
Bad command or file name. Bad, bad command! Sit! Stay! Staaaay...
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
OriginalGriff wrote: Have these people never heard of databases?
That's our job.
Got memory consumption down to no more than 350mb and it only takes 9 minutes to process my three sample files, for a total of 426,000 records. I'm going to look awesome on Tuesday. Upside, this app replaces a large perl script that was doing the same job, and everyone in the shop can maintain it because - well - it's not perl.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
John Simmons / outlaw programmer wrote: I'm going to look awesome on Tuesday.
That only if the other morons people appreciate your work, not the first time awesome tools that are real improvements get dumped because a couple of idiots co-workers say:
- We have always done it this way
- That is not going to work (without even giving a try)
- Or similar crap arguments...
and not even give a damned "Thank you"
M.D.V. 
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
