The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.
I was working on improving its performance even more, but why?
There is a point after which any further improvement is a waste of your time. If you use data only on your own platform, store it in the most convenient binary format. If you need to transfer it to another machine in a portable format, then JSON is an option. However, the time required for a network transfer even over a dedicated 1Gbps line will overwhelm the time required for parsing.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
There is a point after which any further improvement is a waste of your time.
Yeah. The issue was, it *wasn't* a waste of time on my older machine. I had no context for how much faster a modern machine was.
Daniel Pfeffer wrote:
However, the time required for a network transfer even over a dedicated 1Gbps line will overwhelm the time required for parsing.
This is true, but for files the calculus is different, and part of what my library was designed to do was to parse through huge data dumps, typically in line delimited "JSON" format, so network times weren't really the bottleneck in that case. Even I/O wasn't entirely, even on my old machine, which surprised me.
The performance is already decent on my old machine, searching through JSON at almost 600MB/s
it also will run happily on an 8-bit CPU with <8kB of RAM, probably 4kB or less.
So I take your point, but on balance I think it's good. My rationale is this. If you need superfast JSON bulk parsing, you're going to buy a decent machine. If you're okay with an older machine, you're probably okay with 600MB/s of throughput. I think that's reasonable.
If you're okay with an older machine, you're probably okay with 600MB/s of throughput. I think that's reasonable.
I would be already happy with 100MB/s so...
honey the codewitch wrote:
If you need superfast JSON bulk parsing, you're going to buy a decent machine. If you're okay with an older machine, you're probably okay with 600MB/s of throughput. I think that's reasonable.
Yes it is. If the performance is a big step, then it is worthy. For just 1% or 2% or even a 5% faster... it is really to think if the time struggling with the improvement is worth or not
If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.
To be honest, this is why I don't develop on an uberPC - if it runs at an acceptable pace on my dev hardware, then it'll run well on any client kit.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
I feel the *exact* same way and have for as long as I've written code professionally.
However, my old machine was dying. I needed a new one, and buying a low to mid range machine is throwing good money after bad. Getting a midhigh or high end machine means it can remain viable for longer.
Although given my last machine died before the performance got unusable maybe I should take that into account.
I don't know. On my original machine my code ran at maybe 30MB/s before I got it up to almost 600MB/s.
That was worth it.
The other thing is the code will run on a machine with 4kB of RAM. I optimized for memory usage, primarily, not speed, so that this code could run on arduinos. Without optimization that wouldn't have been possible.
Certainly however, a lot of times it's not worth it.
Performance is a clear functional requirement that must be met, energy usage is a secondary quality attribute (a.k.a. non-functional requirement) that has gained attention in the last 2 decades.
Computers, servers, internet, cloud, big data etc already use (fill in your favorite number)% of our total energy consumption and that number is very likely to rise. Energy efficient algorithms help to slow that growth.
Another area where energy efficiency is important is mobile devices. Some of you may recall apps on your phone that drained the battery. And IOT, Arduino etc, think devices that run on battery or solar.
As a rule of thumb, faster algorithms use less energy, so improvements in performance might still be valuable.
Another argument for improving performance of algorithms is the mental exercise which is fun for some of us.
Energy efficient algorithms help to slow that growth.
That's a very good point. Considering my JSON library is meant to run on devices like 8-bit Arduinos, ESP32s, hopefully android phones at least, and windows, linux and apple PCs it pays to make it scale down to the smaller devices while still being efficient
I do that, and my JSON parser is so small the average query takes far less than 4kB. Between the locality and the small RAM usage, I expect the power consumption to be relatively small on small machines.
On PCs the code uses memory mapped files for performance, which of course would increase energy use despite the speed increase due to paging and such, but it would be hard to tell without profiling for energy use - a kinda tricky thing if you've never done it before.
Reminds me a bit of the ISP's hawking their wares. Lot's of TV adverts, emails, etc., to upgrade that 1Gb/s line yo always needed. Now, if you're a corporation with lots of users and lots of concurrent traffic - maybe.
But for us mere mortals? I noticed no difference when they upgraded me (oddly, to keep the price down with a new 1yr sprecial) to 300Mb/s . As that deal wound down I managed a new one, good until the end of 2022, for $10 less by dropping to 200Mb/s (and letting them charge my credit card automatically). Still no difference.
I mean, in the great reality of things - who can even send you data that fast? And, with net traffic, how would it get to you. And that's the parallel.
Scores of years ago, when I spent some time on Chemical Kinetics, that was always an importing thing to consider - or discover: "The Rate Determining Step" - and whatever is done around it will have no real effect. In layman's terms, it the intrinsic idea of a bottleneck.
This is a GREAT question, and it is SOLUTION Dependent!
If you could make the GNU Compilers twice as fast, PLEASE DO. Every time!
Well, until the time savings is so tiny, nobody cares.
We have a phrase: "The right answer late... Is the wrong answer!"
To us, it means that timing is part of the design/interface.
A Ping response that is normally > network timeout is useless to the recipient, as they gave up!
A page that takes 1 second longer than a user will wait, is too slow.
Sometimes, it's about "feedback" (Working... Spinners, Changing Screens, see Travelocity, LOL)
Sometimes, it's simply too slow to feel useable. Or I can do it by hand.
When I can launch EXCEL faster than find my calculator... You can bet I will use excel.
Therefore, either FIND a situation that REQUIRES twice as fast JSON parsing (Apps/Databases/etc), or move on.
Personally, I won't optimize beyond "acceptable" without a business requirement. (NOTE: A business requirement can be indirect, like the impact having 1 million of these running every day at full load).
As an example, we have an update statement taking 2 seconds. It feels slow to me. It's processing almost a million records. Part of a bigger process that is taking too long by 30 minutes. Not in a loop. I think I'll pass. [There is an 8 hour solution that would remove this. I logged a note, not in this pass. Making that update twice as fast is meaningless in this context]
Last Visit: 31-Dec-99 18:00 Last Update: 23-Sep-21 20:53