|
I removed the line and column counting from my JSON processor because I would have had to sacrifice a massive optimization to keep it.
Even though I used this information to debug my parser... 
I thought I would be okay because the only code I had left to add was already tested and running in my older codebase.
I copied and pasted it. It no longer works.
I'm up a creek! *headdesk*
Real programmers use butterflies
|
|
|
|
|
There are worse endeavors.
|
|
|
|
|
Daaaaaaaamn 
|
|
|
|
|
Edit: I figured it out. duh. and it's really too bad because only counting { and } is much faster.
Hey, you're a thinker. Riddle me this:
I want to scan JSON documents quickly, so I have a special function that works like string.IndexOfAny() does in .NET except at gigabyte/s speeds on modern machines.
I am looking for the end of the current object.
I stop at [ or { and increment a depth count.
I stop at ] or } and decrement a depth count.
Once depth hits zero I break.
Depth starts at zero so if there's no [ or { it will break at the next }
Why do I have to stop at [ and ] and count depths? Why can't I just count { and } since it's twice as fast?
For example
{
"foo": "bar",
"baz": [ { "a": 1 } ]
}
if i start in the inner most object i want to stop at the nearest end curly brace. If i start in the outermost object i want to stop at the final curly brace.
If I stop and count [ ] and it works. If I only count { } it doesn't. Why?
Real programmers use butterflies
modified 29-Dec-20 0:37am.
|
|
|
|
|
honey the codewitch wrote: I'm up a creek! *headdesk*
It seems, from your last post in the thread (at this time) you have solved it. Solutions not withstanding, I can give you a tip on some efficiency enhancements. We'll start with this one:
This does help solve some problems, but it can take quite a bit of bang and make noise that may disturb others. Now padding won't help because, although it muffles the noise it it defeats the purpose. A viable solution is to use the corner of the room (you're already there from the subject line) - thereby having the ability to impact both surfaces simultaneously and cut "runtime" in half! Moreover, if the impact is just slightly non-concurrent, those who are listening get the vague impression of being in a large chapel!
And a happy new year to you !
Now, much as the temptation may spring up, I do not recommend practice. Aside from the other obvious inconveniences, you made to repaint the wall in the corner, as well.
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
Have you tried adding curly braces? 
|
|
|
|
|
No, because I need GB/s throughput. 
Can't afford curly braces slowing me down.
Real programmers use butterflies
|
|
|
|
|
Been porting some JPL/Nasa C code to C#. Code is generally nice; originally ported from FORTRAN; what you might expect under the circumstances.
Then there was the latest "high precision" upgrade that required passing 2 extra doubles.
This "guy" creates a 4 element array and at various times passes the address of [0], then [2]; then in the called routines, works with 2 elements; in the calling with all 4; just to implement a double-double precision number.
You can't even do that in C#: pass a reference to the "middle" of an array.
Why would any "pro" dream up something like that?
(It was completely contrary to all the other code "patterns").
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
It has happened to me, and I've seen it happen to more programmers than I can count.
Besides, that code may be ugly, but it's not this:
static bool lexNumber(lex::LexSource& ls,JsonLexState& st) {
int32_t cp;
if(!ls.ensureStarted())
return false;
switch(st.flags.state) {
case 0:
st.flags.accept=0;
st.flags.eCount=0;
st.flags.fracCount=0;
st.flags.fracPart=0;
st.flags.negative=0;
st.flags.overflow=0;
st.integer = 0;
st.real = 0;
if(ls.more()) {
if (((cp=ls.current()) == '-')) {
if(!ls.capture(cp)) {
return false;
}
st.flags.negative =1;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 1;
return true;
}
if ((cp == '0')) {
if(!ls.capture(cp)) {
return false;
}
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 2;
return true;
}
if (((cp >= '1')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
st.integer=cp;
st.real = cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 8;
return true;
}
}
goto error;
case 1:
if(ls.more()) {
if (((cp=ls.current()) == '0')) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
if(0!=st.flags.negative)
cp=-cp;
st.real=cp;
st.integer=cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 2;
return true;
}
if (((cp >= '1')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
if(0!=st.flags.negative)
cp=-cp;
st.real=cp;
st.integer=cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 8;
return true;
}
}
goto error;
case 2:
if(ls.more()) {
if ((((cp=ls.current())== 'E')
|| (cp == 'e'))) {
if(!ls.capture(cp)) {
return false;
}
st.flags.fracPart = 2;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 5;
return true;
}
if ((cp == '.')) {
if(!ls.capture(cp)) {
return false;
}
++st.flags.fracCount;
st.flags.fracPart=1;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 3;
return true;
}
}
st.flags.accept=1;
return false;
case 3:
if(ls.more()) {
if ((((cp=ls.current()) >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
if(st.flags.fracCount<63) {
cp-='0';
if(0!=st.flags.negative)
cp=-cp;
st.real+=(cp*(pow(10,-st.flags.fracCount)));
++st.flags.fracCount;
}
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 4;
return true;
}
}
goto error;
case 4:
if(ls.more()) {
if ((((cp=ls.current())== 'E')
|| (cp == 'e'))) {
if(!ls.capture(cp)) {
return false;
}
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 5;
return true;
}
if (((cp >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
if(st.flags.fracCount<63) {
cp-='0';
if(0!=st.flags.negative)
cp=-cp;
st.real+=(cp*(pow(10,-st.flags.fracCount)));
++st.flags.fracCount;
}
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 4;
return true;
}
}
st.flags.accept=2;
return false; case 5:
if(ls.more()) {
if ((((cp=ls.current()) == '+')
|| (cp == '-'))) {
if('-'==cp)
st.flags.fracPart = 3; if(!ls.capture(cp)) {
return false;
}
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 6;
return true;
}
if (((cp >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
if(st.flags.eCount>6553) {
st.flags.overflow = 1;
}
st.flags.eCount=st.flags.eCount*10+cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 7;
return true;
}
}
goto error;
case 6:
if(ls.more()) {
if ((((cp=ls.current()) >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
if(st.flags.eCount>6553) {
st.flags.overflow = 1;
}
st.flags.eCount=st.flags.eCount*10+cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 7;
return true;
}
}
goto error;
case 7:
if(ls.more()) {
if ((((cp=ls.current()) >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp-='0';
if(st.flags.eCount>6553) {
st.flags.overflow = 1;
}
st.flags.eCount=st.flags.eCount*10+cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 7;
return true;
}
}
st.flags.accept=2;
return false; case 8:
if(ls.more()) {
if ((((cp=ls.current())== 'E')
|| (cp == 'e'))) {
if(!ls.capture(cp)) {
return false;
}
st.flags.fracPart = 2;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 5;
return true;
}
if (((cp >= '0')
&& (cp <= '9'))) {
if(!ls.capture(cp)) {
return false;
}
cp=cp-'0';
if(st.flags.negative)
cp=-cp;
st.integer=st.integer*10+cp;
if((0>st.integer && 0==st.flags.negative)||(0<st.integer && 0!=st.flags.negative)) {
st.flags.overflow = 1;
}
st.real = floor(st.real*10.0+0.5)+cp;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 8;
return true;
}
if ((cp == '.')) {
if(!ls.capture(cp)) {
return false;
}
++st.flags.fracCount;
st.flags.fracPart=1;
if(!ls.advance()) {
if(ls.hasError())
return false;
}
st.flags.state = 3;
return true;
}
}
st.flags.accept =1;
return false;
}
error:
ls.capture(ls.current());
return false;
}
That's a DFA generated from a regular expression into a C# state machine, I then ported to C++ and added numeric computations in the middle of it.
It's because strtod() doesn't stream and neither does atoll() or atoi() - you have to have the entire string loaded before you can parse, and my DFA above supports infinite precision in lexical space but imposes limits in value space, so that you can round trip numbers of any precision.
I can defend the code based on it originating from machine generated source, for which I have a graphical flow chart also that goes with this routine.
It's less buggy than writing it all by hand, despite how ugly it is.
So sometimes there's a reason for nasty code. Maybe there was a reason.
Real programmers use butterflies
modified 28-Dec-20 13:26pm.
|
|
|
|
|
Reminds me of looking at core dumps. But at least one can generate different core dumps. (Progress).
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
The real question is why they use doubles instead of decimals. Doing math with doubles is NOT as precise as doing math with decimals.
".45 ACP - because shooting twice is just silly" - JSOP, 2010
-----
You can never have too much ammo - unless you're swimming, or on fire. - JSOP, 2010
-----
When you pry the gun from my cold dead hands, be careful - the barrel will be very hot. - JSOP, 2013
|
|
|
|
|
Because range was more important?
Wrong is evil and must be defeated. - Jeff Ello
Never stop dreaming - Freddie Kruger
|
|
|
|
|
It was converted to C in 1993; converting to decimal is too risky IMO; they publish the Almanac and plan space missions with this.
The "high precision" model uses "millions" of coefficients (doubles) from prior space observations for 15 solar system objects; the decimal would also double the file requirements.
They effectively model space with static "data" (snap shots: past, present, future); then interpolate; updating the data every few years (ephemerides). That's how they get precision.
(I agree; use decimals. I would wind up with .0000000001 somewhere even when doing a few small integer operations.)
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
#realJSOP wrote: Doing math with doubles is NOT as precise as doing math with decimals.
What they are using are expansions, where a number is represented as a sum of doubles. You lose some range at the lower end, but - with careful attention to detail - gain precision in the rest of the range.
I have implemented a C++ <cmath> library that gives precision of 102-106 bits for all functions except atan() (~90 bits). I presume that NASA can do even better.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
Just one thing - after more years than I dare count:
Is this Brexit thing finally out of the news? Are they really over and one with it?
You know what I mean - will those clowns in Brussels have anything to do (aside from hara$$ US Based Companies) or will they go home and try to find some meaningful employment?
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
Well ... yes and no. (What a surprise!)
We've yet to see how places like France will actually implement it, and I'm sure there will be legal challenges etc. to wade through yet.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
The whole things seems to bring to mind that song "Suicide is Painless". Perhaps they should have just cleared off *that* continent and started fresh?
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
I think the Germans tried that a couple of times last century without much success ... 
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
W∴ Balboos, GHB wrote: Perhaps they should have just cleared off that continent and started fresh?
Some of them tried. they got 'Murica 
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
Judging by the accent, that lot settled in Louisiana.
Clearly, however, those that made the choice, in general, made the right choice.
Now, in these times, we need a place for the EU to flood with its inhabitants. Some suggest Atlantis - but I won't be judgemental at this point.
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
W∴ Balboos, GHB wrote: Clearly, however, those that made the choice, in general, made the right choice.
That's a matter of opinion. Reading the news coming from the US, it seems to be falling apart at the seams.
W∴ Balboos, GHB wrote: we need a place for the EU to flood with its inhabitants.
Agreed. 
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.
|
|
|
|
|
Daniel Pfeffer wrote: Reading the news coming from the US, it seems to be falling apart at the seams.
Actually, when I get news from other parts of the world it seems they, too, are falling apart at the seems. The reality is, at least in my opinion, is that it's a steady and consistent process - fed occasionally with the fuel of hopes things are getting better - soon to be dashed by new problems that make the improvements seem so distant.
Except in China - where, according to CCP News, everything really is wonderful. Only criminals would speak otherwise and they, of course, are jailed to prevent further crimes and slander.
The Who had it quite right in their little ditty "Won't Get Fooled Again[^]" wherein they sang: "Meet the new boss"; a stark contrast (or compliment to?) Joni Michell's 'Big Yellow Taxi[^]' where, no matter what the change, someone will claim "Don't it always seem to go, you don't know what you've got 'till it's gone".
Note that even the music of our species can't seem to agree on anything. In which case, Mr. Hendrix really had a point and awesome coping method, here[^].
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
Nice musical reference. Two of my very favorite songs.
Joni Mitchell's song is mostly about lost lovers, however.
I got to see the Who live once and the Who 2.0 once.
Louisiana is OK. I live here. It's pretty good outside of New Orleans where the crime rate is significantly lower.  
Wear a mask! You are not invincible.
|
|
|
|
|
Slow Eddie wrote: Louisiana is OK. I live here.
One great truth I've observed: people tend to like the place where they grew up. I think it's because adaption was built in.
From NYC, and it's suburbs, I spent years in Chicago and even Morgantown, WV.
The former I described then as "a small town of two million people". Timing, if of interest to you, is when Daly was there, died, and a non-Daly was elected as emperor (mayor). I was a newlywed and in graduate school.
As for the latter - we, now with two smalll kids in tow (and a bun in the oven) thought we couldn't wait to leave NY. Well, after the initial honeymoon, reality set it: the place just plain sucked. A person has no idea how much they miss (from their former venue of life) until it's gone - like the Big Yellow Taxi. There are the obvious ones: bagels, real pizza, knishes, Asian groceries. This, in a university town or it would have been far worse. Seventy miles to Pissburg in order to get anything like civilization and ethnic delights.
But the think I most dislike about the rural communities? Well, there are two things, actually. One of them is that religion interferes too much with the freedoms they pretend to support with flag-waving. This can and does exist at a state-wide level. The other is that the friendly smiles and greetings are also empty. The only real people that the locals would ever have social interaction with are those who went to school with them and "church friends".
And - as the backwaters come close to being the back yards - the (well, I'll drop it as I'd probably violet some Lounge rules).
I look at it this way:
- The US has Alabama
- The Canadians have Quebec
- Europe has France
No one has bragging rights, anywhere.
| Ravings en masse^ |
|---|
| "The difference between genius and stupidity is that genius has its limits." - Albert Einstein | | "If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010 |
|
|
|
|
|
Quote: Is this Brexit thing finally out of the news?
Probably, but they might start the Brentry
Mircea
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
