|
trΓΈnderen wrote: If you have any reason at all to relate to the generated code. Trading readability and maintainability for "slightly faster code" is generally a bad move. . I generally agree with you. However, as we both know there are exceptions, which is why you used the word "generally" I'm sure. This is one of those cases, as lexing is always in a critical code path, and a generalized lexer must be able to handle bulk input as efficiently as possible.
My input to visual fa is one or more regular expressions. Literally just that. Here is the full input for that generated lexer, in my .rl Rolex lexer format, but it should be easy enough to discern the grammar below without knowing the format.
Object = "{"
ObjectEnd = "}"
Array = "["
ArrayEnd = "]"
FieldSeparator = ":"
Comma = ","
Number = '-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?'
Boolean = 'true|false'
Null = "null"
String = '"([^\n"\\]|\\([btrnf"\\/]|(u[0-9A-Fa-f]{4})))*"'
WhiteSpace = '[ \t\r\n]+'
The table driven code is run on a flat array of integers. It might be more efficient to unflatten it in this case - maybe? I used to run a more complicated array of structs for this, and I don't remember there being a performance difference. But anyway, there is also an array of int arrays for a feature called block ends, which simulate lazy matching on a DFA. (I have the details of all of it documented in my Visual FA series). It's also simpler in operation than it looks. I do actually use gotos in a couple of places here to restart the state machine. It was much less complicated than orchestrating a while with breaks. I should state that I didn't comment the code here because it wouldn't help me. It may help others, but I didn't really care about that. This pattern is burned into my brain after writing more than half a dozen lexers that follow the same. It honestly would just clutter it for me, as the code makes immediate sense to me despite how it looks, and I didn't write it for a team.
private FAMatch _NextImpl(
#if FALIB_SPANS
ReadOnlySpan<char> s
#else
string s
#endif
)
{
int tlen;
int tto;
int prlen;
int pmin;
int pmax;
int i;
int j;
int state = 0;
int acc;
if (position == -1)
{
++position;
}
int len = 0;
long cursor_pos = position;
int line = this.line;
int column = this.column;
int ch = -1;
Advance(s, ref ch, ref len, true);
start_dfa:
acc = _dfa[state];
++state;
tlen = _dfa[state];
++state;
for (i = 0; i < tlen; ++i)
{
tto = _dfa[state];
++state;
prlen = _dfa[state];
++state;
for (j = 0; j < prlen; ++j)
{
pmin = _dfa[state];
++state;
pmax = _dfa[state];
++state;
if (ch < pmin)
{
state += ((prlen - (j + 1)) * 2);
j = prlen;
}
else if (ch <= pmax)
{
Advance(s, ref ch, ref len, false);
state = tto;
goto start_dfa;
}
}
}
if (acc != -1)
{
int sym = acc;
int[] be = (_blockEnds != null && _blockEnds.Length > acc) ? _blockEnds[acc] : null;
if (be != null)
{
state = 0;
start_be_dfa:
acc = be[state];
++state;
tlen = be[state];
++state;
for (i = 0; i < tlen; ++i)
{
tto = be[state];
++state;
prlen = be[state];
++state;
for (j = 0; j < prlen; ++j)
{
pmin = be[state];
++state;
pmax = be[state];
++state;
if (ch < pmin)
{
state += ((prlen - (j + 1)) * 2);
j = prlen;
}
else if (ch <= pmax)
{
Advance(s, ref ch, ref len, false);
state = tto;
goto start_be_dfa;
}
}
}
if (acc != -1)
{
return FAMatch.Create(sym,
#if FALIB_SPANS
s.Slice(unchecked((int)cursor_pos), len).ToString()
#else
s.Substring(unchecked((int)cursor_pos), len)
#endif
, cursor_pos, line, column);
}
if (ch == -1)
{
return FAMatch.Create(-1,
#if FALIB_SPANS
s.Slice(unchecked((int)cursor_pos), len).ToString()
#else
s.Substring(unchecked((int)cursor_pos), len)
#endif
, cursor_pos, line, column);
}
Advance(s, ref ch, ref len, false);
state = 0;
goto start_be_dfa;
}
return FAMatch.Create(acc,
#if FALIB_SPANS
s.Slice(unchecked((int)cursor_pos), len).ToString()
#else
s.Substring(unchecked((int)cursor_pos), len)
#endif
, cursor_pos, line, column);
}
while (ch != -1)
{
var moved = false;
state = 1;
tlen = _dfa[state];
++state;
for (i = 0; i < tlen; ++i)
{
++state;
prlen = _dfa[state];
++state;
for (j = 0; j < prlen; ++j)
{
pmin = _dfa[state];
++state;
pmax = _dfa[state];
++state;
if (ch < pmin)
{
state += ((prlen - (j + 1)) * 2);
j = prlen;
}
else if (ch <= pmax)
{
moved = true;
}
}
}
if (moved)
{
break;
}
Advance(s, ref ch, ref len, false);
}
if (len == 0)
{
return FAMatch.Create(-2, null, 0, 0, 0);
}
return FAMatch.Create(-1,
#if FALIB_SPANS
s.Slice(unchecked((int)cursor_pos), len).ToString()
#else
s.Substring(unchecked((int)cursor_pos), len)
#endif
, cursor_pos, line, column);
}
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
We're running a heavy process that's causing some of our servers to creak a little. Sorry about any slowness you might experience today. It'll be over soon, I hope.
cheers
Chris Maunder
|
|
|
|
|
No problem, I had to attend a funeral today
(no, not the funeral of the CodeProject website)
|
|
|
|
|
Sorry to hear that, mate.
cheers
Chris Maunder
|
|
|
|
|
Thanks Chris, it was my mothers funeral, everything went well, the weather was perfect, and I'm sure she would have appreciated all the kind words said by the four speakers. I concluded with some lines from an old poem:
Quote: Farewell to thee, farewell to thee
Thou charming one who dwells in shaded bowers
One fond embrace ere I depart
Until we meet again.
|
|
|
|
|
I will do my part to alleviate the server burden by not posting this. Oops.
|
|
|
|
|
Cheers Chris it wasn't slow for me - it was unavailable for hours
In a closed society where everybody's guilty, the only crime is getting caught. In a world of thieves, the only final sin is stupidity. - Hunter S Thompson - RIP
|
|
|
|
|
I thought the server hamsters seemed a little haggard this morning.
Check out my IoT graphics library here:
https://honeythecodewitch.com/gfx
And my IoT UI/User Experience library here:
https://honeythecodewitch.com/uix
|
|
|
|
|
Honestly, I thought it was the solar flares.
|
|
|
|
|
The introduction to The Lounge says:
Quote: The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.
I am focused on the sentence: Trolling, (political, climate, religious or whatever) will result in your account being removed.
However, is bashing and trolling M$, Apple, Google, Meta, etc. when they, as companies, act stupid with their technology/programming/API offerings, OK?
Of course it is
|
|
|
|
|
It's not bashing. It's discussing, in painful detail, the issues that we face using our tools as part of a thoughtful dialogue.
To be honest, the state of software today is doing my head in.
(and it's been a while - how are you?)
cheers
Chris Maunder
|
|
|
|
|
Chris Maunder wrote: discussing, in painful detail, the issues that we face using our tools as part of a thoughtful dialogue I think I will save that wording, and use it to tag some of the future Lounge postings that the OP is referring to
Religious freedom is the freedom to say that two plus two make five.
|
|
|
|
|
Seems like discussing the developer that was fired from Google for claiming that the Google AI is sentient would be border line on that?
|
|
|
|
|
Strange boy he was !
In a closed society where everybody's guilty, the only crime is getting caught. In a world of thieves, the only final sin is stupidity. - Hunter S Thompson - RIP
|
|
|
|
|
The AI or the human?
|
|
|
|
|
Hi Chris! Long time!
That's a very nice, "PC" way of putting it
But to be fair, I've seen plenty of M$ bashing and trolling (and yes, I know it is out of love) --- and they deserve it somewhat
Regards,
Brian Hart
|
|
|
|
|
Wordle 1,057 3/6*
π¨β¬β¬β¬β¬
β¬π¨π¨β¬π¨
π©π©π©π©π©
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
β¬β¬π¨β¬β¬
β¬π©β¬β¬β¬
π©π©β¬β¬π¨
π©π©π©π©π©
In a closed society where everybody's guilty, the only crime is getting caught. In a world of thieves, the only final sin is stupidity. - Hunter S Thompson - RIP
|
|
|
|
|
Wordle 1,057 4/6
🟨🟨β¬β¬β¬
🟨🟨🟨β¬β¬
β¬🟨🟨β¬🟨
🟩🟩🟩🟩🟩
Ok, I have had my coffee, so you can all come out now!
|
|
|
|
|
Wordle 1,057 3/6
π¨π¨π¨β¬β¬
π¨π©π©π©β¬
π©π©π©π©π©
|
|
|
|
|
Wordle 1,057 3/6
β¬β¬β¬β¬π¨
β¬β¬π¨π¨β¬
π©π©π©π©π©
|
|
|
|
|
Wordle 1,057 3/6
β¬β¬β¬π©β¬
β¬β¬β¬β¬π¨
π©π©π©π©π©
WordleBot
Skill 86/99
Luck 76/99
I have no idea how I found the word today. According to WordleBot there was just one possible solution after my second guess. I guessed in less than 10 seconds and⦠solved!
If you can't explain something to a six year old, you really don't understand it yourself. (Albert Einstein)
|
|
|
|
|
Wordle 1,057 5/6*
🟨🟨β¬β¬β¬
🟩β¬🟨β¬β¬
🟩🟨β¬🟨β¬
🟩🟩β¬β¬🟨
🟩🟩🟩🟩🟩
|
|
|
|
|
7 weeks to go and I have SERIOUS short timer disease. I know it will pass, but at the moment...
Very heavy weather the last few days in the southeastern US... tornadoes up where one set of grandkids live, tornadoes a bit north of me, but....
530am - thunderstorm moving through the area, nice breeze blowing through the office, I just need to sedate the geriatric golden that I know has less than 6 months to live Hips are failing.
Charlie Gilley
βThey who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.β BF, 1759
Has never been more appropriate.
|
|
|
|
|
Back when I was working with a lot of USAF officers, they called the syndrome FIGMO: F___It - Got My Orders. Is this a change of jobs, or actual retirement? I retired in 2016 then, last year, got a call from an old boss asking me to come back to help build an infrastructure for a new Tribe. So far it's been way more fun than retirement!
Will Rogers never met me.
|
|
|
|
|