Welcome to the Lounge

3-Feb-20 6:25

So I finally got the regex optimizing compiler to spit the output i wanted

And it's almost perfect, save the jmp at the beginning the but that's negligible. This is nearly a pure DFA.

The tricky part is at the root some of my lexer expressions may not be able to be converted into this DFA (optimized) code. And while order of evaluation doesn't matter for DFAs it does matter for NFAs (any of the non-optimizable expressions)

So basically what I had to do was scan through my lex expressions top to bottom, looking for entire runs of optimizable expressions

Those I'd transform into a single DFA expression with multiple different match instructions.

And then I intersperse those with the unoptimized expressions in the same order they came to me in.

And boom it allows me therefore to go from

L0000: save 0
L0001: jmp L0002, L0008, L0019, L0022
L0002: set "A".."Z", "_", "a".."z"
L0003: jmp L0004, L0006
L0004: set "0".."9", "A".."Z", "_", "a".."z"
L0005: jmp L0003
L0006: save 1
L0007: match 0
L0008: jmp L0009, L0011
L0009: char "0"
L0010: jmp L0017
L0011: jmp L0012, L0013
L0012: char "-"
L0013: set "1".."9"
L0014: jmp L0015, L0017
L0015: set "0".."9"
L0016: jmp L0014
L0017: save 1
L0018: match 1
L0019: set "\t", "\n", "\v", "\f", "\r", " "
L0020: save 1
L0021: match 2
L0022: any
L0023: save 1
L0024: match -1

Which requires lots of fibers and therefore multiple scans of a character per pass

to this, which requires only two scans per pass max (error checking accounts for the extra jmp):

L0000: save 0
L0001: jmp L0002, L0008
L0002: switch case "\t".."\r", " ":L0003, case "-":L0003, case "0":L0005, case "1".."9":L0004, case "A".."Z", "_", "a".."z":L0005
L0003: switch case "1".."9":L0004
L0004: switch case "0".."9":L0004, default:L0006
L0005: switch case "0".."9", "A".."Z", "_", "a".."z":L0005, default:L0006
L0006: save 1
L0007: match 0
L0008: any
L0009: save 1
L0010: match -1

The only thing left i have to potentially do is eliminate that jmp at the beginning, and add the second operand as the default in the switch/case that follows.

Edit: Perf testing shows me maybe i don't need to that, as it's already on par with the DFA. WOO!

I'm proud of myself. $Shucks | :-\$

Edit: Whoops, bit of a bug i just noticed but minor.

Real programmers use butterflies

Eddy Vluggen3-Feb-20 7:27

Eddy Vluggen

3-Feb-20 7:27

So I just downloaded the code from your latest article to see how you (ab?)use fibers and learn from it Cool | :cool:

First of all, there's obvious a lot of work in there, and the resulted votes seem way of. Second, "rolex" might not be the best of names, as some company might object to it.

Still looking for the fibers; I get distracted quickly. Any chance you're going to dedicate an article to the subject?

--edit
As you already explained, it's a struct. It just has the name fiber; I assumed a call to CreateFiber function (winbase.h) - Win32 apps | Microsoft Docs[^]

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

modified 3-Feb-20 13:44pm.

honey the codewitch3-Feb-20 7:48

3-Feb-20 7:48

Eddy Vluggen wrote:
Second, "rolex" might not be the best of names, as some company might object to it.

They can object all they like. Trademark law says I'm on solid footing. This isn't a watch or anything related to timepieces so it's not in the same industry, ergo, names can be the same without infringement, at least last time i talked to a copyright legal eagle about it.

Eddy Vluggen wrote:
Any chance you're going to dedicate an article to the subject?

Here's an article on the subject of the PikeVM specifically, that should cover what you're after (i think?):
Regex as a Tiny "Threaded" Virtual Machine[^]

Eddy Vluggen wrote:
As you already explained, it's a struct. It just has the name fiber; I assumed a call to CreateFiber function (winbase.h)

No, it's not to do with the Win32 API. It's a fiber in concept though, and uses the same principles as CreateFiber does. It's a cooperatively scheduled Program fragment with its own instruction pointer (Index)

Big Grin | :-D

Real programmers use butterflies

Eddy Vluggen3-Feb-20 8:03

Eddy Vluggen

3-Feb-20 8:03

honey the codewitch wrote:

Here's an article on the subject of the PikeVM specifically, that should cover what you're after (i think?):

Tx; I downloaded the first "FA-Master" I found and it contains a lot of projects (all of them distracting) Big Grin | :-D

honey the codewitch wrote:
No, it's not to do with the Win32 API

I hear duck, so I think ducks. Never crossed my mind that someone might breed a new kind of duck, "just" because you need some kind of duck to solve some other problem.

honey the codewitch wrote:
It's a cooperatively scheduled Program fragment with its own instruction pointer (Index)

I had to read that a few times.

Now exploring why VS demanded I install VS extensions when opening the project.

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

honey the codewitch3-Feb-20 8:57

3-Feb-20 8:57

That's weird that it demanded that of you. Try unloading the XXXXDevStudio projects.

VSIX projects are bogus and weird, but you have to use them if necessary.

The reason my projects are bundled is they're interdependent and I post them on GitHub that way.

The way I develop these days relies on a lot of pre-build steps and a lot of my projects build themselves with related projects, so it's tied together.

The project you want for the virtual machine is Lex.

The project you want for the demo of that is LexDemo.

Lexly is a tokenizer generator using Lex.

BTW I fixed a bug in the codebase, ~~it will be up in 10 minutes. It might not impact your build though, i'm not sure.~~ - found some other stuff to fix which delays me but it won't impact your build

Real programmers use butterflies

modified 3-Feb-20 15:40pm.

Eddy Vluggen3-Feb-20 9:41

Eddy Vluggen

3-Feb-20 9:41

honey the codewitch wrote:
That's weird that it demanded that of you. Try unloading the XXXXDevStudio projects.

Yup, RolexDevStudio.

honey the codewitch wrote:
Lexly is a tokenizer generator using Lex.

..and a compiler, and an assembler.

honey the codewitch wrote:
BTW I fixed a bug in the codebase, it will be up in 10 minutes. It might not impact your build though, i'm not sure.

Just reading atm Big Grin | :-D

Bastard Programmer from Hell Suspicious | :suss:

If you can't read my code, try converting it here[^]
"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

honey the codewitch3-Feb-20 10:07

3-Feb-20 10:07

Lex is the thing that's the compiler and assembler. Lexly uses that engine to build the tokenizer.

I'm still working on fixes.

Real programmers use butterflies

Thought of the Day

OriginalGriff3-Feb-20 4:36

OriginalGriff

3-Feb-20 4:36

Did the PowerPoint presentation cross the road to get to the other slide?

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
AntiTwitter: @DalekDave is now a follower!

littleGreenDude3-Feb-20 5:21

littleGreenDude

3-Feb-20 5:21

I'm not any good at presentations, but when it comes to spreadsheets, I Excel.

“The palest ink is better than the best memory.” - Chinese Proverb

OriginalGriff3-Feb-20 5:26

OriginalGriff

3-Feb-20 5:26

We won't tell anyone - Mum's the Word, eh?

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
AntiTwitter: @DalekDave is now a follower!

honey the codewitch3-Feb-20 5:47

3-Feb-20 5:47

I didn't see your reply before i made mine. and here i thought i was clever. Big Grin | :-D

Real programmers use butterflies

Gary Wheeler3-Feb-20 7:21

Gary Wheeler

3-Feb-20 7:21

You guys are ruining my Outlook on life, with your Access to this crap. We should speak to your Publisher about it and your lack of Visio[n].

Software Zen: delete this;

honey the codewitch3-Feb-20 5:46

3-Feb-20 5:46

Word.

Real programmers use butterflies

Jacquers3-Feb-20 7:11

Jacquers

3-Feb-20 7:11

What data did it want to Access?

W∴ Balboos, GHB3-Feb-20 7:31

W∴ Balboos, GHB

3-Feb-20 7:31

More likely it was simply it's chance to escape from an Open Office.

Morally and ethically, bringing up P...P... is not KSS - in fact, I'd consider it unsafe for SoapBox III, as well.

Ravings en masse^
"The difference between genius and stupidity is that genius has its limits." - Albert Einstein
"If you are searching for perfection in others, then you seek disappointment. If you seek perfection in yourself, then you will find failure." - Balboos HaGadol Mar 2010

Daniel Pfeffer3-Feb-20 22:05

Daniel Pfeffer

3-Feb-20 22:05

It wanted to get to the Libre(ated) Office Smile | :)

Freedom is the freedom to say that two plus two make four. If that is granted, all else follows.
-- 6079 Smith W.

New workplace!

Sander Rossel3-Feb-20 1:14

Sander Rossel

3-Feb-20 1:14

So, as some of you know, I'm my own boss since last year.
For the first few months I've worked from home, which got very lonely (I have no girlfriend/wife, kids, roommates, etc.).
So I rented a place for a few days a week.
Unfortunately, they were in the middle of moving.
They got kicked out of their old building and the new building, an old coach house, had to be renovated.
The temporary space was an old school building... Just that, old and school-y.
But after three months of old school building I'm finally in the renovated coach house as of today!
It's really nice, good desks and chairs, smartphone access to the building, floor heating, really high ceiling with original beams and modern, warm, LED lighting...
The outside looks cool too, original coach house, but with a new characteristic driveway and a couple of parking spaces.
The owner even wants to ask the municipality to replace the regular (boring) street lighting in front of the building with some beautiful and characteristic lighting.
I share the room and building with some other people and companies, so it's never dull (except today because I'm the only here Laugh | :laugh:

).

Goodbye dirty radiators with gum stuck to it, hello professional building where I could invite potential customers if need be Big Grin | :-D

Best,
Sander

sanderrossel.com
Migrating Applications to the Cloud with Azure
arrgh.js - Bringing LINQ to JavaScript
Object-Oriented Programming in C# Succinctly

honey the codewitch3-Feb-20 1:20

3-Feb-20 1:20

Sounds like a cool setup. Smartphone access? spiffy.

I used to call forward the door buzzer of an apartment i lived at to my cell phone to get a lowtech dumbphone version of the same thing.

Real programmers use butterflies

RickZeeland3-Feb-20 1:28

RickZeeland

3-Feb-20 1:28

What a "moving" story $Shucks | :-\$

Richard MacCutchan3-Feb-20 2:36

Richard MacCutchan

3-Feb-20 2:36

Sounds nicer than any workspace I had to endure in 40+ years. (one or two were just about acceptable).