Click here to Skip to main content
15,895,799 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: A little light reading Pin
Randor 27-Dec-20 15:11
professional Randor 27-Dec-20 15:11 
GeneralRe: A little light reading Pin
Rick York27-Dec-20 23:05
mveRick York27-Dec-20 23:05 
GeneralRe: A little light reading Pin
W Balboos, GHB28-Dec-20 5:50
W Balboos, GHB28-Dec-20 5:50 
GeneralRe: A little light reading Pin
Rick York28-Dec-20 6:53
mveRick York28-Dec-20 6:53 
GeneralRe: A little light reading Pin
harold aptroot27-Dec-20 20:32
harold aptroot27-Dec-20 20:32 
GeneralRe: A little light reading Pin
honey the codewitch27-Dec-20 20:35
mvahoney the codewitch27-Dec-20 20:35 
GeneralMessage Removed Pin
27-Dec-20 8:26
Tanmoy8727-Dec-20 8:26 
GeneralSo there's still a use for bit twiddling Pin
honey the codewitch27-Dec-20 2:28
mvahoney the codewitch27-Dec-20 2:28 
Edit: This might be useless. I just found out simdjson's new API supports demand streaming like my library does. The only advantage mine has is it will run on tiny devices.

---------- SNIP

and it's called eliminating branches.

I'm creating a utf8 decoding routine based on some wicked code I found here
branchless-utf8/utf8.h at master · skeeto/branchless-utf8 · GitHub[^]

Black magic it is. I wouldn't be surprised if the compiler vectorizes this and turns it into simd instructions

So now I just need to do some 4 byte windowing without branches. I think i just need between 1 and 4 bit shifts to do it. I need to think on it though. I love this stuff.

I've already sped it up:
utf8 scanned 20383269 characters in 120.909000 ms at 157.142975MB/s

It could be a lot faster though: (currently my code branches)
raw ascii i/o 20383269 characters in 57.367000 ms at 331.200865MB/s

This is the holy grail. I'd be happy with 1GB/s:
raw ascii block i/o 19 blocks in 3.070000 ms at 6188.925081MB/s

I'm using memory mapped files now for speed, which helps significantly but branching is killing me in my decoding routine.

Optimizing still matters.

Watch me hit 1GB/s soon... hopefully.

The reason I'm doing this by the way, is I think I have a better high level algorithm for processing JSON. I've built a better mousetrap, I'm pretty sure.

The problem is to bench it I need to compete with simdjson at a lower level (i/o and decoding) since it's the gold standard in performance for JSON processing so far.
Real programmers use butterflies


modified 27-Dec-20 12:27pm.

GeneralRe: So there's still a use for bit twiddling Pin
pkfox27-Dec-20 2:59
professionalpkfox27-Dec-20 2:59 
GeneralRe: So there's still a use for bit twiddling Pin
honey the codewitch27-Dec-20 4:09
mvahoney the codewitch27-Dec-20 4:09 
GeneralRe: So there's still a use for bit twiddling Pin
pkfox27-Dec-20 11:12
professionalpkfox27-Dec-20 11:12 
GeneralRe: So there's still a use for bit twiddling Pin
honey the codewitch27-Dec-20 12:47
mvahoney the codewitch27-Dec-20 12:47 
GeneralScreen Capture Library Recommendations Pin
W Balboos, GHB27-Dec-20 1:40
W Balboos, GHB27-Dec-20 1:40 
GeneralRe: Screen Capture Library Recommendations Pin
#realJSOP27-Dec-20 1:43
mve#realJSOP27-Dec-20 1:43 
RantRe: Screen Capture Library Recommendations Pin
Gary R. Wheeler27-Dec-20 3:56
Gary R. Wheeler27-Dec-20 3:56 
GeneralRe: Screen Capture Library Recommendations Pin
Rick York27-Dec-20 13:48
mveRick York27-Dec-20 13:48 
GeneralRe: Screen Capture Library Recommendations Pin
W Balboos, GHB28-Dec-20 1:47
W Balboos, GHB28-Dec-20 1:47 
GeneralRe: Screen Capture Library Recommendations Pin
Maximilien27-Dec-20 2:17
Maximilien27-Dec-20 2:17 
GeneralRe: Screen Capture Library Recommendations Pin
Richard MacCutchan27-Dec-20 2:29
mveRichard MacCutchan27-Dec-20 2:29 
GeneralRe: Screen Capture Library Recommendations Pin
Slacker00727-Dec-20 3:59
professionalSlacker00727-Dec-20 3:59 
GeneralRe: Screen Capture Library Recommendations Pin
Gerry Schmitz27-Dec-20 4:57
mveGerry Schmitz27-Dec-20 4:57 
GeneralRe: Screen Capture Library Recommendations Pin
Rick York27-Dec-20 9:41
mveRick York27-Dec-20 9:41 
GeneralRe: Screen Capture Library Recommendations Pin
Marc Clifton27-Dec-20 11:22
mvaMarc Clifton27-Dec-20 11:22 
GeneralRe: Screen Capture Library Recommendations Pin
W Balboos, GHB28-Dec-20 1:42
W Balboos, GHB28-Dec-20 1:42 
GeneralRe: Screen Capture Library Recommendations Pin
#realJSOP4-Jan-21 10:30
mve#realJSOP4-Jan-21 10:30 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.