|
My lexer generator generates over 8000 lines of source code (though mostly whitespace), and about 405k of source code going by raw size, for this input specification:
keyword='abstract|add|as|ascending|async|await|base|bool|break|byte|case|catch|char|checked|class|const|continue|decimal|default|delegate|descending|do|double|dynamic|else|enum|equals|explicit|extern|false|finally|fixed|float|for|foreach|get|global|goto|if|implicit|int|interface|internal|is|lock|long|namespace|new|null|object|operator|out|override|params|partial|private|protected|public|readonly|ref|remove|return|sbyte|sealed|set|short|sizeof|stackalloc|static|string|struct|switch|this|throw|true|try|typeof|uint|ulong|unchecked|unsafe|ushort|using|var|virtual|void|volatile|while|yield'
identifier='[A-Z_a-z][0-9A-Z_a-z]*'
lineComment='//[^\n]*'
blockComment<blockEnd="*/">="/*"
stringLiteral='"([^"\\]|\\.)*"'
characterLiteral='\'([^\'\\]|\\.)\''
integerLiteral = '(0x[0-9A-Fa-f]{1,16}|(0|\-?[1-9][0-9]*))([Uu][Ll]?|[Ll][Uu]?)?'
floatLiteral= '\-?(0|[1-9][0-9]*)(\.[0-9]+)?([Ee][\+\-]?[0-9]+)?[DdMm]?'
booleanLiteral = 'true|false'
nullLiteral = 'null'
whitespace<hidden>='\s+'
Most of it is initialization of a large static nested array with over 500 entries.
But it compiles down to where the entire assembly is only 67k.
I know the CIL code is relatively compact, but still this surprised me since it's initializing arrays with const values you'd think all those values would bloat the assembly size. I was expecting over 100k for this binary, not under 70k with additional source involved. Neat!
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Yawn. 
I just spent the night designing a discrete VGA card with a 80 x 60 character text mode, 320 x 240 VGA in 8, 4 and 1 bit color depth. Plus the same for a 160 x 120 resolution. 8 bit computers are fun!
I have lived with several Zen masters - all of them were cats.
His last invention was an evil Lasagna. It didn't kill anyone, and it actually tasted pretty good.
|
|
|
|
|
not bad for 8 bit, or even 16 bit, but your palette definitions must be a pain, assuming you have them.
Reminds me of the turbographics 16 - an old game console with better graphics than its little CPU really warranted.
Not to knock your project - quite the opposite. I love using things in ways they weren't really designed for.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
No palettes, just direct RGB values and direct conversion to analog via a small resistor network. 8 bit RGB is easy, just as is 1 bit. I just hope that 4 bit RGB is somewhat useful. 4 bit RGBI looks usable. It provides the obligatory screaming colors for an 8 bit computer.
honey the codewitch wrote: an old game console with better graphics than its little CPU really warranted.
That's the real problem here. If you choose too high a resolution or color depth, the poor CPU can't keep up the frame rate. The second bottleneck is pumping out the graphics data to the monitor fast enough. The direct approach quickly falls apart when the memory access time becomes more than an insignificant fraction of the length of the pixel clock cycles. But yes, even ordinary SRAMs are quite fast these days and the pixel clock is going to be at a relatively low frequency to support the very low resolutions.
I have lived with several Zen masters - all of them were cats.
His last invention was an evil Lasagna. It didn't kill anyone, and it actually tasted pretty good.
|
|
|
|
|
Got me thinking...Umm is your beastie going to use analog or digital video, analog would get around the bottle neck I think but these days you are going to have to hunt something with UHF input, there not so common these days. DVI is a pain to get licences for (vendor ID's and some nonesense), VGA is pretty much the only way to go...
|
|
|
|
|
Actually I'll probably just be ripping off a subset of C#
It will target the codeDom because I'm sick of ugly codedom building code.
So instead you'll be able to write your codeDom code in a subset of C#. I've got to figure out how to context switch or otherwise introduce dynamism into it though. Should be interesting.
Or maybe I'll just make a visitor such that you can build the tree out in the C# subset and then use the visitor to modify, trim or otherwise add to it.
If any of you use the codedom and have ideas, feel free to chime in. It doesn't have to be C# specific, that's rather the point. =)
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I, for one, I am keen on that Honey# language! :P
BTW you might want to check out starklang! 
|
|
|
|
|
What I endeavor to do is write it in a similar fashion to how C# was written, as a recursive descent parser, meaning I'm using the hand-rolled stuff from PCK but I'm including a tokenizer.
That link you gave me led me here Stark - Tokens specification and Tokenizer implementation | xoofx[^]
Which gives me a good starting point for my own lex specs used with this parser.
Basically I'm building a hand written parser, but one that uses a tokenizer/lexer
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I'm not very familiar with the codeDOM but you might want to check out the Expression Tree API. It allows building dynamic low-level code using C# code which is similar to what I think you want to do. It also takes the approach of immutable classes that represent code with various visitors to traverse and modify the tree. Might give you some ideas! 
|
|
|
|
|
That's not quite what I want as those are binary, and I'm looking to do source level integration. It also cannot be entirely immutable/stateless
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Ah! I think what you're doing is a wee bit outside of my area of experience then 
|
|
|
|
|
It's kind of the house that jack built. Currently to render my shared codebase (such as table tokenizing support) in a language independent manner I have to render the entire thing using the codedom, as expression trees won't give me source code, and can't define whole classes.
Here's what I'm doing now:
var result = CD.Struct("Token", false,
CD.Field(typeof(int), "Line", MemberAttributes.Public),
CD.Field(typeof(int), "Column", MemberAttributes.Public),
CD.Field(typeof(long), "Position", MemberAttributes.Public),
CD.Field(typeof(int), "SymbolId", MemberAttributes.Public),
CD.Field(typeof(string), "Value", MemberAttributes.Public));
result.CustomAttributes.Add(_GeneratedCodeAttribute);
return result;
in order to declare this:
public struct Token
{
public int Line;
public int Column;
public long Position;
public int SymbolId;
public string Value;
}
And never mind methods like this:
result.Statements.AddRange(new CodeStatement[] {
CD.If(CD.Invoke(input,"MoveNext"),
CD.IfElse(CD.NotEq(state,CD.Literal(_BeforeBegin)), new CodeStatement[] {
CD.Let(position,CD.Add(position,CD.One)),
CD.IfElse(CD.Eq(CD.Literal('\n'),current),new CodeStatement[] {
CD.Let(column,CD.One),
CD.Let(line,CD.Add(line,CD.One))
},
CD.IfElse(CD.Eq(CD.Literal('\t'),current),new CodeStatement[]
{
CD.Let(column,CD.Add(column,CD.Literal(_TabWidth)))
},
CD.Let(column,CD.Add(column,CD.One))))
},
CD.IfElse(CD.Eq(CD.Literal('\n'),current),new CodeStatement[] {
CD.Let(column,CD.One),
CD.Let(line,CD.Add(line,CD.One))
},
CD.If(CD.Eq(CD.Literal('\t'),current),
CD.Let(column,CD.Add(column,CD.Literal(_TabWidth-1))))
)
),
CD.Return(CD.True)),
CD.Let(state,CD.Literal(_InnerFinished)),
CD.Return(CD.False)
});
return result;
to declare this:
bool _MoveNextInput()
{
if (_input.MoveNext())
{
if (_BeforeBegin != _state)
{
++_position;
if ('\n' == _input.Current)
{
_column = 1;
++_line;
}
else if ('\t' == _input.Current)
_column += _TabWidth;
else
++_column;
} else
{
if ('\n' == _input.Current)
{
_column = 1;
++_line;
}
else if ('\t' == _input.Current)
_column += _TabWidth-1;
}
return true;
}
_state = _InnerFinished;
return false;
}
It's ridiculous, and I'd like to move away from it toward something more natural.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Oof. From working with expression trees, I do know that static examples like that are usually the simplest, too. I can only imagine the monstrosity you'd have to write to support any method or struct 
|
|
|
|
|
That's already abbreviated immensely.
normally you have to do like
var state = new CodeFieldReferenceExpression(new CodeThisReferenceExpression(),"_state");
var letStateOne = new CodeAssignStatement(state,new CodePrimitiveExpression(1));
just to render
this._state=1;
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Quote: So I've decided to make my own programming language
I was pretty sure about that. 
Good luck!
|
|
|
|
|
|
and I'm using Rolex for my lexing. Got it set up as a pre-build event in my new project and everything.
You baited me. I blame you.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
|
So obvious!  
|
|
|
|
|
|
=/
It was either take it up again or stay home for thanksgiving as I wouldn't have been good company. So I smoked.
My hubby is disappointed. He'll survive.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the codewitch wrote: He'll survive.
A bit longer. 
|
|
|
|
|
Then you weren't ready yet - and you have to have your mind on side with this: if you aren't ready to give up, you won't. You'll find reasons - any reason - to give up giving up and go back to it.
So don't get discouraged: try again later. Perhaps join a support group - most US states have a helpline that can get you in touch with free ones. Just knowing you aren't the only one going through it can help.
But for ![Elephant | [mastadon]](https://codeproject.global.ssl.fastly.net/script/Forums/Images/elephant.gif) 's sake: don't give up as a new year resolution. Those rarely last the week ...
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
#trueFacts
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I'm quite disappointed as well 
But [insert encouraging words here].
So I just know you can do it 
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
