The Lounge - CodeProject

It's not something I would want to tackle. I suppose that handling the basics like you show could be doable, but things like subqueries and CTEs would be problematic.

Do you think you'll need to support things like
SELECT X + Y AS Z ... ?

And which flavor of SQL? PL/SQL, T-SQL, etc.

Probably something that I would enjoy doing but not sure I would attempt to say it was "small".

I agree with your examples.

Not to mention is the SQL only going to come from SQL scripts? Or also from other code bases.

I found these parsers written in C# on GitHub:
sqlparser · GitHub Topics · GitHub[^]

thanks for the link. I prefer SQL datatable one.

diligent hands rule....

A good example no, sorry.

But next question I recommend you to write it in the proper section Smile | :)

M.D.V.

If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.

I just look for some thoughts here, not a specific language implementation.
I may need to implement my request by other scripting languages such as SAS.

diligent hands rule....

Sorry, for me looks like a question. Not a question about code, but a still a programming question.

M.D.V.

If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about?
Help me to understand what I'm saying, and I'll explain it better to you
Rating helpful answers is nice, but saying thanks can be even nicer.

Southmountain wrote:
I may need to implement my request by other scripting languages such as SAS.

Huh?

You want to try to implement a SQL Parser using 'SAS' (whatever that is)?

Seems like a bad idea to me.

I have written interpreters. And one compiler.
I have been using Perl for 40 years and I have parsed SQL using that before. Multiple times.
I have also taken a class on compiler theory so I do know what goes into a parser.

And in my estimation with your general description you are setting yourself up for failure. Or for writing a lot more code than you expect.

Thing that can impact the actual project
1. How structured is the SQL? For example if this is dumped from a SQL database tool the form is exact, but if it is human created files then the form will be all over the place.
2. How much SQL exists? 1,000 lines? 10,000? 1 million? The more lines the more likely variations will exist.
3. How often will this need to happen. Every day? Every day for each customer? Only one time and never again?

Consider the following case
1. You are parsing some SQL script files. By that I mean that there is nothing in those files except SQL.
2. Those script files are in use. So you know absolutely that the SQL is valid.
3. This is one time project
4. The number of files are low (total number of lines are probably less than 10,000)
6. You can add a manual step in the process to deal with the more complex examples.

Given the above, and that I really know how to do the above in Perl, this would be a fairly easy project for me.

I however would NOT attempt to do it in Python. Because I have never used Python.
I would not choose to do it in C# nor Java even though I know those languages very well because I know that Perl is going to be better at doing the real work that is needed.

If the project was much bigger than what I suggested above then I would look at making a real parser. Probably in C# or Java. C++ is an option but since it has been so long since I used that I would prefer one of the others. I would do that even though I has been a long time since I made a real parser (versus just munging a solution.) But that is going to become a problem if I could not find an actual BNF type description of the SQL being targeted. There is no way I would attempt to write my own BNF for any of the major databases without vastly increasing the timeline for the work. (And I have written a BNF before and read others.)

Note also that if there are different types of SQL then one needs a parser for each regardless of how one implements it.

Quote:
You are parsing some SQL script files. By that I mean that 1. there is nothing in those files except SQL.
2. Those script files are in use. So you know absolutely that the SQL is valid.
3. This is one time project
4. The number of files are low (total number of lines are probably less than 10,000)
6. You can add a manual step in the process to deal with the more complex examples.

yes, my case meets all of above points. thanks for your info! Rose | [Rose]

most cases, my SQL line count will be less than 500.

I figured out that my path will be: table driven LL(1) syntax directed parsing.
SAS can do table easily. also I figured out how to implement stack and queue today.
So it is very hopeful to me.

diligent hands rule....

Southmountain wrote:
most cases, my SQL line count will be less than 500.

Note that I said total line count. The more sources there are the more chance for variability which will break any hack.

Southmountain wrote:
table driven LL(1) syntax directed parsing.

To do formal parsing you need a formal SQL language definition.

Also been a while but seems that I recall that these days languages might require adjustments (manual) in the parser code to deal with variations that formal language definitions were never designed to describe. But I might not be recalling that correctly. So although the formal spec for SQL itself is fairly simple you are unlikely to be dealing with SQL but rather PL/SQL, TSQL, etc.

My case is Teradata SQL.

thanks for the great info! Rose | [Rose]

diligent hands rule....

Southmountain wrote:
Teradata SQL.

I could not find a BNF for that.

In contrast and for comparison it was rather easy to find one for TSQL.

I think my need is not a full bloom SQL parser, I just need to get all field names. So I plan to manually craft a parser to fit my situation.

diligent hands rule....

#Worldle #602 3/6 (100%)
🟩🟩🟩🟨⬜⬅️
🟩🟩🟩🟩⬜⬇️
🟩🟩🟩🟩🟩🎉
https://worldle.teuteuf.fr

"A little time, a little trouble, your better day"
Badfinger

Wordle 819 4/6

🟩⬛⬛🟨🟨
🟩🟩⬛🟩⬛
🟩🟩🟩🟩⬛
🟩🟩🟩🟩🟩

Wordle 819 4/6

🟩⬜⬜🟩⬜
🟩⬜⬜🟩⬜
🟩🟨⬜🟩⬜
🟩🟩🟩🟩🟩

Wordle 819 2/6

⬜🟨🟨⬜🟨
🟩🟩🟩🟩🟩

Wordle 819 2/6*

🟩⬜⬜⬜🟨
🟩🟩🟩🟩🟩

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

Wordle 819 4/6

🟩⬜⬜⬜⬜
🟩🟨⬜🟩⬜
🟩⬜🟨🟩⬜
🟩🟩🟩🟩🟩

“That which can be asserted without evidence, can be dismissed without evidence.”

― Christopher Hitchens

Wordle 819 5/6

⬜⬜🟨🟨🟨
⬜⬜⬜⬜⬜
🟨🟨🟨🟨🟨
🟩🟩🟩🟨🟨
🟩🟩🟩🟩🟩

Wow I've never had all 5 correct letters and not got the word before

In a closed society where everybody's guilty, the only crime is getting caught. In a world of thieves, the only final sin is stupidity. - Hunter S Thompson - RIP

Wordle 819 5/6

⬜⬜⬜🟨⬜
⬜⬜🟨⬜⬜
⬜🟨🟨⬜🟨
🟩🟩⬜⬜🟩
🟩🟩🟩🟩🟩

Best,
Sander

Azure DevOps Succinctly (free eBook)
Azure Serverless Succinctly (free eBook)
Migrating Apps to the Cloud with Azure
arrgh.js - Bringing LINQ to JavaScript

Wordle 819 3/6

🟩⬛⬛⬛🟨
🟩🟨🟨🟨⬛
🟩🟩🟩🟩🟩

Ok, I have had my coffee, so you can all come out now!

Wordle 819 3/6

⬛⬛🟨🟨🟨
⬛🟨🟨🟩⬛
🟩🟩🟩🟩🟩

Latest CodeProject post:
Quick look into Machine Learning workflow
How to solve Word Ladder Problem?

To read all my blog posts, visit: Learn by Insight...

Wordle 819 4/6

⬜🟨🟨⬜⬜
⬜⬜🟨🟨⬜
🟩🟩⬜🟨🟨
🟩🟩🟩🟩🟩

"A little time, a little trouble, your better day"
Badfinger

Is Curiosity avoiding those rocks on Mars on its own or is it getting directions from Mission Control on Earth?
Atlas from Boston Dynamics scores high on dexterity ( it can jump while running and do other operations that require body dexterity ) but there is nothing displayed about operations that require planning. Things like picking something in one room and leaving it in another or navigating through obstacles (for example all sorts of stuff laying on the ground)

Welcome to the Lounge

“That which can be asserted without evidence, can be dismissed without evidence.”