Parsing a string into tokens

Question

0.00/5 (No votes)

See more:

C++

Hi How can i break a file into tokens.
Is there any method to parse the input file on basis of strings ?

Posted 19-Sep-11 19:39pm

typedefcoder

Updated 23-Sep-11 7:49am

v2

Add a Solution

4 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

ShilpiP · Accepted Answer · 2011-09-19T20:16:00

Solution 3

You can use "strtok" or "strtok_s" to tokenize the string.

Posted 19-Sep-11 20:16pm

ShilpiP

Richard MacCutchan · Accepted Answer · 2011-09-23T08:39:00

Solution 4

Take some time to learn about string[^]s.

Posted 23-Sep-11 8:39am

Richard MacCutchan

tolw · Accepted Answer · 2011-09-19T20:15:00

Well you have a couple of options to choose from:

1. Use the strtok/wcstok[^] functions. You can aplay them to strings by calling c_str, but it is a bit of a pain really.

2. You can get yourself a tokenizer written for strings like this one here at CP C++ String Toolkit (StrTk) Tokenizer[^]. I just picked this one at random - haven't used it so you will have to give it a go and see if you like it. If not - you can always use the article to help you create your own solution.

3. You can use the Boost tokenizer. This is pretty close to the 2nd solution, but, you know, Boost :) Boost::Tokenizer[^]

Having said all of that I have to admit I only use the first approach. It has grown on me over the years I guess :) Best of luck to you!

KingsGambit · Accepted Answer · 2011-09-19T20:11:00

Solution 1

Hi,

Take a look at the following link on strtok http://msdn.microsoft.com/en-us/library/2c8d19sb(v=vs.71).aspx[^]

Posted 19-Sep-11 20:11pm

KingsGambit

Comments

ShilpiP 20-Sep-11 2:18am

Correct +5 !! I am late :)