Extracting specific fields from a log file and export it to csv file using C#

Question

4.50/5 (2 votes)

See more:

This is the code that I have written in C# which shows the contains in the sample.log file which is 110MB in size. I have added the while loop which will check till end of file and print all the records, but I am not getting the first record and also not getting each record in new line.

using System;
using System.IO;
using System.Text;

    class Program
    {
        static void Main(string[] args)
        {

            FileStream fs = new FileStream("sample.log", FileMode.Open, FileAccess.Read);
            StreamReader sr = new StreamReader(fs);
            string StrFromFile = sr.ReadLine();

            StringBuilder ResultStr = new StringBuilder();

          while ((StrFromFile = sr.ReadLine()) != null)
          { 
            // your separator char seems to be char 1
            string[] SplitStrs = StrFromFile.Split(new char[] {(char)1});        
            for (int i = 0; i < SplitStrs.Length; i++)
            {   
                if (SplitStrs[i].StartsWith("52="))
                {
                    ResultStr.Append(SplitStrs[i] + " ");
                }
                else if (SplitStrs[i].StartsWith("55="))
                {
                    ResultStr.Append(SplitStrs[i] + " ");
                }
                else if (SplitStrs[i].StartsWith("132="))
                {
                    ResultStr.Append(SplitStrs[i] + " ");
                }
                else if (SplitStrs[i].StartsWith("133="))
                {
                    ResultStr.Append(SplitStrs[i] + " ");
                }
                else if (SplitStrs[i].StartsWith("35="))
                {
                    ResultStr.Append(SplitStrs[i] + " ");
                }   
            }  
         }
            Console.WriteLine(ResultStr);
            sr.Close();
            fs.Close();
            Console.ReadKey();
        }
    }

This is some data from sample.log file :
**sample.log:**

8=FIX.4.39=6335=049=SAXOQUOTE56=IDE34=457=FX52=20101219-18:06:32369=310=003
8=FIX.4.39=6135=034=449=IDE50=FX52=20101219-18:06:32.13056=SAXOQUOTE10=169
8=FIX.4.39=6335=049=SAXOQUOTE56=IDE34=557=FX52=20101219-18:07:02369=410=003
8=FIX.4.39=6135=034=549=IDE50=FX52=20101219-18:07:02.50156=SAXOQUOTE10=170

Also I am not getting why it is not showing the first record and how do I write this result to CSV file?

I am getting output like this,
35=0 52=20101219-18:06:32.130 35=0 52=20101219-18:07:02 35=0 52=20101219-18:07:02.501

but I want the output like this,
35=0 52=20101219-18:06:32.130
35=0 52=20101219-18:07:02
35=0 52=20101219-18:07:02.501

Please help me with this.

Posted 17-Feb-11 22:57pm

vpatil09

Updated 21-Feb-11 17:04pm

v2

Add a Solution

3 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

OriginalGriff · Answer 1 · 2011-02-17T23:28:00

Initially, I thought your data was fixed format, and that string.Substring would do it fairly easily. But a closer inspection shows it is already delimited: each field ends with CTRL-A.
You can just use string.Split to break the log line into it's component parts:

C#

string[] parts = logEntry.Split('\x01');

Then, all you have to do is examine each part in turn, and determine from the bit to the left of the "=" if you want to keep it or not:

C#

string logEntry = "8=FIX.4.39=6135=534=149=IDE50=FX52=20101219-18:05:01.52256=SAXOQUOTE10=171";
string[] parts = logEntry.Split('\x01');
StringBuilder sb = new StringBuilder(logEntry.Length);
string prefix = "";
foreach (string part in parts)
    {
    string[] breakdown = part.Split('=');
    switch (breakdown[0])
        {
        case "9":
        case "52":
            sb.Append(prefix + part);
            prefix = ",";
            break;
        }
    }

#realJSOP · Answer 2 · 2011-02-17T23:29:00

Just call string.Split on the separator character (I can't tell what it is), and then retrieve the strings that start with the desired string:

C#

string[] parts = logLine.Split(' ');
var fields = (from field in parts 
              where field.StartsWith("9=") || 
                    field.StartsWith("35=") || 
                    field.StartsWith("52=") 
              select field);
StringBuilder newString = new StringBuilder("");
foreach(String extracted in fields)
{
    newString.AppendFormat("{0},", extracted); 
}
newString = newString.Trim();

Finally, save the new data into a csv file.

CS2011 · Answer 3 · 2011-02-21T18:25:00

Fisrt thing first

"but I am not getting the first record"

MSIL

using System;
using System.IO;
using System.Text;

    class Program
    {
        static void Main(string[] args)
        {

            FileStream fs = new FileStream("sample.log", FileMode.Open, FileAccess.Read);
            StreamReader sr = new StreamReader(fs);
            string StrFromFile = sr.ReadLine(); Removing this line will solve ur prob
            StringBuilder ResultStr = new StringBuilder();

          while ((StrFromFile = sr.ReadLine()) != null)
         { ......

Now your second problem

U can use any of the first two answers

Extracting specific fields from a log file and export it to csv file using C#

3 solutions

Solution 1

Solution 2

Solution 3

Add your solution here

Preview 0