|
Hi, I think it is not wise to have log files that large.
Why not create a series of normal sized log files instead ?
Just start a new file each day/each hour/whatever is appropriate.
Simply include the date/time in the file name to keep them apart.
You can keep them all in one folder, and if you need to transfer them
as a single entity, a Zip utility would take care of that
(as well as reducing overall size for you).
BTW Notepad is probably not the optimal answer to your question.
Luc Pattyn
|
|
|
|
|
Thanks for the advice but that presents some problems of its own because there are other tools that have to make use of these logs.
Most of the time I will be processing 3 log files at a time of around 900Mb each. Needless to say it takes a while... You should see how long it takes to FTP them...every Monday.
I could break the logs up into smaller files, but I still end up having to process the entire volume as a whole anyway, as well as change the setup of the other tools. So I would like to avoid this if I can.
David Wilkes
|
|
|
|
|
Well, you could surely improve things:
1) before file transfer, try compression; again a ZIP utility is useful, even for a single file.
On text files it would reduce size by a factor of about 3 to 5.
2) if you can modify the app that logs, you could leave everything as is, but add something
that creates another file containing exactly (or approx) what you are really interested in.
3) I dont know what the underlying business logic is, but requiring that amount of text
to be collected, transfered, and analyzed seems very strange to me. I would say
the overall process deserves reconsidering.
Luc Pattyn
|
|
|
|
|
This is a system that I sometimes think that Rube Goldberg designed.
We are talking about a Cellular Phone system running on a UNIX platform (messing around on it is not an option). The available tools and interfaces are archaic at best, and my knowledge of UNIX…well lets say I know just enough to get into trouble. The volume of data while unfortunate is still only 1/3rd of what is actually collected, and I’ve reduce about as far as I can while still maintaining its integrity and validity.
So, does it suck? Sure. But one of the reasons I like this job is that it is always a challenge…and this is today’s…tomorrow? Who knows, maybe I’ll have to solve Cold Fusion.
David Wilkes
|
|
|
|
|
I've run large log files for new processes, full throttle on Debug level. Also, unknown or old code may have large log files. Size is not the problem here.
What I've done is read the articles on making an xml file or database (after xml) of a logfile. There are good articles on using .NET regex to parse a file and put it into xml.
An overview article, a bit light on details, is
http://msdn2.microsoft.com/en-us/library/ms972965.aspx
This way you can rationalize the files, normalize them to a dB, then really look at the contents.
Good luck.
|
|
|
|
|
Finally got a chance to look at the link you provided.
This looks like it will work. I never expected I’d be able to simply locate the desired data based upon its pattern, but I was able to write a RegEx using Expresso that does it (at least on a small sample log).
Now all I need to do is code it and see if it will work for the big files.
Thanks!
David Wilkes
|
|
|
|
|
Also, rereading your original post, you need some grammatical structure here. Read the articles on Yacc, or for .NET anything that is a Yacc-like parser. The article on
http://www.codeproject.com/csharp/minossecc.asp
seems helpful. Search around, I can't put my finger on it but there are other non-Java .NET parsing meta-languages to help with the file structure.
There's always MKS Lex and Yacc -- I've used a long time ago to great effect, not too hard to learn (days). But, find something free if possible.
Cheers.
|
|
|
|
|
I'm overriding an Equals() method in .NET which a type specialization in mind but don't want to kill my code in stupid ways. So:
public override bool Equals(object obj)
{
if ( obj.GetType() == typeof(double) )
// foo
else
return base.Equals(obj)
}
The line to optimize is "if ( obj.GetType() == typeof(double) )". The constant typeof(double) calls is a waste. Tried "if ( obj.GetType().Name == "System.Double" )" with horrible results.
Any optimizations out there? Thx!
|
|
|
|
|
You could store the result of this call to a variable that's used by the comparison.
Christian Graus - C++ MVP
'Why don't we jump on a fad that hasn't already been widely discredited ?' - Dilbert
|
|
|
|
|
if (obj is double) ... else ...
Luc Pattyn
|
|
|
|
|
if (obj is double) wins!
Gives me 2x speed over obj.GetType() == typeof(double).
Thanks!
|
|
|
|
|
As far as performance goes, there are only two rules:
1) shorter code is always faster
2) rule 1 is false
Luc Pattyn
|
|
|
|
|
I didn't think that 'is' would work for a value type ? Or is that just 'as' that won't work ( as it can't return null ) ???
Christian Graus - C++ MVP
'Why don't we jump on a fad that hasn't already been widely discredited ?' - Dilbert
|
|
|
|
|
Sure is is always OK (it produces a bool) whereas as is not.
Luc Pattyn
|
|
|
|
|
Hi, I want to replace any 2 words if found in a line. The 2 words can be on the same line or one on each line.
string line ="hello World, expression";
Regex regx = new Regex("hello"); //how can i optionally put to search either, "hello" and "expression" in a line
if (regx.IsMatch(line))
{
sb.AppendLine(regx.Replace(line, "replaced"));//if "hello" put "H" or if "expression" put "E"
}
Thanks
|
|
|
|
|
You can use the replace method on the string class, you don't need regex for this. There is a RegEx.Replace if you prefer.
Christian Graus - C++ MVP
'Why don't we jump on a fad that hasn't already been widely discredited ?' - Dilbert
|
|
|
|
|
Hi again!
Last Problem get solved very fast - thx to Luc! This time, i had another problem with printing...
How do i get the last page (number of page) that will be printed at PrintPage - Event?
i want something like that:
Page 1 from 9 (where 9 is last page that will be printed)
i want to print Page x from x on any page... at footer line - anyone does know an solution for?
i did something like that
private int LinesThatFit(PrintPageEventArgs e, PointF curPosition)
{
int CalcSum = (e.MarginBounds.Bottom - (int)curPosition.Y);
int CalcRows = CalcSum / newFont.Height;
return CalcRows;
}
this works - but only for a static height - or better a static font height - i draw a string by DrawString - but if it's wrapped, the LinesThatFit doesn't work correct...
Thx again for any solution - i searched for this problem around 2 days... and found nothing - the LinesThatFit is an own creation... but that was when i just had line for line with none wrapped text!
Hope someone get's it!
Bye!
FreewareFire is in the House!
|
|
|
|
|
put a counter if you are calling your print function last time then use if condition and then
e.Drawstring(.....);
|
|
|
|
|
hi i whant to parse the following code
<br />
<!body><br />
some line<br />
more lines<br />
<!/body><br />
<code><br />
<br />
the following doesnt match<br />
<pre>HtmlSource is the hole content<br />
part is "body"<br />
</pre><br />
<code><br />
string parse = Regex.Match(this.HtmlSource, "<!" + part + ">(.*)<!/" + part + ">").Value;<br />
what does i make wrong ?
greetings gigo
|
|
|
|
|
|
ok the problem is that the query cant run multinline / one option will be to replace all linefeeds but this sucks.
has somebody any idea how can i match over more lines ?
|
|
|
|
|
If you use SingleLine mode, the regex works. Just tested in Expresso
Christian Graus - C++ MVP
'Why don't we jump on a fad that hasn't already been widely discredited ?' - Dilbert
|
|
|
|
|
Is it pssoble to use a VS.net wizard at runtim?
For example I want to use "TableAdapter Configuration Wizard" at runtime to generate Dataset.
|
|
|
|
|
No, it's part of VS.NET, not part of the runtime.
Christian Graus - C++ MVP
'Why don't we jump on a fad that hasn't already been widely discredited ?' - Dilbert
|
|
|
|
|
Hello,
I have a listview on my form, but I do not want it to respond to keyboard input as I want to use the keys for something else.
Currently, when the listview is populated with items and I press a key (say, the 'm' key) the listview will change selections to the first item that starts with an M. I would like it not to do that please.
I am guessing this is where I can use 'protected override' (or something like that) however I am new to C# and I haven't grasped that yet.
All help is most appreciated. Thank you
... Andy
photocafe.ca
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
