|
Thanks Christian!
My database is over 20 years old and grows by about 512K every day.
I don't know anything about SQL except that I run circles around people who do use it.
My compiler is set to "any CPU" on my 64 bit machine. I understand all that stuff.
What I don't understand AT ALL is the inability to read/write a struct/class directly to disk without a whole lot of overhead involved.
I only get paid if I am the first one with the answer, therefore speed is critical.
From my perspective, this whole DOT NET thing is a throwback to the old "interpeter" days with this JIT thing.
I don't understand why the JIT compiler "chokes" (only at 64 bit, not at 32) the first time a large function is called.
I can write a native console app to retrive the data and write a small temp file that a C# app can then read.
The problem with that is the original 32 bit app is still faster and there's every indication that the native app is running @ 32 bits.
It's also likely that I've got my head screwed on backwards and am missing something simple.
thanx
bigchump
|
|
|
|
|
bigchump wrote: My database is over 20 years old and grows by about 512K every day.
Oh, a custom format ?
bigchump wrote: I don't know anything about SQL except that I run circles around people who do use it.
Well, if you're not using SQL, you're using some specialised code written for your purpose, no surprise it may work faster than a more generalised approach ( but still impressive )
bigchump wrote: What I don't understand AT ALL is the inability to read/write a struct/class directly to disk without a whole lot of overhead involved.
Well, did C++ just write a class for you ? You can store a class the same as you did in C++, by writing code to stream the component parts, if you like. Serialisation actually works mostly for free, if you're only using simple types that know how to serialise themselves.
bigchump wrote: I don't understand why the JIT compiler "chokes" (only at 64 bit, not at 32) the first time a large function is called.
Because it's compiling it. Why is your function so large ? What do you mean by 'chokes' ?
bigchump wrote: It's also likely that I've got my head screwed on backwards and am missing something simple.
I suspect that I'm not fully understanding you just yet
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
Christian Graus wrote: bigchump wrote:
My database is over 20 years old and grows by about 512K every day.
Oh, a custom format ?
Yes, a relatively small file on HD is indexed to database on tape drives to find records that match.
bigchump wrote:
I don't know anything about SQL except that I run circles around people who do use it.
Well, if you're not using SQL, you're using some specialised code written for your purpose, no surprise it may work faster than a more generalised approach ( but still impressive )
There's no comparison to SQL. I wrote all the code many years ago before SQL even existed.
bigchump wrote:
What I don't understand AT ALL is the inability to read/write a struct/class directly to disk without a whole lot of overhead involved.
Well, did C++ just write a class for you ? You can store a class the same as you did in C++, by writing code to stream the component parts, if you like. Serialisation actually works mostly for free, if you're only using simple types that know how to serialise themselves.
IF I UNDERSTAND YOU, COMPONENT PARTS MEANS WRITING ONE ITEM AT A TIME TO A BINARY FILE.
The problem with that is one of these classes has over 1500 different items.
That would mean writing 1500 lines of code to write and another 1500 to read????
bigchump wrote:
I don't understand why the JIT compiler "chokes" (only at 64 bit, not at 32) the first time a large function is called.
Because it's compiling it. Why is your function so large ? What do you mean by 'chokes' ?
I tried breaking it down into several smaller functions. It just ran slower. "Chokes" means serious delay.
Why does it choke at only 64 bits and not at 32??????
bigchump wrote:
It's also likely that I've got my head screwed on backwards and am missing something simple.
I suspect that I'm not fully understanding you just yet
That's because I'm not a programmer by profession. It took me a while to figure out what "component parts" meant.
|
|
|
|
|
yes, in C++ you would write your own code that uses a stream to store the different properties you need to restore a class, and you'd write code to read the values back out and assign them to properties in a new class instance.
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
bigchump wrote: From my perspective, this whole DOT NET thing is a throwback to the old "interpeter" days with this JIT thing.
CSharp is an Imperative language not Interpreted. Here[^] in an explanation.
bigchump wrote: I don't understand why the JIT compiler "chokes" (only at 64 bit, not at 32) the first time a large function is called.
Because the machine code doesn't exist the first time a large function is called. It lives as IL code and has to be JIT'ed into machine code. If you've come from C programming and did a port, then you probably kept the nuances of C which don't work so well in C#. I think in order to pinpoint your performance issues, you may want to consider a profiler[^]. It'll be nigh on impossible to tell where you're going wrong here.
bigchump wrote: What I don't understand AT ALL is the inability to read/write a struct/class directly to disk without a whole lot of overhead involved.
Have a look at this[^].
Scott P
“It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.”
-Edsger Dijkstra
|
|
|
|
|
Hi Carbon:
"It'll be nigh on impossible to tell where you're going wrong here."
Going wrong? I don't think so. My 32 bit app beats everything else out there.
The problem is it's written in c/c++ with some assy. that doesn't compile with VS8.
So I re-wrote the entire thing in C#. Now I've got a 64 bit dog. That's all I know.
I never liked c++ and hate to use it but I'm becoming desperate and need some advice in the King's ENGLISH I can understand.
I read that thing ("have a look at this") and hardly understood a word of it.
My app is really simple. Read a few struct/class from disk and run about 3000 "if" statements on it and come up with an answer.
thanx for any help
bigchump
|
|
|
|
|
<quote>Going wrong? I don't think so. My 32 bit app beats everything else out there.
Well obviously something is going wrong. If you 3000k if statement wonder program can run circles around everything else ever written by mankind, why don't you just use your old VS version? I don't mean to be hostile, but it seems like you're being hostile to everyone that's trying to help you. You're going to have to learn correct terminology if you're going to a) get your point across and b) understand what people are trying to tell you.
Perhaps the sound of your own awesome is distracting you?
|
|
|
|
|
Hey Tom!
Sure my 32 bit stuff is fine. Just like a Model T Ford. Sooner or later I will get beaten if I don't keep up.
By the way, it ain't a "wonder program". It's the result of many years of very hard work.
I read the King's English just fine but words like "extensibility" and "com object" are just jargon to me...
If I were to say "Quitters stretching out with a big ML should be included in your exotics" is my jargon and may not mean a thing to you.
The difference is -- I know when to use my jargon and when not to.
When the title of the post has the word "amateur" in it - it might mean that someone doesn't understand the jargon.
Of course, it is my choice whether to continue with C# or not. I was hoping for some help. I didn't get it.
All I did was get chastised.
Not one person explained to me why a 27mb file should become over 400mb. Not one.
Oh, if you're wondering about my "wonder program" you can find it on TVG.COM under "TVG PICKS\Free Handicapping Info".
So, C# and Bill Gates can go to the same place.
I'll re-write in c++ much as I hate it.
bigchump
|
|
|
|
|
bigchump wrote:
Going wrong? I don't think so.
Going wrong with the C#. Nobody is disputing the performance of your other program. Sheesh.
bigchump wrote: I never liked c++ and hate to use it but I'm becoming desperate and need some advice in the King's ENGLISH I can understand.
Granted the link I sent you is a mildly technical post, but I think it's the direction you should take. I suggest reading through it again and try to make some sense of it. I don't think there's an easier answer with the performance you want. I appreciate your plight here, but you'll have to take the bull by the horns and make a better attempt at learning how to use the language. If you don't have reference material, I highly suggest getting CLR via C# by Jeffrey Richter, and Programming Visual C# 2005: The Language by Donis Marshall.
If you posted the code that's loading the data from the binary file, it may be useful.
Scott P
“It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.”
-Edsger Dijkstra
|
|
|
|
|
Hi Carbon:
"Going wrong with the C#. Nobody is disputing the performance of your other program. Sheesh."
Brother, I think you're missing the point. When should a 32 bit app outperform a 64????????
Somethings gotta be wrong somewhere. Maybe my head's screwed on sideways. That's a possibility I don't discount.
I PORTED ALL MY APPS TO C# AND ONLY ONE IS CAUSING THIS MISERY.
When I say all, I mean over 14 different apps and only one doesn't work properly.
Even the one with problems works OK just EXTREMELY SLOW. But slow is a killer in my business.
What I need is a way to read/write structs/classes to file without the MAJOR OVERHEAD.
sorry if I sounded hostile - I just need some help without any wisecracks..
bigchump
|
|
|
|
|
"If you posted the code that's loading the data from the binary file, it may be useful."
Here's the code but the problem is not loading data but the SIZE of the file that is created.
public void DeSerializeNow()
{
header c1=new header(); //change to header - horse - results
horse[] c2=new horse[29];
Stream s=File.Open(@"C:\baduga\manure.tmp",FileMode.Open);
BinaryFormatter b=new BinaryFormatter();
while(true)
{
try
{
c1=(header)b.Deserialize(s);
label1.Text+=c1.track+" "+c1.poly+" "+c1.entries+" "+c1.par2f+" "+c1.par4f+" "+c1.par6f+" "+c1.parspeed+"\n";
for(int y=0;y<c1.entries;y++)
{
c2[y]=(horse)b.Deserialize(s);
label1.Text+=c2[y].post+",";
}
label1.Text+="\n";
}
catch
{
s.Close();
break;
}
}
}
Note the label stuff is just for debugging - it ain't used in real life...
thanx
bigchump
|
|
|
|
|
bigchump wrote: run about 3000 "if" statements
Are they all in one function? Could you post a code snippet with 10-20 of these statements?
|
|
|
|
|
If you're not an expert after 20 years of doing it, you should hire someone.
"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001
|
|
|
|
|
OK John,
If you can make my 64bit C# app run faster than my c++ 32 bit one you can write your own ticket.
LITERALLY--
bichump
|
|
|
|
|
Hi,
I've read the rest of this thread, and hopefully I understand what your problem is. As far as I can tell, your main problem is the size of the file created by your program. (Please correct me if I am wrong)
Firstly, I suspect you are using XML serialization, this will likely explain the size. Can you post the code you use to create the file, and a small sample (just a few lines) from an output file so I can take a look at what's going on.
If my guess is correct, and you are using XML serialization, you might consider using binary serialization instead. This will almost certainly drastically reduce the size of the saved file. If you're not sure how to do this, post your code and I'll point out what needs to be changed.
Simon
Simon
|
|
|
|
|
Hi Simon!
Yes, the file size is at least 16x larger than it should be!
I am using binary serialization as far as I know.
I already tried using char arrays (like the original app) instead of strings.
That doesn't help at all.
Here's the start of the code that writes the first small class:
public void SerializeNow()
{
header c=new header();
Stream s=File.Open(@"C:\baduga\manure.tmp",FileMode.Append);
BinaryFormatter b=new BinaryFormatter();
FileStream hread=new FileStream(@"c:\baduga\races.txt",FileMode.Open); //read races.txt
StreamReader hread1=new StreamReader(hread);
string temp;
string[] t=new string[1435];
int a=0,x=0,y;
while((temp=hread1.ReadLine())!=null)
{
if(temp.StartsWith("JJGENDRACE"))
{
c.entries=a;
b.Serialize(s,c);
for(y=0;y<c.entries;y++)>
{
b.Serialize(s,h[y]);
}
a=x=0;
}
else
{
t[x]=temp;
if(t[x]=="") t[x]="0";
x++;
if(x==1435)
{
if(a==0)
{
c.track=t[0];
c.date=t[1];
c.race=t[2];
c.distance=int.Parse(t[5].Trim('-'));
c.distanceD=c.distance/220;
c.surface=t[6];
c.type=t[8];
c.sex=t[9];
c.formtype=t[10];
c.purse=int.Parse(t[11]);
c.purseD=c.purse/5000;
if(c.purseD>50) c.purseD=50;
c.claimprice=int.Parse(t[12]);
c.trackrecord=float.Parse(t[14]);
c.simulT=t[20];
c.simulR=t[21];
c.breed=t[22];
c.poly=t[24];
c.par2f=int.Parse(t[213]);
c.par4f=int.Parse(t[214]);
c.par6f=int.Parse(t[215]);
c.parspeed=int.Parse(t[216]);
c.parLf=int.Parse(t[217]);
}
|
|
|
|
|
Check that your problem is not with your code logic. Step through carefully and check that you aren't accidently serializing more than you meant too. (Like the same object twice).
If you switch to xml serialization you can look at the data and check that only the correct stuff is there. Then once your happy with the data being outputed, switch back to binary again.
Comments on the code (Numbers indicate line in code I'm talking about):
1) The for loop here is incomplete. I'm assuming you just want to loop through all objects in the array h, buy maybe more code is missing.
2) the variable a is never assigned a value other than 0. What is it's purpose?
3) For every line read in that starts with JJGENDRACE you are serializing the whole of the array h. I don't know whats in this array as you haven't included it's defination or population code, but why are you serializing the same array every time? This will be a huge waste of space if you are just serializing the same data over and over again.
General comment: The variable names c, s, b, t, a, x and y are not very useful. If you used more descriptive veriable names that would help other people to read your code.
public void SerializeNow()
{
header c = new header();
Stream s = File.Open(@"C:\baduga\manure.tmp", FileMode.Append);
BinaryFormatter b = new BinaryFormatter();
FileStream hread = new FileStream(@"c:\baduga\races.txt", FileMode.Open);
StreamReader hread1 = new StreamReader(hread);
string temp;
string[] t = new string[1435];
int a = 0, x = 0, y;
while ((temp = hread1.ReadLine()) != null)
{
if (temp.StartsWith("JJGENDRACE"))
{
2) c.entries = a;
b.Serialize(s, c);
1) for (y = 0; y
{
3) b.Serialize(s, h[y]);
}
a = x = 0;
}
else
{
t[x] = temp;
if (t[x] == "")
t[x] = "0";
x++;
if (x == 1435)
{
if (a == 0)
{
c.track = t[0];
c.date = t[1];
c.race = t[2];
c.distance = int.Parse(t[5].Trim('-'));
c.distanceD = c.distance / 220;
c.surface = t[6];
c.type = t[8];
c.sex = t[9];
c.formtype = t[10];
c.purse = int.Parse(t[11]);
c.purseD = c.purse / 5000;
if (c.purseD > 50)
c.purseD = 50;
c.claimprice = int.Parse(t[12]);
c.trackrecord = float.Parse(t[14]);
c.simulT = t[20];
c.simulR = t[21];
c.breed = t[22];
c.poly = t[24];
c.par2f = int.Parse(t[213]);
c.par4f = int.Parse(t[214]);
c.par6f = int.Parse(t[215]);
c.parspeed = int.Parse(t[216]);
c.parLf = int.Parse(t[217]);
}
}
}
}
}
Simon
|
|
|
|
|
For some reason it seems that I've got difficulties to understand the logic of the web service:
I need that a webpage or an application on client side will validate user's details (username, password). Validation is through db that seats on a server application. I want that the procedure will be:
calling a validation method through web service that will use the server's db. I understand how to connect the client or the website to the web service but don't understand how the web service connects to the server to use his db (or to call a function on the server that will do that).
Can someone help please?
thanks
y
|
|
|
|
|
IF it's a webpage, use AJAX, you can't call a webservice from the client. Unless you want to do it from the server, of course.
The webservice uses a standard mechanism to make requests via HTTP. It's all hidden from you, that's kind of the point.
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
thanks Christian,
but to sharpen my misunderstanding :
if i want to use a function that on my server with an application that seats in the client computer :
1.i need a webrefference to the webservice in my client's application to create an instance of it right?
2. the webservice shld have a method that invokes method on the server
so how do i connect to the method on the server with the webservice (using a dll?)
sorry if i'm all upside down ...
|
|
|
|
|
johland wrote: need a webrefference to the webservice in my client's application to create an instance of it right?
Correct.
johland wrote: the webservice shld have a method that invokes method on the server
Well, every method that is marked as such in the webservice, invokes a method on the server. Automatically. .NET hides those details from you, so you call the method and create class instances as if they were local, and the framework handles the job of making calls to the webservice and getting the result. The only difference is you need to anticipate exceptions that can occur if the framework cannot find the webservice to call.
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
If you don't like my answer, ask more questions so I understand where the problem is.
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
i think my first problem is phrasing 
do these steps are right?
1.i want to use a method that seats on my server with another application
2.so i'm creating a webservice that call that method
this webservice will be part of the server solution so it will have access to the server db.
3.i want the client app to use this webservice so i add webreference to the client app and create instance of it and use its methods.
did i get it?
.
|
|
|
|
|
I was commenting on how my response got a 1/bad answer vote. If it wasn't you, then fair enough, but it seemed likely to me that it was.
Yes, you got it right. The framework handles all the messy stuff for you.
Christian Graus
Please read this if you don't understand the answer I've given you
"also I don't think "TranslateOneToTwoBillion OneHundredAndFortySevenMillion FourHundredAndEightyThreeThousand SixHundredAndFortySeven()" is a very good choice for a function name" - SpacixOne ( offering help to someone who really needed it ) ( spaces added for the benefit of people running at < 1280x1024 )
|
|
|
|
|
sorry...didnt even noticed this feature exsits
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
