|
Hi All,
I have a question regarding C# data structure.
My scenario is following:
I have a file that contains millions of today's viewing records(>2GB). What I need to do is -- for each viewing record, I need to find/search the matching viewing record from yesterday's file ( these viewing files are created daily ). Using SQL Server is just too slow for this( and I just need to find one matching record among millions ), and the files are probably too big to load into memory. What is the best approach to balance memory and speed? Any suggestion would be much appreciated.
|
|
|
|
|
I don't think you can beat SQL Server.
Alternatively, you could make your own indexing system for the files.
|
|
|
|
|
cheq326 wrote: Using SQL Server is just too slow for this
SQL is to slow for this --- for what it is designed to do?
Seriously, anything you do won't beat SQL.
I'm going to become rich when I create a device that allows me to punch people in the face over the internet.
"If an Indian asked a programming question in the forest, would it still be urgent?" - John Simmons / outlaw programmer
|
|
|
|
|
Justin Perez wrote:
Seriously, anything you do won't beat SQL.
I can think of specialized ways to outperform SQL, especially if the records aren't in SQL to begin with. The question is whether it's worth the money (and time) to do it.
Anyone who thinks he has a better idea of what's good for people than people do is a swine.
- P.J. O'Rourke
|
|
|
|
|
I'm no database guru, but isn't this exactly the kind of thing you'd want to use SQL server for?
Standards are great! Everybody should have one!
|
|
|
|
|
cheq326 wrote: Using SQL Server is just too slow for this
Have you tried defining some indexes? If the database is designed properly there shouldn't be a performance problem for this.
Loading a file with that much data is going to be very costly in terms of memory use and performance as well.
Scott.
—In just two days, tomorrow will be yesterday.
—Hey, hey, hey. Don't be mean. We don't have to be mean because, remember, no matter where you go, there you are. - Buckaroo Banzai
[ Forum Guidelines] [ Articles] [ Blog]
|
|
|
|
|
If a Relational Database is to slow for you then you better give the job to Google
led mike
|
|
|
|
|
Maybe I didnot state it clearly enough. I am looking for something like a HashTable, where I can find the record with a provided key, so that I dont have to join tables and then do a select statement to select that one record from the DB.
I personally have not tried the SQL Server method cuz I was told that it was too slow for the task(The team has tried it before). Is there any data structure out there that might fit this scenerio? or SQL Server is the only way?
|
|
|
|
|
cheq326 wrote: or SQL Server is the only way?
Of course not. It is however the best conventional method.
cheq326 wrote: I was told that it was too slow for the task(The team has tried it before)
From my experience I would want to confirm that. If they already did a research project to look at that approach you should be able to look at the project and run it etc. You may find that that they did not do something as simple as creating the appropriate index(s) to support the requirements.
Also what about the hardware, OS and Database tuning? Perhaps the machine was not appropriate to the requirements. If you have 2 gigs of data then you could use a large server (memory and processors) and possibly configure the Database to use enough physical memory to keep the records in memory at all times. I don't know squat about tuning a Relational Database and possible the OS to run one, but I would find an expert and exhaust that approach before I jumped into considering anything else.
led mike
|
|
|
|
|
Hi,
I would like to get the value of a cell in a datagrid. I tried the ff code but it gave me an error:
string value = dgOutput.CurrentCell.Value.ToString();
Please help.
Mvelo Walaza
Developer
Telkom SA
|
|
|
|
|
What was the error?
Maybe check to see if the currentcell value is null?
string value;
if(dgOutput.CurrentCell.Value == null)
{
value = "null";
}
else
{
value = dgOutput.CurrentCell.Value.ToString();
}
|
|
|
|
|
If you want to access a cell other than the currently selected
dgOutput.Rows[rowindex].Cells[columnindexORname]
Eslam Afifi
|
|
|
|
|
Not sure if this is 100% the right place but...
A year ago i released and app using clickonce and a test certificate. I have just made a change and found that the certificate had expired, so i created a new one. Now none of the users can use the app (it trys to download the new version on startup) unless i uninstall and reinstall. This is not a good option as most of our users are on remote sites
How can I get round not having to reinstall it for everybody?
|
|
|
|
|
I'm not sure. I'd try posting this question at the MSDN forum for ClickOnce.
|
|
|
|
|
Hi,
I want to open a Word(.doc) document in a Multiline textbox. Is this possible?
Thank you
Illegal Operation
Making Computer Software Talk
|
|
|
|
|
Illegal Operation wrote: I want to open a Word(.doc) document in a Multiline textbox. Is this possible?
Not directly.
You would have to write code to read the contents of the word file then transfer it into the text box.
|
|
|
|
|
You can use Word Object model to get all the text from word document but it will be just plain text. You can use this control to display doc file: Word Control for .NET[^]
|
|
|
|
|
how can i change back colors or customize save dialog box or an open dialog box using forms...
is it possible using c#...
or is there any language that supports this customizing
ramya
|
|
|
|
|
|
|
Hi Guys
I have a DatagridViewcomboBox column in DataGridView where i m using EditControlShowing on selection Changed but when i select the same combo twise EditControlShowing is not working means It is not unselecting the EditControlShowing
|
|
|
|
|
That's because you haven't change the row you are in. Thus you did not cause the SelectionChange event to fire.
|
|
|
|
|
NetworkInterface[] networkInterfaces = NetworkInterface.GetAllNetworkInterfaces();
foreach (NetworkInterface networkInterface in networkInterfaces)
{
Console.WriteLine(networkInterface.Description);
}
How can I get the IP address from each networkInterface ?
Thx...
|
|
|
|
|
You can try something like this:
foreach (NetworkInterface adapter in adapters)
{
IPInterfaceProperties props = adapter.GetIPProperties();
UnicastIPAddressInformation unicastAddress = props.UnicastAddresses.FirstOrDefault(address => address.PrefixOrigin == PrefixOrigin.Dhcp);
Console.WriteLine(unicastAddress.Address);
}
Scott.
—In just two days, tomorrow will be yesterday.
—Hey, hey, hey. Don't be mean. We don't have to be mean because, remember, no matter where you go, there you are. - Buckaroo Banzai
[ Forum Guidelines] [ Articles] [ Blog]
|
|
|
|
|
i have to create a pop-up window sliding from top to down and insert values to a table. i have completed sliding pop-up window and insertion but it is scrolling from buttom to top where i have scroll it from top to buttom
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
