|
Hi
Iam parsing very large text files-around 50MB each using split function. I populate a dataset from it and then use sqlBulkCopy to insert the records in the datbase. But im getting a low virtual memory error and it is taking a lot of time? Any other approaches-help?
Amna
|
|
|
|
|
Have you tried using a DTS job to do the same thing ?
Blog Have I http:\\www.frankkerrigan.com
|
|
|
|
|
No i havent. But if it is more efficient i would surely try it. Thanx
|
|
|
|
|
iam using sql server 2005 express edition. It does not have the DTS option. Any other suggestions?
|
|
|
|
|
Look at BULK INSERT in the SQL help file. If your file is in a compatible format, then this will probably be your fastest option. Otherwise, just send it one record at a time using a SQLCommand object; a stored procedure will be slightly faster than sending a bunch of INSERT statements. The DataSet object is probably taking up a lot of your memory. I routinely load text files of up to about 1GB one record at a time. I have a bunch of validation for each record, so it is very slow, but even with the validations, it loads at about 1000 records per second.
|
|
|
|
|
Sorry to bother you with this question. I'm totally new to this and I don't know what I'm doing 
I want to use an Excel spread sheet as a database. Any ideas how to do it? I think I have to use ODBC but I have no idea how to write the appropriate connection string and how to write a query?
I've been googling for some time but I cannot find any information on this topic. Where should I look?
|
|
|
|
|
Search codeproject for Excel ADO. It may be better to use Access if you can, then use Excel to pull updated data from the access db.
Blog Have I http:\\www.frankkerrigan.com
|
|
|
|
|
Thank you for your advice. I've found some information that might be useful.
I want to make an application that uses a simple database that does not need SQL server, Access or MS Office installed on the computer.
I currently use Excel to do my work but I wan't to be able to do it without it.
|
|
|
|
|
You don't need MS access or office on a machine to use an access database with a website.
Blog Have I http:\\www.frankkerrigan.com
|
|
|
|
|
Using an access database is preferable to Excel, and you don't need Access or Office installed. But if you must use Excel, here is an example connection string for you:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source="Path\MyExcelFilename.xls";Extended Properties="Excel 8.0;HDR=NO;IMEX=1;ImportMixedTypes=Text"
I can't remember what IMEX is, but HDR is whether the first row of data is a header row or not.
|
|
|
|
|
Thank you very much for your advice. I've found a sample on this site that seems as an excellent starting point.
I'll think about the access database too but creating an sample access database (or anyone) does require MS Access?
|
|
|
|
|
|
hi,
i have table let in the following format
Col-A col-B col-C col-D col-E
Row-1 1 2 3 4 E1
Row-2 1 2 3 4 E2
Row-3 1 2 3 4 E3
Row-4 1 2 3 4 E4
so i need the output in the format
Col-A col-B col-C col-D col-E
Row-1 1 2 3 4 E1/E2/E3/E4
i cannt able to find the way to write the SQL for it
|
|
|
|
|
I think there is no option to do this using queries.You have to go for the cursors.
Kamal
|
|
|
|
|
SELECT Col-E FROM tblSomething WHERE Col-A >= 1 AND Col-A <=4
Pablo
Sometimes I think there's no reason to get out of bed . . . then I feel wet, and I realize there is.
|
|
|
|
|
Hi, I am using ODBC to connect my VC++.NET application to connect to SQL server. I can connect any server in the LAN using server name. Is it possible to connect remote server over internet using IP address? Currently I am using connection string like
"connStr = "Driver={SQL Server};Server=server_name;Database=db_name;Trusted_Connection=yes";"
Any suggestions?
Thanks in advance.
|
|
|
|
|
Yes you can but the TrustedConnetion will likley not work. You have to use a SQL username and password.
Please bear in mind that exposing SQL server or another db to the internet is highly dangerous.
Blog Have I http:\\www.frankkerrigan.com
|
|
|
|
|
|
Hi,
I am trying to convert Excel Data to SqlServer for that i am using SqlBulkCopy class under System.Data.SqlClient namespace.
While definin the instance for SqlBulkCopy class i am getting error ...
c:\inetpub\wwwroot\ConvertXLStoSQL\WebForm1.aspx.cs(47): The type or namespace name 'SqlBulkCopy' could not be found (are you missing a using directive or an assembly reference?)
It's very urgent please help me to come out from this problem if posible.
Thanks,
Honey
|
|
|
|
|
Try and use DTS as it is fairly easy to connect EXCEL via dts. I do it all the time with no issues.
Blog Have I http:\\www.frankkerrigan.com
|
|
|
|
|
Hi frndz,
I want name of the foreign key column in a table using sqldmo ,can any one help me.
Rupa
|
|
|
|
|
supposed i have this table, the date is in d/m/y format
value date
100 6/11/2006
100 7/11/2006
100 8/11/2006
100 9/11/2006
100 10/11/2006
200 11/11/2006
200 12/11/2006
200 13/11/2006
200 14/11/2006
100 15/11/2006
100 16/11/2006
100 17/11/2006
200 18/11/2006
130 19/11/2006
130 20/11/2006
i want the result to be like this:
value min_date max_date
100 6/11/2006 10/11/2006
200 11/11/2006 14/11/2006
100 15/11/2006 17/11/2006
200 18/11/2006 18/11/2006
130 19/11/2006 20/11/2006
The only way i know to work this out is to make a cursor table, fetch each record and compare the value. If next value is different from first fetch up value then move to prior record, record it's date as max date. Put that value in temporary table.
Is it possible to retrieve the set i want like above example with only one select statement? I have failed to use select with group by clause.
Can anyone help me? I really need to get it done with only 1 select statement
|
|
|
|
|
select value, min_date, max_date
from
(select value, min(date) min_date
from table
group by value) a,
(select value, max(date) max_date
from table
group by value) b
where a.value = b.value
Mike Lasseter
|
|
|
|
|
mr_lasseter wrote: select value, min_date, max_date
from
(select value, min(date) min_date
from table
group by value) a,
(select value, max(date) max_date
from table
group by value) b
where a.value = b.value
Mike Lasseter
thanks for quick reply. Your query not doing exactly as i requested. Value 100 and 200 would be in the same block.
100 6/11/2006 17/11/2006
I need it to be:
100 6/11/2006 10/11/2006
100 15/11/2006 17/11/2006
200 11/11/2006 18/11/2006
I need it to be:
200 11/11/2006 14/11/2006
200 18/11/2006 18/11/2006
|
|
|
|
|
I am not sure in what context you are using this table, but if I was you I would seriously consider rethinking your design. What happens if records get entered into the table out of order? Is there a primary key on this table?
In addition, AFAIK there is no guarantee that select * from table will always pull the results in the same order.
Mike Lasseter
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
