Click here to Skip to main content
15,884,388 members
Please Sign up or sign in to vote.
1.80/5 (2 votes)
See more:
What is the best way to save a 10K row Data table from sever End to Database. Where some row value will be edited if exist in Database.

What I have tried:

Using For loop in c# and get each row and hit DB continuously,
Posted
Updated 9-May-16 3:35am
Comments
Tomas Takac 5-May-16 3:50am    
Where do you get those 10k rows from? A file perhaps?

As koolprasad2003 suggested in solution #1 you should use SqlBulkCopy to insert data form the data table into your database. See the documentation[^] for more details.

You cannot load the data into the target table directly as you need to do some processing. You should load it into a staging table, which will have the same structure as the target table and do the inserts/updates/deletes of the target table after the import. For that you should use the MERGE statement[^].
 
Share this answer
 
Comments
sabuj82 8-May-16 11:53am    
But if i Use staging table then my process will be Fast?
Tomas Takac 8-May-16 16:24pm    
I think it will be faster. But it's up to you to test it and decide.
sabuj82 8-May-16 23:57pm    
Ok thanks very much :)
I think SQLBulkCopy class will help you more, it Lets you efficiently bulk load a SQL Server table with data from another source.
check out below link
SqlBulkCopy- Bulk Insert records and Update existing rows if record exists using C# and VB.Net[^]
 
Share this answer
 
Comments
Tomas Takac 5-May-16 5:33am    
There is a second part to the question, some rows get updated in the database.
koolprasad2003 5-May-16 5:47am    
In SQL you can handle that, you can use IF NOT EXIST statement. see below snippet
BEGIN TRANSACTION;
IF EXISTS (SELECT 1 FROM dbo.table WHERE PK = @PK)
BEGIN
UPDATE ...
END
ELSE
BEGIN
INSERT ...
END
COMMIT TRANSACTION;
you can use SqlBulkCopy for bulk insertion/Updation.

But if you want to filter. you can use User Defined Table Type in SQL.
In which you can pass the datatable to your stored procedure and then filter by your sql query easily.

SQL:
SQL
CREATE PROCEDURE [dbo].[Insert_Customers]
      @tblCustomers CustomerType READONLY
AS
BEGIN
      SET NOCOUNT ON;
     
      INSERT INTO Customers(CustomerId, Name, Country)
      SELECT Id, Name, Country FROM @tblCustomers
END


C#:
C#
using (SqlCommand cmd = new SqlCommand("Insert_Customers"))
           {
               cmd.CommandType = CommandType.StoredProcedure;
               cmd.Connection = con;
               cmd.Parameters.AddWithValue("@tblCustomers", dt);
               con.Open();
               cmd.ExecuteNonQuery();
               con.Close();
           }
 
Share this answer
 
v2

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900