Click here to Skip to main content
15,885,365 members
Please Sign up or sign in to vote.
1.00/5 (2 votes)
C # using ado.net to connect a text file (text file data is large, there are tens of millions of rows of data), and when the connection is fast to read its contents, thank you.
Posted
Comments
Mario Majčica 1-Jul-11 13:13pm    
Please try to express yourself properly. I can't understand what is your problem.
pucx 2-Jul-11 0:48am    
I mean I want to use the same as connecting to the database using ado.net to connect to a text file. At the same time, It can be quickly read out the data from the text file. Thank you eh.
Vivek Krishnamurthy 1-Jul-11 13:19pm    
I suggest you use Improve question widget and explain what are you trying to do. Its not clear.
pucx 2-Jul-11 0:48am    
I mean I want to use the same as connecting to the database using ado.net to connect to a text file. At the same time, It can be quickly read out the data from the text file. Thank you eh.

You can connect to a textfile thru ADO.NET but accessing tens of millions of data cant be read so fast. As mentioned by my community friends, this cant be possible without indexing the data. Also if you use the where clause in your select query, you need to index that column(s) in database.

I would prefer to transfer all the flat file data in staging table using SSIS package and indexing those data. then you can access those data fast. but here fast doesn't mean with milliseconds..u should consider the amount of data.

Thanks
Subhendu
 
Share this answer
 
Comments
pucx 2-Jul-11 4:37am    
But how can I transfer my text file data to stage table by ssis.Thank you.
De, Subhendu 2-Jul-11 5:25am    
Hi,

http://decipherinfosys.wordpress.com/2008/07/28/ssis-importing-data-from-a-text-file-using-a-package/

You should have SSIS installed on your machine/server.

Regards
Subhendu
I have no idea what you're talking about. But, if you intend on using a text file as a database and expect it to be fast, forget it. Tens of millions of unindexed rows is just going to be brutally slow.
 
Share this answer
 
Comments
pucx 2-Jul-11 0:48am    
I mean I want to use the same as connecting to the database using ado.net to connect to a text file. At the same time, It can be quickly read out the data from the text file. Thank you eh.
You simply can't read that much data at once. You're going to have to be more selective iin the data returned from the query. We have a web page that returns no more than 250 records at a time from a 4-million record table.
 
Share this answer
 
I mean I want to use the same as connecting to the database using ado.net to connect to a text file. At the same time, It can be quickly read out the data from the text file. Thank you eh.
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900