Click here to Skip to main content
15,884,099 members
Home / Discussions / C#
   

C#

 
QuestionHow to get Millions of files from folder and bulk insert in database Pin
Ankur B. Patel16-Oct-21 1:10
Ankur B. Patel16-Oct-21 1:10 
AnswerRe: How to get Millions of files from folder and bulk insert in database Pin
Eddy Vluggen16-Oct-21 3:10
professionalEddy Vluggen16-Oct-21 3:10 
GeneralRe: How to get Millions of files from folder and bulk insert in database Pin
Richard MacCutchan16-Oct-21 4:00
mveRichard MacCutchan16-Oct-21 4:00 
GeneralRe: How to get Millions of files from folder and bulk insert in database Pin
Eddy Vluggen16-Oct-21 4:18
professionalEddy Vluggen16-Oct-21 4:18 
GeneralRe: How to get Millions of files from folder and bulk insert in database Pin
Ankur B. Patel16-Oct-21 4:59
Ankur B. Patel16-Oct-21 4:59 
GeneralRe: How to get Millions of files from folder and bulk insert in database Pin
Gerry Schmitz16-Oct-21 11:04
mveGerry Schmitz16-Oct-21 11:04 
AnswerRe: How to get Millions of files from folder and bulk insert in database Pin
OriginalGriff16-Oct-21 6:43
mveOriginalGriff16-Oct-21 6:43 
AnswerRe: How to get Millions of files from folder and bulk insert in database Pin
Michael Sydney Balloni16-Oct-21 17:36
professionalMichael Sydney Balloni16-Oct-21 17:36 
Hi Ankur,

I'm going to assume your files are in a bunch of directories, not all in one, or mostly in a few, but leafy. I'd use the System.IO classes Directory and Path and FileInfo. You can use Directory.GetDirectories to get then directories, then loop over those directory paths calling Directory.GetFiles to get file paths to the files, then for each file path call Path.GetFilename to get the filename, and (new FileInfo(filePath)).Length to get the file length. I'd use a DB transaction per directory, and create an INSERT statement per file and execute it inside the transaction, and commit when you're done with the files in that directory.

Things not to do:
Don't get all files in all directories.
Don't build one SQL statement for all the files.

Hope this helps, -Michael

AnswerRe: How to get Millions of files from folder and bulk insert in database Pin
jschell24-Oct-21 6:50
jschell24-Oct-21 6:50 
QuestionPredicateBuilder Question Pin
Kevin Marois15-Oct-21 5:24
professionalKevin Marois15-Oct-21 5:24 
AnswerRe: PredicateBuilder Question Pin
Eddy Vluggen15-Oct-21 14:44
professionalEddy Vluggen15-Oct-21 14:44 
QuestionTask.WhenAll ConfigureAwait Question Pin
Kevin Marois14-Oct-21 12:57
professionalKevin Marois14-Oct-21 12:57 
AnswerRe: Task.WhenAll ConfigureAwait Question Pin
Richard Deeming14-Oct-21 23:15
mveRichard Deeming14-Oct-21 23:15 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Kevin Marois15-Oct-21 4:53
professionalKevin Marois15-Oct-21 4:53 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Richard Deeming15-Oct-21 5:59
mveRichard Deeming15-Oct-21 5:59 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Kevin Marois15-Oct-21 6:29
professionalKevin Marois15-Oct-21 6:29 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Kevin Marois15-Oct-21 8:08
professionalKevin Marois15-Oct-21 8:08 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Richard Deeming17-Oct-21 21:58
mveRichard Deeming17-Oct-21 21:58 
GeneralRe: Task.WhenAll ConfigureAwait Question Pin
Kevin Marois18-Oct-21 4:01
professionalKevin Marois18-Oct-21 4:01 
QuestionCodility fails to compile even though it compiles in VS code? Pin
Cliff ludo13-Oct-21 6:41
Cliff ludo13-Oct-21 6:41 
AnswerRe: Codility fails to compile even though it compiles in VS code? Pin
BillWoodruff13-Oct-21 17:11
professionalBillWoodruff13-Oct-21 17:11 
Generalpass array to function Pin
Cliff ludo12-Oct-21 23:04
Cliff ludo12-Oct-21 23:04 
GeneralRe: pass array to function Pin
Richard Deeming12-Oct-21 23:25
mveRichard Deeming12-Oct-21 23:25 
GeneralRe: pass array to function Pin
Cliff ludo12-Oct-21 23:32
Cliff ludo12-Oct-21 23:32 
QuestionCreating An Async Method Pin
Kevin Marois7-Oct-21 11:02
professionalKevin Marois7-Oct-21 11:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.