Click here to Skip to main content
15,889,858 members
Home / Discussions / C#
   

C#

 
AnswerRe: Import pdf to sql Pin
thatraja16-Nov-10 18:47
professionalthatraja16-Nov-10 18:47 
QuestionNetwork Programming Pin
Hum Dum15-Nov-10 19:55
Hum Dum15-Nov-10 19:55 
AnswerRe: Network Programming Pin
Jacob D Dixon16-Nov-10 6:54
Jacob D Dixon16-Nov-10 6:54 
GeneralRe: Network Programming Pin
Hum Dum17-Nov-10 21:48
Hum Dum17-Nov-10 21:48 
AnswerRe: Network Programming Pin
RobCroll17-Nov-10 4:23
RobCroll17-Nov-10 4:23 
GeneralRe: Network Programming Pin
Hum Dum17-Nov-10 21:43
Hum Dum17-Nov-10 21:43 
GeneralRe: Network Programming Pin
RobCroll18-Nov-10 20:21
RobCroll18-Nov-10 20:21 
QuestionA Question Of Efficiency Pin
Roger Wright15-Nov-10 17:11
professionalRoger Wright15-Nov-10 17:11 
I've been playing...

A short while back I posted about writing an app which reads csv data from a folder of files I save from emailed reports, and adds each record to a SQL Server DB. Due to other work tasks, I haven't done much with it lately, but I'm back to it now. Previously I'd managed to select the files, strip off the ten lines or so of descriptive data at the beginning of each, identify and classify data lines using regexes, then write the cleaned up data to a text file. That's about where I got distracted, and it's just as well, since I was stuck on the best way to transfer the information to a SQL Server instance. The problem is duplicates...

My data source sometimes sends consolidated reports that duplicate data already received, and it's not particularly easy to tell from the emails which files contain duplicates. My code thus far doesn't have that capability, and I really don't look forward to writing it, but neither do I want to deal with SQL Server bouncing my transactions because they contain duplicate data. So I did some digging and found a gem.

I found that the Queue collection has a .Contains method which looked interesting. I dumped the output text file from my previous version, and just enqueued each cleaned up record as I processed it; that worked great. Then I added code to skip a record if OutQueue.Contains(record) returned true. Wow! It worked like a charm, and took all of ten minutes to implement.

Whee! It's fun trying new stuff. But I'm curious, having never used a Queue before... My application will never see more than a few hundred records at a time, so efficiency isn't a big issue. But I'm thinking that, at some level of complexity - thousands, hundred thousands, millions perhaps - the Contains method will become less efficient than other methods of detecting duplicate items. Does anyone have any idea at what point it becomes too inefficient, and any suggestions about what to implement in its stead?


I'm somewhat tempted to build this thing using only techniques I've never used before, and rarely seen in the wild, then publish it just for the fun of it. Big Grin | :-D
Will Rogers never met me.

AnswerRe: A Question Of Efficiency Pin
Luc Pattyn15-Nov-10 17:53
sitebuilderLuc Pattyn15-Nov-10 17:53 
GeneralRe: A Question Of Efficiency Pin
Roger Wright15-Nov-10 18:15
professionalRoger Wright15-Nov-10 18:15 
GeneralRe: A Question Of Efficiency Pin
Luc Pattyn16-Nov-10 2:27
sitebuilderLuc Pattyn16-Nov-10 2:27 
AnswerRe: A Question Of Efficiency Pin
_Erik_16-Nov-10 1:58
_Erik_16-Nov-10 1:58 
GeneralRe: A Question Of Efficiency Pin
Roger Wright17-Nov-10 14:14
professionalRoger Wright17-Nov-10 14:14 
AnswerRe: A Question Of Efficiency Pin
PIEBALDconsult16-Nov-10 2:17
mvePIEBALDconsult16-Nov-10 2:17 
GeneralRe: A Question Of Efficiency Pin
Roger Wright17-Nov-10 14:11
professionalRoger Wright17-Nov-10 14:11 
GeneralRe: A Question Of Efficiency Pin
PIEBALDconsult18-Nov-10 2:14
mvePIEBALDconsult18-Nov-10 2:14 
AnswerRe: A Question Of Efficiency Pin
RobCroll17-Nov-10 4:12
RobCroll17-Nov-10 4:12 
GeneralRe: A Question Of Efficiency Pin
Roger Wright17-Nov-10 14:10
professionalRoger Wright17-Nov-10 14:10 
QuestionContrast problem Pin
pancakeleh15-Nov-10 16:57
pancakeleh15-Nov-10 16:57 
AnswerRe: Contrast problem Pin
Roger Wright15-Nov-10 17:14
professionalRoger Wright15-Nov-10 17:14 
GeneralRe: Contrast problem Pin
pancakeleh15-Nov-10 17:22
pancakeleh15-Nov-10 17:22 
GeneralRe: Contrast problem Pin
Dr.Walt Fair, PE15-Nov-10 18:03
professionalDr.Walt Fair, PE15-Nov-10 18:03 
GeneralRe: Contrast problem Pin
Roger Wright15-Nov-10 18:24
professionalRoger Wright15-Nov-10 18:24 
AnswerRe: Contrast problem Pin
pancakeleh15-Nov-10 19:05
pancakeleh15-Nov-10 19:05 
GeneralRe: Contrast problem Pin
pancakeleh15-Nov-10 21:34
pancakeleh15-Nov-10 21:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.