|
A VB.net app I'm building needs to process very long lists stored in .txt files. My problem occurs when I attempt to avoid duplicate entries while merging lists. Most of the lists go quickly, but the largest (the largest to date has 190,000 lines) take forever. I don't expect that much data to process instantly, but I would like to speed things up. Here's how I'm doing it at present:
' "lbSelFiles" is the listbox containing paths to the files to be processed.
' the "sr" streamreader reads each line of each file. If the line isn't already
'in the collection (addList), it adds it.
' The "getLines" array's only purpose is to provide a max value for the progress bar (progBar).
addList.Clear()
For l = 0 To lbSelFiles.Items.Count - 1
If File.Exists(lbSelFiles.Items.Item(l).ToString) Then
Dim getLines() As String = File.ReadAllLines(lbSelFiles.Items.Item(l).ToString)
progBar.Maximum = getLines.Length
progBar.Value = 0
Dim sr As New StreamReader(lbSelFiles.Items.Item(l).ToString)
Dim line As String
Do
line = sr.ReadLine()
'this appears to be the bottle neck
If Not addList.Contains(line) Then
addList.Add(line)
End If
progBar.Increment(1)
Loop Until line Is Nothing
sr.Close()
End If
Next l
'this copies addList to an array, then sends the array to a richtextbox
'the form containing this code then closes
Dim arr(addList.Count) As String
addList.CopyTo(arr, 0)
Dim newRTF As RichTextBox = CType(main.ActiveMdiChild.Controls(0), RichTextBox)
newRTF.Lines = arr
Me.Close()
Any suggestions on a better (faster) way are appreciated.
|
|
|
|
|
Alan Burkhart wrote: 'this appears to be the bottle neck
If Not addList.Contains(line) Then
erm... well... this is going to iterate over the *entire* list, looking for this value. So, of course it's going to be slow. I'd suggest you build an alphabetised version of the list. That way, you can do a more intelligent search ( for example, you could use a dictionary to create lists that mean you only iterate through the items with the same first letter, or you could start in the middle and search alphabetically by grabbing the middle every time of the appropriate half )
Christian Graus
Driven to the arms of OSX by Vista.
|
|
|
|
|
It looks as though you are reading the whole file into an array, and then reading it line by line. That seems rather wasteful. If you have enough storage to read in everything, I would suggest using Array.Sort on the data you read in; if you do that, duplicates would be consecutive in the array and could thus easily be removed.
Further, you could use something like a Dictionary instead of a List. Even if you don't need a Value for each item in the Dictionary, it will still be much faster and more efficient than using a List. I've not gone up to hundreds of thousands of items with a dictionary, but I've found them to perform well even with thousands of items.
|
|
|
|
|
Christian and supercat9...
Just tried using a dictionary (most of the lists are already sorted). You're both right. I didn't realize how much faster a dictionary was than a list.
Will require some nit-picky rewriting of code, but definitely worth it.
Thank You Both!
AB
|
|
|
|
|
As the others said. Furthermore having progBar.Increment(1) execute for every item is another waste: if you hope your app to handle 190,000 items in say 19 seconds, then you are updating the progress bar every 100 microseconds, you would need bionic eyes to see all these updates (assuming infinite resolution for your display).
I suggest you let the progress bar work at 1/1000 of the current setting, so reduce its Maximum by 1000 and call its Increment() only once every 1000 items handled.
Luc Pattyn [Forum Guidelines] [My Articles]
- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets
|
|
|
|
|
Luc Pattyn wrote: I suggest you let the progress bar work at 1/1000 of the current setting, so reduce its Maximum by 1000 and call its Increment() only once every 1000 items handled.
The progress bar was actually just so I could see that the thing was working at all. It'll eventually be either removed or as you say, have it's increments drastically reduced. The latter is the most likely.
Even with the progBar incrementing every list item, the app now runs through 150,000 items in about 30 seconds - instead of 3 or 4 minutes. Ultimately, the app will be pulling data from about a dozen lists, most with less than 30,000 entries and some with only a couple hundred. There will be only two or three "giant" lists. But I didn't want it to take all day when using them.
Thanks!
AB
|
|
|
|
|
Dear All
I ma using late binding to Done word parsing
I have word document i want to change the style of paragraphs using template docuemtn
For exmaple
Par(1) style ="This style 1"
Par(2) style ="This style 2"
My code :
Dim o_nullobject As Object = System.Reflection.Missing.Value
Dim o_filePath As Object = FilePath''file have paragraphs
Dim oTemplate As Object = TemplePath
Dim oFalse As Object = False
Dim oTrue As Object = True
wordApp = CreateObject("Word.Application")
doc = CreateObject("Word.Document")
alse, o_nullobject, o_nullobject)
doc = wordApp.Documents.Open(o_filePath, o_nullobject, oFalse, o_nullobject, o_nullobject, o_nullobject, o_nullobject, o_nullobject, o_nullobject, o_nullobject, o_nullobject, oFalse, o_nullobject, o_nullobject, o_nullobject, o_nullobject)
doc.Activate()
ParseWordStyle()
Private Sub ParseWordStyle()
Try
Dim i As Int64
'' Dim style As String
Dim parsText As String
''Dim p As Object = doc.Styles[""]
For i = 1 To doc.Paragraphs.Count
doc.Paragraphs(i).Range.Style ="some style".......
'''''''''''''''''''''''''''''''
my problem
How to set paragaphs text of styles from template
Yes. CreatiVity withOuT limiTs
modified on Wednesday, March 18, 2009 6:02 AM
|
|
|
|
|
Hi.
I have got a DLL with from one of our software providors that queries a particular info from an IP. This code has been written in c++, and i like to call the function from VB.
I have registered the DLL with regsvr32.
I have created a new vb project, and added reference to the COM object ( found afer i registered it)
The referenced DLL are now found in object browser named "PrinterAccessLib"
The class i would like to use is named CONFIG
When i look in the object browser, i can see the function GetStatus like this:
Sub GetStatus(ByVal P_szIpAddress As String, ByRef P_szBuffer As String)
To me is looks like the sub should update the variable status, but when the sub has been called, it is still ""
I do not get any errors, and i can see with a network sniffer that the DLL comunicates with the printer.
What am i doing wrong ?
Here is the code using the dll sub:
Public Sub Main()
Dim status As String = ""
Dim ip As String = "155.209.118.123"
Dim ObjPrinterStatus As New PrinterAccessLib.CONFIG
ObjPrinterStatus.GetStatus(ip, status)
Console.WriteLine(status)
End Sub
Thanks
Best regards
Bjorn.
|
|
|
|
|
A COM object is a black box. If there's a method and you're calling it, we can't help you any further, you need to talk to the vendors of the dll to find out why it's not responding to your call in the manner you expect.
Christian Graus
Driven to the arms of OSX by Vista.
|
|
|
|
|
Hello ,
I am loading some data to SQL server using my MS Access application. Sometimes loading may hangup with some errors. So partial loading may happens. So what i thinking is that if any error occurs, I way to get restore bac the server to old state(means before loading state) . So i can try loading data again
How can i do that
|
|
|
|
|
Look into database transactions. I recommend this site as a starting point
|
|
|
|
|
I would like to make my an application that will allow it's user the change active audio devices on the fly without having to open control panel, go into Sound and Audio devices ect... I've new to VB and only a Beginner/Intermediate coder. Could someone show me where I should start? I couldn't find any examples online.
Thanks!
|
|
|
|
|
This is no way a beginner's job.
AFAIK you will need to use P/Invoke win32 api for this.
जय हिंद
|
|
|
|
|
this may help
http://www.codeproject.com/KB/cs/Enum_Recording_Devices.aspx[^]
TVMU^P[[IGIOQHG^JSH`A#@`RFJ\c^JPL>;"[,*/|+&WLEZGc`AFXc!L
%^]*IRXD#@GKCQ`R\^SF_WcHbORY87֦ʻ6ϣN8ȤBcRAV\Z^&SU~%CSWQ@#2
W_AD`EPABIKRDFVS)EVLQK)JKQUFK[M`UKs*$GwU#QDXBER@CBN%
R0~53%eYrd8mt^7Z6]iTF+(EWfJ9zaK-iTV.C\y<pjxsg-b$f4ia>
-----------------------------------------------
128 bit encrypted signature, crack if you can
|
|
|
|
|
Hi,
I have a Treeview with 3 nodes under the root. My aim is to check the 2nd node whenever any child nodes under 2nd node gets checked. How could you do that?
Thanks.
What a curious mind needs to discover knowledge is noting else than a pin-hole.
|
|
|
|
|
There's an event that gets fired when the node is checked. Loop back through the .Parent property until you reach the node at the second level, and check it
I are troll
|
|
|
|
|
use BeforeCheck or AfterCheck event
TVMU^P[[IGIOQHG^JSH`A#@`RFJ\c^JPL>;"[,*/|+&WLEZGc`AFXc!L
%^]*IRXD#@GKCQ`R\^SF_WcHbORY87֦ʻ6ϣN8ȤBcRAV\Z^&SU~%CSWQ@#2
W_AD`EPABIKRDFVS)EVLQK)JKQUFK[M`UKs*$GwU#QDXBER@CBN%
R0~53%eYrd8mt^7Z6]iTF+(EWfJ9zaK-iTV.C\y<pjxsg-b$f4ia>
-----------------------------------------------
128 bit encrypted signature, crack if you can
|
|
|
|
|
Hello all
I'm trying to include the a Grammer check into my app; one of the approaches that I've come across is using MS.Words' but it seem that I'm facing somedefficulties that I can't figure out;
mports System.Runtime.InteropServices
Public Class Form1
Inherits System.Windows.Forms.Form
' Invokes either the spell or grammar checker.
Private Sub SpellOrGrammarCheck(ByVal blnSpellOnly As Boolean)
Try
' Create Word and temporary document objects.
Dim objWord As Object
Dim objTempDoc As Object
' Declare an IDataObject to hold the data returned from the
' clipboard.
Dim iData As IDataObject
' If there is no data to spell check, then exit sub here.
If TextBox1.Text = "" Then
Exit Sub
End If
objWord = New Word.Application() // My problem is here as it shows that it is not defined
objTempDoc = objWord.Documents.Add
objWord.Visible = False
' Position Word off the screen...this keeps Word invisible
' throughout.
objWord.WindowState = 0
objWord.Top = -3000
' Copy the contents of the textbox to the clipboard
Clipboard.SetDataObject(TextBox1.Text)
' With the temporary document, perform either a spell check or a
' complete
' grammar check, based on user selection.
With objTempDoc
.Content.Paste()
.Activate()
If blnSpellOnly Then
.CheckSpelling()
Else
.CheckGrammar()
End If
' After user has made changes, use the clipboard to
' transfer the contents back to the text box
.Content.Copy()
iData = Clipboard.GetDataObject
If iData.GetDataPresent(DataFormats.Text) Then
TextBox1.Text = CType(iData.GetData(DataFormats.Text), _
String)
End If
.Saved = True
.Close()
End With
objWord.Quit()
MessageBox.Show("The spelling check is complete.", _
"Spell Checker", MessageBoxButtons.OK, _
MessageBoxIcon.Information)
' Microsoft Word must be installed.
Catch COMExcep As COMException
MessageBox.Show( _
"Microsoft Word must be installed for Spell/Grammar Check " _
& "to run.", "Spell Checker")
Catch Excep As Exception
MessageBox.Show("An error has occured.", "Spell Checker")
End Try
End Sub
Private Sub btnSpellCheck_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles btnSpellCheck.Click
SpellOrGrammarCheck(True)
End Sub
Private Sub btnGrammarCheck_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles btnGrammarCheck.Click
SpellOrGrammarCheck(False)
End Sub
End Class
>
any help? I've added the reference Microsoft Word 12.0 object but still has the same problem undefined ..!!!?
or if any has another idea; I will highly appreciate it...
cheers
|
|
|
|
|
Are you getting any error message?
I guess you need to use fully qualified names.
जय हिंद
|
|
|
|
|
Yes it is an error message ...
|
|
|
|
|
You need to specify the fully qualified name as I had mentioned before ie use
Microsoft.Office.Interop.Word.Application .
जय हिंद
|
|
|
|
|
I've done what you have mentioned; I appreciate your help...and now it says on the error message "Error Expected" what does it mean?
|
|
|
|
|
Hi All,
I want to work with some fonts which are created by me. But i don't want to install those in system Font folder.Which are accessed by only my software.
Pls help.How can it Possible.
Thanks
mailto: anubhava.prodata@gmail.com
|
|
|
|
|
|
Very cool, didn't know that existed.
Any suggestions, ideas, or 'constructive criticism' are always welcome.
"There's no such thing as a stupid question, only stupid people." - Mr. Garrison
|
|
|
|