Thanks to Richard I was able to make it one step further and incorporated his additional coding into my current project.
Public Sub Match_Merge(I As Long)
Is the method I am call for the parallel for.
inside this method I am comparing my temp table which is my import table and by current database table.
For I = 1 To temptable.Rows.Count - 1
countI = countI + 1
For I2 = I3 To dt.Rows.Count - 1
Next I
Next i2
if there is not a match then it the information goes to
removeduptable.ImportRow(temptable.Rows(I))
if there a match then the information is stored in a separate table.
duplicaterow = Duplicatetable.NewRow()
duplicaterow.Item("First_name") = dt.Rows(I2).Item("First_name")
duplicaterow.Item("Middle_name") = dt.Rows(I2).Item("Middle_name")
duplicaterow.Item("Last_name") = dt.Rows(I2).Item("Last_name")
duplicaterow.Item("Duplicatecount") = count
Duplicatetable.Rows.Add(duplicaterow)
This is the method I am using for my parallel for:
Public Sub ParallelForEach(ByVal startOfIteration As Integer, ByVal endOfIteration As Integer, ByVal subFunctionName As String)
processorcount.MaxDegreeOfParallelism = 4
Const flags As BindingFlags = BindingFlags.Public Or BindingFlags.NonPublic Or BindingFlags.Static Or BindingFlags.Instance
Dim method As MethodInfo = Me.GetType().GetMethod(subFunctionName, flags, Nothing, New Type() {GetType(Long)}, Nothing)
If method Is Nothing Then
Throw New ArgumentException(String.Format("Method '{0}' was not found.", subFunctionName), "subFunctionName")
End If
Dim del As System.Delegate
If method.IsStatic Then
del = System.Delegate.CreateDelegate(GetType(Action(Of Long)), method)
Else
del = System.Delegate.CreateDelegate(GetType(Action(Of Long)), Me, method)
End If
Dim action As Action(Of Long) = DirectCast(del, Action(Of Long))
System.Windows.Forms.Control.CheckForIllegalCrossThreadCalls = False
System.Threading.Tasks.Parallel.For(1, 2, processorcount, action)
End Sub
If I leave the parallel for like this:
System.Threading.Tasks.Parallel.For(1, 2, processorcount, action)
it runs as fast as a sequential.
if I change it to
System.Threading.Tasks.Parallel.For(1, 10, processorcount, action)
it runs quicker, but the parallel for adds in duplicate rows.
So if the record count is 150,000 it will show up as 300,000 I can not figure out a way to stop this from happening.
What I have tried:
I tried to cut off the row add when the count made it to the amount of temptable rows. I also tried to synclock both tables and that did not help either.