|
I have a project that needs the ability to modify dozens, hundreds, and potentially thousands of file modifications which are fairly intensive and affect associated resource files as well. To speed things up and maximize the ability to process multiple threads to take advantage of parallel processing I decided to use the System.Threads.Tasks.Parallel class to drive these file changes. There are a few things that I have learned and discovered along the way that I would like to better understand.
First, before I go any further, my project has a BIG need to track all changes in a log file that occur BEFORE they happen to minimize the risk of losing data when something goes wrong. That log file is then parsed for undo actions. This requires the chain of events to be tracked; and logging each change before it happens requires several sub-tasks that use .NET's await feature. A basic picture of the process used to change the files looks something like this:
public class MainFileType
{
internal async void DoSomeMajorChanges(RichTextboxBuilder builder, StreamWriter changeLog)
{
bool result;
await Task.Run(new Action(() => changeLog.LogAction(this))).ConfigureAwait(false);
await Task.Run(new Action(()=> result = coreFile.DoChanges())).ConfigureAwait(false);
builder.Control.BeginInvoke(new Action() builder.NotifyUser("Some Change Occurred", Color.Red));
foreach (ResourceFile file in this.AssociatedFiles)
{
await Task.Run(new Action(() => changeLog.LogAction(file))).ConfigureAwait(false);
await Task.Run(new Action(() => result |= file.DoChanges())).ConfigureAwait(false);
builder.Control.BeginInvoke(new Action() builder.NotifyUser("Some Change Occurred", Color.Blue));
}
return result;
}
}
This code is called by a UI that is shown any time a single or multiple files are modified. The UI regularly reports to the user:
public class ChangeManagerUI : Form
{
private bool processed;
object task;
StreamWriter parseableActionLog;
private void OnFormShown(object sender, EventArgs e)
{
if (!processed)
{
processed = true;
MainFileType file;
RichTextboxBuilder builder = null;
List<MainFileType> batch = task as List<MainFileType>;
Refresh();
if (batch != null)
{
RichTextboxBuilder.BeginConcurrentAppendProcess(this, batch.Count);
ReportCaption = "Conversion Progress";
progressBar.Visible = true;
progressBar.Maximum = batch.Count;
ConcurrentOutputManager.ConcurrentMode = true;
Task.Run(() =>
{
Parallel.For(0, batch.Count, i =>
{
file = batch[i];
builder = RichTextboxBuilder.BeginConcurrentAppend(i);
file.DoSomeMajorChanges(builder, parseableActionLog)
RichTextboxBuilder.EndConcurrentAppend(i);
});
});
if (builder != null)
{
builder.AppendNewLine();
builder.BeginColor(Color.Purple);
builder.AppendStyled("Batch Conversion Complete!", FontStyle.Bold);
builder.EndColor();
}
Finalize(false);
}
else
{
file = task as MainFileType;
ReportTextBuilder = new RichTextboxBuilder(this);
Finalize(file.DoSomeMajorChanges(mod, ReportTextBuilder).Result);
}
}
}
}
The things that I have noticed that I would like to understand are as follows:
1. If I remove the Task.Run( (line 24) that encapsulates the Parallel.For statement the program/UI locks up (stops responding) even though I am using ConfigureWait(false) in the await commands... I know that using await in the UI thread can lead to this kind of issue but as I understand, Parallel methods use separate threads.
2. Sometimes Parallel.For seems to run the action for the same item repeatedly rather than go to the next item. Do I need to explicitly code the increment? Because of this, I switched to the Parallel.Foreach method and that works more consistently.
3. I had a problem in that the code AFTER the Task.Run => Parallel.For block was run before the parallel tasks completed (which led to my discovery of 1). I fixed it by using an atomic integer that was initialized to the number of tasks and calling a method that decreased that number inside the Parallel.For each statement that would only run a block of code if the number hit zero. However, I would still like to know why this occurred and if there is another .NET mechanism for achieving the goal executing code after parallel tasks are completed built in.
|
|
|
|
|
Parallel.For doesn't return a Task which can be awaited; it has no choice but to block the current thread until the processing has been completed. By wrapping it in a Task.Run, you're blocking a background thread instead of the UI thread.
The Parallel methods also don't work well with async methods. The delegate you pass in is expected to run synchronously to completion.
You've declared your DoSomeMajorChanges method as async void. Your should avoid async void like the plague:
Avoid async void methods | You’ve Been Haacked[^]
The await Task.Run(...) lines in your DoSomeMajorChanges method serve no purpose. Since there are no other awaits in that method, you can simply make it synchronous.
internal bool DoSomeMajorChanges(RichTextboxBuilder builder, StreamWriter changeLog)
{
bool result;
changeLog.LogAction(this);
result = coreFile.DoChanges();
builder.Control.BeginInvoke(new Action() builder.NotifyUser("Some Change Occurred", Color.Red));
foreach (ResourceFile file in this.AssociatedFiles)
{
changeLog.LogAction(file);
result |= file.DoChanges();
builder.Control.BeginInvoke(new Action() builder.NotifyUser("Some Change Occurred", Color.Blue));
}
return result;
}
The delegate you're passing to Parallel.For is referencing captured variables (file and builder). This is not thread-safe, which is why you are seeing inconsistent results. Move the variable declarations inside the delegate:
Task.Run(() =>
{
Parallel.For(0, batch.Count, i =>
{
var file = batch[i];
var builder = RichTextboxBuilder.BeginConcurrentAppend(i);
file.DoSomeMajorChanges(builder, parseableActionLog)
RichTextboxBuilder.EndConcurrentAppend(i);
});
}); NB: You will not be able to refer to the last RichTextboxBuilder instance outside of the loop.
It's not clear what your RichTextboxBuilder methods are doing. They could potentially be harming your concurrency.
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
Thanks Richard. That helps; particularly the info about Parallel itself being syncronous. I knew about the async void issue from my research while solving issues. My real method is Task<bool> (just like yours). Being relatively new to asynchronous code, I just forgot about it when simplifying my code.
Just to clear things up:
So anything inside the Parallel.For/Foreach will run syncronously; except of course if it calls an asynchronous method correct?
|
|
|
|
|
pr1mem0ver wrote: So anything inside the Parallel.For/Foreach will run syncronously; except of course if it calls an asynchronous method correct?
It will run synchronously. If it calls an asynchronous method, the delegate will return and the loop will terminate before the asynchronous method has completed.
There are ways around that, but they tend to cause serious problems:
Should I expose synchronous wrappers for asynchronous methods? | .NET Parallel Programming[^]
"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer
|
|
|
|
|
I forgot to thank you for your help back in April. Thanks!
|
|
|
|
|
Multi-tasking "file operations" isn't as great as it sounds. Often the whole thing will run slower due to head and channel contention.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
Say for example,
I wanted to send an array of integers from my C++ exe to C# at runtime
|
|
|
|
|
Well I just said it. Now what?
|
|
|
|
|
Did you consider using sockets or (named) pipes? Or some other IPCs?
|
|
|
|
|
i tried using namedpipes it just sends and receives but, My goal is to pass the elements from C++ and get that Elements at runtime in C# and process them. Like, Get n inputs in c++ and process that n elements in C#.
Eg: I am entering 5 inputs, say 1,2,3,4,5 in C++
where c# should get that inputs and process ( eg: inputelement + 1 )
the output would be 2,3,4,5,6.
C++ file
#include <windows.h>
#include <iostream>
HANDLE fileHandle;
int main()
{
fileHandle = CreateFileW(TEXT("\\\\.\\pipe\\pipesample"), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);
const char* msg = "hello from c++\r\n";
WriteFile(fileHandle, msg, strlen(msg), nullptr, NULL);
}
C# File
static void Main(string[] args)
{
var namedPipeServer = new NamedPipeServerStream("pipesample", PipeDirection.InOut, 1, PipeTransmissionMode.Byte);
var streamReader = new StreamReader(namedPipeServer);
namedPipeServer.WaitForConnection();
Console.WriteLine($"read from pipe client: {streamReader.ReadLine()}");
namedPipeServer.Dispose();
}
modified 23-Apr-21 6:19am.
|
|
|
|
|
I use SQL CE as a database file in my project. I need to load hidden ID numbers into my datatable. What is the command for accessing ID in SQL CE?
|
|
|
|
|
There isn't a "command for accessing ID" specifically, it's just a field within your table, so you use a normal SELECT statement.
To fetch the last inserted IDENTITY value (if that is what you are using for the ID, and it probably should be that or a GUID) just issue a SELECT @@IDENTITY on the same Connection object without closing it first, as @@IDENTITY is local to the Session, and closing the Connection will clase the Session as well.
"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!
|
|
|
|
|
Why would you do this? There is virtually never a good reason to do so.
Typically, this is done by noobs who think in terms of a single user using the database, never thinking of the possibility that more than one user can execute this same command at the same time and get the same answer. Now each instance of the code (and the person who wrote it) thinks they've got exclusive use of a number multiple people have.
|
|
|
|
|
Alex Dunlop wrote: What is the command for accessing ID in SQL CE?
SELECT ID FROM table WHERE ...
|
|
|
|
|
I have used the following Class for saveing/reading DataGridView row and text color into/from an XML file. It works fine, but when I use column filters, all those color information got lost. Even when I remove the filters, there is no color in my texts/rows. (*** I use advanced DataGridView***)
My Class for load/save XML:
public static void WriteDataGridViewSettings(System.Windows.Forms.DataGridView dgv)
{
XmlTextWriter writer = new XmlTextWriter(Application.StartupPath + @"\MyGridColor.xml", null);
writer.WriteStartDocument();
writer.WriteStartElement(dgv.Name);
int LastRow = dgv.Rows.Count;
for (int i = 0; i < LastRow; i++)
{
writer.WriteStartElement("Row");
writer.WriteStartElement("CellColor");
writer.WriteString(dgv.Rows[i].DefaultCellStyle.BackColor.ToArgb().ToString());
writer.WriteEndElement();
writer.WriteStartElement("TextColor");
writer.WriteString(dgv.Rows[i].DefaultCellStyle.ForeColor.ToArgb().ToString());
writer.WriteEndElement();
writer.WriteEndElement();
}
writer.WriteEndElement();
writer.WriteEndDocument();
writer.Close();
}
public static void ReadDataGridViewSettings(System.Windows.Forms.DataGridView dgv)
{
XmlDocument xmldoc = new XmlDocument();
XmlNodeList xmlnode;
FileStream fs = new FileStream(Application.StartupPath + @"\MyGridColor.xml", FileMode.Open, FileAccess.Read);
xmldoc.Load(fs);
xmlnode = xmldoc.GetElementsByTagName("Row");
for (int i = 0; i <= xmlnode.Count-1; i++)
{
int cellcolor = int.Parse(xmlnode[i].ChildNodes.Item(0).InnerText.Trim());
int textcolor = int.Parse(xmlnode[i].ChildNodes.Item(1).InnerText.Trim());
if (cellcolor != 0)
{
dgv.Rows[i].DefaultCellStyle.BackColor = Color.FromArgb(Convert.ToInt32(cellcolor));
dgv.Rows[i].DefaultCellStyle.ForeColor = Color.FromArgb(Convert.ToInt32(textcolor));
}
else
{
dgv.Rows[i].DefaultCellStyle.BackColor = Color.White;
dgv.Rows[i].DefaultCellStyle.ForeColor = Color.Black;
}
}
fs.Close();
}
How can I solve this problem?
modified 21-Apr-21 11:18am.
|
|
|
|
|
What you said is: it doesn't work; filter or no filter.
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
Filtering causes redrawing of all the rows and my color information is lost.
Of course, I need to use a hidden column for assigning an index code for each row, so I save the index color and load it. In this way, I can relocate those color information when using filtering.
|
|
|
|
|
I'm "guessing" that filtering is creating a "new" uncolored view; you might try and (re)color after applying the filter.
(It's a guess, so the troll need not get excited)
It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food
|
|
|
|
|
Did you say that I'm trolling?
I just want to get help and solve my problem. It's all.
|
|
|
|
|
No, he didn't call you a troll.
He's making a comment in an attempt to head off the trolls.
|
|
|
|
|
thank you for help. the previous explanation does make sense but when about instance how yo accesss it from other class? thank you
class felhasznalo
{
adatlista adat = new adatlista();
foreach (string elem adat.lista)
{
}
}
|
|
|
|
|
Every time you "new up" adatlista, you are creating a new list, separate from any other instance.
So, if you new up two lists and add items to the one of them, you will NOT see those items in the other list.
|
|
|
|
|
thank you for help. I don't want to create new list just access the existing one from other classes possibly with foreach or for loop, recoed by record, thank you.
|
|
|
|
|
You might want to try something like this:
class felhasznalo
{
private adatlista adat = new adatlista();
public adatlista items { return adat; }
}You can now access adat from outside your class.
The simple idea behind this concept is that you are exposing a property that allows external access to members of your class, but does it in a controlled manner. This is why I wrapped this in a property. You could, just make adatlista a public field and allow classes access to the public field but that's a separate debate.
modified 21-Apr-21 3:13am.
|
|
|
|
|
Hi
thank you for help.
In my modified instance I placed recordlist is a separate class to be able to access it from various classes but access issues occured. please help me to access the record list type thank you.
<pre>using System;
using System.Linq;
using System.IO;
using System.Text;
using System.Collections.Generic;
namespace metjelentes
{
class adatok
{
public string telepules = " ";
public string ido = " ";
public string szeliranyes_erosseg = " ";
public int homerseklet = 0;
}
class adatlista
{
public List<adatok> lista = new List<adatok>();
}
class beolvasas
{
public void beolvas()
{
string olvas = @"c:\Users\Public\textfiles\tavirathu13.txt";
using (StreamReader sr =new StreamReader(olvas, Encoding.Default))
{
int db = 0;
while (!sr.EndOfStream)
{
string sor = sr.ReadLine();
string[] elemek = sor.Split(' ');
adatlista.lista.Add(new adatok());
adatlista.lista[db].telepules=elemek[0];
adatlista.lista[db].ido = elemek[1];
adatlista.lista[db].szeliranyes_erosseg = elemek[2];
adatlista.lista[db].homerseklet = Int32.Parse(elemek[3]);
db++;
}
Random rd = new Random();
int rand_num = rd.Next(1, db);
}
}
}
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
}
}
}
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
