Multithreading, reading seperated from calculating

Question

1.00/5 (1 vote)

See more:

Hi everybody,

This program calculates all the bank account numbers from 9 digits. and all the bank account numbers are written to the file system. There are 82 files and each file has around 1000 000 records(lines). But how to seperate the Calculating of the bank account numbers and the writing to the file system with multithreading. I am trying it like this:

I have this:

C#

public class Program
    {
        public int id;
        public Program(int _id)
        {
            this.id = _id;

        }

        static void Main(string[] args)
        {

            string path = @"C:\BankNumber";
            var bans = BankAcoutNumbers.BANS;
            const int MAX_FILES = 82;
            const int BANS_PER_FILE = 81818182 / 82;
            int bansCounter = 0;
            var part = new List<int>();
            var maxNumberOfFiles = 10;
            Stopwatch timer = new Stopwatch();
            var fileCounter = 0;





            if (!Directory.Exists(path))
            {
                DirectoryInfo di = Directory.CreateDirectory(path);
            }

            try
            {
                while (fileCounter <= maxNumberOfFiles)
                {
                    timer.Start();
                    Console.WriteLine("Start writing the bank numbers to file system");
                    // Console.WriteLine(timer.Start());
                    foreach (var bank in BankAcoutNumbers.BANS)
                    {
                        part.Add(bank);
                        if (++bansCounter >= BANS_PER_FILE)
                        {
                            string fileName = string.Format("{0}.txt-{1}.txt", part[0], part[part.Count - 1]);
                            //string outputToFile = "";// Otherwise you dont see the lines in the file. Just single line!!
                            StringBuilder OutputFile = new StringBuilder("");


                            string subString = System.IO.Path.Combine(path, "nr");//Needed to add, because otherwise the files will not stored in the correct folder!!
                            fileName = subString + fileName;

                            foreach (var partBan in part)
                            {
                                OutputFile.Append(string.Format("{0}{1}", partBan.ToString(), Environment.NewLine));


                            }
                            //Writes to file system.
                            System.IO.File.WriteAllText(fileName, OutputFile.ToString());
                            part.Clear();
                            bansCounter = 0;
                            ThreadPool.QueueUserWorkItem(DoLongTask);


                            if (++fileCounter >= MAX_FILES)
                                break;
                        }
                    }
                }

                timer.Stop();
                Console.WriteLine("Total time of writing the bank numbers to file system " + timer.Elapsed.TotalSeconds + " seconds");
                //Console.WriteLine(BankAcoutNumbers.BANS.Count());
            }
            catch (Exception)
            {

                throw;
            }
            //Console.WriteLine("Main thread ends");            
            System.Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();
        }

        public static void DoLongTask(object input)
        {
            //int index = (int)input;
            Console.WriteLine("Created file : {0}", Thread.CurrentThread.IsAlive);
        }

    }</int>

Thank you

oke, I try it like this:

C#

public class Program
    {
        public int id;
        static List<int> part = new List<int>();
        static int bansCounter = 0;
        const int BANS_PER_FILE = 81818182 / 82;
        static string path = @"C:\BankNumber";
        const int MAX_FILES = 82;
         
        static int fileCounter = 0;
        static ReaderWriterLockSlim _rw = new ReaderWriterLockSlim();
        //public Program(int _id)
        //{
        //    this.id = _id;

        //}


        static void Main(string[] args)
        {
            if (!Directory.Exists(path))
            {
                DirectoryInfo di = Directory.CreateDirectory(path);
            }

            new Thread(Read).Start();
            new Thread(Read).Start();
            new Thread(Read).Start();

            new Thread(Write).Start("A");
            new Thread(Write).Start("B");


            var bans = BankAcoutNumbers.BANS;
            System.Console.WriteLine("Press any key to exit.");
            System.Console.ReadKey();

        }

        static void Read()
        {
            while (true)
            {
                _rw.EnterReadLock();
                foreach (int i in part ) Thread.Sleep(10);
                _rw.ExitReadLock();
            }
        }

        static void Write(object threadID)
        {
            while (true)
            {

                foreach (var bank in BankAcoutNumbers.BANS)
                {
                    part.Add(bank);
                    if (++bansCounter >= BANS_PER_FILE)
                    {
                        string fileName = string.Format("{0}.txt-{1}.txt", part[0], part[part.Count - 1]);
                        //string outputToFile = "";// Otherwise you dont see the lines in the file. Just single line!!
                        StringBuilder OutputFile = new StringBuilder("");


                        string subString = System.IO.Path.Combine(path, "nr");//Needed to add, because otherwise the files will not stored in the correct folder!!
                        fileName = subString + fileName;

                        foreach (var partBan in part)
                        {
                            OutputFile.Append(string.Format("{0}{1}", partBan.ToString(), Environment.NewLine));


                        }
                        //Writes to file system.
                        System.IO.File.WriteAllText(fileName, OutputFile.ToString());
                        part.Clear();
                        bansCounter = 0;
                        //ThreadPool.QueueUserWorkItem(DoLongTask);


                        if (++fileCounter >= MAX_FILES)
                            break;
                    }

                _rw.EnterWriteLock();              
                _rw.ExitWriteLock();
                Console.WriteLine("Thread " + threadID + " added " + part);
                Thread.Sleep(100);
                }            
            }
        }

      
    }

But I get this error:

Additional information: Collection was modified; enumeration operation may not execute.

on this line:

foreach (int i in part ) Thread.Sleep(10);

Posted 25-Jan-15 23:50pm

Member 10225052

Updated 26-Jan-15 6:08am

v4

Add a Solution

Comments

Leo Chapiro 26-Jan-15 5:56am

And what is your question?

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

E.F. Nijboer · Answer 1 · 2015-01-25T23:56:00

Solution 1

Sounds like producer-consumer. Using c#5.0 you can utilize async/await. Have a look at this link for an example:
https://msdn.microsoft.com/en-us/library/hh228601%28v=vs.110%29.aspx[^]

Good luck!

Posted 25-Jan-15 23:56pm

E.F. Nijboer

Comments

[no name] 26-Jan-15 6:03am

Thank you. But how to implement that in the code? Thank you

E.F. Nijboer 26-Jan-15 8:25am

Hopefully you understand I'm not going to rewrite your code. The link shows how to do it. Please try yourself and if you get stuck you can ask again here on codeproject.

[no name] 26-Jan-15 12:09pm

I updated my post

E.F. Nijboer 26-Jan-15 12:17pm

Use Concurrent collections instead:
https://msdn.microsoft.com/en-us/library/system.collections.concurrent%28v=vs.110%29.aspx

[no name] 26-Jan-15 12:39pm

? and what I have to do with: foreach (int i in part)
{
Thread.Sleep(10);
}

E.F. Nijboer 26-Jan-15 14:11pm

You can simply use the concurrentbag for example and add items safely. You could also have a look at the blockingcollection because it has examples how to implement the producer/consumer pattern. It just takes some time to fully understand and get good experience with threading. Simply try the examples shown and go from there. Also, the Sleep method isn't really helpfull in normal concurrent programming. You should avoid it because it is for real low level use only. Maybe one exception is for test purposes to simulate long running tasks. Also, just keep thinking like how it would be in the real world. Sleep won't help with 2 or more people from accessing some resource. Therefore some locking mechanism needed to protect a shared resource. Have a look at the dining philosophers problem (http://en.wikipedia.org/wiki/Dining_philosophers_problem) for example. There are also a lot more examples to find from there.

_Asif_ · Answer 2 · 2015-01-26T00:15:00

You can start with very simple design like below pseudo code and can add multi-threaded capabilities later.

IBankManager
|
| List<string> Calculate()
|->AccountNumberManager
|
| List<string> Calculate()

IOutputManager
|
|-> Write(List<string>)
|
|->FileOutputManager
-> Write(List<string>)

C#

void main()
{
   IBankManager accountNumberManager = new AccountNumberManager()
   IOutputManager fileManager = new FileOutputManager()
   List<string> list =  accountNumberManager.Calculate();
   fileManager.Write( list );
}</string>