Re: C# How to speed up for loop with large data iteration - C# Discussion Boards

I know the program not throwing error and compile properly. the problem is i am expecting that when program will run then 1,2,3....5 should be showing in textbox but it is not occurring properly. sir if you run the code then you will see 1,2,3 not showing in textbox one after one after delay. so i confused when program not behaving properly. please help me and tell me what mistake i made there for which digits not showing in textbox one after one gradually. thanks

Why on earth would you expect it to do that?
You are starting 5 identical tasks, each of which show a message in the same box, wait the same amount of time, and clear it.

Since all the tasks are the same, and they all output to the same textbox, all you will ever see is the final output: whichever task finished last - and that'll be random because they all have the same priority and run in different threads.
Why?
Because they all wait the same time: so after the same 90 seconds they all try to run and all update the same text box. Last one "wins" and that's all you see.

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

Thank you sir pointing the problem.

A BGW makes that simple. Even got examples on MSDN.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

I am iterating in 2509693 data in for loop which is taking long time. i am using VS2013 and .Net v4.5.2

so see my code and suggest some approach or changes for my existing code which speed up execution of my for loop.

This ds.Tables[1] has 2509693 data please guide me how to restructure & speed up below code. Thanks

C#

public static List<ElementHierarchy> GetElementHierarchy(DataSet ds)
 {
     List<ElementHierarchy> _ElmHierarchy = new List<ElementHierarchy>();
    
     string StrPrevDisplayInCSM = "", DisplayInCSM = "", Section = "", LineItem = "", LastGroupName = "", BGColor="",
         BlueMatrix1stElementFormulaText = "", Type = "", Period = "", EarningsType = "", ParentGroup = "", HeadingSubheading = "", Box="";
     int row = 6, EarningID = 0, LineItemID = 0, BMID = 0, ID = 0, ParentID=0;
     bool IsNextElementGroup = false;
     List<ListOfSection> lstData = new List<ListOfSection>();
     bool IsGreenHeader = false;
    
     for (int p = 0; p <= ds.Tables[1].Rows.Count - 1; p++)
     {
         ID = Convert.ToInt32(ds.Tables[1].Rows[p]["ID"].ToString());
         ParentID = Convert.ToInt32(ds.Tables[1].Rows[p]["ParentID"].ToString());
            
         EarningID = 0; 
         Section = (ds.Tables[1].Rows[p]["Section"] == DBNull.Value ? "" : ds.Tables[1].Rows[p]["Section"].ToString());
         LineItem = (ds.Tables[1].Rows[p]["LineItem"] == DBNull.Value ? "" : ds.Tables[1].Rows[p]["LineItem"].ToString());
         DisplayInCSM = ds.Tables[1].Rows[p]["DisplayInCSM"].ToString();
         Type = ds.Tables[1].Rows[p]["Type"].ToString();
         BlueMatrix1stElementFormulaText = (ds.Tables[1].Rows[p]["BlueMatrix1stElementFormulaText"] == null 
             ? "" : ds.Tables[1].Rows[p]["BlueMatrix1stElementFormulaText"].ToString());
         Period = (ds.Tables[1].Rows[p]["Period"] == DBNull.Value ? "" : ds.Tables[1].Rows[p]["Period"].ToString());
         HeadingSubheading = (ds.Tables[1].Rows[p]["HeadingSubheading"] == null ? "" : ds.Tables[1].Rows[p]["HeadingSubheading"].ToString());
         Box = (ds.Tables[1].Rows[p]["Box"] == DBNull.Value ? "" : ds.Tables[1].Rows[p]["Box"].ToString());
         LineItemID = Convert.ToInt32(ds.Tables[1].Rows[p]["LineItemID"].ToString());
         BMID = Convert.ToInt16(ds.Tables[1].Rows[p]["BMID"].ToString());
         BGColor = (ds.Tables[1].Rows[p]["BGColor"] == null ? "" : ds.Tables[1].Rows[p]["BGColor"].ToString());
    
    
         if (BGColor.Contains("ff003300"))
         {
             IsGreenHeader = true;
         }
         else
         {
             IsGreenHeader = false;
         }
    
         if (StrPrevDisplayInCSM != "" && StrPrevDisplayInCSM != DisplayInCSM && (Type == "LINEITEM" || Type=="BM"))
         {
             row++;
         }
    
         if (Type == "GROUP")
         {
             if (IsNextElementGroup)
             {
                 row++;
             }
             else if (p > 0 && !IsNextElementGroup)
             {
                 row++;
    
                 if (p > 0 && HeadingSubheading=="H")
                 {
                     row++;
                 }
    
                 if (p > 0 && HeadingSubheading == "S")
                 {
                     row++;
                 }
             }
             else if (p > 0 && IsGreenHeader)
             {
                 row++;
             }
             else if (p > 0 && ds.Tables[1].AsEnumerable().Any(a => a.Field<int>("ParentID") == ID && a.Field<string>("Type") == "GROUP"))
             {
                 row++;
             }
    
             ParentGroup = DisplayInCSM;
    
             if (HeadingSubheading != "")
             {
                 if (HeadingSubheading == "H")
                 {
                     if (Box != "Y")
                     {
    
                     }
                 }
             }
    
             if(IsGreenHeader)
             {
                 row++;
             }
             else if (ds.Tables[1].AsEnumerable().Any(a => a.Field<int>("ParentID") == ID && a.Field<string>("Type")=="GROUP"))
             {
                 row++;
             }
             IsNextElementGroup = true;
    
         }
         else if (Type == "LINEITEM")
         {
             if (!lstData.Any(a =>
                 a.Section == Section
                 && a.LineItem == LineItem
                 && a.Parent == ParentGroup
                 && a.DisplayINCSM == DisplayInCSM
                 && a.EarningsID == EarningID
                 && a.EarningsType == EarningsType
                 && a.Period == Period
                 ))
             {
    
                 if (!_ElmHierarchy.Any(z => z.RowIndex == row))
                 {
                     _ElmHierarchy.Add(new ElementHierarchy
                     {
                         ID=ID,
                         ParentID=ParentID,
                         RowIndex = row,
                         Section = Section,
                         Lineitem = LineItem,
                         Type = "LI",
                         DisplayInCSM = DisplayInCSM,
                         BMFormula = "",
                         LineitemID = LineItemID,
                         BMID = 0
                     });
                 }
    
                 lstData.Add(new ListOfSection
                 {
                     Section = Section,
                     LineItem = LineItem,
                     DisplayINCSM = DisplayInCSM,
                     Parent = ParentGroup,
                     EarningsID = EarningID,
                     EarningsType = EarningsType,
                     Period = Period
                 });
             }
             IsNextElementGroup = false;
             IsGreenHeader = false;
         }
         else if (Type == "BM")
         {
             IsNextElementGroup = false;
             IsGreenHeader = false;
    
             if (!lstData.Any(a =>
                 a.Section == Section
                 && a.LineItem == LineItem
                 && a.Parent == ParentGroup
                 && a.DisplayINCSM == DisplayInCSM
                 && a.EarningsID == EarningID
                 && a.EarningsType == EarningsType
                 && a.Period == Period
                 ))
             {
                 if (!_ElmHierarchy.Any(z => z.RowIndex == row))
                 {
                     _ElmHierarchy.Add(new ElementHierarchy
                     {
                         ID = ID,
                         ParentID = ParentID,
                         RowIndex = row,
                         Section = Section,
                         Lineitem = LineItem,
                         Type = "BM",
                         DisplayInCSM = DisplayInCSM,
                         BMFormula = BlueMatrix1stElementFormulaText,
                         LineitemID = 0,
                         BMID = BMID
                     });
                 }
    
                 lstData.Add(new ListOfSection
                 {
                     Section = Section,
                     LineItem = LineItem,
                     DisplayINCSM = DisplayInCSM,
                     Parent = ParentGroup,
                     EarningsID = EarningID,
                     EarningsType = EarningsType,
                     Period = Period
                 });
             }
         }
    
         StrPrevDisplayInCSM = DisplayInCSM;
     }
     return _ElmHierarchy;
 }
    
    
 public class ListOfSection
 {
     public string Parent { get; set; }
     public int EarningsID { get; set; }
     public string EarningsType { get; set; }
     public string Section { get; set; }
     public string LineItem { get; set; }
     public string DisplayINCSM { get; set; }
     public string Period { get; set; }
 }

Which lines are taking the longest to execute?

The difficult we do right away...
...the impossible takes slightly longer.

I have no idea what you are trying to do or why you need to update so many items. However, Stop calling ToString on items that are already strings. And look at the following line (just one example of a few):

C#

ID = Convert.ToInt32(ds.Tables[1].Rows[p]["ID"].ToString());

Why are you converting a value to a string, just so you can convert it to an integer?

But you need to understand that processing just over 2.5 million items will take some time. The laws of physics are immutable.

Richard is right:

Quote:
Which lines are taking the longest to execute?

And so is Richard:

Quote:
you need to understand that processing just over 2.5 million items will take some time.

The first part of any speed improvement exercise is to find out what you have: and that means timing your code to find out what takes a "long time" and what is "quick" - there is no point in trying to squeeze improvements out of fast code because that will make marginal difference; the slow code is teh bit that needs speeding up.
So start with the Stopwatch Class (System.Diagnostics) | Microsoft Docs[^] and start finding out where that code runs quick, and where it runs slow.
We can't do that for you: we don't have any access to your data, and it's likely to be very relevant to any speed analysis.

When you have that, you also have a metric for much improvement you are making as you go.

It's possible that you might be able to multithread it, to spread the load out over the whole processor - but that's not as simple as it may sound: although thinks like Parallel.Foreach exist, using threading blindly is as likely to slow down your whole computer as speed up processing, particularly when you start dealing with 2,000,000 items each of which probably don't take much time so the additional thread setup / processing overhead starts to overweigh the actual task.
You may get some good "throughput" improvements by splitting the for loop into a small number N of threads, each of which does 1/N th of the total task. But remember: List and DataTable are not considered thread safe, so you'll probably need to think very carefully before you start coding that if you don't want really hard to spot and track down data bugs.
As a general rule, I wouldn't recommend trying to use more threads than you have cores ...

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

Since it involves a dataset / datatable, I'm guessing there's a database and an SQL query out there that you should be focusing on instead.

It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food

I've had similar situations - large (but not a large as this requesters) datasets going to grid views and have timed them. In my cases, issuing the SQL and getting results was quite quick. The delays was filling in and displaying the grid view.

Things to try (which I admit I haven't actually tried in my cases)
* Switch off rendering of the form whilst writing data to the grid view and switch on once all of the grid view has been populated
* Do partial updates (e.g. write 1000 rows, update the UI, append 1000 more rows, update the UI, etc). This would actually slow down the changes but would appear to the user that things were happening and he / she could do some work with the displayed data whilst the rest was being processed. The only way I know of to do partial updates includes using Application.DoEvents which is not a good method; I am sure that someone here could suggest a better way.

OR: Target a subset of the data, Allow the user to request data by whatever criteria is relevant. We (by default) only supply the current day's data. Users soon learn that if they want more, they will have to ask for it (filling in more selection criteria fields), and they will have to wait longer as a consequence.

He's not displaying at this point ... he's doing "database operations" on the client.

But, yes, comparatively speaking, the UI is "slow", and will slow down your app if you do UI updates synchronously.

It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food

Thanks - my mistake. It was the business about green headers etc that I had misinterpreted.

I'm wondering if there is a way other than executing cmd commands in my windows form app to enable or configure an ad hoc network in windows 10?
I'm looking for a direct way like using windows's own libraries which the cmd command for ad hoc (netsh wlan) uses to configure the network.

Note: I don't want to execute cmd commands in my app.

Any help will be appreciated.

I think using the "network" classes (Ping; TcpClient; etc) in the .NET framework would be easier than trying to figure out what the Windows API equivalent of a command line utility is.

You need to be more specific about what you mean by "configure".

It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food

How can I show the sql server 2019 instances in a combo box with C #?

See here: SmoApplication.EnumAvailableSqlServers Method (Microsoft.SqlServer.Management.Smo) | Microsoft Docs[^]

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

A Google would have been quicker. DuckDuckGo would have been quicker.

This is not a tech-secret or anything comparable, there's quite a lot of copy and paste code out there doing just what you asked.

And you didn't find it.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

student management system program in csharp console app

Uhhh... WHAT?

If you're looking to have an app written for you, you've come to the wrong site.

Asking questions is a skill
CodeProject Forum Guidelines
Google: C# How to debug code
Seriously, go read these articles.

Dave Kreskowiak

We are more than willing to help those that are stuck: but that doesn't mean that we are here to do it all for you! We can't do all the work, you are either getting paid for this, or it's part of your grades and it wouldn't be at all fair for us to do it all for you.

So we need you to do the work, and we will help you when you get stuck. That doesn't mean we will give you a step by step solution you can hand in!
Start by explaining where you are at the moment, and what the next step in the process is. Then tell us what you have tried to get that next step working, and what happened when you did.

If you are having problems getting started at all, then this may help: How to Write Code to Solve a Problem, A Beginner's Guide[^]

"I have no idea what I did, but I'm taking full credit for it." - ThisOldTony
"Common sense is so rare these days, it should be classified as a super power" - Random T-shirt
AntiTwitter: @DalekDave is now a follower!

Buying or selling?

It was only in wine that he laid down no limit for himself, but he did not allow himself to be confused by it.
― Confucian Analects: Rules of Confucius about his food

Really? When you are hungry do you just grunt "food" at anyone?

WTF | :WTF:

Yes. Twice Big Grin | :-D

"Console app" means homework; no one use the console for presentation, only for excercises to cut out complexity.

Don't get me wrong, I all in favor of posting your homework questions here, if you have questions. Just don't expect us to cheat for you.

Also, even with our help and cheating, this would take you some hours of work to bring it together. I try to encourage people to let us help with homework; meaning this site has enough pro's that can point you in a good direction; but we will not do what is assigned to you.

We can point, assist, show documentation, or ask what your code does. But there it stops.

So, from your question; my only reasonable response is you in trouble and not up to the task. That is even with international help, you still screwed.

Bastard Programmer from Hell Suspicious | :suss:

"If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.

Drop the course and look for something more in line with your skills and interest.

Never underestimate the power of human stupidity -
RAH
I'm old. I know stuff - JSOP

Good morning. As you may have gathered by the replies you have already had here, people are less than impressed with a question that appears to be asking for someone to give you a working program. In general, the people who answer questions here are professional programmers who earn their living writing code. You wouldn't go up to a plumber and say "leaking pipe in bathroom" and expect them to fix it for you, would you?

Let us put this one down to youthful folly and try to give you a bit of a hint as to what you should be doing. The first thing you have to consider is that your "question" is just a statement. It sketches out the bare minimum of information, namely what the title is and the language/library type behind it is. I can't believe that this is all you got but, if it is, you need to go back to the person who asked you for this and start asking questions.

As I said, the people who answer questions here are, generally, professional programmers. This means that they all approach program development using roughly similar techniques. The first thing that they will all do is look at what the requirements are - this is the starting point for any development - and look to see what they can know as facts in there, and where there are gaps that require assumptions to be made. Where there are assumptions, the professional developer will go back to the origin of the requirements and ask them questions to work out whether those assumptions are correct, or whether something else was meant. Here's a practical example of a requirement that leads to assumptions. First the requirement:

As a Business Owner
I want the customers details saved
So they can be retrieved later on

This is a typical requirement that raises a lot more questions than are answered and, as a developer, the first thing you need to do is identify the questions in there and get answers to them. Here are a list of some of the questions that this requirement pose.

What make up the customer details?
Can the record be saved regardless?
If we need to validate the record before saving, what are the rules?
When we say saved here, what does this mean? If we're saving this into a reporting system on AWS, this gives a different meaning to saving?

So, what are your next steps? Well, dare I say it, your next step is to start acting like a programmer. Write down what you know and what the assumptions are. Seek clarifications. Once you know what all your requirements are, it becomes an easier matter for you to start solving the problem you were presented with. All programming is, is breaking big problems down into small logical problems and then solving those small problems. It requires thought and analytical skills but now is the time to learn how to do this.

Advanced TypeScript Programming Projects