Re: slow listview refresh - C# Discussion Boards

15-Oct-04 6:49

I have a listview in detail mode which I use to display a subset of records from an SQL table. If the user changes the subset parameter I refresh the list with another set of lines.
My problem is that while the original population of the list is very fast (it is done in the dialog's constructor), the refreshes are quite slow - about 10 seconds for a 500 line data set.
I know the problem lies with the listview since if I comment out the listview code, the method executesd quite fast. I have tried:
begin/end update
setting enable=false and then true
setting visible=false and then true
None seem to have more than a small effect (about 10% difference).

Any suggestions?

the method's code:

...<br />
sqlCmd.CommandType = CommandType.StoredProcedure;<br />
SqlDataReader reader = sqlCmd.ExecuteReader();<br />
<br />
lvwLogDays.Items.Clear();<br />
      <br />
while( reader.Read() )<br />
{<br />
	DateTime Date = reader.GetDateTime( reader.GetOrdinal( "LogDate" ) );<br />
	string strTemp = Date.ToString( "MM/dd/yyyy" );<br />
	OpenDayLogData	DayLogData = new OpenDayLogData();<br />
<br />
	ListViewItem item = lvwLogDays.Items.Add( strTemp );<br />
        strTemp = reader["CallLetters"].ToString().Trim();<br />
        item.SubItems.Add( strTemp );<br />
<br />
	strTemp = reader["LogStatus"].ToString().Trim();<br />
        item.SubItems.Add( strTemp );<br />
<br />
	DayLogData.LogID = Convert.ToInt32(reader["DF_LogID"].ToString(), 10);<br />
	DayLogData.LogDayID = Convert.ToInt32(reader["DF_LogDayID"].ToString(), 10);<br />
        item.Tag = DayLogData;<br />
}<br />
...

Heath Stewart15-Oct-04 7:00

15-Oct-04 7:00

Calling ListView.BeginUpdate before clearing and adding items, and ListView.EndUpdate should help performance quite a bit.

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]

DF_Bandit15-Oct-04 7:10

15-Oct-04 7:10

Actually I have the method (and one that follows this method - a sorter) bracketed by BeginUpdate() and EndUpdate() to no avail. As I wrote in the original, there seems to be about a 10% gain by using such a technique.

so the code goes something like this:
lvwLogDays.BeginUpdate();
RefreshList();
SortList();
lvwLogDays.EndUpdate();

Optionally, I have put lvwLogDays.Enabled=false/true bfroe/after the update calls.

Heath Stewart15-Oct-04 8:44

15-Oct-04 8:44

You could also get a pretty good gain if you fetch all records first (say, into a DataSet) and then build your list. This really depends on your wire speed and format, though (whether or not you'll see any gains).

One other thing is to make sure that your ListView doesn't specify a sort already if you're going to sort it later a la SortList. Do one or the other, but not both. Sorting the list after is most often faster when you have a large number of items.

Is the code you use for the first and future operations exactly the same (with the call to Clear), i.e. you're using a DataReader to fetch the rows, etc.?

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]

DF_Bandit15-Oct-04 9:39

15-Oct-04 9:39

Thanks for the help.

I did replace the reader with a load into a DataTable and then 'foreached' out of it - no noticible difference in speed.

I have ensured the listview does not sort. We opted to sort after so we can perform sorts on the various columns if the user clicks a column head.

The method I cliped from (LoadLogList) is used both by the dialog's constructor and then by the refresh method, so it is used in 'first and future operations'. So I call the Clear() when I know the listview's item list is empty. I just moved the clear out of the method - no noticable difference.
I now do this:

lvwLogDays.Enabled=false;<br />
lvwLogDays.BeginUpdate();<br />
lvwLogDays.Items.Clear();<br />
LoadLogList();<br />
Resort( 0 );<br />
lvwLogDays.EndUpdate();<br />
lvwLogDays.Enabled=true;

Heath Stewart15-Oct-04 15:42

15-Oct-04 15:42

Out of curiousity, are you by changing collecting data in a separate thread other than the one the ListView was created? If so, undefined problems can result. In such a case, you need to use Control.Invoke. The SDK documentation gives clear details and examples.

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]

afinnell18-Oct-04 6:17

18-Oct-04 6:17

This sorta reminds me of something I just came across also.

I really like the way items are added to the list when doing a Find in Microsoft Outlook 2003. The UI is very responsive, the CPU usuage is low and the items get added quickly. It would be nice if this was possible in .NET.

However, when iterating over items and adding them to a listview in .NET it is very slow. I have another thread that adds the items so that the main UI thread isn't blocked. I did use Control.Invoke to add each of the items and that was a bad idea to do an Invoke for each item that is to be added. It completely bogged down the message pump and locked up the UI.

If anyone has any suggestions I'm all ears. I believe the problem lies in the synchronization and invokations in .NET that need to occur when adding items to a list view in a seperate thread.

BeginUpdate and EndUpdate wasn't really an option for me becaues that prevents the UI from being drawn. The search process I am performing can take some time so I want to provide immediate feedback about items that I have found.

-
Drew

Heath Stewart18-Oct-04 6:46

18-Oct-04 6:46

Thread synchronization isn't a problem specific to .NET. Remember that most of the Windows Forms controls simply encapsulate the Windows Common Controls. This is a "problem" (actually, it makes a lot of sense if you understand threading in Windows; other platforms have these restrictions as well) with native threading.

To communicate with a control you must communicate with the control on the thread on which it was created. If you don't, undefined (i.e., unpredictable) results can happen. Some times you may not notice a problem. Other times it may appear to work but in actually you'll get memory leaks. Most times it just won't work.

If you'd rather batch your updates, then define your own method on your derivative ListView that, perhaps, takes an array or collection of ListViewItems. Gather X-number of ListViewItems then call Invoke using a delegate for the method defined above.

Like it or, at some point your UI will seem to lag but making your code effectient as possible won't be noticable - or at least as noticable - on most systems.

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]

afinnell18-Oct-04 7:50

18-Oct-04 7:50

This isn't really a thread synch problem, but a .NET invoke synch problem. In a native application you would normally call SendMessage to add items. This will handle synching the messages from a thread to the UI thread.

However in .NET a little more has to happen. A Control.Invoke needs to happen to dispatch a delegate. In the delegate you then need to invoke the Items.Add that in turn calls the SendMessage because the basic .NET controls wrap the Win32 API. This being the case it involves more calls then a native app invoking SendMessage. It involes two synch calls (Control.Invoke and SendMessage) and a marshalled call (The SendMessage).

I wonder how the performance of a native app would peform when adding many items in real-time to a list view. Microsoft Office 2003 does a great job of it when using the find feature so I know it can be done.

-
Drew

Heath Stewart18-Oct-04 10:06

18-Oct-04 10:06

I didn't mean thread sync in the traditional sense, but synchronization with threads as you said.

The fact that adding items uses SendMessage is something I know well - almost everything you call on controls in Windows Forms calls SendMessage. It is for this reason - as I explained before when talking about the encapsulation of the Windows Common Controls - that you must use Control.Invoke. Believe me, I know.

The performance issues all come down to the efficiency of your code. I could build a massive (around 2,000 nodes on average) tree (data-fead, polymorphic design) in just a couple of seconds from a DataSet. Adding ListViewItems to a ListView wasn't really a problem either.

Keep in mind that the original poster is reading data for the ListViewItems from a DataReader, which must be connected since it pulls results across the wire record-by-record, which is closer to the traditional ADO record sets than ADO.NET's DataSet.

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]

afinnell18-Oct-04 11:11

18-Oct-04 11:11

I didn't mean thread sync in the traditional sense, but synchronization with threads as you said.

Yeah I read too much into that statement. I should have known that is what you meant.

The fact that adding items uses SendMessage is something I know well - almost everything you call on controls in Windows Forms calls SendMessage. It is for this reason - as I explained before when talking about the encapsulation of the Windows Common Controls - that you must use Control.Invoke. Believe me, I know.

To be fair I wasn't trying to educate you on the internals of the common controls in .NET but rather explain what is happening so that others that read the posts, that might not know, can have a better understanding. I've read quite a bit of your posts and you are a regular contributor to Code Project, plus you work at Microsoft. It was in no way an attempt to educate you on the internals but more of a reference for someone who doesn't know.

The performance issues all come down to the efficiency of your code. I could build a massive (around 2,000 nodes on average) tree (data-fead, polymorphic design) in just a couple of seconds from a DataSet. Adding ListViewItems to a ListView wasn't really a problem either.

The number of items I expect to add are from 1,000 - 70,000. For the cases that I need to display 70,000 and I already know ahead of time what they are I use virtual listview so that it doesn't have to create or load each item until it is displayed on the screen. But that doesn't work when showing search results in real-time.

Another thing that I will bring up is that I haven't ran a profile on the code in question yet. It is always important to profile code before making assumptions on why it is so slow.

-
Drew

Heath Stewart18-Oct-04 11:41

18-Oct-04 11:41

afinnell18-Oct-04 12:03

18-Oct-04 12:03

For 70,000 you might consider a virtual list. This will be supported in .NET 2.0, which takes advantage of functionality that already exists in the List-View common control now. If you haven't downloaded 2.0 yet, you should do so (beta 1 as of this posting) and see how it works using ildasm.exe or .NET Reflector, or just read the common controls documentation

Yeah I have created a virtual listview control in .NET 1.1 It is wonderful for loading that many items but I found an issue when you want to add a single item to the list repeatably. When you set the count with the Win32 API it resets the whole list and causes a slow redraw problem again. It's what I use when I have a list of 70,000 already computed and have them available.

I will say thought that there appears to be a problem with the virtual listview on Windows 98 SE. The text of the items don't show up when using a Virtual list view. But that is a seperate problem that I had to deal with.

I have VS 2k5 Beta 1 and will definately check out how they implement the virtual list view and see if it works on Win98 SE. That would be great if it did and I could figure out what I did differently from them.

You know when VS 2k5 Beta 1 is going Gold?

-
Drew

DF_Bandit18-Oct-04 8:16

Splitter & multiple panels

18-Oct-04 8:16

Thanks Heath,

I am not sure if the data are being gathered on another thread, so I altered my code to use a delegat data formater and used the listview's Invoke method. So the code looks like this:

		<br />
protected void LoadLogList()<br />
{<br />
...<br />
SqlCommand		m_sqlCmd = new SqlCommand( "DF_Get_DayLog_List",<br />
MainForm.Instance.FDSConnection.DBConnection.Connection );<br />
<br />
m_sqlCmd.CommandType = CommandType.StoredProcedure;<br />
m_sqlCmd.Parameters.Add("@DF_TrafficID", iTrafID );<br />
<br />
SqlDataReader reader = m_sqlCmd.ExecuteReader();<br />
<br />
myMethodDelegate myD1 = new myMethodDelegate( this.LoadLogItem );<br />
object[] param = {reader};<br />
<br />
while( reader.Read() )<br />
{<br />
	lvwLogDays.Invoke(myD1, param);<br />
}<br />
<br />
reader.Close();<br />
	<br />
}<br />
<br />
public void LoadLogItem(SqlDataReader reader)<br />
{<br />
	DateTime Date = reader.GetDateTime( reader.GetOrdinal( "LogDate" ) );<br />
	string strTemp = Date.ToString( "MM/dd/yyyy" );<br />
	OpenDayLogData	DayLogData = new OpenDayLogData();<br />
<br />
	ListViewItem item = lvwLogDays.Items.Add( strTemp );<br />
<br />
	strTemp = reader["LogStatus"].ToString().Trim();<br />
	item.SubItems.Add( strTemp );<br />
<br />
	DayLogData.LogID = Convert.ToInt32(reader["DF_LogID"].ToString(), 10);<br />
	DayLogData.LogDayID = Convert.ToInt32(reader["DF_LogDayID"].ToString(), 10);<br />
	item.Tag = DayLogData;<br />
}<br />

I experienced no speed change. Am I being unreasonable to expect 500 lines to be loading into the list view in much less than Sleepy | :zzz:

10 seconds (on a 2.x GHz P4 machine, lots of RAM - nothing else - much - going on)?

DennisMetz15-Oct-04 6:47

DennisMetz

15-Oct-04 6:47

Hi,

I've been struggling with the following problem for quite some time. Google wasn't very helpfull, too... Mad | :mad:

Is there an easy way to accomplish having three (or more) panels which can be resized by a splitter? So, if you increase Panel3, Panel2 should be decreased by the same amount. This obviosly can not be done with the default splitter control, because there are more than two panels involved.

Do you have an idea how this could be done?

--------------------
|
| Panel1
|
--------------------
|
| Panel2
|
--------------------
|
| Panel3
|
--------------------

Thanks,
Dennis

Stanciu Vlad15-Oct-04 7:42

Stanciu Vlad

15-Oct-04 7:42

I think you have to use 2 spliters and dock your panels as following :

------
| Panel 1 - Dock Top
-----
<----- spliter 1
-----
| Panel 2 - Dock Center (Fill)
-----
<---- spliter 2
-----
| Panel 3 - Dock Bottom
-----

I hope you understand...
By the way... visit http://nehe.gamedev.net[^]

DennisMetz15-Oct-04 8:14

DennisMetz

15-Oct-04 8:14

I already gave that a try... it works fine for three panels. But increase that number to 5... D'Oh! | :doh:

btw, I don't know the number of panels that will be displayed. The user can determine this at runtime.

Dennis

Heath Stewart15-Oct-04 9:02

15-Oct-04 9:02

And you definitely don't want to start adding panels for each case, either. This adds a lot of overhead.

The Splitter class has both SplitterMoved and SplitterMoving events you could handle. Do handle these intelligently (i.e., writing intelligent code - not duplicating it) use the same event handler for each splitter and iterate through the splitters to change their size using the appropriate location and/or size properties:

private void resizeYoungerSiblings(object sender, EventArgs e)
{
  Splitter split = sender as Splitter;
  if (split != null)
  {
    int index = Controls.GetChildIndex(split);
    if (index >-1)
    {
      for (int i=index; i<Controls.Count; i++)
      {
        Splitter sibling = Controls[i] as Splitter;
        if (sibling != null)
        {
          // Appropriate resizing code here
        }
      }
    }
  }
}

You don't have to worry about this event handler being called for resizing the other splitters since the event is only fired by mouse events (OnMouseMove calls OnSplitterMoving).

This posting is provided "AS IS" with no warranties, and confers no rights.

Software Design Engineer
Developer Division Sustained Engineering
Microsoft

[My Articles] [My Blog]