Re: Large memory usage. - C# Discussion Boards

Luc Pattyn8-Apr-09 15:04

8-Apr-09 15:04

Hi,

Task Manager is not providing the right information. It shows how much memory an app is currently entitled to use, not how much is actually used. When a .NET app needs memory, it either looks for free memory inside the process, or asks Windows for more. When objects die, their memory is garbage collected but not necessarily returned to Windows.

One experiment you could do is this: minimize your main form, then watch Task Manager. The number will be lower now. If that reduced number stays somewhat constant over time (each time minimizing), then as a first approximation your app isn't loosing/wasting memory. If that number keeps growing, your app probably has some memory leaks.

I often include a permanent memory observer in the app itself: have a timer regularly update a label and have it show the value of Environment.WorkingSet (divide by 1024*1024 to get megabytes). That is more comfortable and more reliable than the TM numbers.

PS: large is a relative notion. If your system has ample memory and Windows is willing to give 500MB of memory to your app, then the app will use that to its advantage, i.e. the need to garbage collect will be reduced.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

- before you ask a question here, search CodeProject, then Google
- the quality and detail of your question reflects on the effectiveness of the help you are likely to get
- use the code block button (PRE tags) to preserve formatting when showing multi-line code snippets

Jacob Dixon8-Apr-09 15:10

8-Apr-09 15:10

So I open and close my window, looking to see if the memory usage gets smaller each time and stays constant?

I will try the WorkingSet tomorrow when I get to work and see how it does.

I know you have helped me before and I appreciate it!

Luc Pattyn8-Apr-09 15:15

8-Apr-09 15:15

Jacob Dixon wrote:
So I open and close my window

actually minimize and restore.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Jacob Dixon8-Apr-09 16:52

8-Apr-09 16:52

Lol thats what I meant Smile | :)

Actually I use a notifyicon and when minimizing I hide the form and restore it with a doubleclick of the notifyicon (or right-click restore). I did find one memory leak. When loading the Inventory Form(the editing of an item part), I made it use threading to load the image (due to the fact that the image could take a while to load and I didn't want the whole form freezing at this time, this is only an issue when working remotely). Now I did notice that the thread still runs on exiting everything. So I need to abort that thread no matter what during the closing event right?

I'm still learning on the progressbar for reading using the sequentialaccess for the VARBINARY column. The problem is, I only see the SequentialAccess being used with the TEXT column (BLOB). I didn't use the TEXT column.

Luc Pattyn8-Apr-09 17:17

8-Apr-09 17:17

Hi Jacob,

yes tasks that may take several hundreds of milliseconds or more should not be organized on the main or GUI thread, since that adversely affects the GUI's responsiveness.
The solution basically is using a thread, i.e. either a Thread class instance, or a ThreadPool item, or a BackgroundWorker.

Threads are either foreground or background; background means they don't prevent your app from exiting. Therefore, real background stuff, things that are relevant to the user as long as the app is running, but not when the app stops, should be organized in a background thread. Then, an explicit abort is not needed. Aborting a thread can be messy, you don't know in general what exactly it is/was doing at the moment you issue the abort command, hence a lot of objects and resources could be in an unknown state.

BackgroundWorkers are always running in the background. ThreadPool threads are not, and should never be aborted (well aborting them on app exit may be OK).

Things that are important to finish, should not be run in background, e.g. a database update. You don't want your app to exit in the midst of a DB update, so you may have to take special precautions for that.

I can't help you with the VARBINARY stuff.
I am not sure what you want about the progress bar; in general operations that you order as a single command will not allow you to see any progress, they are either ongoing or done, without any intermediate quantization.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

harold aptroot8-Apr-09 15:35

8-Apr-09 15:35

Ah the minimize trick, but I wonder, how would you do "the same" (force the program to release unused ram) for console apps and other programs without a window to minimize? Surely it's not the minimized state that is somehow magic but a call the framework does under the hood?

Luc Pattyn8-Apr-09 15:58

8-Apr-09 15:58

harold aptroot wrote:
how would you do "the same"

I would not. As I said, when relevant, I install a label (*) that permanently shows the working set, and I don't minimize; I only tell the minimize trick to put people's mind at ease after they got worried by Task Manager.

(*) Actually, the label often is a status bar showing working set, number of GDI objects, current time, and whatever else I consider relevant.

Furthermore, I seldom use Console apps, when I do they are short, simple and short lived. And my windowless Windows apps have very constrained functionality and a very stable memory need.

Having a solid background in real-time and limited-resource systems, I tend to be aware of the objects I create; but then I seldom use third-party libraries, I oversee all the software on a system, and I don't just throw things around without caring at all.

Knowing the large-object-heap never gets compacted, I am fully aware the GC has its limitations, and you may well end up fragmenting your memory, so you may get an OutOfMemoryException even when Task Manager would tell you you are hardly using any memory at all.

For testing purposes, it makes a lot of sense to have a "fast forward" mode, where the app creates objects as intended, but speeds or skips its business logic, so you can simulate a long lasting exceution in a matter of minutes.

In the end, building a reliable and long-lived app with strong variations in memory needs remains a challenge under .NET

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

harold aptroot8-Apr-09 16:37

8-Apr-09 16:37

Ok, then let me ask it like this, how should I do "the same"?
Putting peoples mind at ease is of course far more important than actual memory usage - they will believe TaskManager sooner than some label you put somewhere (it could be lying, right? and it's not in TaskManager's interest to do so - how many people don't think like this?)

I have a bit of a background in z80 assembly programming (TI 83/84 calcs during highschool), so I tend to be careful as well Smile | :)

(except in my article, I just messed around there Laugh | :laugh:

)
But I still want to know, you know, the minimize trick makes me curious as to how it manages to do whatever it is it's doing - and what is is doing? Forcing a GC? Releasing unused pages? Something else?

Luc Pattyn8-Apr-09 17:05

8-Apr-09 17:05

Hi Harold,

that's an excellent question, and I don't know the answer for a fact. My best guess is Windows, not the app, decides it is going to take away as much as possible the resources the app has, so they become available for other processes, since, well you did minimize, showing a disinterest.
I'm not sure at all whether the GC is activated (that might take too long), maybe/probably all that happens is that memory pages that are known to not contain live data (the far end of the small objects heap, and maybe the gaps in the large-object-heap) are recovered, directly lowering the working set as shown by TM. If all or most of the above is correct, the app has no say (unless it is capable and willing of minimizing itself).

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Dave Kreskowiak8-Apr-09 17:47

Dave Kreskowiak

8-Apr-09 17:47

harold aptroot wrote:
then let me ask it like this, how should I do "the same"?

You don't. You're worried about something that happens completely automatically. The .NET CLR does a very good job of managing memory. If your code frees a bunch of objects, the memory isn't freed immediately. The Garbage Collector gets around to freeing them eventually, whenever it needs to. When an object is finally freed, the memory is returned to the Managed Memory Pool. The reason it doesn't get returned to Windows immediately is because future allocations are faster out of the Managed Pool if there is sufficient memory in the pool. If not, the memory manager has to go to Windows to allocate another block of memory, add it to the pool, then allocate the new object. Now, if Windows needs the memory back, the .NET CLR is more than happy to remove any free memory from the Managed Pool and return it to Windows - entirely automatically. You don't have to do anything.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

harold aptroot9-Apr-09 4:59

9-Apr-09 4:59

I know we shouldn't, but mostly I'm just curious Smile | :)

Ok so basically what you say here, when you minimize a program, windows will dump part of it in the swap file. Correct?
What happens if you don't have a swap file, like me? Nothing?

Dave Kreskowiak9-Apr-09 13:16

Dave Kreskowiak

9-Apr-09 13:16

harold aptroot wrote:
What happens if you don't have a swap file, like me?

Why would you not run with a swap file?? You're only hurting performance doing that.

harold aptroot wrote:
when you minimize a program, windows will dump part of it in the swap file. Correct?

Like Senthil said, Windows will try to free up an much physical RAM as possible by swapping out pages not being used, not not free either, to disk. If there is no swap file, the stuff stays in memory, reducing the amount of RAM Windows can use for caching and buffers.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007, 2008

harold aptroot9-Apr-09 13:38

9-Apr-09 13:38

Dave Kreskowiak wrote:
Why would you not run with a swap file?? You're only hurting performance doing that.

I have 4GB ram, swap space is not needed, the reduced amount for caching and buffers is usually between 3 and 3.5 GB - so there is really no need for a file of at least 4GB (but defaulting to 6GB) that will cause extra disk-rattling
Going lower than 4GB gives sporadic warnings - I wouldn't mind having a 128MB pagefile or so, but windows complains
edit: actually it seems to be slower with a swap file

And this is using XP of course, I wouldn't touch Vista with a 100yard pole - I heard Vista has more trouble without a swap file

S. Senthil Kumar8-Apr-09 23:03

S. Senthil Kumar

8-Apr-09 23:03

harold aptroot wrote:
But I still want to know, you know, the minimize trick makes me curious as to how it manages to do whatever it is it's doing - and what is is doing? Forcing a GC? Releasing unused pages? Something else?

Windows does this, not .NET. When you minimize a GUI application, Windows trims the working set of the process (SetProcessWorkingSetSize[^], which writes out unused memory pages to disk, reducing the physical memory consumption of the application. What Task Manager shows in the "Mem Usage" column is the working set size, so that goes down. Windows does this so that the applications which appear in the foreground get some more physical memory to play with.

But in reality, there is no decrease in virtual memory consumption.

Regards
Senthil [MVP - Visual C#]
_____________________________
My Home Page |My Blog | My Articles | My Flickr | WinMacro

harold aptroot9-Apr-09 5:00

9-Apr-09 5:00

Thanks, precisely what I wanted to know Smile | :)

Jacob Dixon9-Apr-09 5:16

9-Apr-09 5:16

Ok I have done as you suggested. I have also converted the Threads I was using into background workers due to them only needing to work when that form was loaded and no where else. I think this increased the memory usuage during that form being loaded, but disposed of everything when I closed it.

I noticed that I could get it to jump to 130MB, close the edit form and it would still be there. BUT, when I minimized the entire application the memory jumped down to like 5-10MB from 130MB. Is that good or bad? It doesn't seem to go down right then unless I minimize it.

Luc 6480119-Apr-09 5:39

Luc 648011

9-Apr-09 5:39

It all sounds normal.

Minimizing the main form causes Windows to reclaim part of the Working Set.

Just closing some forms won't achieve anything right away: the GC only runs when it has to, as in your code trying to create a new object and not enough memory being available. Closing a window is unlikely to cause that, unless you create objects in the FormClosing/FormClosed handlers.

Anubhava Dimri8-Apr-09 19:57

Anubhava Dimri

8-Apr-09 19:57

Are You using Thread Under thread ?

mailto: anubhava.prodata@gmail.com

Jacob Dixon9-Apr-09 3:20

9-Apr-09 3:20

When the form loads it loads information from SQL. One information that takes a while remotely is a picture. Since the pictures put in there are usually 1-3MB in size. So I use thread to call a method to do this.

Thread newThread = new Thread(new ThreadStart(GetPicture));
newThread.Start();

This is how I call it. The TAG (Identity) is passed to the form when it is loaded from another form. So how I get the picture I want is by using that Tag.

private string TAG;

public EditInventory(string TAG)
{
this.TAG = TAG;
InitializeComponent();
}