.NET (Core and Framework)

24-Aug-07 4:33

Hello,

I posted the following in the C++/CLI forum, because the application I was writing when I noticed the problem is written in C++/CLI. However, the thought occured to me that maybe this is something .NET developers in C# or VB.NET have also experienced. Please take a read through and let me know what you think. Thanks!

Yesterday I tried an experiment.... I took a C++/CLI application of mine and disabled my menu-disabling code so that my full menu system (with fly-outs) would work even when no application data was open. Using Task Manager to monitor my memory use, I noticed that every time I expanded a top-level menu or moused over a sub-menu that triggered a fly-out, more memory was used. No shock there. BUT, when I click back in the main window area to make all the fly-outs disappear, the memory use does NOT decrease! In fact, I sat there for 10 minutes waiting for Task Manager to show some drop in memory use, but it never did. Meanwhile, when I triggered the same fly-out again, even MORE memory was used.

I noticed the same behavior when I triggered the Open File Dialog (.NET out of the box -- no overrides or anthing weird).

The application I am writing is very dialog-intensive. The base application uses about 64MB. But at the consumption rates I'm seeing, I could very well imagine that after an hour or so I'd be up to 512MB from invoking dialogs and the GC failing to reclaim memory.

What the heck is going on? Is my Garbage Collector broken? Should I somehow manually invoke garbage collection to clear some memory? Does anyone know if the GC works on a time interval or a certain percentage of consumed memory, or what?

Thanks for your help.

Luc Pattyn24-Aug-07 5:17

24-Aug-07 5:17

Hi,

you may not have a problem at all.

when you instantiate some class, an array, a dialog, whatever, you would need more
memory; either your Operating System allows you that, or your gc will try and free
up some of the memory that is already yours. There is nothing periodic about it,
it's on a need-to-run base.

So if your app is allowed 100MB, it will grow until it starts to reach such numbers,
only then will gc run, and you may fall back to as low as a few MB. Who cares ?

If the number it falls back to keeps growing, then there is a potential problem,
since that indicates more and more objects don't get freed, either because you
really need more and more objects, or because you have a memory leak.

If you are not using unmanaged code, the typical way to get memory leaks is by
forgetting to call Dispose() for objects instantiated from a class that has such
method. There are many, one that people often seem to forget is Form (and hence
also OpenFileDialog). For dialogs, you should create and show them, then
collect the results and dispose.

Hope this helps.

BTW: there are lots of articles on memory, dispose and the like, here on CP and
elsewhere.

Luc Pattyn [Forum Guidelines] [My Articles]

this weeks tips:
- make Visual display line numbers: Tools/Options/TextEditor/...
- show exceptions with ToString() to see all information
- before you ask a question here, search CodeProject, then Google

Xpnctoc24-Aug-07 8:54

24-Aug-07 8:54

So in other words, I'm paranoid.

The application I'm writing requires a potentially unlimited amount of user data. Well of course this is impossible because a computer only has a finite amount of memory. But what I've been worried about is that someday I'll be using the app and I'll hit memory capacity because the GC didn't reallocate 100MB+ that was used up in temporary dialog boxes in menus. But if it's just because the GC is lazy and doesn't kick in until absolutely necessary, then I guess I should be OK.

Dave Kreskowiak24-Aug-07 9:01

Dave Kreskowiak

24-Aug-07 9:01

Well, that doesn't exactly guarantee you won't run out of memory. You can still cause leaks, like resources and handles. For example, with all these dialogs, are you showing them by calling .ShowDialog()?? If so, are you then calling .Dispose() on those dialogs when you're done with the data in them?? If not, you'll eventually run the system out of resources and cause an OutOfMemory exception.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

Lutosław25-Aug-07 21:35

Lutosław

25-Aug-07 21:35

Hey, I though that a code

openFileDialog.ShowDialog();
openFileDialog.Dispose();
openFileDialog.ShowDialog();

throws "object disposed" excpt or sth like that, doesn't it? So if I use this dialog all the time...

Greetings - Gajatko

Luc Pattyn25-Aug-07 23:27

25-Aug-07 23:27

of course you only dispose of things you don't need any longer.

Luc Pattyn [Forum Guidelines] [My Articles]

Dave Kreskowiak26-Aug-07 4:06

Dave Kreskowiak

26-Aug-07 4:06

gajatko wrote:
So if I use this dialog all the time...

You create a new instance of it...

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

Luc Pattyn24-Aug-07 12:57

24-Aug-07 12:57

Hi,

there are good reasons why the gc is lazy, and should not be called explicitly:

1. it is an expensive operation, since it needs to check all the data in your app, figuring
out which objects still have references to them (are "alive") and which don't. There can
be a lot of data involved, it does not fit in the cache, and it trashes the cache
(i.e. whatever was rightfully in there now is gone).

2. while doing its job, the gc suspends all your threads, to make sure the stacks
and objects don't change while it is examining them; so your app is temporarily dead.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Dave Kreskowiak24-Aug-07 6:17

Dave Kreskowiak

24-Aug-07 6:17

Xpnctoc wrote:
Using Task Manager to monitor my memory use,

Stop right there. You're looking in the wrong place. Managed code (.NET CLR) apps run inside a virtual machine, just like a Java app. What Task Manager is showing you is the RAM reserved for the entire virtual machine your app is running in, not just your app. So, even though your app isn't using that memory, it's still being held by the virtual machine for future use. That's why the memory doesn't get freed.

Xpnctoc wrote:
Is my Garbage Collector broken?

Nope. It's just being lazy. It will hang on to that memory, so long as Windows doesn't want it back. When Windows starts to get low on memory, it'll ask the virtual machine to release whatever it can back to Windows.

Xpnctoc wrote:
Should I somehow manually invoke garbage collection to clear some memory?

This won't solve the "problem". This will tell the GC to free up any managed memory that hasn't already been freed. But, this memory goes back into the Managed Heap (reserved for future use), NOT back to Windows.

Xpnctoc wrote:
Does anyone know if the GC works on a time interval or a certain percentage of consumed memory, or what?

The GC is extremely good at it's job and runs whenever it detects the need to. There is no set schedule and it tunes it's own performance based on how it sees your app running and what it expects your app to do in the future.

A guide to posting questions on CodeProject[^]

Dave Kreskowiak
Microsoft MVP
Visual Developer - Visual Basic
2006, 2007

Xpnctoc24-Aug-07 8:59

24-Aug-07 8:59

Thanks for the insight, Dave. I have never heard that .NET was actually running on a virtual machine. I knew there was JIT and such involved, but that puts things into a little different perspective.

And like I told Luc, I think I'm coming to the realization that I'm just paranoid because of the extreme data memory requirement of the app I'm writing. But I'd even take it one step farther and suggest to myself that I'm also only working in the theoretical, because it would take a long time using the app to develop the need for so much memory.

By then we'll probably be talking terabytes of memory on standard computer models anyway Smile | :)

Luc Pattyn25-Aug-07 0:04

25-Aug-07 0:04

You may want to display (and periodically update) the value of Environment.WorkingSet
inside your app; that allows you to see memory usage by your process.

Luc Pattyn [Forum Guidelines] [My Articles]

Xpnctoc25-Aug-07 4:14

25-Aug-07 4:14

Thanks for the tip. I wasn't aware of that one. I'm sure that will help.

#realJSOP25-Aug-07 3:13

#realJSOP

25-Aug-07 3:13

A side note - don't allocate more than about 85k of memory to any single object unless you're prepare to handle cleanup manually. Anything bigger than that gets shoved onto the "large object heap", and the GC doesn't clean that up for you.

"Why don't you tie a kerosene-soaked rag around your ankles so the ants won't climb up and eat your candy ass..." - Dale Earnhardt, 1997
-----
"...the staggering layers of obscenity in your statement make it a work of art on so many levels." - Jason Jystad, 10/26/2001

Xpnctoc25-Aug-07 4:14

25-Aug-07 4:14

Got that covered. All of my core logic I'm writing in unmanaged code. I found early on that I didn't like the overhead of some of the System.Collections.Generic classes -- especially the bulky disk usage in serialization compared to doing a straight binary write of unmanaged data structures. So I chose to write the app in C++/CLI, using .NET for the ease of creating Windows forms and controls while keeping my core logic and processing in native C++. So I'm fully prepared to do my unmanaged allocation and cleanup. Like I said, the behavior I'm calling into question occurs with things I have little or no control over, such as Menu controls and dialogs.

Luc Pattyn25-Aug-07 8:12

25-Aug-07 8:12

Hi John,

I don't fully agree, AFAIK objects on the large object heap are managed like all other
managed objects, with a single difference: the LOH does not get reshuffled, hence
there is a risk of fragmentation.

Example: allocate 1MB, then 100MB, repeat this N times until memory is almost full,
then throw away the references to all 100MB objects and try to get one really big
object of 300MB.

IMO the threshold (85KB?) is not documented, is not available through some API,
may vary over time, and may depend on your system's memory situation.
I feel the Framework should be more specific about this, it should guarantee a
reasonable minimum value for the threshold, so one can make sure fragmentation will
not become an issue by keeping objects smaller than that.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Scott Dorman26-Aug-07 3:09

Scott Dorman

26-Aug-07 3:09

Luc Pattyn wrote:
the threshold (85KB?) is not documented, is not available through some API,
may vary over time, and may depend on your system's memory situation.

In general this is true, the 85KB limit isn't publicly documented. I did find it in Rotor[^] (SSCLI) in the gc.h header file for the garbage collector's native code:

/* misc defines */
#define LARGE_OBJECT_SIZE   85000

Scott.

—In just two days, tomorrow will be yesterday.

[Forum Guidelines] [Articles] [Blog]

Vaibhav Sharma26-Aug-07 16:54

Vaibhav Sharma

26-Aug-07 16:54

Hi,

I am amazed that after reading all the replies and posts on this article no one mentions of the Using...End Using code block.

I am sure you must be knowing it but just a little bit of detailing on the using statements:

'VB Version, C# no different, just lang. changes<br />
Using dlgOpen As New OpenFileDialog()<br />
    dlgOpen.Filter="*.*"    <br />
    'Any other properties<br />
    dlg.ShowDialog()<br />
End Using

Now what the above code fragment does is that the CLR will create the dlgOpen object and will be responsible for its disposal, not YOU.
This has serious benefits in terms of Performance & Memory Utilization since as soon as the code execution reaches End Using the dlgOpen will be Disposed off without requiring you to do any thing at all.

:::One Catch Though While using the Using statements:::
the dlgOpen will not be accessible outside the Using block since its scope is limited is to the Using...End Using block.

So i strongly recommend you to go ahead and use the Using...End Using blocks for all the DialogBoxes since most of them times they return only one value which can easily be stored on a local/global variable.

Awaiting reply eagerly.

Regards,
Vaibhav Sharma

Xpnctoc26-Aug-07 17:24

Embedded resources -- pros & cons

26-Aug-07 17:24

I appreciate the tip. I am aware of the "using" syntax, but in my case that won't work. As I mentioned in my response to John Simmons's post above, my application is written in C++/CLI, which has no such construct. There were multiple reasons for that language choice, some of which I mentioned and others that are irrelevant to this conversation. And even if I was in C#/VB, I'd still have the same problem with the Menu control. Can't use "using" for something like a menu that has to be present for the duration of execution.

Regardless, I think it was one of Luc Pattyn's responses that directed me to try examining memory usage through the Environment.WorkingSet (thank's again, Luc). This provides memory use much closer to what I would expect based on my interaction with the application. I still find it alarming that the .NET virtual machine continues to eat up additional memory when it's not needed, but there I just have to trust that the GC will do its job when push comes to shove.

Xpnctoc24-Aug-07 4:16

Re: Embedded resources -- pros & cons

24-Aug-07 4:16

Hello. I hope these are some simple questions:

1. Is there a way to read an embedded resource out of a managed DLL without having to load the whole assembly?

2. If, as is my understaning, the answer to #1 is "NO", what is the point of embedding resources instead of loading your resources from individual files at run-time? In other words, if I have a DLL that contains 128MB of images, and I only want to extract a 64KB image from it, what is the point of embedding when I'd have to take the 128MB hit on my memory use -- especially when an assembly can't be unloaded?

Thanks for your input.

Luc Pattyn24-Aug-07 4:54

Re: Embedded resources -- pros & cons

24-Aug-07 4:54

Hi,

AFAIK:
1. No
2a. You can make a single-file app that holds images, fonts, whatever.
2b. You can protect your data by putting it in as a resource (possibly using encryption).
2c. You can group things that belong together into a single DLL, have several of those
and load the one you need (e.g. language selection, regionalization)
2d. If you are that concerned about memory footprint, make smaller DLLs; at one resource
per DLL, it does not make much sense any more.

Smile | :)

Luc Pattyn [Forum Guidelines] [My Articles]

Xpnctoc24-Aug-07 8:43