|
Hello,
I am working on a project that requires me to generate long reports that can have 150 pages or more - Each of these pages are generated as an A4 image (~3500x2480pixels), which is then inserted into an ArrayList...
The problem however is that I'm repeatedly getting OutOfMemory exceptions that I seem to be unable to get rid of despite extensively clearing all objects i'm not using (using the Image.Dispose() method for example) as well as forcing the garbage collector (GC.Collect()) every now and then...
I'm constructing each of these images in the 1bpp pixel format that would mean that each of these images would be ~1MB on average... For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)??
Any one has any ideas on how I can get rid of, or even handle, these OutOfMemory exceptions??
Thanks,
Sarab
|
|
|
|
|
How are you using the report when it's finished, as you create the pages as images? Do they really have to be images?
Do you really have to create all images at the same time? Is there no way of handling the data of the report in some other way, and generate each image when you need it?
You could convert each image into GIF or PNG format, that would substantially reduce the size without losing any information.
---
b { font-weight: normal; }
|
|
|
|
|
If it were me, I'd try just saving each bitmap as it's created, and then just keeping an arraylist of filenames (files in a temporary location), accessing individual files as needed.
|
|
|
|
|
Just tried this...
I write each image to disk after it is created, dispose it and force the garbage collector after every 20 images...
No success however; I keep the Windows Task Manager on during processing, and the page file size still continues to gradually increase till it hits the limit, and once again I get hit with OutofMemory exceptions, which only lead to more when handled...
Interestingly however, the moment the control comes out of the routine - the page file size immediately drops back to normal...
|
|
|
|
|
Heritos is right. I would save the images to and only hold a reference to them.
sarabjs wrote: For 200 images, this (crudely) corresponds to a 200MB ArrayList.. I agree this is a size larger than many RAMs, but shouldn't my computer be able use the pagefile (my pagefile is set to 512MB)??
No. Because .Net will use much more memory, as the code is managed and everything is encapsulate in Objects.
I tried it with a 3D-scene of 100 MB and old C++ only needs a little bit more than 100 MB for the whole programm while .Net used more than 200 MB.
Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore.
Greetings,
Ingo
|
|
|
|
|
ihoecken wrote:
Another Point is the Garbage Collection, which isn't a real one. If you generate an object and drop it afterwards its hold in the memory quiet a time. So there are objects which aren't used anymore.
That's not an accurate statement, you appear to be laboring under one of two incorrect assumptions. .net's garbage collector is generational which means that it attempts to collect younger objects far more frequently than older ones. If memory is needed it will however collect the older generations.
Secondly when an object is GCed the memory is returned to the application stack/heap, not to the operating system. Standard c/c++ runtimes behave the same way because requesting/returning memory to the OS is an expensive operation. Consequently the task manager doesn't display the actual ammount of ram the program is using. This is more apparent in .net since it's runtime requests a larger default chunk size than standard C++ ones. Excess memory is returned if the OS asks for it (ie appB requests more than the OS has available so the OS asks other running apps if they can return thier excess), minimizing the app will also trigger this release under .net. IIRC it will eventually do this on it's own if it concludes it asked for more than it'll ever likely need. If the systems apps are using more total ram than is physically available, the excess will be parked on the swap file because of inactivity so while it may make the task manager numbers look scary it doesn't have a real effect on performance.
|
|
|
|
|
A good garbage collection begins to release unused memory when there is time, .Net Garbage collections releases memory when memory is needed. So its much slower and sometimes such memory isn't released properly, even if the object isn't used any more.
Greetings,
Ingo
|
|
|
|
|
The .NET GC is self-tuning. It runs on both a semi-regular (self-adusted) schedule and on-demand when either the applications calls for it or the operating system does and wants memory returned. Memory is never released "improperly" as you call it. The GC tries, and does a very good job, at keeping a resonably sized pool of memory available to the application for future allocations. Like I said, the GC is self-tuning and can only improve it's allocation strategy by over-estimating what it needs to start with then watching how your application allocates and releases memory.
But, yes, the GC is a bit lazy when application activity is low, and for good reason. In order for the GC to run, the threads of your application must be stopped before the collection takes place. This is because objects in memory can be moved by the GC to compact and defrag the heaps and pointers to those objects must be updated to reflect their new locations. This cannot be done while the code is still running! So, yeah, the GC tries to be as lazy as possible, giving your app more time to execute.
Garbage Collection and object allocation, when dealing with the Managed Heap, goes VERY, VERY fast. It slows down considerably if memory has to be allocated from the system first, adding it to the Managed Heap.
BTW, this is all documented on MSDN:
Programming for Garbage Collection[^]
Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework[^]
Garbage Collection—Part 2: Automatic Memory Management in the Microsoft .NET Framework[^]
Improving .NET Application Performance and Scalability[^]
RageInTheMachine9532
"...a pungent, ghastly, stinky piece of cheese!" -- The Roaming Gnome
|
|
|
|
|
Hi Sarab,
I think that your problem is regarding the number of handles you create!!! If you know, Windows restricts each process in creating handles. For example, it is 10,000 handles for XP. But my experience says that this equation is that straight forward!!! It is also dependent on the memory the process occupies and the number of GDI objects being created by your app.
To confirm this, try seeing the number of GDI objects and Handles being created by your app through the taskmanager->View->Select Columns... And if the number of Handles is going beyond 500 and GDI objects more than 300 or 400, then i think that you have located the problem!!!
Even i had a similar problem, but not yet found the solution as such!!! The work around which i did was not storing such a huge data in memory which creates lots of GDI objects!!!! Though im able to get control over it, i feel that this is not the correct solution to it!!!
Regards,
Rajesh
|
|
|
|
|
Thanks folks!! I appreciate everyone's help...
Turns out, I didn't have to cross the Seven Seas:
I managed to solve the issue by explicitly disposing (Image.Dispose()) some images I was creating during processing... I used to think that .NET automatically clears the memory being held by an image object once all references to it are removed - Guess that isn't the case...
Learnt a lot from these discussions though - Thanks!
-- modified at 16:03 Friday 3rd February, 2006
|
|
|
|
|
I have a .Net application which at one point repeats a task several times. Each time the task is performed, it pops up a progress window, updates the window while working, and then closes the window.
This should all work fine, but what I am finding is that if I have a loop that repeats the task 5 times, the progress windows get displayed and updated, but the screen doesn't seem to redraw the parent window after they are 'closed', thus it looks as though all windows remain open until the last task completes, at which point the screen redraws, and correctly see that all windows have been closed.
Is there a function that I can call to ensure that all window events take place and the screen redraws after each task completes? I have this problem in several spots where a parent window calls a computation intensive task, and then does not get redrawn correctly until the task ends.
Thanks in advance for any pointers,
Spencer
|
|
|
|
|
If you call the Invalidate() method of the control which isn't redrawing properly, followed by Application.DoEvents(), this should force the control to process its pending events and redraw.
If you're doing process intensive stuff and dont want the application to lock up, then you might want to consider using a seperate thread.
Cheers,
Will H
-- modified at 10:41 Monday 30th January, 2006
|
|
|
|
|
Thanks, this worked perfectly.
In this case, I don't want the user interface to be active, just need the dialogs to clear correctly when finished, so I don't need the seperate thread here.
Thanks again,
Spencer
|
|
|
|
|
Hi,
Can anybody suggest a good .NET Profiler which gives the details of the memory taken up by the application and also regarding the performance details of the application!!!
Thanks anyway,
|
|
|
|
|
|
|
Hey Thanks Buddy.
I had used ANTS Profiler and i found it to be very slow(atleast on my system)... But the other one was a very handy, and it made my life easy!!!
And can i have some dedicated Performance profilers also??
Thanks anyway,
Rajesh
|
|
|
|
|
I'm working with Visual Studio 2003 and Framework 1.1.
Someone tells me that it is possible to install Framework 2.0 SDK and developing applications with Framework 2.0 in Visual Studio 2003.
I've tried to install Framework 2.0 SDK, but the new objects are not visible from Visual Studio 2003 .
Someone can help me  ???
nikolag
|
|
|
|
|
nikolag wrote: Someone tells me that it is possible to install Framework 2.0 SDK and developing applications with Framework 2.0 in Visual Studio 2003.
No, it's not. 2003 will ONLY work with the .NET Framework version 1.1. 2005 will ONLY work with the .NET Framework 2.0.
RageInTheMachine9532
"...a pungent, ghastly, stinky piece of cheese!" -- The Roaming Gnome
|
|
|
|
|
ok, now it is clear.
Thank you
"Tutti sanno che una cosa è impossibile da realizzare, finchè non arriva uno sprovveduto che non lo sa e la inventa"
(Albert Einstein)
|
|
|
|
|
hi all,
Recently I devised this bit of code (C#) that works wonders for the memory usage reported in task manager:
Process currentProcess = System.Diagnostics.Process.GetCurrentProcess();
currentProcess.MinWorkingSet = currentProcess.MinWorkingSet;
With one of my more complicated programs, this reduced the initial memory usage (measured once the form loaded completely) from 34690 KB to 8980 KB (!).
I've tried various other solutions, most of which rely on forcing a garbage collection or reducing the process' working set to the minimum working set. I found neither to be very effective. I've never seen this approach anywhere else (small wonder - who would think that setting a value to itself would make such a difference?!).
I haven't had time to research why/how this code works, but this solution changes no values and has an immediate effect (just like minimizing and restoring a .NET program does). Place it in your code after your form initializes and watch your memory usage go down!
Hope this helps someone!
|
|
|
|
|
|
And the working set is precisely what I want to reduce, not just the specific amount that the app is actually using but the entire chunk of memory that the runtime reserves. That's what makes a difference from an outside perspective.
|
|
|
|
|
Maybe, but how much slower is your application when it needs more memory from somewhere and has to ask for more, rather than just use some from its reserve.
ColinMackay.net
"Man who stand on hill with mouth open will wait long time for roast duck to drop in." -- Confucius
"If a man empties his purse into his head, no man can take it away from him, for an investment in knowledge pays the best interest." -- Joseph E. O'Donnell
|
|
|
|
|
Notice though that the runtime has no problem reducing the memory reserve when you minimize and restore an application, so apparently there is an excess available. As discussed elsewhere, at startup the runtime needs a larger amount of memory, but not so much once everything has loaded, so reducing the working set after initialization is a reasonable solution.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
