Trouble with the Large Object Heap

• Download source files - 6.3 Kb
• Download demo project - 2.24 Kb

Introduction

Unlike most articles here at CodeProject that show you how to make something work, this one shows you how to make something break. Specifically, I am speaking of the Large Object Heap inside the .NET framework. Why you ask? Because like myself, you too might be experiencing this issue. If you are not yet, you might be tomorrow...

Details

In the problem domain I typically work (image processing), we need to load and display very large JPEG or TIFF images. They can range up to 500 MB when rendered as bitmaps. As I said, very large. The problem we encounter is that after a while, we would hit an OOM (out of memory) condition. It doesn't seem to matter how you manage yourself, if you need large objects you can fall victim to this issue.

If you study this topic on MSDN and the Internet, there are lots of good references to proper memory management practices. This article does not attempt to rehash that info here. Instead, it simply attempts to add a small footnote to your working knowledge set, as it relates to the CLR in all its glory.

Just to bring you up to speed with .NET and imaging. The mainstays of this problem space are the PictureBox control, the abstract Image class, as well as its derived class Bitmap. There are lots of great features available to you, and for most applications, everything will work fine. However, if you study the classes, you will note that in most cases, you do not have control over the underlying buffers used by these classes. Hence, their use of memory, and that my friends, leads us to how I found the weak spot in the CLR's heap management.

Within the managed heap, there are actually several heaps the CLR uses when allocating memory for managed objects. In general, these heaps can be divided into two classes, the SMO (small object heaps) and the LOH (large object heap). The small object heaps are fully managed by the CLR in quite an elaborate dance when studied in detail. Object lifetimes are tracked, object memory is freed as necessary, and finally, heaps are compacted if necessary to keep the available memory flowing so to speak. The LOH is treated to almost the same dance by the CLR with one exception. The LOH is never compacted. This is so, so to speak for performance reasons; after all, moving lots of objects around in memory can and does burn lots of CPU cycles.

Unfortunately, Microsoft has not provided a backdoor mechanism to force a LOH compaction cycle. So if you are like me and you simple must create and destroy lots of very large objects, your LOH might become fragmented. Once the LOH fragments, there is no way back from the edge, you must simply jump off. In other words, restart your process.

Here is a brutal application that illuminates the issue:

using System;
using System.Text;
using System.Threading;

namespace Burn_LOH
{
    class Program
    {
        static void Main (string[] args)
        {
            for (int count = 1; count <= 5; ++count)
                AllocBigMemoryBlock (count);
            Console.Write ("\nPress any key to exit...");
            while (Console.KeyAvailable == false)
                Thread.Sleep (250); 
        }

        static void AllocBigMemoryBlock (int pass)
        {
            const int MB = 1024 * 1024;
            byte[] array = null;
            long maxMem = 0;
            int arraySize = 0;
            MemoryFailPoint memFailPoint;
            while (Console.KeyAvailable == false)
            {
                try
                {
                    arraySize += 10;
                    array = new byte[arraySize * MB];
                    array[arraySize * MB - 1] = 100; 
                    maxMem = GC.GetTotalMemory (true);
                    Console.Write ("Pass:{0}  Max Array " + 
                        "Size (MB): {1:D4}  {2:D4}\r", 
                        pass, arraySize, 
                        Convert.ToInt32 (maxMem / MB));
                }
                catch (OutOfMemoryException)
                {
                    Console.Write ("\n");
                    maxMem = GC.GetTotalMemory (true);
                    if (arraySize < 20)
                        Console.Write ("Pass:{0} Small " + 
                            "Array Size (MB): {1:D4}" + 
                            "  {2:D4} {3}\r\n\n", 
                            pass, arraySize, 
                            Convert.ToInt32 (maxMem / MB), 
                            "Insufficient Memory...");
                    else
                        Console.Write ("Pass:{0} Failed " + 
                            "Array Size (MB): {1:D4}" + 
                            "  {2:D4} {3}\r\n\n", 
                            pass, arraySize, 
                            Convert.ToInt32 (maxMem / MB), 
                            "Out Of Memory...");
                    break;
                }
                finally
                {
                    array = null;
                    GC.Collect ();
                    GC.WaitForPendingFinalizers ();
                }
            }
        }
    }
}

As you can see from the code, the routine AllocBigMemoryBlock does all of the damage. This routine simply tries to create a bigger and bigger array until it reaches an out of memory condition. At that point, it writes what it has got to the console and returns. Specifically, it loops through a create / destroy code sequence, creating bigger and bigger arrays while trying to force the CLR to maximize the available memory. In all, not a very interesting piece of code.

Where things do get interesting is when you put the call to AllocBigMemoryBlock inside a loop and watch the LOH disapear. On my machine, it takes less than five cycles to loose all practical use of the LOH. Near the end of the program, it can't even allocate a 20 MB array!

Now, I know what you are thinking, memory is not being freed. Oh yea, try it! You will find your memory usage, VM usage, CLR memory management performance counters, et. al. all indicate there is no real memory being consumed by the program. So if the program doesn't have it, where is it?

Fragmented to bits is my best guess. However, it turns out that there is an issue with the heap manager in the 32 bit XP version of the 2.0 CLR. Under Vista RC1 and later, or under 64 bit versions of Windows, the issue does not arise.

If you would like to see some strange behavior, try replacing the main loop with the following code snippet. All it does is keep looping until you stop it. However, you will notice that the available memory comes and goes. On my machine, this cycle occurs about once a second. With the maximum array size I can create switching between 910 MB down to 10 MB, and back again.

static void Main (string[] args)
{
    int pass = 0; 
    Console.Write ("Press any key to stop...\n\n");
    while (Console.KeyAvailable == false)
    {
        ++pass;
        AllocBigMemoryBlock (pass);
    }
    Console.Write ("\nPress any key to exit...");
    Console.ReadKey ();
    while (Console.KeyAvailable == false)
    {
        Thread.Sleep (250);
    }
}

However, there is a workaround

Microsoft introduced a new class in the 2.0 framework, MemoryFailPoint. A MemoryFailPoint object is used to test if a memory allocation can succeed or not. Throwing an InsufficientMemoryException if an allocation of the requested size would result in an OutOfMemoryException. The interesting part is that so long as your program never generates an OOM condition, the heap continues to function as you would expect it to. To see this in action, replace the AllocBigMemoryBlock function with the following code snippet. The resultant program will continue to cycle, allocating as big of an array as your system will allow it to. Ah, once again all is well in la la land.

static void AllocBigMemoryBlock (int pass)
{
    const int MB = 1024 * 1024;
    byte[] array = null;
    long maxMem = 0;
    int arraySize = 0;
    MemoryFailPoint memFailPoint;
    while (Console.KeyAvailable == false)
    {
        try
        {
            arraySize += 10;
            using (memFailPoint = new MemoryFailPoint (arraySize))
            {
                array = new byte[arraySize * MB];
                array[arraySize * MB - 1] = 100; 
                maxMem = GC.GetTotalMemory (true);
                Console.Write ("Pass:{0}    " + 
                    "Max Array Size (MB): {1:D4}  {2:D4}\r", 
                    pass, arraySize, 
                    Convert.ToInt32 (maxMem / MB));
            }
        }
        catch (InsufficientMemoryException)
        {
            Console.Write ("\n");
            maxMem = GC.GetTotalMemory (true);
            if (arraySize < 20)
                Console.Write ("Pass:{0}  Small Array" + 
                    " Size (MB): {1:D4}  {2:D4} {3}\r\n\n",
                    pass, arraySize, 
                    Convert.ToInt32 (maxMem / MB), 
                    "Insufficient Memory...");
            else
                Console.Write ("Pass:{0} Failed Array Size" + 
                    " (MB): {1:D4}  {2:D4} {3}\r\n\n", 
                    pass, arraySize, 
                    Convert.ToInt32 (maxMem / MB), 
                    "Out Of Memory...");
            break;
        }
        catch (OutOfMemoryException)
        {
            Console.Write ("\n");
            maxMem = GC.GetTotalMemory (true);
            if (arraySize < 20)
                Console.Write ("Pass:{0}  Small Array" + 
                    " Size (MB): {1:D4}  {2:D4} {3}\r\n\n", 
                    pass, arraySize, 
                    Convert.ToInt32 (maxMem / MB), 
                    "Insufficient Memory...");
            else
                Console.Write ("Pass:{0} Failed Array" + 
                    " Size (MB): {1:D4}  {2:D4} {3}\r\n\n", 
                    pass, arraySize, 
                    Convert.ToInt32 (maxMem / MB), 
                    "Out Of Memory...");
            break;
        }
        finally
        {
            array = null;
            GC.Collect ();
            GC.WaitForPendingFinalizers ();
        }
    }
}

Conclusions

So what can I do, you ask?

If you can follow the rules of good CLR memory management:

• Let the CLR manage the heap.
• Don't force collection cycles.
• Create object pools and reuse your big objects (the longer lived the better).

If you can't follow the rules of good CLR memory management:

• Watch out!
• Move to a version of Windows that works well for big objects.

But in either case, you should consider using MemoryFailPoint objects to protect your large objects from causing OOM conditions. Especially if your programs are designed to be long running. In that no matter what, if your heaps become unstable, there is no real way to recover them, short of restarting your process.