Detecting memory leaks: by using CRT diagnostic functions - Part1

• Download source files - 1.83 Kb
• Download demo project - 14.2 Kb

Introduction

One of the nightmares of a programmer is having their program's memory leaked. This is an important and noticeable concern in this crazy world of corporate politics, where your program may undergo several reviews, just to prove that your program is a buggy one (code reviews don’t focus on QA!!). Irrespective of the size of the application, it is very common for a programmer to make some mistakes. This article is for those who want their programs to be memory leak free.

Why you need to go through this

There are many tools that are available, and the debuggers can detect the memory leaks. Then why should you go through this article?? Debuggers don’t give the detailed output, and this is the first step to create a full fledged tool. And one more point that motivated me to write this article is, I have seen some postings in MSDN forums asking ways to detect memory leaks. This is currently in proc version, to make use of this you have to include the files available as download with this article. This is the first article in this series, and might span to 3-4 articles based on users feed back.

Brief introduction to heap memory

You might already be aware of the fact that, heap is a chunk of memory from which your program requests and releases memory on the fly. Windows heap manager processes the requests made by your program. Windows offers a variety of functions to deal with the heap, and supports compaction and reallocation. I am not going to discuss advanced heap memory management, and I reserve it for my future articles. This article concentrates on hooking the heap allocations, reallocations, and de-allocations.

Win32 systems support a logical address space of 4 GB, out of which 2 GB is reserved for OS itself, and the remaining 2 GB is left for the user programs. If your application requires more than 2 GB address space, you can request the OS to make room for one more GB, so that the OS adjusts itself to 1 GB, and allots the remaining 3 GB for your program to meet your requirements. Your program can run in a maximum of 3 GB address space, and needs little physical memory to support it.

By default your program's heap memory reserves 1 MB (256 pages of size 4 KB), and commits 4 KB (1 page). Whenever your program requests for some more memory, the heap manager tries to process your request by allocating it from the committed 4 KB, if it crosses the 4 KB boundary it will commit one more page. If your application requests for memory more than the default 1 MB, then it will reserve one more MB. Heap manager keeps this process until your program runs out of address space.

In the days of WIN16, Windows maintained two kinds of heap memory, one is global heap, and the other is local heap. Each Win16 application has one global heap, and one local heap. Applications are free to request the chunk of memory from any heap. But WINNT removed the concept of global and local heap, and introduced the new concept of one default heap, and a number of dynamic heaps. But WINNT still supports the Win16 heap functions like GlobalAlloc, LocalAlloc for backward compatibility. If your application requests the Win16 heap functions, WINNT maps them to the default heap.

By default your application owns a default heap, and you can create as many number of dynamic heaps as your applications needs. (I think there is some limitation on the number of handles like 65,535, but I am not sure!!!) Default Heap is the area of memory specific to the process. Usually you don’t need to get the handle, but you can get the handle by using GetProcessHeap(). Usual malloc, new calls will map to the default heap. Dynamic heap is the area of memory which can be created and destroyed by your application at runtime. There are some set of functions like HeapCreate, AllocHeap to work with the dynamic heaps. I will give you a detailed description on heap memory management in my later article.

This article uses the CRT diagnostics functions available as part of MS Visual Studio. Whenever I call the memory, treat it as Heap memory, don’t confuse it with primary memory.

About the debug heap

Debug heap is the extension to the base heap. It provides powerful ways to manage your heap allocations in debug builds. You can track any kind of problem from detecting memory leaks to validating and checking for the buffer overruns. When your application allocates memory by using malloc () or new, it will actually be mapped to its debug equivalents like _malloc_dbg(). These debug heap functions further rely on the base heap functions to process the user request.

Debug heap maintains a variety of information to keep track of the memory allocations. Suppose when you request 20 bytes, the debug heap functions actually allocate more memory than you requested. Then this extra memory will be used by the debug heap functions to perform some validation checks and bookkeeping. The debug header is stored in a structure _CrtMemBlockHeader, defined in dbgint.h file.

The structure of _CrtMemBlockHeader is as follows:

typedef struct _CrtMemBlockHeader
{
    struct _CrtMemBlockHeader * pBlockHeaderNext;
    struct _CrtMemBlockHeader * pBlockHeaderPrev;
    char*  szFileName;
    int    nLine;
    size_t nDataSize;
    int nBlackUse ;
    long lRequest;
    unsigned char gap[nNoMansLandSize];
    /* followed by 
    * unsigned char data[nDataSize] ;
    * unsigned char anotherGap[nNoMansLandSize];
    */
} _CrtMemBlockHeader;

This header will be maintained as an ordered linked list. The first two parameters point to next and previous blocks:

• szFileName holds the filename that requested the memory.
• nLine holds the line number in the file, where memory requests happen.
• nDataSize holds the size of the block request.
• nBlockUse indicates the type of the block requested. This may be one of the following:
  • _CRT_BLOCK: CRT (C Runtime) functions make use of this type of blocks for internal use.
  • _NORMAL_BLOCK: This is a general type of block, allocated when requested using malloc or new.
  • _CLIENT_BLOCK: To keep track of a certain type of allocation you can call this type of blocks. It includes the subtypes. MFC internally uses this type of blocks to allocate memory for the CObject derived classes. Refer MSDN for more information.
  • _FREE_BLOCK: Blocks that are freed. You can keep the free blocks still in the list to check for threshold memory requirements, by using the _CRTDBG_DELAY_FREE_MEM_DF flag. These blocks will be filled with 0xDD to keep track of free blocks.
  • _IGNORE_BLOCK: When you turnoff the debug heap for a certain period of time, allocations that are recorded during this time will be marked as ignore blocks.
• lRequest holds the block number (block numbers are assigned in the order of request) in the list.
• Data holds the actual data.
• gap and anotherGap surround the actual data. This is a 4 byte area (Microsoft calls it NoMansland buffer) filled with a known value (0xFD) to identify the buffer overruns.

So, whenever you request for some memory in the debug versions, you are actually allocated with some extra memory for bookkeeping information.

Detecting memory leaks by taking memory snapshots

_CrtMemState can be used to hold the memory state. When we call _CrtMemCheckpoint by passing _CrtMemState variable as parameter, it fills the state of the memory at that point. The following code snippet shows how to set the checkpoint:

_CrtMemState memstate1 ;
_CrtMemCheckpoint(&memstate) ;

You can find the memory leaks by comparing the different check points. Usually you need to take the first checkpoint at the start of the program and next checkpoint at the end of the program, and by comparing the two checkpoints, you can get the memory leak information. Like:

CrtMemState memstate1, memstate2, memstate3 ;
 
_CrtMemCheckpoint(&memstate1) // call at the start of your program
.............
............
_CrtMemCheckpoint(&memstate2) // call at the end of the function

Use the function _CrtMemDiffernce() to find the memory leak. Its syntax is as follows:

_CRTIMP int __cdecl _CrtMemDifference(
    _CrtMemState *diff,
    const _CrtMemState *oldstate,
    const _CrtMemState *newstate,
);

It takes two memory state variables and compares them, and fills the difference in the third variable. Use like:

_CrtMemeDifference(&memstate3, &memstate1, &memstate2) ;

If it finds the memory leak it returns true, otherwise it returns false.

For dumping the memory leak information, you can either use _CrtDumpMemoryLeaks(), or _CrtMemDumpAllObjectsSince() to dump the allocations from a specific checkpoint.

Example:

#include <stdio.h>
#include <string.h>
#include <crtdbg.h>
#ifndef _CRTBLD
#define _CRTBLD
#include <dbgint.h>
#endif

int main(void)
{
   _CrtSetReportMode(_CRT_WARN, _CRTDBG_MODE_FILE);
   _CrtSetReportFile(_CRT_WARN, _CRTDBG_FILE_STDOUT );


   _CrtMemState memstate1, memstate2, memstate3 ; // holds the memory states
   _CrtMemCheckpoint(&memstate1) ; //take the memory snapshot


    int *x = new int(1177) ;// allocated
    char *f = new char[50] ; // allocated
    strcpy(f, "Hi Naren") ;
    delete x ; // freed, 
    _CrtMemCheckpoint(&memstate2) ; //take the memory snapshot
   //compare two snapshots, we didnt free the char *f block. It should catch 
   //by the debug heap

   if(_CrtMemDifference(&memstate3, &memstate1, &memstate2)) 
   {    
      printf("\nOOps! Memory leak detected\n") ;
      _CrtDumpMemoryLeaks() ; 
      // alternatively you can use _CrtMemDumpAllObjectsSince for 
      //dumping from specific check point
   }
   else
      printf("\nNo memory leaks") ;
      return 0 ;
}

Output:

OOps! Memory leak detected
Detected memory leaks!
Dumping objects ->
{42} normal block at 0x002F07E0, 50 bytes long.
Data: <Hi Naren        > 48 69 20 4E 61 72 65 6E 00 CD CD CD CD CD CD CD
Object dump complete.

Detecting memory leaks by using hooks

This is the procedure that I followed to keep track of the allocations and deallocations. CRT debug offers functions to hook the allocations. When you call the hook function by passing it the pointer of your own function handler, it will be called whenever your program requests and releases the memory.

_CrtSetAllocHook allows you to set the hook. Its syntax is as follows:

_CRTIMP _CRT_ALLOC_HOOK __cdecl _CrtSetAllocHook(
_CRT_ALLOC_HOOK hookFunctionPtr
);

hookFunctionPtr is a pointer of your function that handles the allocations.

It should have the following syntax:

int CustomAllocHook(int nAllocType, void *userData, size_t size, 
                             int nBlockType, long requestNumber, 
                                  const unsigned char *filename, 
                                                 int lineNumber)

Here, nAllocType indicates the type of operation. It can be either of the following:

• _HOOK_ALLOC - for allocations.
• _HOOK_REALLOC – for reallocations.
• _HOOK_FREE - for free requests.

userData is the header of type _CrtMemBlockHeader. It is valid for free requests and holds NULL value for allocation requests.

size holds the number of bytes requested.

nBlockType indicates the type of block (like _NORMAL_BLOCK). For the _CRT_BLOCK allocations, you better return the function with the return value true, otherwise you may get struck in the loop. You better don’t handle _CRT_BLOCK.

requestNumber holds the block number.

Filename is the name of the file that sent the request.

lineNumber is the line number in the above file, to pinpoint where the request happens.

The basic skeleton for the hook function is as follows:

_CrtSetAllocHook(CustomAllocHook)

int CustomAllocHook( int nAllocType, void *userData, size_t size, 
                              int nBlockType, long requestNumber, 
                                   const unsigned char *filename, 
                                                  int lineNumber)
{
    if( nBlockType == _CRT_BLOCK)
        return TRUE ; // better to not handle
        switch(nAllocType)
        {
            case _HOOK_ALLOC :
            // add the code for handling the allocation requests
            break ;

            case _HOOK_REALLOC:
            // add the code for handling the reallocation requests
            break ;

            case _HOOK_FREE :
            // add the code for handling the free requests
            break ;
        }

    return TRUE ;
}

You can replace the CRT functions with your versions for the tracking of memory management. But I used the hook functions only for logging purposes. Whenever allocation requests come, I fill that information in the ordered linked list, and remove the entry from the list whenever the corresponding free request is called. Here the block number is the main variable to map the free requests to the entries in our own linked list.

Using the code

Two sample files (MLFDef.h and MLFDef.cpp) are available as downloads with this article. To make use of these files do the following:

1. Add the files to your project.
2. Include the MLFDef.h in the file where you want to call the MLF functions.
3. Call EnableMLF() to enable the MLF (memory leak founder). It is better to call at the start of the program (like InitInstance()).
4. Call CloseMLF() to create the log file. The log file will be created in the C: drive. Its hard coded path is c:\MLFLog.txt.

Demo version is available to illustrate how to use the files.

Limitations

I experienced some problems, when used across the ActiveX modules. It is better to include the EnableMLF() and CloseMLF() in the main module until I update this article. It doesn’t work for VC7 but it works fine for VC6. I will update this article to include the support for VC7 and to fix the ActiveX module problems.

Extensions

This is just the in proc version, the first step to build a full fledged tool. You are encouraged to develop your own. This article is also the first part of the series, and you can expect some more articles on advanced memory management. You are suggested to go through MSDN for further information.