Click here to Skip to main content
15,867,308 members
Articles / Programming Languages / XML
Tip/Trick

Not Every Memory Allocation Failure is OOM

Rate me:
Please Sign up or sign in to vote.
5.00/5 (1 vote)
24 Dec 2018CPOL2 min read 5K   22   1   1
It could be a hacking to crash your program.

Introduction

As with many C++ programmers with C background, bring their C habits to C++ programming as shown in the below code where a massive array is allocated and pointer is then checked for failed allocation in presence of null address. It works this way for C malloc. Unfortunately, C++ new does not work like that: in the case of allocation failure, new throws a bad_alloc exception. I have seen this bad example in production code. And I am guilty myself for writing such code as you can seen in this WIC article.

C++
// C code that works
char* ptr = (char*) malloc(0x7fffffff);
if (!ptr) // will
{
    // will come here!
    printf("Error: ptr is null!\n");
}
// C++ code that does not work as intended
char* ptr = new char[0x7fffffff];
if (!ptr) 
{
    // will never come here when allocation fails because bad_alloc exception is thrown
    std::cerr << "Error: ptr is null!" << std::endl;
}

To fix the code, either instruct new not to throw exception by specifying std::nothrow after it or use a try-catch block to catch bad_alloc exception.

C++
char* ptr = new (std::nothrow) char[0x7fffffff];
if (!ptr)
{
    std::cerr << "Error: ptr is null!" << std::endl;
}

Should You Catch bad_alloc Exception?

One school of thought clearly recommends let the bad_alloc exception bubbles up the call stack and let the process die an inglorious death by termination because there is nothing you can do in a process ran out of memory situation. Here, we come to the question of today's topic: is every memory allocation failure OOM?

The answer is no. It could a case of a competitor hiring a hacker to cause a denial of service(DoS). In today's world, your program or service may open a file from user or read a network packet from another service. Inside the file or network packet, there could be an array to be read. In the usual implementation, there is count field preceding the array to let the program know in advance, the array length. Hacker can just manipulate this field to a very large number to cause your program to crash repeatedly. Instead of terminating, the fix is to detect this anomaly and discard reading this file (and flag this as error) and continue business as usual.

Is There a Notorious Vulnerability Out in the Wild Exploiting OOM?

The answer to this question is yes. In the OWASP Top Ten 2017, in rank number 4: XML External Entities (XXE) is a type of vulnerability which exploits XML processors either by injection or expansion. We shan't discuss injection as it is not relevant to our discussion. One notable XML External Entities expansion is Billion laughs attack (See below for example). Except for the first entity, every entity is 10 times expansion of previous entity, causing too big a memory allocation to handle in the end.

XML
<?xml version="1.0"?>
<!DOCTYPE lolz [
 <!ENTITY A "A">
 <!ELEMENT lolz (#PCDATA)>
 <!ENTITY B "&A;&A;&A;&A;&A;&A;&A;&A;&A;&A;">
 <!ENTITY C "&B;&B;&B;&B;&B;&B;&B;&B;&B;&B;">
 <!ENTITY D "&C;&C;&C;&C;&C;&C;&C;&C;&C;&C;">
 <!ENTITY E "&D;&D;&D;&D;&D;&D;&D;&D;&D;&D;">
 <!ENTITY F "&E;&E;&E;&E;&E;&E;&E;&E;&E;&E;">
 <!ENTITY G "&F;&F;&F;&F;&F;&F;&F;&F;&F;&F;">
 <!ENTITY H "&G;&G;&G;&G;&G;&G;&G;&G;&G;&G;">
 <!ENTITY I "&H;&H;&H;&H;&H;&H;&H;&H;&H;&H;">
 <!ENTITY J "&I;&I;&I;&I;&I;&I;&I;&I;&I;&I;">
]>
<lolz>&J;</lolz>

The solution to this type of vulnerability is to disable External Entities in the XML parser or when this is not possible, not to use External Entities, set a limit on the maximum level of expansion.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
Singapore Singapore
Shao Voon is from Singapore. His interest lies primarily in computer graphics, software optimization, concurrency, security, and Agile methodologies.

In recent years, he shifted focus to software safety research. His hobby is writing a free C++ DirectX photo slideshow application which can be viewed here.

Comments and Discussions

 
Questionalways catch bad_alloc Pin
SeattleC++31-Dec-18 14:10
SeattleC++31-Dec-18 14:10 
When an exception is thrown, the stack is unwound step by step looking for a handler. This process may liberate an arbitrary amount of memory. Wherever you catch bad_alloc is a place where your program gets a chance to decide what to do. It might be that retry or fail-and-continue is a more appropriate action than stop.

When you don't catch bad_alloc (or any other exception) C++ is not required to unwind the stack. On Linux, the memory is recovered when the process exits, but Windows is not so generous; the allocated-but-never-freed memory leaks.

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.