A tester for strong exception guarantee in C++11

• Download source code - 5 KB

Introduction

Have you ever wondered how robust your class templates are when faced with exceptions? If your answer is yes, this tester may help you. Adapted from a Matt Arnold’s original idea, a variation of this technique is revamped here for its use with C++11 class templates.

We will first state the problem of the lack of strong safety before exceptions. Later we will give some tips on how to use the tester (you will see it is very straightforward). Finally, we will deepen in how it works and its possible drawbacks. 

The problem

Suppose you have written a class template, say, my_class<T>. This template may hold objects of a great diversity of types T, possibly none of them implemented by you. Somebody else is using your class template, instantiated for a certain third_party type. Now this person calls a method on your class, say:

my_class<third_party> my_class_object;  // Object created
my_class_object.my_method(arguments…);  // Method called

You probably were very careful at the time you wrote my_method, so your own implementation cannot throw any exceptions. But what if an internal operation in third_party throws an exception inside my_method? How do you think your client (that “somebody else” using your class template) will react?

You can analyze this situation from two different points of view:

1. Should I catch exceptions of this kind inside my_method or should I let them go upstream?
2. What is the state of my container (my_class_object) after the exception has been thrown? 

First note we are using the term “container” in a very broad sense. Your class template is a container either of one or multiple T objects. 

In my opinion, if my_class<T> is a general container template (intended for a wide range of types T), the best option is to let exceptions of this kind go upstream. Otherwise we might be obscuring the exception cause to our client, and probably we had to handle exceptions differently according to their types. This may lead to inelegant non-sturdy code.

Regarding the second question, our container will stay after the exception in one of these three states:

1. Broken. This is the worst situation. Probably some resources have leaked. There could be dangling pointers. The original state of the container (before the exception) is unrecoverable. It may be even impossible to destroy it safely. This is a symptom of an implementation bug. 
2. Unusable but safe. No resources have leaked. It is not possible to know the contents of the container unless we inspect it. The original state of our container is unrecoverable but at least it can be destroyed safely. In this case our container is said to fulfill the basic exception guarantee.
3. Untouched. This is the ideal situation. The container keeps the state and contents prior to the exception. The container has shown to be transparent to third party exceptions. In this case our container is said to fulfill the strong exception guarantee.   

There is an even stronger guarantee. If my_method does not throw itself and call methods of T that cannot throw either (the destructor, for instance), my_method is said to fulfill the nothrow guarantee. 

Is it always possible to write a strong exception safe method? Yes. For methods other than constructors of any type, these are the steps to be taken:

1. Make a copy of your container. If this process throws, your container will remain intact.
2. Carry out the desired method on the copy. If this process throws, your original container will remain intact again.
3. Swap the original container guts with the transformed container ones. Make sure these swapping operations do not throw exceptions (this is guaranteed for primitive and pointer types and can be achieved for more complex ones). 

If you are really interested in these techniques (and those affecting constructors), reference [2] is a very reputable reading.

So why bother if there is a canonical way of writing strong exception safe methods? Well, although safe, the final method implementation may turn out to be fairly inefficient; think that we are making a deep and probably costly copy in step number one above.

Your goal must be to write elegant, strong exception safe code unless efficiency gets seriously compromised. Achieve a basic exception guarantee only as a minimum. The tester herein included will help you reach this goal.  

The tester

Using the tester is very easy. As an example, let’s test some methods of std::list<>.

First of all, include some necessary headers and create a non-empty list to be tested: 

#include <list>
#include "strong_tester.h"
#include "third_party.h"

int main(int argn, char *argc[])
{
    std::list<third_party> tested{9, 5, 3, 7, 1, 16, 34, 56, 32, -12, -34};
    // …

Observe that the list is instantiated for the third_party type. third_party objects are initialized from plain integers. The name third_party is intentional: it represents the “worst” class (which you have not written and over which you do not have any control) your tested container can hold. third_party is an evil class that will throw exceptions once and again. 

Now let’s test the reverse method, for instance:

    // Test of void reverse() noexcept
    strong_test("void reverse() noexcept",        // Method signature 
                tested,                           // Container to be tested
                &std::list<third_party>::reverse  // Method to be tested
                );
 
    return 0;
}

If we compile and run this program, we will obtain the following output on the screen: 

Test of void reverse() noexcept

And that’s all. After seeing this output, we can be (say) 99% sure the reverse method is strong exception safe (in fact it is because reverse fulfills the nothrow guarantee). We will discuss that 1% uncertainty later.

Let’s test the assignment operator now. operator= has three signatures, namely:

list& operator= (const list& x);                    // Signature 1
list& operator= (list&& x);                         // Signature 2
list& operator= (initializer_list<value_type> il);  // Signature 3

Let’s test them all. For signature 1 we need another list to copy from:

std::list<third_party> other{4, 6, 2, 5, 90, -32, -5, 67, 45, -11, 59, -6, -32, 12, 11};
 
// Test of list& operator= (const list& x)
typedef std::list<third_party>&
        (std::list<third_party>::*assignment_ptr_type)
        (const std::list<third_party>&);                                  // [1]

assignment_ptr_type assignment_ptr=&std::list<third_party>::operator=;    // [2]

strong_test("list& operator= (const list& x)",    // Method signature
            tested,                               // Container to be tested
            assignment_ptr,                       // Method to be tested
            other                                 // Method argument
            );                                                            // [3]

First, we typedef an assignment_ptr_type type matching the method signature [1]. Second, we create a member function pointer (called assignment_ptr) to operator= [2]. And last, we carry out the test [3].

For signature 2 we need a temporary list:

// Test of list& operator= (list&& x)
typedef std::list<third_party>& 
        (std::list<third_party>::*move_assignment_ptr_type)
        (std::list<third_party>&&);
 
move_assignment_ptr_type move_assignment_ptr=&std::list<third_party>::operator=;
 
strong_test("list& operator= (list&& x)",        // Method signature         
            tested,                              // Container to be tested
            move_assignment_ptr,                 // Method to be tested
            std::list<third_party>               // Method argument
              {4, 6, 2, 5, 90, -32, -5, 67, 45, -11, 59, -6, -32, 12, 11}
            );

For signature 3 we need an initialization list:

// Test of list& operator= (initializer_list<value_type> il)
typedef std::list<third_party>& 
        (std::list<third_party>::*il_assignment_ptr_type)
        (std::initializer_list<third_party>);
 
il_assignment_ptr_type il_assignment_ptr=&std::list<third_party>::operator=;
 
strong_test("list& operator= (initializer_list<value_type> il)",  // Method signature
            tested,                                               // Container to be tested
            il_assignment_ptr,                                    // Method to be tested
            std::initializer_list<third_party>                    // Method argument
              ({4, 6, 2, 5, 90, -32, -5, 67, 45, -11, 59, -6, -32, 12, 11})
            );

These are the results corresponding to the three previous tests:

Test of list& operator= (const list& x)
Strong exception guarantee NOT fulfilled. Sources: [ASSIGNMENT] [COPY_CONSTRUCTOR]
 
Test of list& operator= (list&& x)
 
Test of list& operator= (initializer_list<value_type> il)
Strong exception guarantee NOT fulfilled. Sources: [ASSIGNMENT] [COPY_CONSTRUCTOR]

Wait, wait. Are you telling me that the copy (in opposition to move) assignment operator is not strong exception safe for Standard Template Library lists? Well, I guess it depends on vendor’s implementation but, for the sake of efficiency, it is usually not (if not always).

So, how do we interpret the results above? If the output matches the following pattern:

Test of <method_signature>

you can be almost sure your method is strong exception safe. If, on the contrary, the result matches the following pattern:

Test of <method_signature>
Strong exception guarantee NOT fulfilled. Sources: [<source_1>] [<source_2>]…

you can be 100% sure your method is not strong exception safe if the contained class parameter T launches exceptions in any of its internal operations: source_1, source_2,...

Thus, std::list<T>’s copy assignment operator is not strong exception safe as long as T’s assignment operator or copy constructor themselves may throw. 

These are the lack-of-guarantee sources we have considered:

• [CONSTRUCTOR]: The container’s tested method is not strong exception safe when faced … with an exception in T’s constructor.
• [NEW_ALLOCATION]: ... with a memory exhaustion when trying to allocate a T object. Probably you will never see this source.
• [COPY_CONSTRUCTION]: … with an exception in T’s copy constructor.
• [MOVE_CONSTRUCTOR]: … with an exception in T’s move constructor.
• [ASSIGNMENT]: … with an exception in T’s copy assignment operator.
• [MOVE_ASSIGNMENT]: … with an exception in T’s move assignment operator.
• [OP== OR OP!=]: … with an exception in T’s operator== or operator!=. We do not individualize them because one operator is usually implemented in terms of the other.
• [OP<= OR OP>]: … with an exception in T’s operator<= or operator>. We do not individualize them because one operator is usually implemented in terms of the other.
• [OP>= OR OP<]: … with an exception in T’s operator>= or operator<. We do not individualize them because one operator is usually implemented in terms of the other. 

So far we have tested methods with zero or one argument at most. You can test methods with as many arguments as written in theirs signatures. Just make sure you respect the correct argument order.

The machinery

The function template strong_test_impl does the tests, once at a time. This is how it is written:

template <typename Container, typename Operation, typename... Arguments>
void strong_test_impl(
            std::set<std::string>& failure_sources,
            const Container& tested,
            const Operation& operation,
            Arguments&&... arguments
            )
{
    Container copy(tested);          // [1]

    try
    {
        thrower::enable_throw();     // [2]
        // Operation that throws...
        (copy.*operation)(std::forward<Arguments>(arguments)...);  // [3]
        thrower::disable_throw();    // [4]
    }
    catch (std::exception& ex)
    {
        thrower::disable_throw();    // [5]

        // Strong exception guarantee test
        if(copy!=tested)             // [6]
            failure_sources.insert(ex.what());  // [7]
    }
}

First, we make a copy of our tested container [1]. During this operation, the throwing mechanism into third_party is disabled, so no exception can be launched, save an extremely unlikely NEW_ALLOCATION, at most. Second, exceptions into third_party are enabled [2]. We call our tested method in [3] with its required arguments. Notice that the method is called on the copy, not on the tested container. If no exception is thrown we disabled the throwing mechanism into third_party, waiting for the next test [4]. If, on the contrary, an exception is thrown, this is caught in the catch block. We then disable exceptions [5] to immediately carry out the actual test: comparing the original and copied containers [6]. If these are not equal it means the strong exception safety guarantee has been violated and the lack-of-guarantee source is stored in a std::set [7].

As you can see, the idea is quite simple. But what if (inside the catch block) copied and tested containers are equal? What does it mean? Well, you may conclude that the strong exception safety has been safeguarded, but this is not necessarily true. To see why, consider a sorting method, for example. Probably the first important operation will be to compare two elements, before taking further actions. If this first comparison throws, the copied container has not been modified yet. But what prevents it from being modified if the same kind of exception occurs at a later time?

One possible key to palliate the problem above is randomness. In effect, exceptions in third_party are thrown at random, with a probability of 25% each time a method is called. But once the randomness has been brought in, it is necessary to repeat each test many times and to see the overall behavior. That is a task for strong_test function template:

template <typename Container, typename Operation, typename... Arguments>
void strong_test(
            const std::string& test_name,
            const Container& tested,
            const Operation& operation,
            Arguments&&... arguments
            )
{
    // Number of runs for a single operation to be tested
    const size_t number_of_runs=1000;
 
    // If strong exception guarantee is not fulfilled,
    // the source is stored in failure_sources.
    std::set<std::string> failure_sources;
 
    // The same operation is tested number_of_runs times.
    for(size_t i=0; i< number_of_runs; ++i)
            strong_test_impl(failure_sources,
                             tested,
                             operation,
                             std::forward<Arguments>(arguments)...
                             );
 
    //
    // Failure sources printing
    //
    
    // (non-relevant code here)
}

The strong_test function template is self-explanatory. number_of_runs has been chosen to be 1000.

Limiting the uncertainty

Can the tester fail in detecting a lack of strong exception guarantee? Yes, and that is because of the problem random nature. You can though limit this drawback very much by simply following these tips:

1. Populate your containers as much as you can before starting the tests; the bigger the number of operations, the higher the probability of detecting anomalies.
2. Make argument containers (other) bigger than tested containers (tested).
3. Make number_of_runs even greater if necessary. See strong_test.h. 
4. Vary the throwing probability. See thrower.h and consider multiples of numbers other than 4. 

Follow the tips above as a rule of thumb.

Source code

The strong exception guarantee tester implementation code has been attached, along with a main sample code. The code has been written with Code::Blocks and compiled with MinGw (mingw-get-inst-20120426).

References

1. Lessons learned from Specifying Exception-Safety for the C++ Standard Library. David Abrahams. Boost C++ Libraries. (http://www.boost.org/community/exception_safety.html).
2. Exceptional C++. 47 Engineering Puzzles, Programming Problems and Solutions. Herb Sutter. Addison-Wesley. Items 8 to 17.