Tracking heap memory use in Windows

Introduction

I recently encountered a problem with a C++ program that was allocating more memory than it should. It wasn’t leaking memory– that is, the program was well-behaved in the sense of eventually releasing every byte of memory that it allocated. But it allocated a lot, enough so that running multiple instances of the program simultaneously on cluster nodes would exceed physical memory capacity.

It should be relatively easy to troubleshoot memory issues like this. However, let’s suppose that our application runs on Windows, compiled in Visual Studio, and for mostly stupid reasons, the built-in Diagnostic Tools are unavailable, and we have neither administrative privilege nor support to install any of the other various existing tools for doing this sort of thing. How can we reinvent this wheel with a minimal amount of effort?

This turned out to be an interesting exercise, with a relatively simple implementation that solved the problem at hand, but also raised some questions that I don’t have good answers for.

Implementation

The result is available here, as well as on GitHub. It’s pretty easy to use: there is no need to modify application source code at all, just compile and link with mem_log.cpp in a debug build. There is only one function exposed in the corresponding header: mem::print_log(std::ostream&), which you can call yourself if desired, but it will be called automatically at program termination, printing a log of heap memory usage to stdout. For example, the following program:

int main()
{
    for (int i = 10; i > 0; --i)
    {
        int *a = new int[i];
        delete[] a;
    }
}

yields the following output:

                    Heap            Freed      Max. Alloc.
==========================================================
Caller: c:\users\username\test_mem\test_mem.cpp(5):main
Blocks:                0               10                1
Bytes:                 0              220               40

The first column indicates the current (and in the usual case, final) state of the heap, as the number of allocated blocks (i.e., calls to operator new) and corresponding total number of bytes. Non-zero values here suggest a memory leak… although more on this shortly.

The second column indicates the number of previously allocated blocks and bytes released by eventual calls to operator delete. Finally, the third column is the one that helped solve this problem, indicating the maximum number of bytes (and corresponding blocks) allocated at any one time.

There is one tricky note: in general there will be multiple records in the log, one for each line of application source code that calls operator new. Such a call may not be explicit as in the above example, but instead perhaps an indirect result of, say, std::vector::push_back, for example. We want to identify this call with the application’s code, not the standard library’s. We do this by walking the stack trace, starting from the globally replaced implementation of operator new, inspecting each corresponding source code filename until the first one is found containing the case-sensitive prefix specified by the preprocessor definition MEM_LOG_PATH (whose default is c:\users). If your application source code lives somewhere else, define MEM_LOG_PATH accordingly.

Limitations and questions

Although this code helped to solve this particular problem, it is still of limited use:

  1. It only works on Windows. I have indicated in source code comments where the Windows-specific stuff is, but have made no effort to implement the corresponding Linux calls to backtrace, etc.
  2. It is not thread-safe.
  3. It is not a leak detector. I only replaced operator new and delete, which is all that we really need to track memory usage in a well-behaved program that doesn’t care about alignment. We would also need to replace the array forms new[] and delete[] (which otherwise call our non-array implementations) to properly check for common undefined-behavior errors like freeing an array allocated via new[] with the non-array delete.
  4. Along the same lines as (3), I didn’t bother with the arguably much harder problem of tracking application memory usage via std::malloc and std::free.

Finally, the special, automagic global replacement behavior of implementing our own operator new and delete presents an interesting puzzle that I’m not sure how to solve, or whether it is even solvable. First, consider that we want to track every memory allocation and release, no matter how early (or late) it occurs during program execution. Since that tracking involves manipulation of our own internal data structure, we need to ensure that our internal object gets fully constructed– and stays constructed– by the time we need it to record any such unpredictably timed allocation.

One easy way to do this is with the construct on first use idiom: we instantiate our internal object as a function static variable, allocated on the heap the first time we need it, and never released. This intentionally “leaks” our object, but (1) so what? And (2) we have to do this, since there is no guarantee– as far as I can tell– that any particular non-trivial static object’s lifetime would persist beyond the last call to the allocation operators we are trying to track.

This works, in the sense that we can confidently track all heap allocations and releases, no matter when they occur… but now consider the final automatic call to mem::print_log(std::cout), which is realized in the destructor of a “dummy” static object whose only purpose is to try to be the last thing that happens in the program. But because of the static initialization order fiasco, there is still a possibility that we might “miss” a subsequent heap allocation or release during some later static object’s destruction. (That is, although we will track it, we won’t see it in the final printed log.) Is there any way to fix this? I think the answer is no, but I’m not sure.

This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to Tracking heap memory use in Windows

  1. wilmington says:

    That’s impressive! I don’t know what to think of a guy that groks generator functions and low level C++ stuff too! My head is spinning from reading 10 pages of generatorfunctionology.

    I’ll toss two ideas out.
    1. I’ve got a book by John Robbins, (2003) “Debugging Applications for Microsoft .NET and Microsoft Windows”. Yon about page 667, he discusses the Debug C Run-Time Library and Memory Management. It has built in support to find leaks with both new/delete and malloc/free. It’s an old book, and I don’t know whether this functionality still exists.
    2. You asked about how to make sure “that any particular non-trivial static object’s lifetime would persist beyond the last call to the allocation operators we are trying to track.” One way might be to make a trivial static object like an array of bytes, and then allocate the non-trivial object on top of it. You can do this with the placement variant of the operator new.

    Caveat: I haven’t tried any of this.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.