16.1 Getting started

The first example we'll look at is when the argument in a call to free() doesn't match the return value from malloc(), even though the intention is to free the memory that was allocated by malloc(). This example is in tests/fail/test1.c and causes many existing malloc() implementations to crash.

Along the way, I'll try to describe as many features of the mpatrol library as possible, and illustrate them with examples. Note that the output from your version of the library is likely to vary slightly from that shown in the examples, especially on non-UNIX systems.

     23  /*
     24   * Allocates a block of 16 bytes and then attempts to free the
     25   * memory returned at an offset of 1 byte into the block.
     26   */
     
     
     29  #include "mpatrol.h"
     
     
     32  int main(void)
     33  {
     34      char *p;
     
     36      if (p = (char *) malloc(16))
     37          free(p + 1);
     38      return EXIT_SUCCESS;
     39  }

Note that I've removed the copyright message from the start of the file and added line numbers so that the tracing below makes more sense.

After compiling and linking the above program with the mpatrol library, the MPATROL_OPTIONS environment variable should be set to be LOGALL and the program should be executed, generating the following output in mpatrol.log.

     @(#) mpatrol 1.5.1 (08/12/16)
     Copyright (C) 1997-2008 Graeme S. Roy
     
     This is free software, and you are welcome to redistribute it under
     certain conditions; see the GNU Lesser General Public License for
     details.
     
     For the latest mpatrol release and documentation,
     visit http://sourceforge.net/projects/mpatrol.
     
     operating system:       UNIX
     system variant:         Linux
     processor architecture: Intel 80x86
     processor word size:    32-bit
     object file format:     BFD
     dynamic linker type:    SVR4
     
     Log file generated on Tue Dec 16 10:10:12 2008
     
     read 310 symbols from /usr/lib/libmpatrol.so.1.5
     read 647 symbols from /usr/lib/libbfd-2.9.5.0.22.so
     read 2634 symbols from /lib/libc.so.6
     read 1142 symbols from /usr/lib/libstdc++-libc6.1-1.so.2
     read 695 symbols from /lib/libm.so.6
     read 178 symbols from /lib/ld-linux.so.2
     read 158 symbols from ./test1
     
     ALLOC: malloc (52, 16 bytes, 4 bytes) [main|test1.c|36]
             0x0804942F main+31
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33
     
     returns 0x080620E8
     
     FREE: free (0x080620E9) [main|test1.c|37]
             0x08049457 main+71
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33
     
     ERROR: [MISMAT]: free: 0x080620E9 does not match allocation of 0x080620E8
         0x080620E8 (16 bytes) {malloc:52:0} [main|test1.c|36]
             0x0804942F main+31
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33
     
     system page size:  4096 bytes
     default alignment: 4 bytes
     overflow size:     0 bytes
     overflow byte:     0xAA
     allocation byte:   0xFF
     free byte:         0x55
     allocation stop:   0
     reallocation stop: 0
     free stop:         0
     unfreed abort:     0
     small boundary:    32 bytes
     medium boundary:   256 bytes
     large boundary:    2048 bytes
     lower check range: 0
     upper check range: 0
     check frequency:   1
     failure frequency: 0
     failure seed:      972951591
     prologue function: <unset>
     epilogue function: <unset>
     handler function:  <unset>
     log file:          mpatrol.log
     profiling file:    mpatrol.out
     tracing file:      mpatrol.trace
     program filename:  ./test1
     symbols read:      5764
     autosave count:    0
     freed queue size:  0
     allocation count:  52
     allocation peak:   20 (427512 bytes)
     allocation limit:  0 bytes
     allocated blocks:  7 (1528 bytes)
     marked blocks:     0 (0 bytes)
     freed blocks:      0 (0 bytes)
     free blocks:       4 (432648 bytes)
     internal blocks:   33 (540672 bytes)
     total heap usage:  974848 bytes
     total compared:    0 bytes
     total located:     2 bytes
     total copied:      32176 bytes
     total set:         582856 bytes
     total warnings:    0
     total errors:      1

Ignoring the copyright blurb and target environment information at the top of the file, let's first take a look at the initial log message from the library. I've annotated each of the items with a number that corresponds to the descriptions below.

      (1)    (2)   (3)    (4)       (5)     (6)    (7)  (8)
       |      |     |      |         |       |      |    |
       V      V     V      V         V       V      V    V
     ALLOC: malloc (52, 16 bytes, 4 bytes) [main|test1.c|36]
      (9) -> 0x0804942F main+31
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33 <- (10)
     
     returns 0x080620E8 <- (11)

1. Allocation type. This generalises the type of dynamic memory operation that is being performed, and can be one of `ALLOC', `REALLOC' or `FREE'. This should make looking for all allocations, reallocations or frees in the log file a lot easier. Alternatively, if a memory operation function was called then this can also be one of `MEMSET', `MEMCOPY', `MEMFIND' or `MEMCMP'.
2. Allocation function. This is the name of the function that has been called to allocate the memory, in this case `malloc'.

3. Allocation index. This is incremented every time a new memory allocation is requested, and persists even if the memory allocation is resized with realloc() and its related functions, so can be useful to keep track of a memory allocation, even if its start address changes. The mpatrol library may use up the first few allocation indices when it gets initialised.
4. Size of requested allocation.
5. Alignment for requested allocation. This is normally the default system alignment for general-purpose memory allocations, but may be different depending on the type of function that is used to allocate the memory.

The following information contains source file details of where the call to malloc() came from, but is only available if the source file containing the call to malloc() included mpatrol.h; otherwise the fields will all be `-'¹. Because of the convoluted way this information is obtained for the C++ operators, you may encounter some problems in existing C++ programs when making direct calls to operator new for example. However, if you want to disable the redefinition of the C++ operators in mpatrol.h you can define the preprocessor macro MP_NOCPLUSPLUS before the inclusion of that file. Alternatively, you may wish to define the MP_NONEWDELETE preprocessor macro in order to use MP_NEW, MP_NEW_NOTHROW and MP_DELETE instead of new and delete. That way you can combine calls to mpatrol's operators and the standard operators. Just make sure you don't mix them!

If you are running on a system on which mpatrol supports full symbolic stack tracebacks the following information may still be useful if the source files were compiled with optimisation turned on. This is because the calling function may have been inlined, in which case you will only see the name of the function into which the calling function was expanded in the stack traceback.

6. Function where call to malloc() took place. This information is only available if the source file containing the call to malloc() was compiled with gcc or g++.
7. Filename in which call to malloc() took place.
8. Line number at which call to malloc() took place.

The following information contains function call stack details of where the call to malloc() came from, but is only available if the mpatrol library has been built on a platform that supports this. The top-most entry should be the function which called malloc() and the bottom-most entry should be the entry-point for the process.

9. Address of function call. This is normally the address of the machine instruction immediately after the function call instruction, also known as the return address.

10. Function where call took place. This information is only available if the mpatrol library has been built on a platform that supports reading symbol table information from executable files, and then only if there is an entry in the symbol table corresponding to the return address. C++ function names may still be in their mangled form, but this can be easily rectified by processing the log file with a C++ name demangler. The number after the plus sign is the offset in bytes from the beginning of the function.

The following information is only available when the allocation type is `ALLOC' or `REALLOC' since it makes no sense when applied to `FREE'.

11. The address of the new memory block that has been allocated by malloc().

As you can see, there is quite a lot of information that can be displayed from a simple call to malloc(), and hopefully this information has been presented in a clear and concise format in the log file.

The next entries in the log file correspond to the call to free(), which attempts to free the memory allocated by malloc(), but supplies the wrong address.

The first four lines should be self-explanatory as they are very similar to those described above for malloc(). However, the next lines signal that a terminal error has occurred in the program, so I've annotated them as before.

     FREE: free (0x080620E9) [main|test1.c|37]
             0x08049457 main+71
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33
     
      (1)     (2)      (3)
       |       |        |
       V       V        V
     ERROR: [MISMAT]: free: 0x080620E9 does not match allocation of 0x080620E8
            (4)        (5)       (6)  (7)(8)  (9)    (10) (11)
             |          |         |    |  |    |      |    |
             V          V         V    V  V    V      V    V
         0x080620E8 (16 bytes) {malloc:52:0} [main|test1.c|36]
     (12) -> 0x0804942F main+31
             0x4007C9CB __libc_start_main+255
             0x08049381 _start+33

1. Error severity. The mpatrol library has two different severities of error: `WARNING' and `ERROR'. The first is always recoverable, and serves only to indicate that something is not quite right, and so may be useful in determining where something started to go wrong. The second may or may not be recoverable, and the library terminates the program if it is fatal, displaying any relevant information as it does this.

2. Error abbreviation code. This is a code that is different for each type of error that is detected by the mpatrol library. Some warnings and errors that are not directly related to the program being run will not contain this field. See the appendix on diagnostic messages (see Diagnostic messages) for a complete list of all possible error abbreviation codes and their descriptions.
3. Allocation function. This is the name of the function used to allocate, reallocate or free memory where the error was detected. This may be omitted if an error is detected elsewhere in the library.

The following information is related to the information that the library has stored about the relevant memory allocation. This information is always displayed in this format when details of individual memory allocations are required. If any information is missing then it simply means that the library was not able to determine it when the memory block was first allocated.

4. Address of memory allocation.
5. Size of memory allocation.
6. Allocation function. This is the name of the function that was called to allocate the memory block, in this case `malloc'. If the memory allocation has been resized then this will be either `realloc', `reallocf', `recalloc', `expand' or `xrealloc'.
7. Allocation index.
8. Reallocation index. This is used to count the number of times a memory allocation has been resized with realloc() and its related functions.
9. Function where original call to malloc() took place. If the memory allocation has been resized then this will be the name of the function which last called realloc() and its related functions.
10. Filename in which original call to malloc() took place. If the memory allocation has been resized then this will be the filename in which the last call to realloc() and its related functions took place.
11. Line number at which original call to malloc() took place. If the memory allocation has been resized then this will be the line number at which the last call to realloc() and its related functions took place.
12. Function call stack of original memory allocation. If the memory allocation has been resized then this will be the call stack of the last call to realloc() and related functions.

So, the mpatrol library detected the error in the above program and terminated it. When the library terminates it always displays a summary of various memory allocation statistics and settings that were used during the execution of the program.

The various settings and statistics displayed by the library for the above example have been numbered and their descriptions appear below.

      1  system page size:  4096 bytes
      2  default alignment: 4 bytes
      3  overflow size:     0 bytes
      4  overflow byte:     0xAA
      5  allocation byte:   0xFF
      6  free byte:         0x55
      7  allocation stop:   0
      8  reallocation stop: 0
      9  free stop:         0
     10  unfreed abort:     0
     11  small boundary:    32 bytes
     12  medium boundary:   256 bytes
     13  large boundary:    2048 bytes
     14  lower check range: 0
     15  upper check range: 0
     16  check frequency:   1
     17  failure frequency: 0
     18  failure seed:      972951591
     19  prologue function: <unset>
     20  epilogue function: <unset>
     21  handler function:  <unset>
     22  log file:          mpatrol.log
     23  profiling file:    mpatrol.out
     24  tracing file:      mpatrol.trace
     25  program filename:  ./test1
     26  symbols read:      5764
     27  autosave count:    0
     28  freed queue size:  0
     29  allocation count:  52
     30  allocation peak:   20 (427512 bytes)
     31  allocation limit:  0 bytes
     32  allocated blocks:  7 (1528 bytes)
     33  marked blocks:     0 (0 bytes)
     34  freed blocks:      0 (0 bytes)
     35  free blocks:       4 (432648 bytes)
     36  internal blocks:   33 (540672 bytes)
     37  total heap usage:  974848 bytes
     38  total compared:    0 bytes
     39  total located:     2 bytes
     40  total copied:      32176 bytes
     41  total set:         582856 bytes
     42  total warnings:    0
     43  total errors:      1

1. System page size. This value is used on some platforms when allocating and protecting system memory.
2. Default alignment. This value is the minimum alignment required for general purpose memory allocations, and is usually the alignment required by the most restrictive datatype on a given system. It is used when allocating memory that has no specified alignment. It can be changed at run-time using the DEFALIGN option, but setting this value too small may cause the program to crash due to bus errors which are caused by reading from or writing to misaligned data.
3. Overflow size. This value is the size used by one overflow buffer. If this is non-zero then every memory allocation will have two overflow buffers; one on either side. These buffers are used by the library to detect if the program has written too many bytes to a memory allocation, thus overflowing into one of the buffers, but these extra checks can slow down execution speed. It can be changed at run-time using the OFLOWSIZE option.
4. Overflow byte.
5. Allocation byte.
6. Free byte. These values are used by the library to pre-fill blocks of memory for checking purposes. The overflow byte is used to fill overflow buffers, the allocation byte is used to fill newly-allocated memory (except from calloc() or recalloc()), and the free byte is used to fill free blocks or freed memory allocations. These can be changed at run-time using the OFLOWBYTE, ALLOCBYTE and FREEBYTE options.
7. Allocation stop.
8. Reallocation stop.
9. Free stop. These values are used by the library to halt the program when run inside a debugger whenever a specified allocation index is allocated, reallocated or freed. These can be changed at run-time using the ALLOCSTOP, REALLOCSTOP and FREESTOP options.

10. Unfreed abort. This value is used when the program terminates and is used by the library to check if there are more than a given number of unfreed memory allocations. If there are then the library will cause the program to abort with an error. It can be changed at run-time using the UNFREEDABORT option.
11. Small boundary.
12. Medium boundary.
13. Large boundary. These values are used in memory allocation profiling and specify the boundaries in bytes between small, medium, large and extra large allocations. These can be changed at run-time using the SMALLBOUND, MEDIUMBOUND and LARGEBOUND options.
14. Lower check range.
15. Upper check range.
16. Check frequency. These values specify the range of allocation indices through which the library will physically check every area of free memory and every overflow buffer for errors, along with the frequency at which to make the checks. A dash specifies that either the lower or upper range is infinite, but if they are both zero then no such checking will ever be performed, thus speeding up execution speed dramatically. The check frequency indicates the number of memory allocation events that must occur in between checking the heap. The library defaults to performing no such checks. This can be changed at run-time using the CHECK option.
17. Failure frequency.
18. Failure seed. These values are used to specify if random memory allocation failures should occur during program execution, for the purposes of stress testing a program. If the failure frequency is zero then no random failures will occur, but if it is greater than zero then the higher the number, the less frequent the failures. The failure seed is used internally by the mpatrol library when generating random numbers. If it is zero then the seed will be set randomly, but if it is greater than zero then it will be used to generate a predictable sequence of random numbers; i.e. two runs of the same program with the same failure frequencies and the same failure seeds will generate exactly the same sequence of failures.

19. Prologue function.
20. Epilogue function.
21. Handler function. These values contain addresses or names of functions that have been installed as callback functions for the library. These functions, if set, will be called from the library at appropriate times during program execution in order to handle specific events. These can be changed at compile-time using the __mp_prologue(), __mp_epilogue() and __mp_nomemory() functions.
22. Log file. Simply contains the name of the file where all mpatrol library diagnostics go to. It can be changed at run-time using the LOGFILE option.
23. Profiling file. Contains the name of the file where all of the mpatrol library memory allocation profiling information goes when the PROF option is used. It can be changed at run-time using the PROFFILE option.
24. Tracing file. Contains the name of the file where all of the mpatrol library memory allocation tracing information goes when the TRACE option is used. It can be changed at run-time using the TRACEFILE option.
25. Program filename. Contains the full pathname to the program's executable file. This is used by the mpatrol library to read the symbol table in order to provide symbolic information in function call stacks. It can be changed at run-time using the PROGFILE option.
26. Symbols read. This value contains the total number of symbols read from a program's executable file and/or the dynamic linker, if applicable.
27. Autosave count. This value contains the frequency at which the mpatrol library should periodically write the profiling data to the profiling output file. When the total number of profiled memory allocations and deallocations is a multiple of this number then the current profiling information will be written to the profiling output file. It can be changed at run-time using the AUTOSAVE option.

28. Freed queue size. This value contains the maximum number of freed memory allocations that will be stored in the freed queue if the NOFREE option is used. Once the freed queue becomes full then the oldest freed allocation in the queue will be returned to the free memory pool for reuse every time an existing memory allocation is freed. If this value is zero then the freed queue will never contain any freed allocations. It can be changed at run-time using the NOFREE option.
29. Allocation count. This value contains the total number of memory allocations that were created by the mpatrol library. This value may be more than expected if the mpatrol library makes any memory allocations during initialisation.

30. Allocation peak. This value contains the peak memory usage set by the program when running; the peak number of memory allocations, and also the peak number of bytes allocated in parentheses (the two numbers may peak at different times throughout the lifetime of the program). This value may be more than expected if the mpatrol library makes any memory allocations during initialisation.
31. Allocation limit. This value is used to limit the amount of memory that can be allocated by a program, which can be useful for stress-testing in simulated low memory conditions. It can be changed at run-time using the LIMIT option.

32. Allocated blocks.
33. Marked blocks.
34. Freed blocks.
35. Free blocks. These values contain the total number of allocated, marked, freed and free blocks at the time the summary was produced. A marked block is an allocated block that the user has instructed (via the __mp_setmark() function) the mpatrol library should remain allocated for the rest of the lifetime of the program and should never be freed or counted as a memory leak. A freed block is an allocated block that has been freed but has not been returned to the free memory list for later allocation. These values may be different from those expected if the mpatrol library makes any memory allocations during initialisation. In this example a large amount of memory is used by the system object file access library which is used for reading the symbols from the program's executable file and any shared libraries that it requires.

36. Internal blocks. This value contains the total number of memory blocks (of varying sizes) that have been allocated from the system for the mpatrol library to use internally. These memory blocks will be write-protected on systems that support memory protection in order to prevent the program from corrupting the library's data structures. This can be overridden at run-time using the NOPROTECT option in order to speed up program execution slightly.

37. Total heap usage. This value contains the total amount of system heap memory that has been allocated by the mpatrol library.
38. Total compared.
39. Total located.
40. Total copied.
41. Total set. These values contain the total number of bytes that have been tracked by the mpatrol library in byte comparison operations (such as memcmp()), byte location operations (such as memchr(), byte copy operations (such as memcpy()) and byte set operations (such as memset()) respectively. They do not take into account any other such operations that occur outwith these functions, such as loading and storing from machine instructions.
42. Total warnings.
43. Total errors. The library keeps a count of the total number of warnings and errors it has displayed so that you can quickly work out this information at program termination.