AddressSanitizer

https://github.com/google/sanitizers/wiki/AddressSanitizer

ASan is built into gcc. You can pass the options gcc needs for ASan to cmake via CMAKE_CXX_FLAGS:

cmake .. -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS="-Werror -fsanitize=address -fno-omit-frame-pointer"

Then build and run as usual. Nothing else special is needed.

Currently we have the very helpful static init order checking enabled this way in main.cpp:

// ASAN_OPTIONS.
extern "C" ROSEGARDENPRIVATE_EXPORT const char *__asan_default_options() {
    // Turn on static init order fiasco detection for ASan.
    return "check_initialization_order=true:strict_init_order=true";
}

Options can also be passed to ASan via the ASAN_OPTIONS environment var. E.g. to accomplish the same as above:

ASAN_OPTIONS=check_initialization_order=true:strict_init_order=true build/rosegarden

ASan will end Rosegarden if a plugin is misbehaving at startup. The logging will show the offending plugin as the last one it tried to load before exiting:

DSSIPluginFactory::discoverPlugin(): /usr/lib/dssi/libzynaddsubfx_dssi.so
=================================================================
==340386==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x7f48df95f5a0 in thread T9 (QThread)

Rosegarden supports a ROSEGARDEN_PLUGIN_BLACKLIST environment variable (see LADSPAPluginFactory::discoverPlugins()). E.g. to disable plugins with matrix or pitch in their names:

$ ROSEGARDEN_PLUGIN_BLACKLIST="matrix|pitch" ./rosegarden

To disable all plugins (safe mode if you will):

$ ROSEGARDEN_PLUGIN_BLACKLIST=".*" ./rosegarden

Another option is to uninstall the offending plugin.

I did dig into ASan's suppression capabilities and for the life of me I couldn't figure out how to get it to work. I was able to get a suppressions file to be parsed, but it never had any effect on the detection of plugin errors.

In a typical run, ASan reports multiple leaks in libraries that we have no control over. We can suppress these reports using LSAN_OPTIONS. First, create an LSAN.supp suppression file someplace predictable:

leak:libfontconfig.so
leak:libfreetype.so
leak:libdbus-1.so
leak:libraptor2.so
leak:liblrdf.so

Then refer to it in LSAN_OPTIONS when running:

$ LSAN_OPTIONS="suppressions=/home/ted/LSAN.supp" ./rosegarden

Can we add this at runtime like we do with asan_default_options()?

Just run Rosegarden as usual. It will run at about half its normal speed. When a problem is encountered, Rosegarden will end and stderr will have the details.

ASan reports hundreds of memory leaks in Rosegarden and the libraries it uses. This is quite overwhelming. Either pipe the output to a file:

./rosegarden 2>asan.out

Or run in an IDE that captures the output (e.g. Eclipse).

You can find the Rosegarden memory leaks in the output by searching for “Rosegarden::”.

Running “make test” on an ASan build is a good way to find problems. However, memory leaks outside of Rosegarden will cause the tests to fail. This requires examination of the logs after a test run to confirm there are no problems we can solve.

I do wonder whether these “external” leaks aren't actually being caused by Rosegarden somehow. Might be worth doing a little more digging.

Do an ASan build as described above, then run make test as usual. Some tests will fail. You can either examine the test log:

grep "Totals:" Testing/Temporary/LastTest.log

Or re-run individual tests and examine the output of each:

test/convert |& less

As of this writing (May 2023) I'm only seeing memory leaks related to libfontconfig.so:

=================================================================
==386526==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4352 byte(s) in 17 object(s) allocated from:
    #0 0x7f39ce2b4867 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f39c08d5a21  (/lib/x86_64-linux-gnu/libfontconfig.so.1+0x1fa21)

Indirect leak of 1056 byte(s) in 33 object(s) allocated from:
    #0 0x7f39ce2b4a37 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f39c08d851e  (/lib/x86_64-linux-gnu/libfontconfig.so.1+0x2251e)

Indirect leak of 321 byte(s) in 32 object(s) allocated from:
    #0 0x7f39ce25b9a7 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
    #1 0x7f39c08d54f7 in FcValueSave (/lib/x86_64-linux-gnu/libfontconfig.so.1+0x1f4f7)

SUMMARY: AddressSanitizer: 5729 byte(s) leaked in 82 allocation(s).

It might be possible to suppress these leaks via an LSAN suppressions file. I had no success with using an ASAN suppressions file, so I'm not sure how successful we might be here. And, of course, we would need to be sure this external leak isn't somehow being caused by Rosegarden.

Add the no_sanitize_address attribute to functions that appear to be causing false positives, or that are getting in the way of tracking down others:

__attribute__((no_sanitize_address))
void foo()
{
    // Do something...
}

We'll need to periodically review these and hopefully remove them all in the end.