Bug #83034 MySQL is not ThreadSanitizer-clean
Submitted: 18 Sep 2016 16:35 Modified: 26 Jul 2017 4:14
Reporter: Laurynas Biveinis (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Compiling Severity:S3 (Non-critical)
Version:8.0 OS:Any
Assigned to: CPU Architecture:Any
Tags: tsan

[18 Sep 2016 16:35] Laurynas Biveinis
Description:
cmake ... -DWITH_TSAN=ON -DWITH_DEBUG=ON ...
...
./mtr --debug-server 1st
...
Installing system database...
(two days later)
mysql-test-run: *** ERROR: Error executing mysqld --initialize

var/log/bootstrap.log is 57MB and contains 5110 ThreadSanitizer warnings.

Let's try reducing that a bit:
cmake ... -DWITH_TSAN=ON -DWITH_DEBUG=ON -DMUTEXTYPE=sys ...
...
./mtr --debug-server 1st
Installing system database...
(two minutes (not days!) later)
mysql-test-run: *** ERROR: Error executing mysqld --initialize

var/log/bootstrap.log is 8MB and contains only 629 warnings.

How to repeat:
See above

Suggested fix:
Obviously Thread Sanitizer support is currently experimental, and it will take non-trivial amount of work to clean the codebase, nevertheless that is a good goal to achieve. Even in the current state TSan has already proven to be useful in catching particular bugs - when it's known in advance what to grep for in the log file - and in zero-warning state would be even more useful.
[19 Sep 2016 6:02] Umesh Shastry
Hello Laurynas,

Thank you for the report.

Thanks,
Umesh
[19 Sep 2016 6:29] Umesh Shastry
test results

Attachment: 83034.results (application/octet-stream, text), 1.05 MiB.

[19 Sep 2016 19:13] Mark Callaghan
We would be thrilled to see this get done
[13 Apr 2017 14:21] Laurynas Biveinis
An update for 8.0.1. 

CMake options -DWITH_DEBUG=ON -DWITH_TSAN=ON -DMUTEXTYPE=sys

On Linux, one has to use clang, not gcc, or the build fails with

-- The C compiler identification is GNU 6.2.0
-- The CXX compiler identification is GNU 6.2.0
...
[  8%] Built target comp_err
Scanning dependencies of target GenError
[  8%] Generating ../include/mysqld_error.h, ../sql/share/english/errmsg.sys
FATAL: ThreadSanitizer: unexpected memory mapping 0x559059659000-0x559059a01000
utilities/CMakeFiles/GenError.dir/build.make:63: recipe for target 'include/mysqld_error.h' failed
make[2]: *** [include/mysqld_error.h] Error 66

Likewise with GCC 5.4.

With clang, it builds. "./mtr 1st" results in:

Installing system database...
mysql-test-run: *** ERROR: Error executing mysqld --initialize

bootstrap.log contains 723 tsan warnings
[13 Apr 2017 14:45] Laurynas Biveinis
There has been some effort in 8.0.1:

commit 4fcf61bc8041439a43620a8ea266aac049c66c74
Author: Jon Olav Hauglid <jon.hauglid@oracle.com>
Date:   Fri Oct 28 16:22:25 2016 +0200

    Bug#24970905: ADD THREAD SANITIZER BLACKLIST
    
    In order to support incrementally fixing issues identified by
    Thread Sanitizer (-DWITH_TSAN=1), add support for a blacklist
    which can be used to suppress issues yet to be fixed.
    
    The initial version of the blacklist contains suppressions
    required to get through main.1st.
    
    Also the patch extends the --sanitize MTR option so that
    error logs are scanned for Thread Sanitizer warnings.

commit 0b3825f2426f7ba019b62eb481fe1a8068fecd18
Author: Jon Olav Hauglid <jon.hauglid@oracle.com>
Date:   Tue Nov 8 11:36:16 2016 +0100

    Bug#25027185: TSAN: DATA RACE IN MY_TIMER_MICROSECONDS
    
    Fix data race in my_timer_microseconds() - use std::atomic
    for static variable modified by multiple threads.
    
    Also add more suppressions needed to run main.alter_table.
[26 Jul 2017 4:14] Laurynas Biveinis
Related bug 83601