Bug #53593 Add some instrumentation to improve Valgrind sensitivity

• Submitted: 12 May 2010 9:34
• Modified: 14 Oct 2010 15:19
• Reporter: Marko Mäkelä
• Status: Closed
• Category: MySQL Server
• Severity: S3 (Non-critical)
• Version: 5.1+
• OS: Any
• Assigned to: Marko Mäkelä
• CPU Architecture: Any
• Tags: valgrind TRASH 0x8f

[12 May 2010 9:34] Marko Mäkelä

Description:
I added some Valgrind instrumentation when implementing compression in the InnoDB Plugin. Later, I ported the macros to the built-in InnoDB 5.1 too. Now that HAVE_purify enables UNIV_DEBUG_VALGRIND, we have found (and fixed) some bugs in InnoDB and even got 100% repeatability for Bug #38999 in the server.

In the MySQL core, there is the TRASH macro that can fill memory with 0x8f. Also, the _myfree function will fill freed memory with 0x8f. Both these fills are enabled by -DSAFEMALLOC. The BUILD/compile-*valgrind* scripts pass -USAFEMALLOC, disabling these fills, so that Valgrind will not think that the memory is initialized. But, this is not enough. See "Suggested fix".

How to repeat:
Note that MySQL source code (other than InnoDB) does not refer to <valgrind/memcheck.h> or any VALGRIND_ macros.

Suggested fix:
Something like this:

#ifdef HAVE_purify
# ifdef HAVE_VALGRIND_MEMCHECK_H
#  include <valgrind/memcheck.h>
#  define USE_VALGRIND_MEMCHECK
# endif
#endif

#ifdef USE_VALGRIND_MEMCHECK
# define MEM_UNDEFINED(addr,size) VALGRIND_MAKE_MEM_UNDEFINED(addr,size)
#else
# define MEM_UNDEFINED(addr,size) ((void) 0)
#endif

In the definitions of TRASH and _myfree, declare the memory uninitialized with MEM_UNDEFINED(). This should improve Valgrind results.

See also storage/innobase/include/univ.i for further ideas. InnoDB posts client check requests too, to check that certain buffers are fully initialized, and writes diagnostics.

[12 May 2010 13:05] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108129

[12 May 2010 13:05] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108130

[12 May 2010 13:17] Marko Mäkelä

On a second thought, we should not remove the -UFORCE_INIT_OF_VARS from the build scripts. If we remove it, we should also change the definition of LINT_INIT to mark the variable uninitialized for Valgrind (but to do that, taking the address of the variable would force it to stack).

Removing the -USAFEMALLOC should be OK, because the Valgrind instrumentation would be added to all SAFEMALLOC code.

[14 May 2010 13:43] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108343

[14 May 2010 13:43] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108344

[19 May 2010 14:32] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108644

[19 May 2010 14:32] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108645

[20 May 2010 10:41] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108762

[20 May 2010 10:41] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108763

[20 May 2010 11:45] Marko Mäkelä

Pushed to mysql-5.1-innodb and mysql-trunk-innodb. I could not figure out how to add the HAVE_VALGRIND stuff to CMakeLists.txt, so this will only work in GNU Autotools based builds for now.

[31 May 2010 8:28] Bugs System

Pushed into 5.1.48 (revid:[e-mail]-20100531082307-9x08gg1g7zybx2jy) (version source revid:[e-mail]-20100531082307-9x08gg1g7zybx2jy) (merge vers: 5.1.48) (pib:16)

[16 Jun 2010 10:55] Marko Mäkelä

I don't think that this needs any documentation. This would only make a difference when running a Valgrind-instrumented binary under Valgrind.

[16 Jun 2010 11:02] Paul DuBois

No changelog entry needed.

Setting report to Need Merge pending further pushes.

[17 Jun 2010 6:13] Bugs System

Pushed into 5.5.5-m3 (revid:[e-mail]-20100615145247-8bj0vmuqlotbqsn9) (version source revid:[e-mail]-20100520104042-ma2nsscqdvwoph8k) (merge vers: 5.1.48) (pib:16)

[17 Jun 2010 6:17] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20100615150216-cubqoyn1fj9b6a2p) (version source revid:[e-mail]-20100520104042-ma2nsscqdvwoph8k) (pib:16)

[21 Jun 2010 10:54] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111677

[21 Jun 2010 10:54] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111678

[21 Jun 2010 11:41] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111689

3109 Vladislav Vaintroub	2010-06-21
      Bug#53593: Add some instrumentation to improve Valgrind sensitivity.
      Implement WITH_VALGRIND for the CMake build.

[4 Aug 2010 7:52] Bugs System

Pushed into mysql-trunk 5.5.6-m3 (revid:[e-mail]-20100731131027-1n61gseejyxsqk5d) (version source revid:[e-mail]-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)

[4 Aug 2010 8:10] Bugs System

Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:[e-mail]-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)

[4 Aug 2010 8:26] Bugs System

Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:[e-mail]-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)

[4 Aug 2010 9:05] Bugs System

Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:[e-mail]-20100622115215-kxtzx7xuugcxd375) (pib:20)

[4 Aug 2010 22:36] Paul DuBois

No changelog entry needed.

[14 Oct 2010 8:37] Bugs System

Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:[e-mail]-20101014082627-jrmy9xbfbtrebw3c) (version source revid:[e-mail]-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)

[14 Oct 2010 8:52] Bugs System

Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.39 (revid:[e-mail]-20101014083757-5qo48b86d69zjvzj) (version source revid:[e-mail]-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)

[14 Oct 2010 9:07] Bugs System

Pushed into mysql-5.1-telco-6.2 5.1.51-ndb-6.2.19 (revid:[e-mail]-20101014084420-y54ecj85j5we27oa) (version source revid:[e-mail]-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)

[14 Oct 2010 15:19] Jon Stephens

Setting back to Closed -- see above.