Bug #53593 Add some instrumentation to improve Valgrind sensitivity
Submitted: 12 May 2010 9:34 Modified: 14 Oct 2010 15:19
Reporter: Marko Mäkelä Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S3 (Non-critical)
Version:5.1+ OS:Any
Assigned to: Marko Mäkelä CPU Architecture:Any
Tags: valgrind TRASH 0x8f

[12 May 2010 9:34] Marko Mäkelä
Description:
I added some Valgrind instrumentation when implementing compression in the InnoDB Plugin. Later, I ported the macros to the built-in InnoDB 5.1 too. Now that HAVE_purify enables UNIV_DEBUG_VALGRIND, we have found (and fixed) some bugs in InnoDB and even got 100% repeatability for Bug #38999 in the server.

In the MySQL core, there is the TRASH macro that can fill memory with 0x8f. Also, the _myfree function will fill freed memory with 0x8f. Both these fills are enabled by -DSAFEMALLOC. The BUILD/compile-*valgrind* scripts pass -USAFEMALLOC, disabling these fills, so that Valgrind will not think that the memory is initialized. But, this is not enough. See "Suggested fix".

How to repeat:
Note that MySQL source code (other than InnoDB) does not refer to <valgrind/memcheck.h> or any VALGRIND_ macros.

Suggested fix:
Something like this:

#ifdef HAVE_purify
# ifdef HAVE_VALGRIND_MEMCHECK_H
#  include <valgrind/memcheck.h>
#  define USE_VALGRIND_MEMCHECK
# endif
#endif

#ifdef USE_VALGRIND_MEMCHECK
# define MEM_UNDEFINED(addr,size) VALGRIND_MAKE_MEM_UNDEFINED(addr,size)
#else
# define MEM_UNDEFINED(addr,size) ((void) 0)
#endif

In the definitions of TRASH and _myfree, declare the memory uninitialized with MEM_UNDEFINED(). This should improve Valgrind results.

See also storage/innobase/include/univ.i for further ideas. InnoDB posts client check requests too, to check that certain buffers are fully initialized, and writes diagnostics.
[12 May 2010 13:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108129
[12 May 2010 13:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108130
[12 May 2010 13:17] Marko Mäkelä
On a second thought, we should not remove the -UFORCE_INIT_OF_VARS from the build scripts. If we remove it, we should also change the definition of LINT_INIT to mark the variable uninitialized for Valgrind (but to do that, taking the address of the variable would force it to stack).

Removing the -USAFEMALLOC should be OK, because the Valgrind instrumentation would be added to all SAFEMALLOC code.
[14 May 2010 13:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108343
[14 May 2010 13:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108344
[19 May 2010 14:32] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108644
[19 May 2010 14:32] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108645
[20 May 2010 10:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108762
[20 May 2010 10:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108763
[20 May 2010 11:45] Marko Mäkelä
Pushed to mysql-5.1-innodb and mysql-trunk-innodb. I could not figure out how to add the HAVE_VALGRIND stuff to CMakeLists.txt, so this will only work in GNU Autotools based builds for now.
[31 May 2010 8:28] Bugs System
Pushed into 5.1.48 (revid:vasil.dimov@oracle.com-20100531082307-9x08gg1g7zybx2jy) (version source revid:vasil.dimov@oracle.com-20100531082307-9x08gg1g7zybx2jy) (merge vers: 5.1.48) (pib:16)
[16 Jun 2010 10:55] Marko Mäkelä
I don't think that this needs any documentation. This would only make a difference when running a Valgrind-instrumented binary under Valgrind.
[16 Jun 2010 11:02] Paul DuBois
No changelog entry needed.

Setting report to Need Merge pending further pushes.
[17 Jun 2010 6:13] Bugs System
Pushed into 5.5.5-m3 (revid:alexey.kopytov@sun.com-20100615145247-8bj0vmuqlotbqsn9) (version source revid:marko.makela@oracle.com-20100520104042-ma2nsscqdvwoph8k) (merge vers: 5.1.48) (pib:16)
[17 Jun 2010 6:17] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100615150216-cubqoyn1fj9b6a2p) (version source revid:marko.makela@oracle.com-20100520104042-ma2nsscqdvwoph8k) (pib:16)
[21 Jun 2010 10:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111677
[21 Jun 2010 10:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111678
[21 Jun 2010 11:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111689

3109 Vladislav Vaintroub	2010-06-21
      Bug#53593: Add some instrumentation to improve Valgrind sensitivity.
      Implement WITH_VALGRIND for the CMake build.
[4 Aug 2010 7:52] Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100731131027-1n61gseejyxsqk5d) (version source revid:marko.makela@oracle.com-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:10] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:marko.makela@oracle.com-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 8:26] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:marko.makela@oracle.com-20100622115215-kxtzx7xuugcxd375) (merge vers: 5.1.49) (pib:18)
[4 Aug 2010 9:05] Bugs System
Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:marko.makela@oracle.com-20100622115215-kxtzx7xuugcxd375) (pib:20)
[4 Aug 2010 22:36] Paul DuBois
No changelog entry needed.
[14 Oct 2010 8:37] Bugs System
Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:martin.skold@mysql.com-20101014082627-jrmy9xbfbtrebw3c) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)
[14 Oct 2010 8:52] Bugs System
Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.39 (revid:martin.skold@mysql.com-20101014083757-5qo48b86d69zjvzj) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)
[14 Oct 2010 9:07] Bugs System
Pushed into mysql-5.1-telco-6.2 5.1.51-ndb-6.2.19 (revid:martin.skold@mysql.com-20101014084420-y54ecj85j5we27oa) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:21)
[14 Oct 2010 15:19] Jon Stephens
Setting back to Closed -- see above.