Bug #59148 'INSTALL PLUGIN rpl_semi_sync_master' fails in release build with debug binaries
Submitted: 24 Dec 2010 7:40 Modified: 9 Feb 2011 19:21
Reporter: Alexander Nozdrin Email Updates:
Status: Closed Impact on me:
None 
Category:Tools: MTR / mysql-test-run Severity:S7 (Test Cases)
Version:5.6.1, 5.5,mysql-trunk OS:Any
Assigned to: Bjørn Munch CPU Architecture:Any
Tags: pb2, test failure

[24 Dec 2010 7:40] Alexander Nozdrin
Description:
A few tests fail with the following symptom in release build
with debug binaries:

mysqltest: At line 29: query 'INSTALL PLUGIN rpl_semi_sync_master SONAME '$SEMISYNC_MASTER_PLUGIN'' failed: 1126: Can't open shared library '/export/home2/pb2/test/sb_2-2710269-1293149042.86/usr/lib/mysql/plugin/semisync_master.so' (errno: 2 undefined symbol: my_pthread_fastmutex_lock)

The following tests are affected:
sys_vars.all_vars
sys_vars.rpl_semi_sync_master_enabled_basic
sys_vars.rpl_semi_sync_master_timeout_basic
sys_vars.rpl_semi_sync_master_trace_level_basic
sys_vars.rpl_semi_sync_master_wait_no_slave_basic

The tests fail on the following platforms:
linux2.6-x86-32bit community rpm
linux2.6-x86-32bit community tar-gz
linux2.6-x86-64bit community tar-gz
rhel5-ia64-64bit community rpm
rhel5-x86-32bit community rpm
sles10-ia64-64bit community rpm
sles10-x86-32bit community rpm
sles11-x86-32bit community rpm

Might be related to Bug#49022.

How to repeat:
http://pb2.norway.sun.com/?template=show_pushes&branch=mysql-5.6.1-m5-release
[10 Jan 2011 9:53] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128258

3226 Georgi Kodinov	2011-01-10
      Bug #59148 - made tests experimental
[24 Jan 2011 4:06] Anitha Gopi
rpl.rpl_semi_sync_event and rpl.rpl_semi_sync are also failing with this bug in daily-trunk. See http://pb2.norway.sun.com/?template=mysql_show_test_failure&search=yes&push_id=1950378&tes...
[27 Jan 2011 10:52] Joerg Bruehe
AIUI, the problem is just another effect of the decision to have some internal objects be defined different in debug vs optimized builds.
This base issue is discussed in greater length in bug#49022.

The fix for that bug is to tell a "debug" server on startup (in "mysqld_safe") to search for the plugins in a different directory, by appending "/debug" to the directory given as "--plugin-dir=".
For more details, see the changeset:
http://lists.mysql.com/commits/101974

(Other parts of that bug fix affect packaging, to ensure that the debug plugins are really distributed - this is not relevant here.)

This bug fix seems to work for customers and users.

However, it does not help when we run the test suite: MTR does start the server (be it optimized or debug) withut using "mysqld_safe", so that fix is bypassed.

The clean solution should be to revert that old design of having different mutex object implementations in debug vs optimized builds, in order to get the identical ABI between server and plugin for both build modes (which I strongly propose!).
Until we get that, the workaround seems to be that MTR should provide a modified "--plugin-dir=" option to the debug server.
[27 Jan 2011 14:58] Jonathan Perkin
Summary of this task then is to change MTR so that when passed --debug it looks for debug plugins in a similar manner to mysqld_safe (but needs to support Windows too), and then passes --plugin-dir= to the appropriate mysqld.
[28 Jan 2011 10:17] Bjørn Munch
Back to the drawing board! This is NOT related to mtr --debug and the debug server. I look at the PB2 logs and see that

1) mtr is not called with --debug
2) The core file is produced by mysqld, not by mysqld-debug

In fact, trying to run mtr --debug on the release build fails due to Bug #47141.

I try to tun manually but can't make those tests fail. This remains a mystery.
[28 Jan 2011 10:38] Bjørn Munch
OK it has been explained. The mysqld-debug binary was renamed to mysqld before running mtr, possibly as a workaround. mtr still needs to be fixed so that it would work when thing are done the right way using --debug.
[28 Jan 2011 14:59] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/129885

3160 Bjorn Munch	2011-01-28
      Bug #59148 'INSTALL PLUGIN rpl_semi_sync_master' fails in release build with debug binaries
      Do as mysqld_safe: if running mysqld-debug, plugins are in debug subdirs
      NB mtr --debug won't work in this context until 47141 is fixed
      Also moved read_plugin_defs; no point running this in all worker threads
[28 Jan 2011 15:43] Bjørn Munch
Pushed to 5.5-mtr and trunk-mtr
[30 Jan 2011 11:52] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:bjorn.munch@oracle.com-20110130115129-js8oloj0kke23y2u) (version source revid:bjorn.munch@oracle.com-20110130115042-u671iduxcr5uw04c) (merge vers: 5.6.2) (pib:24)
[30 Jan 2011 11:53] Bugs System
Pushed into mysql-5.5 5.5.10 (revid:bjorn.munch@oracle.com-20110130110933-9rcq2ic58w06ae30) (version source revid:bjorn.munch@oracle.com-20110130110933-9rcq2ic58w06ae30) (merge vers: 5.5.10) (pib:24)
[30 Jan 2011 18:13] Paul DuBois
Changes to test suite. No changelog entry needed.
[8 Feb 2011 14:09] Bugs System
Pushed into mysql-5.5 5.5.10 (revid:jonathan.perkin@oracle.com-20110208140736-1173xnoipufbhowh) (version source revid:jonathan.perkin@oracle.com-20110208135903-jhzy6wq16b2fx7pg) (merge vers: 5.5.10) (pib:24)
[8 Feb 2011 16:45] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:georgi.kodinov@oracle.com-20110208155412-tfy4l5hqxi0g7o41) (version source revid:georgi.kodinov@oracle.com-20110208154951-gzqgr74u4bndswi8) (merge vers: 5.6.2) (pib:24)