Bug #49170 Inconsistent placement of semisync plugin prevents it from getting tested
Submitted: 27 Nov 2009 16:42 Modified: 13 Feb 2010 3:05
Reporter: Joerg Bruehe Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Installing Severity:S3 (Non-critical)
Version:5.5.0-beta OS:Any
Assigned to: Zhenxing He CPU Architecture:Any

[27 Nov 2009 16:42] Joerg Bruehe
Description:
The new (in 5.5.0) "semisync" plugin is built on all platforms,
but rarely participates in tests:
In all "tar.gz" builds, on Windows, and with the "specific" RPMs the respective test is skipped because the plugin is not found.

This is due to different issues:

1) "mysql-test-run.pl" currently checks for the plugin presence using this code:
=====
    my $semisync_master_filename;
    my $semisync_slave_filename;
    if (IS_WINDOWS)
    {
       $semisync_master_filename = "semisync_master.dll";
       $semisync_slave_filename = "semisync_slave.dll";
    }
    else
    {
       $semisync_master_filename = "libsemisync_master.so";
       $semisync_slave_filename = "libsemisync_slave.so";
    }
    my $lib_semisync_master_plugin=
      mtr_file_exists(vs_config_dirs('plugin/semisync',$semisync_master_filename),
              "$basedir/plugin/semisync/.libs/" . $semisync_master_filename,
                      "$basedir/lib/mysql/plugin/" . $semisync_master_filename);
    my $lib_semisync_slave_plugin=
      mtr_file_exists(vs_config_dirs('plugin/semisync',$semisync_slave_filename),
              "$basedir/plugin/semisync/.libs/" . $semisync_slave_filename,
                      "$basedir/lib/mysql/plugin/" . $semisync_slave_filename);
    if ($lib_semisync_master_plugin && $lib_semisync_slave_plugin)
    {
      $ENV{'SEMISYNC_MASTER_PLUGIN'}= basename($lib_semisync_master_plugin);
      $ENV{'SEMISYNC_SLAVE_PLUGIN'}= basename($lib_semisync_slave_plugin);
      $ENV{'SEMISYNC_PLUGIN_OPT'}= "--plugin-dir=".dirname($lib_semisync_master_plugin);
    }
    else
    {
      $ENV{'SEMISYNC_MASTER_PLUGIN'}= "";
      $ENV{'SEMISYNC_SLAVE_PLUGIN'}= "";
      $ENV{'SEMISYNC_PLUGIN_OPT'}="--plugin-dir=";
    }
=====

2) In the created packages, the "libsemisync" files are present at different places:

2a) "tar.gz" packages (and derived, like "depot", "pkg", "dmg")
have these files:
    lib/plugin/libsemisync_{master,slave}.{a,la,so,so.0,so.0.0}
(extensions vary a bit, according to platform).
Similar, Windows "noinstall" packages have
    lib/plugin/semisync_{master,slave}.dll
Notice that the path "lib/plugin/*semisync_*" does not match any of locations checked by MTR, so the plugin is not found, and the test is skipped.

2b) All "server" RPMs ("generic" or "specific") have these files:
    /usr/lib/mysql/plugin/libsemisync_{master,slave}.{so,so.0,so.0.0}
This seems to match the last location,
and in the builds of "generic" RPMs the test is taken (and passes).

2c) However, the build of "specific" RPMs is done in several phases, and a consequence is that there is no "libsemisync_*.so" present at the moment the tests are run.

How to repeat:
Just do a release build ...

Suggested fix:
1) Align the placement of the "semisync" stuff between "tar.gz" packages and the MTR assumptions.

2) Solve the phase problem in building the "specific" RPMs, so that the semisync plugin takes part in the tests.

When working on this, see the related bug#48016 and bug#48351,
take care that all these changes are merged early so that they don't conflict.
[3 Dec 2009 11:27] Joerg Bruehe
I installed a "specific" RPM on a clean machine
and successfully ran the test for the "semisync" plugin.

I will attend to the spec files soon and know how to fix them,
so that this test will be taken during the build of these RPMs.
[3 Dec 2009 11:43] Joerg Bruehe
I consider it unfortunate that the storage locations for the "semisync" plugin (and any other plugins) differ
between RPMs
   /usr/lib/mysql/plugin/*semisync_{master,slave}.so*
and tar.gz packages
   /usr/local/mysql/lib/plugin/*semisync_{master,slave}.so*
and build trees
   .../plugin/semisync/.libs/*semisync_{master,slave}.so*

However, the effort to change this (and to align the structure) may be too high.

So *if* these paths remain, then we need to add the path used with tar.gz packages to MTR:
    "$basedir/lib/plugin/" . $semisync_slave_filename,
(same for "master") should do it.

I propose you run a local build (on some Unix or Linux),
then do a "scripts/make_binary_distribution",
take the resulting tarball and follow the instructions in the manual:
    2.2. Installing MySQL from Generic Binaries on Unix/Linux

If you can run the "rpl_semi_sync" test there (which will need the MTR change), it should be fine.
[3 Dec 2009 12:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92672

3125 He Zhenxing	2009-12-03
      Bug#49170 Inconsistent placement of semisync plugin prevents it from getting tested
      
      Add $basedir/lib/plugin to the search paths for semisync plugins.
[3 Dec 2009 13:46] Joerg Bruehe
I am not aware of anything else (yet).

If it passed your local test with "make_binary_distribution", it should also work in our builds of tar.gz packages.
[3 Dec 2009 17:52] Joerg Bruehe
Installed 5.5.0-beta locally (on Linux),
bug could be reproduced,
after adding this patch the test passed.

Approved.

I cannot say anything about Windows, though.
[3 Dec 2009 18:48] Elena Stepanova
Checked on Windows Server 2008, with mysql-noinstall-5.5.0-beta-winx64.zip (from 5.5.0 release)

After applying the change to MTR, the test runs with all binlog_format values, although it fails with result contents mismatch with differences like

< Rpl_semi_sync_master_no_tx    0
> Rpl_semi_sync_master_no_tx    1

< Rpl_semi_sync_master_yes_tx   301
> Rpl_semi_sync_master_yes_tx   300

etc.

Setting status to 'Patch approved' as agreed with Joerg, the mismatch issue will be handled separately.
[4 Dec 2009 1:44] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92805

3124 He Zhenxing	2009-12-04
      Bug#49170 Inconsistent placement of semisync plugin prevents it from getting tested
      
      Add $basedir/lib/plugin to the search paths for semisync plugins.
[4 Dec 2009 2:06] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92810

3125 He Zhenxing	2009-12-04 [merge]
      Auto Merge fix for Bug#49170
[5 Dec 2009 2:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92953

2918 He Zhenxing	2009-12-05 [merge]
      Merge patch for Bug#49170 from 5.1-rep-semisync
[11 Dec 2009 6:01] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091211055901-yp18b3c7xuhl87rf) (version source revid:alik@sun.com-20091211055401-43rjwq7gjed6ds83) (merge vers: 6.0.14-alpha) (pib:13)
[11 Dec 2009 6:04] Bugs System
Pushed into 5.6.0-beta (revid:alik@sun.com-20091211055628-ltr7fero363uev7r) (version source revid:alik@sun.com-20091211055453-717czhtezc74u8db) (merge vers: 5.6.0-beta) (pib:13)
[15 Dec 2009 20:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/94270

2917 Alexander Nozdrin	2009-12-15
      Backporting a patch for Bug#49170 (Inconsistent placement of semisync
      plugin prevents it from getting tested) from mysql-next-mr-bugfixing
      to mysql-trunk-bugfixing.
      
      Original revision:
      ------------------------------------------------------------
      revision-id: zhenxing.he@sun.com-20091204014339-2m06r42vajhm9vke
      committer: He Zhenxing <zhenxing.he@sun.com>
      branch nick: 5.1-rep-semisync
      timestamp: Fri 2009-12-04 09:43:39 +0800
      message:
        Bug#49170 Inconsistent placement of semisync plugin prevents it from getting tested
        
        Add $basedir/lib/plugin to the search paths for semisync plugins.
      ------------------------------------------------------------
[16 Dec 2009 2:58] Paul DuBois
Changes to test suite. No changelog entry needed.
[21 Dec 2009 15:39] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091221153807-80nxoli1tw1z9bxn) (version source revid:alik@sun.com-20091215205224-v4xxef6r433ha367) (merge vers: 6.0.14-alpha) (pib:15)
[21 Dec 2009 15:40] Bugs System
Pushed into 5.5.1-m2 (revid:alik@sun.com-20091221153538-ifi3mxf3y10ozxin) (version source revid:alik@sun.com-20091215202301-4xhqs9yl317hyrw1) (merge vers: 5.5.0-beta) (pib:15)
[21 Dec 2009 15:41] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20091221153659-d7bt0fh6mhhclxuf) (version source revid:alik@sun.com-20091215204708-2ttqmc4r279i26a2) (pib:15)
[5 Jan 2010 11:35] MC Brown
Changes to test suite, no changelog entry needed
[12 Feb 2010 17:41] Bugs System
Pushed into 5.5.2-m2 (revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (version source revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (merge vers: 5.5.2-m2) (pib:16)