MySQL Bugs: #44937: backup.backup_external_non_win fails if it cannot detect DBD::mysql

Bug #44937	backup.backup_external_non_win fails if it cannot detect DBD::mysql
Submitted:	18 May 2009 16:44	Modified:	10 Jul 2009 15:18
Reporter:	Patrick Crews	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Tests: Server	Severity:	S3 (Non-critical)
Version:	6.0	OS:	Any
Assigned to:	Patrick Crews	CPU Architecture:	Any
Tags:	Backup, DBD::MySQL, external driver, pushbuild, test failure

Description:
The test backup.backup_external_non_win fails if it cannot detect DBD::mysql. 

NOTE:  This is also interesting as DBD::mysql *is* installed on my machine(!)
This is the same situation as:  Bug#43909	The test backup_external_non_win fails with perl error in team trees.  However, the fix for this bug was only applied to 6.0+ and should be backported to 5.1

backup.backup_external_non_win           [ fail ]
        Test ended at 2009-05-18 11:34:14

CURRENT_TEST: backup.backup_external_non_win
install_driver(mysql) failed: Can't locate DBD/mysql.pm in @INC (@INC contains: /sw/lib/perl5 /sw/lib/perl5/darwin /Library/Perl/Updates/5.8.8 /System/Library/Perl/5.8.8/darwin-thread-multi-2level /System/Library/Perl/5.8.8 /Library/Perl/5.8.8/darwin-thread-multi-2level /Library/Perl/5.8.8 /Library/Perl /Network/Library/Perl/5.8.8/darwin-thread-multi-2level /Network/Library/Perl/5.8.8 /Network/Library/Perl /System/Library/Perl/Extras/5.8.8/darwin-thread-multi-2level /System/Library/Perl/Extras/5.8.8 /Library/Perl/5.8.6 /Library/Perl/5.8.1 .) at (eval 3) line 3.
Perhaps the DBD::mysql perl module hasn't been fully installed,
or perhaps the capitalisation of 'mysql' isn't right.
Available drivers: DBM, ExampleP, File, Proxy, SQLite, Sponge.

How to repeat:
Run the test on a machine without any of the listed drivers and observe the failure:

./mtr --force --suite=backup backup_external_non_win

Suggested fix:
A test-suite test should *NOT* fail is external connector is not present.

At a minimum, the test should perform such checks as a prerequisite and *skip* the test as it cannot successfully complete.  As it currently stands, the 'fail' is not accurate and is an 'incomplete' at best.

I would suggest adding a file to mysql-test/include to perform these checks so that the test can skip gracefully if it cannot detect all of the components needed to run.

I made a mistake in my initial description of the problem.

This problem is still present in 6.0 (not 5.1) despite the changes pushed by the previous bug.

Discovered the include file: mysql-test/suite/backup/include/have_dbi.inc  *is* properly detecting that I have these modules installed.

It appears that the problem is in the .test file itself - it is failing at line 80 or so.  Will investigate further.

Related to BUG#45117

In my environment, a Solaris machine in the Trondheim lab, I see a similar complaint (error) about missing perl modules after the include file has checked that it can find the modules. 

The error occurs when calling (--exec) the mysql_convert_table_format perl script, and it happens because that script has the following SheBang:

#!/usr/bin/perl

On Solaris hosts in the TRD lab, the newest Perl installation with the DBI/DBD modules are installed in /opt/csw/bin, so that is the perl that needs to be first in PATH. However, that won't help if the SheBang points to a different Perl installation which does not have these modules.

Modifying the mysql_convert_table_format script's SheBang (top line) to 

#!/usr/bin/env perl

will make it use whatever Perl is first in path, and then the test passes for me.

Not sure if this is the same issue, though, but it might be worth looking into. As far as I know my description is only relevant for the Solaris platforms.

This is caused by having multiple perl installations on the machine.
The $MYSQL_CONVERT_TABLE_FORMAT script uses /usr/bin/perl in the shebang line.

However, the include file that is designed to detect the presence of dbd::mysql will detect *any* installation (mine came from a package manager and is installed in the /opt directory).

This causes a problem as the prerequisite check passes, but the subsequent test will fail (the /opt dbd::mysql install is not in the search path - different perl's).

This can be fixed by changing the "--exec $MYSQL_CONVERT_TABLE_FORMAT" calls in the test to "--exec perl $MYSQL_CONVERT_TABLE_FORMAT" (verified on my machine).

However, it might also be possible to make the include file more stringent.
Submitting a patch with the 'adding "perl"' approach shortly.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75633

3306 Patrick Crews	2009-06-04
      Bug#44937: backup.backup_external_non_win fails if it cannot detect DBD::mysql
      
      Proposed patch for fixing this.  Found the problem was caused by multiple perl
      installations.  The pre-req checking include file detects dbd::mysql for one
      perl installation, but the $MYSQL_CONVERT_TABLE_FORMAT script uses the /usr/bin/perl
      version.
      
      Adding 'perl' before the calls to the script will correct this.  However, it might
      also be possible to change the include file to be more strict.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75642

3341 Patrick Crews	2009-06-04
      Bug#44937: backup.backup_external_non_win fails if it cannot detect DBD::mysql
      
      Fix provided by John E.
      
      Changing the script's shebang line allows the prerequisite and the 
      mysql_convert_table_format.sh scripts to use the same perl and ensures
      that the test will be skipped if dbd::mysql is not detected.
      
      This problem was caused by having two perl installations.  The file
      mysql-test/suite/backup/include/have_dbd.inc was checking one install, while
      the script was using /usr/bin/perl.

Using "perl" explicitly is much safer than messing with the shebang line. The Perl interperter may be located anywhere on the machine, the shebang line is a platform-specific hack.

Patch approved

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/76231

3359 Patrick Crews	2009-06-13
      Bug#44937: backup.backup_external_non_win fails if it cannot detect DBD::mysql
      
      Re-submitting my initial patch rather than altering the shebang line in the 
      mysql_convert_table_format script based on pstoev's feedback.
      
      Altered the test to explicitly call perl in the test.
      This bug was caused by having two installations of perl on a machine.
      The include/have_dbd.inc file checked one version (based in /opt dir)
      while the script was using /usr/bin/perl due to the shebang line + how
      the test called the script.
      
      This patch ensures that the check and the test both use the same version of perl.

Pushed into 5.4.4-alpha (revid:anozdrin@bk-internal.mysql.com-20090710111017-bnh2cau84ug1hvei) (version source revid:patrick.crews@sun.com-20090613101202-stfv5urrxz177ozl) (merge vers: 5.4.4-alpha) (pib:11)

Test case changes. No changelog entry needed.