MySQL Bugs: #45877: Implementation of backup_external_non

Bug #45877	Implementation of backup_external_non_win does not work
Submitted:	1 Jul 2009 14:02	Modified:	20 Feb 2010 19:00
Reporter:	Jørgen Løland	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Tests: Backup	Severity:	S3 (Non-critical)
Version:	5.4, 6.0	OS:	Any
Assigned to:	Rafal Somla	CPU Architecture:	Any
Tags:	experimental

Description:
The test backup.backup_external_non_win does not work as it should in PB:

1) It seems to never be executed is PB1
2) It seems to always fail in PB2

Note that the same test does not fail on my Ubuntu boxes.

[17 Jun 10:44] Daniel Fischer, in original bug report for this problem, BUG#45117: "The test was skipped as expected, due to absence of DBD::mysql. We can't provide DBD::mysql on pushbuild hosts because it requires a MySQL installation, which we want to avoid in the test environment to prevent cross-contamination. Hence, we can't fix this bug."

How to repeat:
Look at PB1 and PB2:

1) In PB1, you'll see this:
backup.backup_external_non_win           [ skipped ]  DBI & DBD modules not installed.

2) In PB2, you'll see this:
backup.backup_external_non_win           [ fail ]
        Test ended at 2009-05-27 03:06:46

CURRENT_TEST: backup.backup_external_non_win
sh: --user=root: not found
mysqltest: At line 79: command "$MYSQL_CONVERT_TABLE_FORMAT --user=root -S $MASTER_MYSOCK
--port=$MASTER_MYPORT  --type=innodb db1" failed

Output from before failure:
exec of ' --user=root -S
/export/home/pb2/test/sb_7-545255-1243383348.05/tmp/HscMkIWb4F/mysqld.1.sock --port=13010 
--type=innodb db1' failed, error: 256, status: 1, errno: 29

Suggested fix:
Rewrite test to not rely on DBD::mysql

Thank you for the report.

Verified as described by reading test source code.

It is not possible to write test without using DBD modules. The backup_external_non_win test uses external program mysql_convert_table_format which essentially depends on DBD::mysql installation. 

It is not possible to write the test by taking inspiration of backup_object_coverage.test. The have_rqg.inc file is sourced in the test file and while execution of test, we define the RQG_HOME to direct to RQG installation. This is completely different than how the backup_external_non_win.test is written.

The only way to avoid using DBI and DBD modules is by removing the mysql_convert_table_format external tool from the test. Ideally I won't prefer to do that.

Since this is an interoperability test for convert_table_format combined with backup/restore, it does not make sense to remove that part from this test. However, I don't think this test is executed on any platform at the moment. If this is the case, the test should probably rather be removed than continue to fail.

It seems that currently test behaves as expected with respect to the DBI module. It is skipped in environments in which DBI is not present and it executes otherwise.

However, if it is not skipped, then the following problem can be seen:

CURRENT_TEST: backup.backup_external_non_win_not_falcon
sh: --user=root: not found
mysqltest: At line 73: command "$MYSQL_CONVERT_TABLE_FORMAT --user=root -S $MASTER_MYSOCK --port=$MASTER_MYPORT  --type=innodb db1" failed

Output from before failure:
exec of ' --user=root -S /export/home2/pb2/test/sb_1-959056-1257965446.33/tmp/OgVycnys0K/mysqld.1.sock --port=13010  --type=innodb db1' failed, error: 256, status: 1, errno: 29

The problem is that variable $MYSQL_CONVERT_TABLE_FORMAT is not set and expands to empty string. Investigating further.

Note: I can see the problem in PB on sol10 platform, but I can not repeat it on my local linux box.

Test script contains the following lines:

source $MYSQLTEST_VARDIR/tmp/mctf.inc;
--exec $MYSQL_CONVERT_TABLE_FORMAT --user=root -S $MASTER_MYSOCK --port=$MASTER_MYPORT  --type=innodb db1

The mysterious mctf.inc file should set the MYSQL_CONVERT_TABLE_FORMAT variable to point at the utility script. It does that on my box. Apparently, it does not work correctly on other platforms.

File mctf.inc is created with perl code in suite/backup/include/backup_ext.inc. This perl code is supposed to detect location of mysql_convert_table_format utility script.

Here is a small script which can be used to check how the variable is set by the prolog code.

----------------------->8----------------------
--source suite/backup/include/backup_ext.inc
--cat_file $MYSQLTEST_VARDIR/tmp/mctf.inc
--echo
--echo ==DONE==
-----------------------8<----------------------

REFINED PROBLEM DESCRIPTION
---------------------------
In PB, tests are run in a binary distribution tree which was created with make_binary_distribution script. This binary distribution does not contain the same files as source tree. In particular, it does not contain scripts/mysql_convert_table_format script which is required by backup_external_non_win test. Because of that suite/backup/include/backup_ext.inc sets $MYSQL_CONVERT_TABLE_FORMAT to empty string and test fails when it tries to execute it.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/90340

2890 Rafal Somla	2009-11-13
      Bug #45877 Implementation of backup_external_non_win does not work
      
      Before: Tests backup_external_non_win* would fail if the mysql_convert_table_format
      utility could not be found in the tree.
      
      After: These tests will be skipped in such case.
     @ mysql-test/collections/default.experimental
        Remove tests from experimental list.
     @ mysql-test/suite/backup/include/backup_ext.inc
        Source temporary file to set $MYSQL_CONVERT_TABLE_FORMAT variable, so that it
        does not have to be done by script including this file.
     @ mysql-test/suite/backup/t/backup_external_non_win.test
        - Skip test if the mysql_convert_table_format utility could not be found.
        - No need to source temporary file - variable MYSQL_CONVERT_TABLE_FORMAT
          is set in backup_ext.inc file.
     @ mysql-test/suite/backup/t/backup_external_non_win_not_falcon.test
        - Skip test if the mysql_convert_table_format utility could not be found.
        - No need to source temporary file - variable MYSQL_CONVERT_TABLE_FORMAT
          is set in backup_ext.inc file.

Pushed into mysql-6.0-backup tree.
revid:rafal.somla@sun.com-20091113121039-6vf0iybzrwvag4v8

Pushed into 6.0.14-alpha (revid:ingo.struewing@sun.com-20100218152520-s4v1ld76bif06eqn) (version source revid:ingo.struewing@sun.com-20100119103538-wtp5alpz4p2jayl5) (merge vers: 6.0.14-alpha) (pib:16)

Test case changes. No changelog entry needed.