Bug #28939 Failing test causes other subsequent tests to fail

Description:
From customer;

When running the testsuite in enterprise5.0.40 with the parameter '--force' and '--extern', the failture of one test case will cause some following test cases to fail because they use the same table names, t1, t2 and so on, and the first failing case will not clean up its environment. I suggest that the clean-up be done just before every case starts to run so that it'll be affected by the previous failling cases. Actually, when I call the following bash script in perl file mysql-test-run before each cases runs, mysql_test_run can work well:

ret=`mysql -uroot -e 'show databases'`
ret_test=`mysql -uroot -D'test' -e 'show tables'`

for db in $ret
do
if [ $db != "Database" -a $db != "information_schema" -a $db != "mysql" -a $db != "test" ]
then
`mysql -uroot -e "drop database $db"`
fi
done
#clean up tables in test database
for tb in $ret_test
do
if [ $tb != "Tables_in_test" ]
then
`mysql -uroot -D'test' -e "drop table $tb"`
fi
done

These codes are easy to implement in perl. mysql-test-run should include them.

How to repeat:
.

Suggested fix:
If this itself is not a bug, perhaps provide an extra option to clean up & re-initialize required data/tables after each test (as this would be much more time consuming).

Thank you for the report.

Please indicate which test case fails first.

The command is
./mysql-test-run --extern --user=root --socket=/var/lib/mysql/mysql.sock --force alias backup analyze

In my environment, alias and analyze pass, and backup fails if I run them respectively. Since I used '--force' here, even though backup fails, the script should continue to run analyze. Actually, it does not. The error is "mysql-test-run: *** ERROR: No snapshot existed". I think I found the defect in mysql-test-run. If I do the following change in mysql-test-run, the problem can be solved:

Change the line 3572 in the function report_failure_and_restart($) from

restore_installed_db($tinfo->{'name'});
To
if ( !$opt_extern )
{
	restore_installed_db($tinfo->{'name'});

}

[root@dixon mysql-test]# ./mysql-test-run --extern --user=root --socket=/tmp/mysql.sock --force backup
Logging: ./mysql-test-run --extern --user=root --socket=/tmp/mysql.sock --force backup
Disable instance manager when running with extern mysqld
Using extern server at '/tmp/mysql.sock'
mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped
=======================================================
Starting Tests in the 'main' suite

TEST                           RESULT         TIME (ms)
-------------------------------------------------------

backup                         [ fail ]

Errors are (from /usr/src/mysql/binary/t/t/usr/share/mysql-test/var/log/mysqltest-time) :
mysqltest: At line 20: query 'select count(*) from t4' failed: 1146: Table 'test.t4' doesn't exist
(the last lines may be the most important ones)
Result from queries before failure can be found in r/backup.log

Stopping All Servers
mysql-test-run: *** ERROR: No snapshot existed

070613  8:09:34      14 Connect     root@localhost on test
                     15 Connect     root@localhost on test
                     16 Connect     root@localhost on test
                     15 Query       set SQL_LOG_BIN=0
                     15 Query       drop table if exists t1, t2, t3
                     15 Query       create table t4(n int)
                     15 Query       backup table t4 to '../bogus'
                     15 Query       SHOW WARNINGS
                     15 Query       backup table t4 to '../tmp'
                     15 Query       SHOW WARNINGS
                     15 Query       backup table t4 to '../tmp'
                     15 Query       SHOW WARNINGS
                     15 Query       drop table t4
                     15 Query       restore table t4 from '../tmp'
                     15 Query       SHOW WARNINGS
                     15 Query       select count(*) from t4
                     15 Quit       
                     14 Quit       
                     16 Quit

Thank you for the report.

With 5.0.41 behaviour is same as described in comment "[13 Jun 0:47] Adam Dixon", but in current BK sources problem is partially fixed: ./mysql-test-run.pl --socket=/tmp/mysql.sock --force --extern --user=root alias backup analyze runs OK, but ./mysql-test-run.pl --socket=/tmp/mysql.sock --force --extern --user=root fails with error "Restoring snapshot of databases
mysql-test-run: *** ERROR: Can't find /Users/apple/bk/mysql-5.0/mysql-test/var/tmp/snapshot_9306//slave-dataNo such file or directory" after failure of test alter_table

Version 5.1 is not affected.