Bug #41054 mtr reports wrong binlog-format for skipped tests when combinations are used
Submitted: 26 Nov 2008 13:59 Modified: 21 Dec 2010 13:44
Reporter: John Embretsen Email Updates:
Status: Won't fix Impact on me:
None 
Category:Tools: MTR / mysql-test-run Severity:S3 (Non-critical)
Version:mtr_version=1 OS:Any
Assigned to: Bjørn Munch CPU Architecture:Any

[26 Nov 2008 13:59] John Embretsen
Description:
Consider the following example (mysql-6.0 bazaar branch):

./mtr --suite=rpl --mysqld=binlog-format=mixed

(...)
rpl.rpl_set_charset 'stmt'     [ pass ]            321
rpl.rpl_sf 'stmt'              [ pass ]              2
rpl.rpl_skip_error 'stmt'      [ skipped ]   Doesn't support --binlog-format='mixed'
rpl.rpl_slave_skip 'stmt'      [ pass ]           2939
rpl.rpl_slave_status 'stmt'    [ pass ]           1082
rpl.rpl_sp 'stmt'              [ skipped ]   Test requires: 'have_binlog_format_mixed'
rpl.rpl_sp004 'stmt'           [ pass ]            806
rpl.rpl_sp_effects 'stmt'      [ pass ]            577
(...)

The two "skipped" messages are contradicting each other, since both tests are run with the same set of options.

Running mysql-test-run.pl (mtr) with the option --script-debug reveals that the following options are passed to mysqld in this case:

--binlog-format=mixed --binlog-format=statement

According to MySQL documentation the last occurrence ('statement') takes precedence. This means that the message "Doesn't support --binlog-format='mixed'" is misleading, because the binlog format used by mysqld is 'statement'. 

"Doesn't support --binlog-format='statement'" would have been a more appropriate message.

This happens because the suite is run with 3 different combinations of options, specified in the file suite/rpl/combinations. The binlog-format option specified on the command line is effectively ignored - which should be taken into consideration by the mtr reporting.

How to repeat:
Branch mysql-6.0, cd to mysql-test and run

perl mysql-test-run.pl --suite=rpl --mysqld=--binlog-format=mixed

Suggested fix:
Make mtr libraries detect the *real* binlog-format used by mysqld. That is, if an option is specified more than once, the last occurrence takes precedence and is the one that should be reported.

Logic for this is here (mtr version 1):
mysql-test$ grep -n 'support --binlog-format=' lib/*
lib/mtr_cases.pl:466:	    "Doesn't support --binlog-format='$::used_binlog_format'";
lib/mtr_cases.pl:492:	    "Doesn't support --binlog-format='$test_binlog_format'";
[21 Dec 2010 13:44] Bjørn Munch
MTR v1 is no longer supported for general use.