MySQL Bugs: #44054: MTR2: --no-reorder does not prevent reordering

Bug #44054	MTR2: --no-reorder does not prevent reordering
Submitted:	2 Apr 2009 20:44	Modified:	17 Jun 2010 19:31
Reporter:	Guilhem Bichot	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Tools: MTR / mysql-test-run	Severity:	S3 (Non-critical)
Version:	5.1+	OS:	Linux
Assigned to:	Bjørn Munch	CPU Architecture:	Any

Description:
I run:
./mtr  --mem --force --max-test-fail=0 --ps-protocol --mysqld=--binlog-format=row --no-reorder rpl.rpl_LD_INFILE main.trigger-compat rpl.rpl_alter main.trigger_notembedded rpl.rpl_alter_db rpl.rpl_auto_increment_11932 main.truncate main.type_binary main.type_bit main.trigger rpl.rpl_bit main.type_blob main.type_date rpl.rpl_backup main.type_datetime main.type_decimal rpl.rpl_blackhole rpl.rpl_bit_npk main.type_float main.type_nchar main.type_enum rpl.rpl_bug31076 --parallel=1
and the order of tests is:
TEST                                      RESULT   TIME (ms)
------------------------------------------------------------

main.trigger-compat                      [ pass ]     59
main.trigger_notembedded                 [ pass ]    302
main.truncate                            [ pass ]     42
main.type_binary                         [ pass ]     57
etc.
So, the first one (rpl.rpl_LD_INFILE) on the command line was not the first run by mtr.
Looks like mtr runs the main suite and then the rpl suite in my experiment.
It is annoying; if I see a test failure without --no-reorder (just "./mtr") and I observe it's dependent on the order of tests ran, I would want to repeat the same order and that's when I would list the tests as they were ran and use --no-reorder.

How to repeat:
see description

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100377

2894 Bjorn Munch	2010-02-15
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder

First patch is incomplete, two problems:

1. It fails if you don't name any tests, only suites
2. It runs only the first combination of a test if using combinations.

I have a simple fix for #1 and will look at #2. Please await next patch.

This appears to require more extensive rewrite in order to support combinations (which it must), setting back to In Progress.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100587

2894 Bjorn Munch	2010-02-17
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder
      Also fix getting array from collect_one_suite() in this case

+our $reorder;

I assume that magically makes the $reorder variable set by 'collect_options' visible also in mysql-test-run.pl

Maybe move the variable to "my $opt_reorder" in mtr.pl and the pass it as argument to the collect function?

Good idea about $opt_reorder passed as argument, I can change that.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100600

2894 Bjorn Munch	2010-02-17
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder
      Also fix getting array from collect_one_suite() in this case
      Amended according to previous comment

Ok

Pushed to 6.0-codebase-mtr, next-mr-mtr, trunk-mtr, 5.1-mtr

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100220092844-jh07ayojyxn8uh9p) (version source revid:bjorn.munch@sun.com-20100217121426-au3ftaznjepx5yyw) (merge vers: 6.0.14-alpha) (pib:16)

Pushed into 5.5.3-m2 (revid:alik@sun.com-20100220092622-wvhh1vfy5tjq4mhu) (version source revid:bjorn.munch@sun.com-20100217120948-xuwoert11kk9485c) (merge vers: 5.5.3-m2) (pib:16)

Pushed into mysql-next-mr (revid:alik@sun.com-20100220092730-paoo5t9gcajs5dq8) (version source revid:bjorn.munch@sun.com-20100217121230-3jo40be856wwhk27) (pib:16)

Test suite changes. No changelog entry needed.

Setting report to Need Merge pending push to 5.1.x.

Pushed into 5.1.45 (revid:joro@sun.com-20100301083827-xnimmrjg6bh33o1o) (version source revid:azundris@mysql.com-20100222175719-viuh0f3gdsrkgv0r) (merge vers: 5.1.45) (pib:16)

No changelog entry needed.

Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609140708-52rvuyq4q500sxkq) (merge vers: 5.1.45-ndb-6.2.19) (pib:16)

Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)