Bug #44054 MTR2: --no-reorder does not prevent reordering
Submitted: 2 Apr 2009 20:44 Modified: 17 Jun 2010 19:31
Reporter: Guilhem Bichot Email Updates:
Status: Closed Impact on me:
None 
Category:Tools: MTR / mysql-test-run Severity:S3 (Non-critical)
Version:5.1+ OS:Linux
Assigned to: Bjørn Munch CPU Architecture:Any

[2 Apr 2009 20:44] Guilhem Bichot
Description:
I run:
./mtr  --mem --force --max-test-fail=0 --ps-protocol --mysqld=--binlog-format=row --no-reorder rpl.rpl_LD_INFILE main.trigger-compat rpl.rpl_alter main.trigger_notembedded rpl.rpl_alter_db rpl.rpl_auto_increment_11932 main.truncate main.type_binary main.type_bit main.trigger rpl.rpl_bit main.type_blob main.type_date rpl.rpl_backup main.type_datetime main.type_decimal rpl.rpl_blackhole rpl.rpl_bit_npk main.type_float main.type_nchar main.type_enum rpl.rpl_bug31076 --parallel=1
and the order of tests is:
TEST                                      RESULT   TIME (ms)
------------------------------------------------------------

main.trigger-compat                      [ pass ]     59
main.trigger_notembedded                 [ pass ]    302
main.truncate                            [ pass ]     42
main.type_binary                         [ pass ]     57
etc.
So, the first one (rpl.rpl_LD_INFILE) on the command line was not the first run by mtr.
Looks like mtr runs the main suite and then the rpl suite in my experiment.
It is annoying; if I see a test failure without --no-reorder (just "./mtr") and I observe it's dependent on the order of tests ran, I would want to repeat the same order and that's when I would list the tests as they were ran and use --no-reorder.

How to repeat:
see description
[15 Feb 2010 12:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100377

2894 Bjorn Munch	2010-02-15
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder
[15 Feb 2010 15:15] Bjørn Munch
First patch is incomplete, two problems:

1. It fails if you don't name any tests, only suites
2. It runs only the first combination of a test if using combinations.

I have a simple fix for #1 and will look at #2. Please await next patch.
[16 Feb 2010 12:43] Bjørn Munch
This appears to require more extensive rewrite in order to support combinations (which it must), setting back to In Progress.
[17 Feb 2010 10:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100587

2894 Bjorn Munch	2010-02-17
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder
      Also fix getting array from collect_one_suite() in this case
[17 Feb 2010 10:55] Magnus Blåudd
+our $reorder;

I assume that magically makes the $reorder variable set by 'collect_options' visible also in mysql-test-run.pl

Maybe move the variable to "my $opt_reorder" in mtr.pl and the pass it as argument to the collect function?
[17 Feb 2010 11:12] Bjørn Munch
Good idea about $opt_reorder passed as argument, I can change that.
[17 Feb 2010 11:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/100600

2894 Bjorn Munch	2010-02-17
      Bug #44054 MTR2: --no-reorder does not prevent reordering
      Some logic would group by suite always
      Disable this if using --noreorder
      Also fix getting array from collect_one_suite() in this case
      Amended according to previous comment
[17 Feb 2010 12:00] Magnus Blåudd
Ok
[17 Feb 2010 15:02] Bjørn Munch
Pushed to 6.0-codebase-mtr, next-mr-mtr, trunk-mtr, 5.1-mtr
[20 Feb 2010 9:31] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100220092844-jh07ayojyxn8uh9p) (version source revid:bjorn.munch@sun.com-20100217121426-au3ftaznjepx5yyw) (merge vers: 6.0.14-alpha) (pib:16)
[20 Feb 2010 9:34] Bugs System
Pushed into 5.5.3-m2 (revid:alik@sun.com-20100220092622-wvhh1vfy5tjq4mhu) (version source revid:bjorn.munch@sun.com-20100217120948-xuwoert11kk9485c) (merge vers: 5.5.3-m2) (pib:16)
[20 Feb 2010 9:37] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100220092730-paoo5t9gcajs5dq8) (version source revid:bjorn.munch@sun.com-20100217121230-3jo40be856wwhk27) (pib:16)
[20 Feb 2010 18:59] Paul Dubois
Test suite changes. No changelog entry needed.

Setting report to Need Merge pending push to 5.1.x.
[1 Mar 2010 8:44] Bugs System
Pushed into 5.1.45 (revid:joro@sun.com-20100301083827-xnimmrjg6bh33o1o) (version source revid:azundris@mysql.com-20100222175719-viuh0f3gdsrkgv0r) (merge vers: 5.1.45) (pib:16)
[1 Mar 2010 15:56] Paul Dubois
No changelog entry needed.
[17 Jun 2010 11:47] Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:24] Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609140708-52rvuyq4q500sxkq) (merge vers: 5.1.45-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:11] Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)