Bug #57840 MTR: parallel execution breaks with smart ordering of test cases

• Submitted: 29 Oct 2010 10:37
• Modified: 20 Nov 2010 0:06
• Reporter: Bjørn Munch
• Status: Closed
• Category: Tools: MTR / mysql-test-run
• Severity: S3 (Non-critical)
• Version: 5.1+
• OS: Any
• Assigned to: Bjørn Munch
• CPU Architecture: Any

[29 Oct 2010 10:37] Bjørn Munch

Description:
MTR v2 sorts test cases according to relevant criteria to reduce needs for server restart between tests. But this does not combine well with parallel test execution. Tests grouped together due to identical configs will be scheduled in sequence, but typically spread across threads.

How to repeat:
./mtr --parallel=2 udf 1st udf 1st

This groups the tests thus: udf udf 1st 1st. But with two parellels, each will first run udf, then 1st after a server restart. The best (for reducing restarts) is to have one run udf udf and the other 1st 1st.

[2 Nov 2010 14:55] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/122553

2948 Bjorn Munch	2010-11-02
      Bug #57840 MTR: parallel execution breaks with smart ordering of test cases
      There were actually more problems in this area:
        Slaves (if any) were unconditionally restarted, this appears unnecessary.
        Sort criteria were suboptimal, included the test name.
      Added logic to "reserve" a sequence of tests with same config for one thread

[3 Nov 2010 11:42] Bjørn Munch

First patch is somewhat dysfunctional when combinations are used; the hash key used for sorting is only the test name, needs to add the combination. I'm testing an update now.

[3 Nov 2010 13:21] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/122679

2948 Bjorn Munch	2010-11-03
      Bug #57840 MTR: parallel execution breaks with smart ordering of test cases
      Improvements from first patch
      Got rid of sort_criteria hash, put it into the test case itself
      This also simplifies tagging similar tests during execution
      This is a new complete patch, not an addendum

[4 Nov 2010 13:21] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/122837

2948 Bjorn Munch	2010-11-04
      Bug #57840 MTR: parallel execution breaks with smart ordering of test cases
      Another updated patch:
      Adds little sanity check that expected worker picks up test
      Fixed some tests that may fail if starting on running server
      Some of these fail only if *same* test is repeated.

[5 Nov 2010 14:29] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/122963

2948 Bjorn Munch	2010-11-05
      Bug #57840 MTR: parallel execution breaks with smart ordering of test cases
      There were actually more problems in this area:
        Slaves (if any) were unconditionally restarted, this appears unnecessary.
        Sort criteria were suboptimal, included the test name.
      Added logic to "reserve" a sequence of tests with same config for one thread
      Got rid of sort_criteria hash, put it into the test case itself
      Adds little sanity check that expected worker picks up test
      Fixed some tests that may fail if starting on running server
      Some of these fail only if *same* test is repeated.
      Finally, special sorting of tests that do --force-restart

[7 Nov 2010 15:00] Bjørn Munch

Pushed to -mtr branches

[13 Nov 2010 16:11] Bugs System

Pushed into mysql-trunk 5.6.99-m5 (revid:[e-mail]-20101113155825-czmva9kg4n31anmu) (version source revid:[e-mail]-20101113152450-2zzcm50e7i4j35v7) (merge vers: 5.6.1-m4) (pib:21)

[13 Nov 2010 16:32] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20101113160336-atmtmfb3mzm4pz4i) (version source revid:[e-mail]-20101113152540-gxro4g0v29l27f5x) (pib:21)

[14 Nov 2010 2:50] Paul DuBois

Changes to test suite. No changelog entry needed.

[15 Dec 2010 5:51] Bugs System

Pushed into mysql-5.1 5.1.55 (revid:[e-mail]-20101215054055-vgwki317xg1wphhh) (version source revid:[e-mail]-20101215054055-vgwki317xg1wphhh) (merge vers: 5.1.55) (pib:23)

[16 Dec 2010 22:35] Bugs System

Pushed into mysql-5.5 5.5.9 (revid:[e-mail]-20101216101358-fyzr1epq95a3yett) (version source revid:[e-mail]-20101216101358-fyzr1epq95a3yett) (merge vers: 5.5.9) (pib:24)