Bug #48240 "Test suite timeout" and "Too many tests failed" is masked from status page

• Submitted: 22 Oct 2009 18:51
• Modified: 17 Jun 2010 19:41
• Reporter: Joerg Bruehe
• Status: Closed
• Category: Tools: MTR / mysql-test-run
• Severity: S3 (Non-critical)
• Version: Any
• OS: Any
• Assigned to: Bjørn Munch
• CPU Architecture: Any

[22 Oct 2009 18:51] Joerg Bruehe

Description:
Test script "mysql-test-run.pl" ("MTR") writes a message
   Test suite timeout
if the timeout for the whole suite hits - so far, so good.

However, since June 2009 the tool also reports statistics of the tests taken before the timeout hit, and for this it uses the same format as it would on normal termination. (Fix for bug#45190)

A side effect of this change is that now the tool generating the status page recognizes a normal end line and does not detect the timeout, so the status page does not display that fact.

This needs to be changed so that the timeout is displayed in the status page.

How to repeat:
Just run a release build on a slow machine.

Suggested fix:
1) Change the format of the summary line in case of timeout.

2) As an alternative, change the order of lines.
This alone will not solve the problem, it needs a corresponding change in "gen-build-status-page" to recognize the timeout.

Approach 1 is the preferred one, because it isolates the change to this script, and it would be helpful to find "timeout" immediately in the summary line.

[23 Oct 2009 23:29] Joerg Bruehe

The same happens when MTR decides to abort running a suite because there were 10 failures:
     Too many tests(10) failed! Terminating...
     mysql-test-run: *** ERROR: Test suite aborted

We definitely need to show this in the status page, with the current implementation we get a false sense of security because the problems are masked.

Until this is fixed, the status page does not correctly display the situation.

[26 Oct 2009 13:30] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/88157

2850 Bjorn Munch	2009-10-26
      Bug #48240 "Test suite timeout" and "Too many tests failed" is masked from status page
      Prepend "Completed", "Timeout" or "Too many failed" to summary line

[26 Oct 2009 13:51] Joerg Bruehe

The change should solve the problem, looks good to me.

[26 Oct 2009 14:06] Daniel Fischer

What could possibly go wrong?

[26 Oct 2009 14:41] Bjørn Munch

Pushed to 5.1-mtr, trunk-mtr, next-mr-mtr and 6.0-codebase-mtr

[1 Mar 2010 8:46] Bugs System

Pushed into 5.1.45 (revid:[e-mail]-20100301083827-xnimmrjg6bh33o1o) (version source revid:[e-mail]-20100222175719-viuh0f3gdsrkgv0r) (merge vers: 5.1.45) (pib:16)

[1 Mar 2010 16:25] Paul DuBois

Test suite changes. No changelog entry needed.

Setting report to Need Merge pending push to Celosia, 6.0.x.

[1 Mar 2010 17:26] Bjørn Munch

Has been pushed to 5.5.3-m2, 6.0.14-alpha on 2010-02-20 (along with other MTR fixes), but for some reason not automatically updated in bug report. Closing.

[17 Jun 2010 12:19] Bugs System

Pushed into 5.1.47-ndb-7.0.16 (revid:[e-mail]-20100617114014-bva0dy24yyd67697) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

[17 Jun 2010 13:06] Bugs System

Pushed into 5.1.47-ndb-6.2.19 (revid:[e-mail]-20100617115448-idrbic6gbki37h1c) (version source revid:[e-mail]-20100609140708-52rvuyq4q500sxkq) (merge vers: 5.1.45-ndb-6.2.19) (pib:16)

[17 Jun 2010 13:47] Bugs System

Pushed into 5.1.47-ndb-6.3.35 (revid:[e-mail]-20100617114611-61aqbb52j752y116) (version source revid:[e-mail]-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)