Bug #43065 main.lock_multi: This test is too big if the disk is slow
Submitted: 20 Feb 2009 17:45 Modified: 13 Apr 2009 15:19
Reporter: Matthias Leich Email Updates:
Status: Closed Impact on me:
None 
Category:Tests: Server Severity:S3 (Non-critical)
Version:5.0, 5.1, 6.0 OS:Any
Assigned to: Matthias Leich CPU Architecture:Any
Tags: random failures, timeout
Triage: Triaged: D3 (Medium)

[20 Feb 2009 17:45] Matthias Leich
Description:
Outocome of tests with lock_multi on my notebook
(Intel Core2Duo 2.2 GHz, 4 GB RAM, SATA disk)
tree mysql-5.0-bugteam last change Feb 2009):
------------------------------------------------
./mysql-test-run.pl --force `perl -e "for (1 .. 30) \
   { print 'lock_multi '}"` | tee prt 2>&1

If there is no parallel load (CPU or disk)
runtime: ~ 23 till 26 seconds

If there is a parallel
root: dd if=/dev/<partition of testfilesystem> \
      of=/dev/null
This generates high load on the disk.
runtime: 800 till 830 seconds

This runtime is too much near the testcases timeout
of 900 seconds.

Search via xref:
Jan-2007 till Feb-2009 about 52 failures of this test
where Test output match (LIKE): [ fail ]  timeout

Typical weak platforms:
powermacg5, vm-win2003-*

The test is not bad.

IMHO the failure is to expect that a combination of
this test with MTR using a testcase timeout of 900
seconds and a testing box which has from whatever 
reason a slow disk perform all time well.

How to repeat:
See above

Suggested fix:
Cut the test into several pieces because this
is the easiest solution.
[20 Feb 2009 18:29] Matthias Leich
Test with high parallel disk load:
The subtest for
   Bug#38499 flush tables and multitable table update
             with derived table cause crash
has a runtim of ~ 490 seconds.
[17 Mar 2009 14:22] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69426

2773 Matthias Leich	2009-03-16
      Fix for Bug#43015 and Bug#43065
      
      Details for Bug#43015 main.lock_multi: Weak code (sleeps etc.)
      -------------------------------------------------------------
      - The fix for bug 42003 already removed a lot of the weaknesses mentioned.
      - Tests showed that there are unfortunately no improvements of this tests
        in MySQL 5.1 which could be ported back to 5.0.
      - Remove a superfluous "--sleep 1" around line 195
      
      Details for Bug#43065 main.lock_multi: This test is too big if the disk is slow
      -------------------------------------------------------------------------------
      - move the subtests for the bugs 38499 and 36691 into separate scripts
      - runtime under excessive parallel I/O load after applying the fix
        lock_multi           [ pass ]          22887
        lock_multi_bug38499  [ pass ]         536926
        lock_multi_bug38691  [ pass ]         258498
[23 Mar 2009 13:57] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/70058

2741 Matthias Leich	2009-03-23
      Fix for Bug#43015 and Bug#43065
      
      Details for Bug#43015 main.lock_multi: Weak code (sleeps etc.)
      -------------------------------------------------------------
      - The fix for bug 42003 already removed a lot of the weaknesses mentioned.
      - Tests showed that there are unfortunately no improvements of this tests
        in MySQL 5.1 which could be ported back to 5.0.
      - Remove a superfluous "--sleep 1" around line 195
      
      Details for Bug#43065 main.lock_multi: This test is too big if the disk is slow
      -------------------------------------------------------------------------------
      - move the subtests for the bugs 38499 and 36691 into separate scripts
      - runtime under excessive parallel I/O load after applying the fix
        lock_multi           [ pass ]          22887
        lock_multi_bug38499  [ pass ]         536926
        lock_multi_bug38691  [ pass ]         258498
[23 Mar 2009 14:21] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/70066

2741 Matthias Leich	2009-03-23
      Fix for Bug#43015 and Bug#43065
      
         Details for Bug#43015 main.lock_multi: Weak code (sleeps etc.)
         -------------------------------------------------------------
         - The fix for bug 42003 already removed a lot of the weaknesses mentioned.
         - Tests showed that there are unfortunately no improvements of this tests
           in MySQL 5.1 which could be ported back to 5.0.
         - Remove a superfluous "--sleep 1" around line 195
      
         Details for Bug#43065 main.lock_multi: This test is too big if the disk is slow
         -------------------------------------------------------------------------------
         - move the subtests for the bugs 38499 and 36691 into separate scripts
         - runtime under excessive parallel I/O load after applying the fix
           lock_multi           [ pass ]          22887
           lock_multi_bug38499  [ pass ]         536926
           lock_multi_bug38691  [ pass ]         258498
[24 Mar 2009 13:35] Matthias Leich
Pushed to
mysql-bugteam-6.0
mysql-bugteam-5.1
mysql-bugteam-5.0

I did not follow the suggestion of Patrick
".. add comments (echo to .result file) to
differentiate the actions taken by connections
 writer and locker in the tests ... specifically
,the bugNNNN extensions ..."
because such comments have no advantages if the
corresponding test alraedy runs with disabled
logging of queries and results.
[27 Mar 2009 14:31] Bugs System
Pushed into 5.0.80 (revid:joro@sun.com-20090327142516-55gumdxj39z6eijj) (version source revid:leonard@mysql.com-20090324072904-7w2lxdxzw8hx1rnm) (merge vers: 5.0.80) (pib:6)
[27 Mar 2009 14:58] Bugs System
Pushed into 5.1.34 (revid:joro@sun.com-20090327143448-wuuuycetc562ty6o) (version source revid:matthias.leich@sun.com-20090323160848-a4gtbtwqnl7bdx2e) (merge vers: 5.1.34) (pib:6)
[27 Mar 2009 15:20] Paul Dubois
Test case changes. No changelog entry needed.

Setting report to NDI pending push into 6.0.x.
[13 Apr 2009 9:22] Bugs System
Pushed into 6.0.11-alpha (revid:alik@sun.com-20090413084402-snnrocwzktcl88ny) (version source revid:matthias.leich@sun.com-20090323185831-13boe36lf6c7pagg) (merge vers: 6.0.11-alpha) (pib:6)
[13 Apr 2009 15:19] Paul Dubois
Test case changes. No changelog entry needed.
[9 May 2009 16:41] Bugs System
Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (merge vers: 5.1.34-ndb-6.2.18) (pib:6)
[9 May 2009 17:38] Bugs System
Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (merge vers: 5.1.34-ndb-6.3.25) (pib:6)
[9 May 2009 18:35] Bugs System
Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (merge vers: 5.1.34-ndb-7.0.6) (pib:6)