Bug #42308 Several server tests do not pass MTR's --check option
Submitted: 23 Jan 2009 17:55 Modified: 18 Jun 2009 15:30
Reporter: Patrick Crews Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Tests Severity:S3 (Non-critical)
Version:5.0+ OS:Any
Assigned to: Matthias Leich CPU Architecture:Any
Tags: check-testcases, cleanup, server tests, test
Triage: Triaged: D3 (Medium)

[23 Jan 2009 17:55] Patrick Crews
Description:
There are a number of tests in the MySQL test suite that do not pass mysql-test-run's --check option.

(Note:  This list was generated with MySQL-5.1-bugteam tree).
You can use the attached Python script to run against MySQL 5.0 (or other versions)

The list follows:
archive.test [ fail ]
  cache_innodb.test [ fail ]
  concurrent_innodb_safelog.test [ fail ]
  concurrent_innodb_unsafelog.test [ fail ]
  events_restart_phase1.test [ fail ]
  events_stress.test [ fail ]
  grant2.test [ fail ]
  group_by.test [ fail ]
  init_file.test [ fail ]
  innodb-autoinc-optimize.test [ fail ]
  innodb.test [ fail ]
  innodb_bug34300.test [ fail ]
  innodb_notembedded.test [ fail ]
  insert_update.test [ fail ]
  mysql_client_test.test [ fail ]
  profiling.test [ fail ]
  query_cache_notembedded.test [ fail ]
  read_many_rows_innodb.test [ fail ]
  status.test [ fail ]

These tests are Folio3 tests and will be fixed as part of other efforts / bugs (but these fail --check as well):
character_set_connection_func.test [ fail ]
  completion_type_func.test [ fail ]
  concurrent_insert_func.test [ fail ]
  delay_key_write_func.test [ fail ]
  func_compress.test [ fail ]
  func_time.test [ fail ]
  general_log_file_basic.test [ fail ]
  innodb_autoextend_increment_basic.test [ fail ]
  innodb_autoinc_lock_mode_func.test [ fail ]
  innodb_fast_shutdown_basic.test [ fail ]
  innodb_max_dirty_pages_pct_func.test [ fail ]
  innodb_table_locks_func.test [ fail ]
  interactive_timeout_func.test [ fail ]
  key_buffer_size_func.test [ fail ]
  log_bin_trust_function_creators_func.test [ fail ]
  log_output_func.test [ fail ]
  log_queries_not_using_indexes_basic.test [ fail ]
  max_allowed_packet_func.test [ fail ]
  max_seeks_for_key_func.test [ fail ]
  max_sort_length_func.test [ fail ]
  myisam_stats_method_func.test [ fail ]
  query_cache_limit_func.test [ fail ]
  query_cache_type_func.test [ fail ]
  query_prealloc_size_func.test [ fail ]
  rpl_max_binlog_size_func.test [ fail ]
  slow_query_log_file_basic.test [ fail ]

How to repeat:
Run any of the above tests with the --check option

Example:
From the mysql-test directory:
./mysql-test-run.pl --force --big --check events_stress

Observe MTR's output for the check of the test case

Suggested fix:
Ensure that the state of the server after a test is the same as it's initial state.
[23 Jan 2009 17:57] Patrick Crews
Python utility script for gather failing tests. Run in mysql-5.1-bugteam

Attachment: mysql_test_check.py (text/x-python-script), 2.00 KiB.

[4 Feb 2009 18:29] Matthias Leich
grant2 was fixed via a push for
Bug#42003 tests missing the disconnect of connections <> default
[4 Feb 2009 20:38] Matthias Leich
"status" will be fixed within the fix for
Bug#42602 main.status: random failures
[17 Feb 2009 17:33] Matthias Leich
Application of a "private" test which runs
- "--check-testcases"
- stability related tests around parallel sessions
in mysql-5.0-bugteam (last changeset ~16-Feb-2009)
-------------------------------------------------
Tests which do not exist in !5.0!
   cache_innodb
   concurrent_innodb_safelog
   concurrent_innodb_unsafelog
   events_restart_phase1
   events_stress
   innodb_bug34300
   profiling
   read_many_rows_innodb
Tests which satisfy "--check-testcases":
   archive
   grant2
   group_by
   query_cache_notembedded
   status
   innodb_notembedded
Tests which do not satisfy "--check-testcases":
   init_file
   innodb-autoinc-optimize
   innodb
   insert_update
   mysql_client_test
Tests which were unstable around paralles sessions:
   innodb_notembedded
   status
[23 Apr 2009 16:21] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72720

2730 Matthias Leich	2009-04-23
      Fix for Bug#43546 Several 5.0 tests do not pass MTR's --check option (moved from Bug#42308)
      
      Details:
      insert_update, innodb-autoinc-optimize
         Add the DROP TABLE which was missing
      innodb_notembedded
         Take care that the disconnects of additional sessions are completed
[23 Apr 2009 16:46] Matthias Leich
The change set mentioned is for a subset
(Bug#43546 Several 5.0 tests do not pass MTR's 
 --check option (moved from Bug#42308).
It will be reviewed there.
Therefore I switch the status back to "In progress".
[24 Apr 2009 18:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72799

2730 Matthias Leich	2009-04-24
      Fix for Bug#43546 Several 5.0 tests do not pass MTR's --check option (moved from Bug#42308)
      
      Details:
      - insert_update
        Add DROP TABLE which was missing, error numbers -> names
      - innodb-autoinc-optimize, varbinary
        Add DROP TABLE which was missing
      - innodb_notembedded
        Take care that the disconnects of additional sessions are completed
      - sp_trans_log
        Add missing DROP function, improved formatting
[28 Apr 2009 15:00] Patrick Crews
Ok to push.
[3 May 2009 13:15] Matthias Leich
Sorry, the message about the changeset belongs
to another bug. -> Resetting some values
[15 May 2009 10:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/74183

2888 Matthias Leich	2009-05-15
      Fix for Bug#42308 Several server tests do not pass MTR's --check option
      
      Details:
      Most tests mentioned within the bug report were already fixed.
      The test modified here failed in stability (high parallel load) tests.
      
      Details:
      1. Take care that disconnects are finished before the test terminates.
      2. Correct wrong handling of send/reap in events_stress which caused
         random garbled output
      3. Minor beautifying of script code
[18 May 2009 21:09] Patrick Crews
Ok to push.
[20 May 2009 20:14] Matthias Leich
Push to
mysql-6.0-bugteam
mysql-5.1-bugteam
[28 May 2009 8:21] Bugs System
Pushed into 5.1.36 (revid:joro@sun.com-20090528073639-yohsb4q1jzg7ycws) (version source revid:matthias.leich@sun.com-20090520132744-als676cukqy24n1j) (merge vers: 5.1.36) (pib:6)
[29 May 2009 1:08] Paul Dubois
Test case changes. No changelog entry needed.

Setting report to NDI pending push into 6.0.x.
[17 Jun 2009 19:28] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090616183122-chjzbaa30qopdra9) (version source revid:matthias.leich@sun.com-20090520185642-sb12yshz0vo816x7) (merge vers: 6.0.12-alpha) (pib:11)
[18 Jun 2009 15:30] Paul Dubois
Test case changes. No changelog entry needed.
[20 Jun 2009 6:14] Sveta Smirnova
Bug #45617 was marked as duplicate of this one
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)
[26 Aug 2009 13:48] Bugs System
Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)
[27 Aug 2009 16:32] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)