Bug #40866 mysql-test-run's check of tests provides false failures due to timestamp
Submitted: 19 Nov 2008 21:23 Modified: 8 Dec 2008 16:07
Reporter: Patrick Crews Email Updates:
Status: Closed Impact on me:
None 
Category:Tests Severity:S3 (Non-critical)
Version:Any OS:Any
Assigned to: Patrick Crews CPU Architecture:Any
Tags: check, mysql-test-run, Tests

[19 Nov 2008 21:23] Patrick Crews
Description:
mysql-test-run's --check option provides false positives due to its inclusion of the timestamp global variable in its checks.

Logging: ./mtr --check --mtr-build-thread=61 key

--- /Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp/check-master0.result	2008-11-20 00:16:24.000000000 +0300
+++ /Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp/check-master0.reject	2008-11-20 00:16:30.000000000 +0300
@@ -234,7 +234,7 @@
 time_format	%H:%i:%s
 time_zone	SYSTEM
 timed_mutexes	OFF
-timestamp	1227129384
+timestamp	1227129390
 tmp_table_size	16777216
 tmpdir	/Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp
 transaction_alloc_block_size	8192

mysqltest: Result content mismatch

--- /Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp/check-master0.result	2008-11-20 00:16:24.000000000 +0300
+++ /Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp/check-master0.reject	2008-11-20 00:16:30.000000000 +0300
@@ -234,7 +234,7 @@
 time_format	%H:%i:%s
 time_zone	SYSTEM
 timed_mutexes	OFF
-timestamp	1227129384
+timestamp	1227129390
 tmp_table_size	16777216
 tmpdir	/Users/pcrews/usr/local/bin/data1/work/5130/mysql-5.1_1/mysql-test/var/tmp
 transaction_alloc_block_size	8192
main.key                       [ pass ]           5676
-------------------------------------------------------
Stopping All Servers
All 1 tests were successful.
The servers were restarted 1 times
Spent 5.676 of 13 seconds executing testcases

Check of testcase failed for: main.key

How to repeat:
Run mysql-test-run with the --check option for any test.  

./mysql-test-run --check key

And observe that the check fails for timestamp differences. (See output in the Description field)

Suggested fix:
Eliminate timestamp from the checks.

If there is a pressing need for checking timestamp (ie to make sure a test hasn't drastically altered system time), perhaps we could check everything *but* the seconds / minutes.

This is a simple alteration of mysql-test/include/check-testcase.test
[20 Nov 2008 17:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/59426

2720 Patrick Crews	2008-11-20
      Bug#40866 mysql-test-run's check of tests provides false failures due to timestamp
      
      Altered check-testcase to exclude timestamp from GLOBAL variables check.
      A similar change was applied in the rpl trees' version of mtr.
      
      Also, altered SQL syntax for style purposes.
[20 Nov 2008 18:47] Matthias Leich
ok to push

BTW I made some experiments.
"timestamp" is a session variable = no danger
[24 Nov 2008 21:54] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/59725

2723 Patrick Crews	2008-11-24
      Bug#40866: mysql-test-run's check of tests provides false failures due to timestamp
      
      Altering how MTR checks global variable status to exclude timestamp
      Changed SQL statements to update style.
[2 Dec 2008 13:01] Bugs System
Pushed into 5.0.74  (revid:patrick.crews@sun.com-20081124215332-pt73sthqowu7ynx2) (version source revid:patrick.crews@sun.com-20081124215332-pt73sthqowu7ynx2) (pib:5)
[2 Dec 2008 18:11] Paul DuBois
Test suite change. No changelog entry needed.

Setting report to NDI pending push into 5.1.x, 6.0.x.
[8 Dec 2008 10:21] Bugs System
Pushed into 5.1.31  (revid:patrick.crews@sun.com-20081124215332-pt73sthqowu7ynx2) (version source revid:patrick.crews@sun.com-20081126180318-v685u61mpgoc176x) (pib:5)
[8 Dec 2008 11:31] Bugs System
Pushed into 6.0.9-alpha  (revid:patrick.crews@sun.com-20081124215332-pt73sthqowu7ynx2) (version source revid:satya.bn@sun.com-20081126062231-h6os2axygjw27wb4) (pib:5)
[8 Dec 2008 16:07] Paul DuBois
Test case changes. No changelog entry needed.
[19 Jan 2009 11:27] Bugs System
Pushed into 5.1.31-ndb-6.2.17 (revid:tomas.ulin@sun.com-20090119095303-uwwvxiibtr38djii) (version source revid:tomas.ulin@sun.com-20090108105244-8opp3i85jw0uj5ib) (merge vers: 5.1.31-ndb-6.2.17) (pib:6)
[19 Jan 2009 13:05] Bugs System
Pushed into 5.1.31-ndb-6.3.21 (revid:tomas.ulin@sun.com-20090119104956-guxz190n2kh31fxl) (version source revid:tomas.ulin@sun.com-20090119104956-guxz190n2kh31fxl) (merge vers: 5.1.31-ndb-6.3.21) (pib:6)
[19 Jan 2009 16:11] Bugs System
Pushed into 5.1.31-ndb-6.4.1 (revid:tomas.ulin@sun.com-20090119144033-4aylstx5czzz88i5) (version source revid:tomas.ulin@sun.com-20090119144033-4aylstx5czzz88i5) (merge vers: 5.1.31-ndb-6.4.1) (pib:6)