Bug #87312 Test main.events_time_zone is fundamentally unstable

Submitted: 4 Aug 2017 3:12 Modified: 5 Nov 2019 21:15
Reporter: Laurynas Biveinis (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:Tests: Server Severity:S7 (Test Cases)
Version:5.5+ OS:Any
Assigned to: CPU Architecture:Any
Tags: event scheduler, mtr

[4 Aug 2017 3:12] Laurynas Biveinis
Description:
Test main.events_time_zone fails intermittently, e.g.

main.events_time_zone                    w7 [ fail ]
        Test ended at 2017-08-03 17:19:54

CURRENT_TEST: main.events_time_zone
--- /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/ubuntu-yakkety-64bit/mysql-test/r/events_time_zone.result	2017-08-03 16:52:30.747832000 +0300
+++ /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/ubuntu-yakkety-64bit/build/mysql-test/var/7/log/events_time_zone.reject	2017-08-03 20:19:53.815657061 +0300
@@ -74,10 +74,10 @@
 5	5	5	e2 should be executed
 6	0	0	<e1>
 6	0	2	<e2>
-6	0	2	Forward +2 step shift, local 0, 1 are skipped, e2 should be executed
+6	1	3	Forward +2 step shift, local 0, 1 are skipped, e2 should be executed
 7	1	1	<e1>
 7	1	3	<e2>
-7	1	3	e2 should be executed
+7	2	4	e2 should be executed
 SET TIME_ZONE= @save_time_zone;
 DROP EVENT e2;
 DROP EVENT e1;

mysqltest: Result content mismatch

How to repeat:
The above happened on Ubuntu Yakkety, MySQL 8.0.2, CMake options -DCMAKE_BUILD_TYPE=Debug -DMYSQL_MAINTAINER_MODE=ON -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DENABLE_DOWNLOADS=1 -DWITH_SSL=system -DDOWNLOAD_BOOST=1 -DWITH_BOOST=mysql-boost -DWITH_ZLIB=system -DDOWNLOAD_BOOST_TIMEOUT=3000

Suggested fix:
The testcase itself admits that it's unstable:

# 1. This test case is sensitive to execution timing.  You may control
#    this sensitivity by the parameter below.  Small values will result
#    in fast but more unstable execution, large values will improve
#    stability at the cost of speed.  Basically, N is a number of seconds
#    to wait for operation to complete.  Should be positive.  Test runs
#    about 25*N seconds (it sleeps most of the time, so CPU speed is not
#    relevant).
let $N = 5;

In Percona Server, we tried bumping N aggressively, and still observed the failures. We ended up disabling the testcase. It will not be completely stable without e.g. rewriting it to use debug builds and injecting deterministic timestamps
[16 Apr 2018 1:10] MySQL Verification Team
Tried to reproduce this bug several times with several server version without luck:

miguel@tikal:~/tmp/2018APR04/mysql-8.0/mysql-test $ ./mtr events_time_zone --big-test
Logging: ./mtr  events_time_zone --big-test
2018-04-16T01:01:24.602247Z 0 [System] [MY-010116] [Server] /home/miguel/tmp/2018APR04/mysql-8.0/runtime_output_directory/mysqld (mysqld 8.0.12-debug) starting as process 4839
MySQL Version 8.0.12
Checking supported features...
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Removing old var directory...
Creating var directory '/home/miguel/tmp/2018APR04/mysql-8.0/mysql-test/var'...
Installing system database...
Using parallel: 1

==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
main.events_time_zone                    [ pass ]  124205
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 124.205 of 421 seconds executing testcases

Completed: All 1 tests were successful.

miguel@tikal:~/tmp/2018APR04/mysql-8.0/mysql-test $
[10 Sep 2019 23:21] MySQL Verification Team
I was able to repeat, it's still valid bug?. Thanks in advance.
[24 Sep 2019 14:20] Laurynas Biveinis
Yes I believe so - It hasn't been substantially changed since the report, including the part where the header comment confirms this bug explicitly. Perhaps it can be confirmed by the testcase source reading.
[5 Nov 2019 21:15] MySQL Verification Team
Thank you for the feedback.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.040 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.