Bug #87312 Test main.events_time_zone is fundamentally unstable
Submitted: 4 Aug 2017 3:12
Reporter: Laurynas Biveinis (OCA) Email Updates:
Status: Open Impact on me:
None 
Category:Tests: Server Severity:S3 (Non-critical)
Version:5.5+ OS:Any
Assigned to: CPU Architecture:Any
Tags: event scheduler, mtr

[4 Aug 2017 3:12] Laurynas Biveinis
Description:
Test main.events_time_zone fails intermittently, e.g.

main.events_time_zone                    w7 [ fail ]
        Test ended at 2017-08-03 17:19:54

CURRENT_TEST: main.events_time_zone
--- /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/ubuntu-yakkety-64bit/mysql-test/r/events_time_zone.result	2017-08-03 16:52:30.747832000 +0300
+++ /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/ubuntu-yakkety-64bit/build/mysql-test/var/7/log/events_time_zone.reject	2017-08-03 20:19:53.815657061 +0300
@@ -74,10 +74,10 @@
 5	5	5	e2 should be executed
 6	0	0	<e1>
 6	0	2	<e2>
-6	0	2	Forward +2 step shift, local 0, 1 are skipped, e2 should be executed
+6	1	3	Forward +2 step shift, local 0, 1 are skipped, e2 should be executed
 7	1	1	<e1>
 7	1	3	<e2>
-7	1	3	e2 should be executed
+7	2	4	e2 should be executed
 SET TIME_ZONE= @save_time_zone;
 DROP EVENT e2;
 DROP EVENT e1;

mysqltest: Result content mismatch

How to repeat:
The above happened on Ubuntu Yakkety, MySQL 8.0.2, CMake options -DCMAKE_BUILD_TYPE=Debug -DMYSQL_MAINTAINER_MODE=ON -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DENABLE_DOWNLOADS=1 -DWITH_SSL=system -DDOWNLOAD_BOOST=1 -DWITH_BOOST=mysql-boost -DWITH_ZLIB=system -DDOWNLOAD_BOOST_TIMEOUT=3000

Suggested fix:
The testcase itself admits that it's unstable:

# 1. This test case is sensitive to execution timing.  You may control
#    this sensitivity by the parameter below.  Small values will result
#    in fast but more unstable execution, large values will improve
#    stability at the cost of speed.  Basically, N is a number of seconds
#    to wait for operation to complete.  Should be positive.  Test runs
#    about 25*N seconds (it sleeps most of the time, so CPU speed is not
#    relevant).
let $N = 5;

In Percona Server, we tried bumping N aggressively, and still observed the failures. We ended up disabling the testcase. It will not be completely stable without e.g. rewriting it to use debug builds and injecting deterministic timestamps
[16 Apr 2018 1:10] Miguel Solorzano
Tried to reproduce this bug several times with several server version without luck:

miguel@tikal:~/tmp/2018APR04/mysql-8.0/mysql-test $ ./mtr events_time_zone --big-test
Logging: ./mtr  events_time_zone --big-test
2018-04-16T01:01:24.602247Z 0 [System] [MY-010116] [Server] /home/miguel/tmp/2018APR04/mysql-8.0/runtime_output_directory/mysqld (mysqld 8.0.12-debug) starting as process 4839
MySQL Version 8.0.12
Checking supported features...
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Removing old var directory...
Creating var directory '/home/miguel/tmp/2018APR04/mysql-8.0/mysql-test/var'...
Installing system database...
Using parallel: 1

==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
main.events_time_zone                    [ pass ]  124205
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 124.205 of 421 seconds executing testcases

Completed: All 1 tests were successful.

miguel@tikal:~/tmp/2018APR04/mysql-8.0/mysql-test $