| Bug #89533 | MTR test rpl_gtid.rpl_multi_source_block_receiver failing testcase check | ||
|---|---|---|---|
| Submitted: | 5 Feb 2018 12:51 | Modified: | 18 Jul 2019 10:17 |
| Reporter: | Laurynas Biveinis (OCA) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Tests: Replication | Severity: | S3 (Non-critical) |
| Version: | 8.0.4 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[23 Mar 2018 8:55]
MySQL Verification Team
Hello Laurynas, Thank you for the report and feedback. I tried reproduce this on almost all available instances(OL7, Ubuntu, CentOS7 etc with src debug builds - mysql-boost-8.0.4-rc.tar.gz) but not seeing any issues. Could you please provide exact cmake with the options used? Thanks! Regards, Umesh
[23 Mar 2018 13:14]
MySQL Verification Team
## ushastry@ArtfulAardvark:~/Downloads$ cmake --version cmake version 3.9.1 CMake suite maintained and supported by Kitware (kitware.com/cmake). ushastry@ArtfulAardvark:~/Downloads$ gcc --version gcc (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ushastry@ArtfulAardvark:~/Downloads$ uname -aa Linux ArtfulAardvark 4.13.0-36-generic #40-Ubuntu SMP Fri Feb 16 20:07:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ushastry@ArtfulAardvark:~/Downloads$ cat /etc/*release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=17.10 DISTRIB_CODENAME=artful DISTRIB_DESCRIPTION="Ubuntu 17.10" NAME="Ubuntu" VERSION="17.10 (Artful Aardvark)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 17.10" VERSION_ID="17.10" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=artful UBUNTU_CODENAME=artful ushastry@ArtfulAardvark:~/Downloads/mysql-8.0.4-rc$ cmake . -DDOWNLOAD_BOOST=1 -DWITH_BOOST=./boost -DWITH_ASAN=ON -DWITH_UBSAN=ON -DWITH_DEBUG=1 -DCMAKE_INSTALL_PREFIX=/home/ushastry/Downloads/8_4_ASAN_UBSAN ushastry@ArtfulAardvark:~/Downloads/mysql-8.0.4-rc$ make -j8 ushastry@ArtfulAardvark:~/Downloads/mysql-8.0.4-rc$ make install ushastry@ArtfulAardvark:~/Downloads$ cd 8_4_ASAN_UBSAN/ ushastry@ArtfulAardvark:~/Downloads/8_4_ASAN_UBSAN$ cd mysql-test/ ushastry@ArtfulAardvark:~/Downloads/8_4_ASAN_UBSAN/mysql-test$ cd .. ushastry@ArtfulAardvark:~/Downloads/8_4_ASAN_UBSAN$ cat docs/INFO_SRC commit: 73b6cff33896a3cd0f0d02d6c30fdaed8b562cf3 date: 2018-01-18 16:43:56 +0100 build-date: 2018-01-18 16:53:08 +0100 short: 73b6cff branch: mysql-8.0.4-rc-release MySQL source 8.0.4 ushastry@ArtfulAardvark:~/Downloads/8_4_ASAN_UBSAN$ cd mysql-test/ ushastry@ArtfulAardvark:~/Downloads/8_4_ASAN_UBSAN/mysql-test$ ./mtr --debug-server rpl_gtid.rpl_multi_source_block_receiver Logging: ./mtr --debug-server rpl_gtid.rpl_multi_source_block_receiver 2018-03-23T13:00:49.423467Z 0 [System] [MY-010116] /home/ushastry/Downloads/8_4_ASAN_UBSAN/bin/mysqld (mysqld 8.0.4-rc-debug-log-asan-ubsan) starting as process 20047 ... MySQL Version 8.0.4 Checking supported features... - SSL connections supported - binaries are debug compiled Collecting tests... - adding combinations for rpl_gtid Checking leftover processes... Removing old var directory... Creating var directory '/home/ushastry/Downloads/8_4_ASAN_UBSAN/mysql-test/var'... Installing system database... Using parallel: 1 ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 rpl_gtid.rpl_multi_source_block_receiver 'mix' [ pass ] 10379 rpl_gtid.rpl_multi_source_block_receiver 'row' [ pass ] 11090 rpl_gtid.rpl_multi_source_block_receiver 'stmt' [ pass ] 10672 -------------------------------------------------------------------------- The servers were restarted 2 times Spent 32.141 of 228 seconds executing testcases Completed: All 3 tests were successful.
[26 Mar 2018 7:57]
Laurynas Biveinis
Umesh - I have lost the exact CMake invocation for this particular reproduction. In any case this should be load-, not cmake-related, and I have suggested an IMHO very plausible cause and fix in the bug report, it could be handled as a testcase improvement request then.
[26 Mar 2018 14:25]
MySQL Verification Team
Thank you for the feedback. Verifying as a testcase improvement request. Regards, Umesh
[27 May 2019 11:23]
Sven Sandberg
Posted by developer: Thank you for a good bug report. We don't need to reproduce this to fix it the way suggested. There is clearly a missing synchronization that will lead to the described symptom, and the suggested fix is obviously the right one. Setting status to verified.
[18 Jul 2019 10:17]
Margaret Fisher
Posted by developer: Test case so no changelog entry. Thanks for reporting.
Description: Intermittently: rpl_gtid.rpl_multi_source_block_receiver 'stmt' w1 [ fail ] Test ended at 2018-02-05 01:34:28 MTR's internal check of the test case 'rpl_gtid.rpl_multi_source_block_receiver' failed. This means that the test case does not preserve the state that existed before the test case was executed. Most likely the test case did not do a proper clean-up. It could also be caused by the previous test run by this thread, if the server wasn't restarted. This is the diff of the states of the servers before and after the test case was executed: mysqltest: Results saved in '/mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/min-centos-7-x64/build/mysql-test/var/1/tmp/check-mysqld_2.result'. mysqltest: Logging to '/mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/min-centos-7-x64/build/mysql-test/var/1/tmp/check-mysqld_2.log'. mysqltest: Connecting to server localhost:13001 (socket /tmp/2cBaIuT0W1/1/mysqld.2.sock) as 'root', connection 'default', attempt 0 ... mysqltest: ... Connected. mysqltest: Start processing test commands from './include/check-testcase.test' ... mysqltest: ... Done processing test commands. --- /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/min-centos-7-x64/build/mysql-test/var/1/tmp/check-mysqld_2.result 2018-02-05 09:34:24.547744599 +0300 +++ /mnt/workspace/mysql-5.7-param/BUILD_TYPE/debug/Host/min-centos-7-x64/build/mysql-test/var/1/tmp/check-mysqld_2.reject 2018-02-05 09:34:28.234790310 +0300 @@ -633,6 +633,7 @@ def test utf8mb4 utf8mb4_0900_ai_ci NULL FILE_ID FILE_NAME FILE_TYPE TABLESPACE_NAME TABLE_CATALOG TABLE_SCHEMA TABLE_NAME LOGFILE_GROUP_NAME LOGFILE_GROUP_NUMBER ENGINE FULLTEXT_KEYS DELETED_ROWS UPDATE_COUNT FREE_EXTENTS TOTAL_EXTENTS EXTENT_SIZE INITIAL_SIZE MAXIMUM_SIZE AUTOEXTEND_SIZE CREATION_TIME LAST_UPDATE_TIME LAST_ACCESS_TIME RECOVER_TIME TRANSACTION_COUNTER VERSION ROW_FORMAT TABLE_ROWS AVG_ROW_LENGTH DATA_LENGTH MAX_DATA_LENGTH INDEX_LENGTH DATA_FREE CREATE_TIME UPDATE_TIME CHECK_TIME CHECKSUM STATUS EXTRA tables_in_test +t2 tables_in_mysql mysql.columns_priv mysql.component mysqltest: Result content mismatch How to repeat: Not sure, probably likely to happen under load Suggested fix: The relevant testcase snippet is --let $rpl_connection_name= server_3 --source include/rpl_connection.inc DROP TABLE t2; --let $rpl_connection_name= server_2 --source include/rpl_connection.inc --let $rpl_channel_name= 'channel_3' --source include/start_slave.inc --let $rpl_skip_sync=1 --source include/rpl_end.inc It looks like the issue is missing sync on channel_3 after starting the slave there.