| Bug #58754 | rpl_corruption failed on mysql-trunk | ||
|---|---|---|---|
| Submitted: | 6 Dec 2010 13:06 | Modified: | 30 Mar 2011 8:10 |
| Reporter: | Andrei Elkin | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Replication | Severity: | S3 (Non-critical) |
| Version: | mysql-trunk | OS: | Any |
| Assigned to: | Andrei Elkin | CPU Architecture: | Any |
[8 Feb 2011 21:46]
Luis Soares
This is failing sporadically on {mysql,daily,weekly}-trunk trees with new
symptoms:
- Symptom #1: Slave stopped with wrong error code
rpl.rpl_corruption 'row' [ fail ]
Test ended at 2011-02-05 00:40:02
CURRENT_TEST: rpl.rpl_corruption
mysqltest: In included file ".\include\wait_for_slave_io_error.inc":
included from .\include\wait_for_slave_io_error.inc at line 69:
At line 68: Slave stopped with wrong error code
The result from queries just before the failure was:
< snip >
master-bin.000001 3917 Delete_rows 1 3957 table_id: 51 flags: STMT_END_F
master-bin.000001 3957 Query 1 4030 COMMIT
master-bin.000001 4030 Query 1 4102 BEGIN
master-bin.000001 4102 Table_map 1 4153 table_id: 51 (test.t1)
master-bin.000001 4153 Write_rows 1 4193 table_id: 51 flags: STMT_END_F
master-bin.000001 4193 Query 1 4266 COMMIT
master-bin.000001 4266 Query 1 4338 BEGIN
master-bin.000001 4338 Table_map 1 4389 table_id: 51 (test.t1)
master-bin.000001 4389 Write_rows 1 4429 table_id: 51 flags: STMT_END_F
master-bin.000001 4429 Query 1 4502 COMMIT
master-bin.000001 4502 Query 1 4574 BEGIN
master-bin.000001 4574 Table_map 1 4625 table_id: 51 (test.t1)
master-bin.000001 4625 Write_rows 1 4665 table_id: 51 flags: STMT_END_F
master-bin.000001 4665 Query 1 4738 COMMIT
**** SHOW RELAYLOG EVENTS on server_1 ****
relaylog_name = 'No such row'
SHOW RELAYLOG EVENTS IN 'No such row';
Log_name Pos Event_type Server_id End_log_pos Info
**** Slave stopped with wrong error code: 1722 (expected 1595) ****
More results from queries before failure can be found in G:\pb2\test\sb_1-2947639-1296862223.87\mysql-5.6.2-m5-winx64\mysql-test\var-normal\log\rpl_corruption.log
- saving 'G:/pb2/test/sb_1-2947639-1296862223.87/mysql-5.6.2-m5-winx64/mysql-test/var-normal/log/rpl.rpl_corruption-row/' to 'G:/pb2/test/sb_1-2947639-1296862223.87/mysql-5.6.2-m5-winx64/mysql-test/var-normal/log/rpl.rpl_corruption-row/'
Retrying test rpl.rpl_corruption, attempt(2/3)...
- Symptom #2: failed in 'select master_pos_wait('master-bin.000001', 3136, 300)'
rpl.rpl_corruption 'row' [ fail ]
Test ended at 2011-02-05 10:37:56
CURRENT_TEST: rpl.rpl_corruption
mysqltest: At line 114: failed in 'select master_pos_wait('master-bin.000001', 3136, 300)': 2013: Lost connection to MySQL server during query
The result from queries just before the failure was:
< snip >
SET GLOBAL debug="-d,corrupt_read_log_event";
# 4. Master read a corrupted event from binlog and send it to slave
SET GLOBAL master_verify_checksum=0;
SET GLOBAL debug="+d,corrupt_read_log_event";
START SLAVE IO_THREAD;
- Symptom #3: query 'START SLAVE' failed
rpl.rpl_corruption [ fail ]
Test ended at 2011-02-03 17:41:15
CURRENT_TEST: rpl.rpl_corruption
mysqltest: In included file ".\include\start_slave.inc":
included from .\include\start_slave.inc at line 34:
At line 34: query 'START SLAVE' failed: 2013: Lost connection to MySQL server during query
The result from queries just before the failure was:
< snip >
SET GLOBAL debug="-d,corrupt_read_log_event";
# 4. Master read a corrupted event from binlog and send it to slave
SET GLOBAL master_verify_checksum=0;
SET GLOBAL debug="+d,corrupt_read_log_event";
START SLAVE IO_THREAD;
[9 Mar 2011 18:49]
Andrei Elkin
Actually among two issues one is a crash with a stack the as as one of Bug #40385.
[12 Mar 2011 12:18]
Andrei Elkin
Here is a stack and coincidently or maybe not the stack appears at time of the SQL thread executes its own exposed to valgrind failure of bug#59060 (see there for its stack). 000000013FFC2F5B mysqld-debug.exe!strcmp()[strcmp.asm:68] 000000013FF0880A mysqld-debug.exe!InList()[dbug.c:1501] 000000013FF08959 mysqld-debug.exe!_db_keyword_()[dbug.c:1718] 000000013FF079E9 mysqld-debug.exe!_db_doprnt_()[dbug.c:1262] 000000013FF545E6 mysqld-debug.exe!vio_read()[viosocket.c:52] 000000013FF547C7 mysqld-debug.exe!vio_read_buff()[viosocket.c:83] 000000013F9AD8FE mysqld-debug.exe!my_real_read()[net_serv.cc:831] 000000013F9AD34C mysqld-debug.exe!my_net_read()[net_serv.cc:1028] 000000013FA5FE3A mysqld-debug.exe!cli_safe_read()[client.c:735] 000000013FA67064 mysqld-debug.exe!cli_read_query_result()[client.c:3821] 000000013FA675F3 mysqld-debug.exe!mysql_real_query()[client.c:3908] 000000013FBFAAE9 mysqld-debug.exe!get_master_version_and_clock()[rpl_slave.cc:1766] 000000013FBF8606 mysqld-debug.exe!handle_slave_io()[rpl_slave.cc:3258] 000000013FEE9FDE mysqld-debug.exe!pfs_spawn_thread()[pfs.cc:1401] 000000013FC1579B mysqld-debug.exe!pthread_start()[my_winthread.c:62] 000000013FFD6075 mysqld-debug.exe!_callthreadstartex()[threadex.c:348] 000000013FFD6048 mysqld-debug.exe!_threadstartex()[threadex.c:331] 00000000777ABE3D kernel32.dll!BaseThreadInitThunk() 00000000778E6A51 ntdll.dll!RtlUserThreadStart()
[30 Mar 2011 8:10]
Jon Stephens
Issue doesn't appear in a release. Closed without further action.
Description: A recent run of rpl_checksum on PB2 showed rpl.rpl_corruption w3 [ fail ] Test ended at 2010-12-05 14:30:43 CURRENT_TEST: rpl.rpl_corruption mysqltest: In included file ".\include\start_slave.inc": At line 18: query 'START SLAVE' failed: 2013: Lost connection to MySQL server during query The result from queries just before the failure was: < snip > SET GLOBAL debug="-d,corrupt_read_log_event_char"; # 3. Master read a corrupted event from binlog and send the error to slave SET GLOBAL debug="+d,corrupt_read_log_event"; START SLAVE IO_THREAD; SET GLOBAL debug="-d,corrupt_read_log_event"; # 4. Master read a corrupted event from binlog and send it to slave SET GLOBAL master_verify_checksum=0; SET GLOBAL debug="+d,corrupt_read_log_event"; START SLAVE IO_THREAD; SET GLOBAL debug="-d,corrupt_read_log_event"; SET GLOBAL master_verify_checksum=1; # 5. Slave. Corruption in network SET GLOBAL debug="+d,corrupt_queue_event"; START SLAVE IO_THREAD; SET GLOBAL debug="-d,corrupt_queue_event"; # 6. Slave. Corruption in relay log SET GLOBAL debug="+d,corrupt_read_log_event_char"; START SLAVE; SET GLOBAL debug="-d,corrupt_read_log_event_char"; # 7. Seek diff for tables on master and slave More results from queries before failure can be found in G:\pb2\test\sb_1-2634914-1291555679.18\mysql-5.6.1-m5-win-x86-test\mysql-test\var-n_mix\3\log\rpl_corruption.log How to repeat: https://central.sun.net/http://pb2.norway.sun.com/?template=mysql_show_test_failure&search...