MySQL Bugs: #79771: found a zombie dump thread with the same UUID. Master is killing the zombie dum

Bug #79771	found a zombie dump thread with the same UUID. Master is killing the zombie dum
Submitted:	25 Dec 2015 2:26	Modified:	3 Jan 2017 7:06
Reporter:	lee zhenhua	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Replication	Severity:	S1 (Critical)
Version:	5.6.24	OS:	CentOS (6.6 x86_64)
Assigned to:		CPU Architecture:	Any

Description:
2015-09-14 14:02:34 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 1, Er
ror_code: 2003
2015-09-14 14:04:35 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 2, Er
ror_code: 2003
2015-09-14 14:06:35 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 3, Er
ror_code: 2003
2015-09-14 14:08:35 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 4, Er
ror_code: 2003
2015-09-14 14:10:35 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 5, Er
ror_code: 2003
2015-09-14 14:12:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 6, Er
ror_code: 2003
2015-09-14 14:12:07 23901 [Note] While initializing dump thread for slave with UUID <b1696644-4a30-11e5-b14e-0a922f8f5927>, found a
zombie dump thread with the same UUID. Master is killing the zombie dump thread(3).
2015-09-14 14:12:07 23901 [Note] Start binlog_dump to master_thread_id(107) slave_server(1), pos(mysql-bin.000090, 103087)
2015-09-14 14:13:06 23901 [Note] Slave: connected to master 'repl@172.31.39.254:3306',replication resumed in log 'mysql-bin.000007' 
at position 30685840
2015-09-14 14:20:06 23901 [Warning] Storing MySQL user name or password information in the master info repository is not secure and 
is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE S
yntax' in the MySQL Manual for more information.
2015-09-14 14:21:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 1, Er
ror_code: 2003
2015-09-14 14:23:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 2, Er
ror_code: 2003
2015-09-14 14:25:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 3, Er
ror_code: 2003
2015-09-14 14:27:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 4, Er
ror_code: 2003
2015-09-14 14:29:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 5, Er
ror_code: 2003
2015-09-14 14:31:03 23901 [Note] While initializing dump thread for slave with UUID <b1696644-4a30-11e5-b14e-0a922f8f5927>, found a
zombie dump thread with the same UUID. Master is killing the zombie dump thread(107).
2015-09-14 14:31:03 23901 [Note] Start binlog_dump to master_thread_id(108) slave_server(1), pos(mysql-bin.000090, 133782)
2015-09-14 14:31:06 23901 [ERROR] Slave I/O: error reconnecting to master 'repl@172.31.39.254:3306' - retry-time: 60  retries: 6, Er
ror_code: 2003
2015-09-14 14:32:06 23901 [Note] Slave: connected to master 'repl@172.31.39.254:3306',replication resumed in log 'mysql-bin.000008' 
at position 31975
2015-09-14 14:32:06 23901 [ERROR] Error reading packet from server: Client requested master to start replication from position > fil
e size; the first event 'mysql-bin.000008' at 31975, the last event read from '/export/data/binlog/mysql-bin.000008' at 4, the last
byte read from '/export/data/binlog/mysql-bin.000008' at 4. ( server_errno=1236)
2015-09-14 14:32:06 23901 [ERROR] Slave I/O: Got fatal error 1236 from master when reading data from binary log: 'Client requested m
aster to start replication from position > file size; the first event 'mysql-bin.000008' at 31975, the last event read from '/export
/data/binlog/mysql-bin.000008' at 4, the last byte read from '/export/data/binlog/mysql-bin.000008' at 4.', Error_code: 1236

How to repeat:
i don't know how to repeat

on the master host i found this
 2015-09-14 14:31:03 3585 [Note] Recovering after a crash using /export/data/binlog/mysql-bin
2015-09-14 14:31:03 3585 [ERROR] Error in Log_event::read_log_event(): 'Event too small', data_len: 0, event_type: 0
2015-09-14 14:31:03 3585 [Note] Starting crash recovery...
2015-09-14 14:31:03 3585 [Note] Crash recovery finished.
2015-09-14 14:31:03 3585 [Note] Crashed binlog file /export/data/binlog/mysql-bin.000008 size is 31975, but recovered up to 31580. B
inlog trimmed to 31580 bytes.

Thank you for the report.

Post Bug #72578 fix,  a NOTE is added on master's error log file with the following information when log_warnings is greater than 1.

"While initializing dump thread for slave with UUID <slave_uuid>,
  found a zombie dump thread with the same UUID.
  Master is killing the zombie dump thread."

Marking this as duplicate of Bug #84358 for now as Bug #84358 is in verified state.

This is just to update you that our internal discussions concluded that it is not a duplicate of BUG#84358(which is sort of enhancement to the reported note to fix the confusing error message) this issue is just the expected result of recovery after a crash, and maybe what really needs to know here is that the reason for the crash. This is currently impossible to debug because it is missing lots of information, e.g. replication topology, configuration, and binlogs.  Also, this is an expected if you are using bidirectional replication and the master is using sync-binlog=0 and crashes. I would suggest you to reopen the issue if you are seeing crash on latest GA builds i.e 5.6.35/5.7.16 along with the missing detail.

If you can provide more information, feel free to add it to this bug and change the status back to 'Open'.

Thank you for your interest in MySQL.

Thanks,
Umesh