MySQL Bugs: #101947: show slave status/io/sql threads is hang

Bug #101947	show slave status/io/sql threads is hang
Submitted:	10 Dec 2020 7:31	Modified:	10 Jan 2021 9:03
Reporter:	peng gao	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	MySQL Server	Severity:	S2 (Serious)
Version:	5.7.17 Enterprise	OS:	CentOS (6.6)
Assigned to:		CPU Architecture:	Any

Description:
Hi all:
  We monitor our slave status use "show slave status",but occasionally "show slave status" is hang!
  And sql thread and io thread is hang! but every time when use pstack to capture thread stack then 
  IO/SQL/SHOW SLAVE STATUS  is go on.We use GTID MTS SLAVE. 
  
  
  show processlist status:
  IO thread:Queueing master event to the relay log
  SQL thread:Reading event form the relay log
  worker thread:Waiting for event for Cooradinator
  show slave status:starting
  
  Io thread stack: 
Thread 7 (Thread 0x7f13627aa700 (LWP 23027)):
#0  0x00007f136f843264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f136f83e53e in _L_lock_995 () from /lib64/libpthread.so.0
#2  0x00007f136f83e4a6 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000000ef57b3 in inline_mysql_mutex_lock ()
#4  0x0000000000f00002 in queue_event(Master_info*, char const*, unsigned long) ()
#5  0x0000000000f074dd in handle_slave_io ()
#6  0x0000000000f5e044 in pfs_spawn_thread ()
#7  0x00007f136f83c9d1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f136e5a59dd in clone () from /lib64/libc.so.6  
  
  sql thread stack:
Thread 6 (Thread 0x7f1362728700 (LWP 23028)):
#0  0x00007f136f843264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f136f83e508 in _L_lock_854 () from /lib64/libpthread.so.0
#2  0x00007f136f83e3d7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000000ef57b3 in inline_mysql_mutex_lock ()
#4  0x0000000000f0af9a in exec_relay_log_event(THD*, Relay_log_info*) ()
#5  0x0000000000f0ca41 in handle_slave_sql ()
#6  0x0000000000f5e044 in pfs_spawn_thread ()
#7  0x00007f136f83c9d1 in start_thread () from /lib64/libpthread.so.0
#8  0x00007f136e5a59dd in clone () from /lib64/libc.so.6  
  
  show  slave status stack:
Thread 16 (Thread 0x7f136fe14700 (LWP 5103)):
#0  0x00007f136f843264 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f136f83e53e in _L_lock_995 () from /lib64/libpthread.so.0
#2  0x00007f136f83e4a6 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000000000f08570 in show_slave_status_send_data(THD*, Master_info*, char*, char*) ()
#4  0x0000000000f09afe in show_slave_status(THD*) ()
#5  0x0000000000f09e33 in show_slave_status_cmd(THD*) ()
#6  0x0000000000d0dfff in mysql_execute_command(THD*, bool) ()
#7  0x0000000000d128c5 in mysql_parse(THD*, Parser_state*) ()
#8  0x0000000000d13ae8 in dispatch_command(THD*, COM_DATA const*, enum_server_command) ()
#9  0x0000000000d149d4 in do_command(THD*) ()
#10 0x0000000000de219c in handle_connection ()
#11 0x0000000000f5e044 in pfs_spawn_thread ()
#12 0x00007f136f83c9d1 in start_thread () from /lib64/libpthread.so.0
#13 0x00007f136e5a59dd in clone () from /lib64/libc.so.6

   woker slave is idle stack:
Thread 3 (Thread 0x7f1362665700 (LWP 23031)):
#0  0x00007f136f8405bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x0000000000f1935c in pop_jobs_item(Slave_worker*, slave_job_item*) ()
#2  0x0000000000f1bb43 in slave_worker_exec_job_group(Slave_worker*, Relay_log_info*) ()
#3  0x0000000000f0a273 in handle_slave_worker ()
#4  0x0000000000f5e044 in pfs_spawn_thread ()
#5  0x00007f136f83c9d1 in start_thread () from /lib64/libpthread.so.0
#6  0x00007f136e5a59dd in clone () from /lib64/libc.so.6   

This look like deadlock,The full stack and processlist picture in attachment.
Thanks!
  

How to repeat:
I don't kown!

Suggested fix:
no suggest

show processlist status

Attachment: showprocesslist.png (image/png, text), 47.92 KiB.

full pstack info

Attachment: 1210.log (text/plain), 26.17 KiB.

Thank you for the bug report. You have not provide a repeatable test case and using a quite older version 5.7.17, please try last released version 5.7.32. Thanks.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".