MySQL Bugs: #63276: skip sleep in srv_master_thread when shutdown is in progress

Bug #63276	skip sleep in srv_master_thread when shutdown is in progress
Submitted:	16 Nov 2011 0:19	Modified:	30 Jul 2013 14:25
Reporter:	Mark Callaghan	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: InnoDB Plugin storage engine	Severity:	S5 (Performance)
Version:	5.1.52, 5.1.60	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	innodb, shutdown, slow

Description:
I traced the overhead for shutdown and one source of latency was a few seconds of stalls in srv_master_thread where nothing was done when the thread should have been flushing dirty pages. The problem is;
1) sleep is done in the one second loop and the "goto background_loop:" is done because srv_shutdown_state > 0
2) after jumping to srv_shutdown_state the "goto loop:' is executed because srv_activity_count != old_activity_count 

How to repeat:
instrument code in srv_master_thread on a busy OLTP server, shutdown mysqld

Suggested fix:
Skip sleep in the one second loop when srv_shutdown_state != 0

Move the "goto background_loop:" in the one second loop below the code that does some work in the loop (let it flush dirty pages)

Thank you for the problem report.

2) after jumping to "background_loop:" the "goto loop:' is executed because
srv_activity_count != old_activity_count and os_thread_sleep is done, ... repeat

Added a changelog entry for 5.1.72, 5.5.34, 5.6.14, 5.7.2:

The "srv_master_thread" background thread, which monitors server activity
and performs activities such as page flushing when the server is inactive
or in a shutdown state, runs on a one second delay loop.
"srv_master_thread" would fail to check if the server is in a shutdown
state before sleeping.

5.1$ bzr log -r 4030
------------------------------------------------------------
revno: 4030
committer: Aditya A <aditya.a@oracle.com>
branch nick: mysql-5.1
timestamp: Mon 2013-07-29 14:45:06 +0530
message:
  Bug#13417564 SKIP SLEEP IN SRV_MASTER_THREAD WHEN 
               SHUTDOWN IS IN PROGRESS 
  
  PROBLEM
  -------
   In the background thread srv_master_thread() we have a 
   a one second delay loop which will continuously monitor
   server activity .If the server is inactive (with out any
   user activity) or in a shutdown state we do some background
   activity like flushing the changes.In the current code
   we are not checking if server is in shutdown state before
   sleeping for one second.
  
  FIX
  ---
  If server is in shutdown state ,then dont go to one second
  sleep.