Bug #63276 skip sleep in srv_master_thread when shutdown is in progress
Submitted: 16 Nov 2011 0:19 Modified: 30 Jul 2013 14:25
Reporter: Mark Callaghan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB Plugin storage engine Severity:S5 (Performance)
Version:5.1.52, 5.1.60 OS:Any
Assigned to: CPU Architecture:Any
Tags: innodb, shutdown, slow
Triage: Needs Triage: D3 (Medium)

[16 Nov 2011 0:19] Mark Callaghan
Description:
I traced the overhead for shutdown and one source of latency was a few seconds of stalls in srv_master_thread where nothing was done when the thread should have been flushing dirty pages. The problem is;
1) sleep is done in the one second loop and the "goto background_loop:" is done because srv_shutdown_state > 0
2) after jumping to srv_shutdown_state the "goto loop:' is executed because srv_activity_count != old_activity_count 

How to repeat:
instrument code in srv_master_thread on a busy OLTP server, shutdown mysqld

Suggested fix:
Skip sleep in the one second loop when srv_shutdown_state != 0

Move the "goto background_loop:" in the one second loop below the code that does some work in the loop (let it flush dirty pages)
[16 Nov 2011 5:08] Valeriy Kravchuk
Thank you for the problem report.
[16 Nov 2011 15:56] Mark Callaghan
2) after jumping to "background_loop:" the "goto loop:' is executed because
srv_activity_count != old_activity_count and os_thread_sleep is done, ... repeat
[30 Jul 2013 14:25] Bugs System
Added a changelog entry for 5.1.72, 5.5.34, 5.6.14, 5.7.2:

The "srv_master_thread" background thread, which monitors server activity
and performs activities such as page flushing when the server is inactive
or in a shutdown state, runs on a one second delay loop.
"srv_master_thread" would fail to check if the server is in a shutdown
state before sleeping.
[27 Sep 2013 7:32] Laurynas Biveinis
5.1$ bzr log -r 4030
------------------------------------------------------------
revno: 4030
committer: Aditya A <aditya.a@oracle.com>
branch nick: mysql-5.1
timestamp: Mon 2013-07-29 14:45:06 +0530
message:
  Bug#13417564 SKIP SLEEP IN SRV_MASTER_THREAD WHEN 
               SHUTDOWN IS IN PROGRESS 
  
  PROBLEM
  -------
   In the background thread srv_master_thread() we have a 
   a one second delay loop which will continuously monitor
   server activity .If the server is inactive (with out any
   user activity) or in a shutdown state we do some background
   activity like flushing the changes.In the current code
   we are not checking if server is in shutdown state before
   sleeping for one second.
  
  FIX
  ---
  If server is in shutdown state ,then dont go to one second
  sleep.