Bug #63276 skip sleep in srv_master_thread when shutdown is in progress
Submitted: 16 Nov 2011 0:19 Modified: 30 Jul 2013 14:25
Reporter: Mark Callaghan Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: InnoDB Plugin storage engine Severity:S5 (Performance)
Version:5.1.52, 5.1.60 OS:Any
Assigned to: CPU Architecture:Any
Tags: innodb, shutdown, slow
Triage: Needs Triage: D3 (Medium)

[16 Nov 2011 0:19] Mark Callaghan
I traced the overhead for shutdown and one source of latency was a few seconds of stalls in srv_master_thread where nothing was done when the thread should have been flushing dirty pages. The problem is;
1) sleep is done in the one second loop and the "goto background_loop:" is done because srv_shutdown_state > 0
2) after jumping to srv_shutdown_state the "goto loop:' is executed because srv_activity_count != old_activity_count 

How to repeat:
instrument code in srv_master_thread on a busy OLTP server, shutdown mysqld

Suggested fix:
Skip sleep in the one second loop when srv_shutdown_state != 0

Move the "goto background_loop:" in the one second loop below the code that does some work in the loop (let it flush dirty pages)
[16 Nov 2011 5:08] Valeriy Kravchuk
Thank you for the problem report.
[16 Nov 2011 15:56] Mark Callaghan
2) after jumping to "background_loop:" the "goto loop:' is executed because
srv_activity_count != old_activity_count and os_thread_sleep is done, ... repeat
[30 Jul 2013 14:25] Bugs System
Added a changelog entry for 5.1.72, 5.5.34, 5.6.14, 5.7.2:

The "srv_master_thread" background thread, which monitors server activity
and performs activities such as page flushing when the server is inactive
or in a shutdown state, runs on a one second delay loop.
"srv_master_thread" would fail to check if the server is in a shutdown
state before sleeping.
[27 Sep 2013 7:32] Laurynas Biveinis
5.1$ bzr log -r 4030
revno: 4030
committer: Aditya A <aditya.a@oracle.com>
branch nick: mysql-5.1
timestamp: Mon 2013-07-29 14:45:06 +0530
   In the background thread srv_master_thread() we have a 
   a one second delay loop which will continuously monitor
   server activity .If the server is inactive (with out any
   user activity) or in a shutdown state we do some background
   activity like flushing the changes.In the current code
   we are not checking if server is in shutdown state before
   sleeping for one second.
  If server is in shutdown state ,then dont go to one second