Bug #67030 Mysql 5.6.5 does not to respond to SIGTERM during recovery (to allow aborting)
Submitted: 1 Oct 2012 7:05 Modified: 2 Oct 2012 6:33
Reporter: Simon Mudd (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S4 (Feature request)
Version:5.6.5 OS:Any (centos 5.x)
Assigned to: CPU Architecture:Any
Tags: windmill

[1 Oct 2012 7:05] Simon Mudd
Description:
I guess this is likely to be true of later versions, but noticed in MySQL 5.6.5.
I had a crash (probably a known issue) on a version of 5.6.5 that runs.
mysqld_safe restarts mysqld and this immediately tries to recover (innodb).

However, the server has a ~300GB database, and 2x16GB innodb redo log files so recovery can take ~2 hours even with the storage being on SSD.
It also appears to not be interruptable as I sent the process SIGTERM and it does not stop, and exit.

I would like to interrupt the process so I can update to a newer version of 5.6, but now I have to wait for the recovery to complete.
I don't want to do kill -9 <mysqld_pid> as the server is configured to enable core dumps and this would dump a 192GB core file which isn't ideal either.

So please add a feature so that SIGTERM is caught during startup Innodb recovery and this will allow the server to terminate "as cleanly as possible", even when in InnoDB recovery mode.

Logging shows:

InnoDB: Doing recovery: scanned up to log sequence number 8978398965248
InnoDB: Doing recovery: scanned up to log sequence number 8978400656369
InnoDB: Transaction 4184556500 was in the XA prepared state.
InnoDB: Transaction 4184556502 was in the XA prepared state.
InnoDB: 3 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 13 row operations to undo
InnoDB: Trx id counter is 4184556800
121001 8:07:52 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 13 14 15 16 17 

How to repeat:
If the server crashes, while doing recovery send it the SIGTERM signal and notice that it does not interrupt the recovery process.

Suggested fix:
Make the recovery thread(s) recognise the SIGTERM signal, and interrupt the recovery process, hopefully making the mysqld process terminate (cleanly).
[1 Oct 2012 8:12] Simon Mudd
Note: InnoDB does see the sigterm (or mysqld does), but only after completing the innodb recovery process, at which point in time it now has stuff (dirty pages) it needs to flush prior to doing a clean shutdown.  This final step also takes some time.