Bug #73341 | cant't shutdown server with --innodb-force-recovery=6 | ||
---|---|---|---|
Submitted: | 21 Jul 2014 13:37 | Modified: | 22 Sep 2014 18:50 |
Reporter: | zhai weixiang (OCA) | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: InnoDB storage engine | Severity: | S3 (Non-critical) |
Version: | 5.7,5.6, 5.6.19, 5.6.21 | OS: | Any |
Assigned to: | CPU Architecture: | Any |
[21 Jul 2014 13:37]
zhai weixiang
[21 Jul 2014 14:09]
zhai weixiang
the shutdown thread hangs in function logs_empty_and_mark_files_at_shutdown. and quoted code: 2235 if (lsn != log_sys->last_checkpoint_lsn 2236 ) { 2237 2238 log_mutex_exit(); 2239 2240 goto loop; 2241 } here log_sys->lsn != log_sys->last_checkpoint_lsn. But since we set innodb-force-recovery=6, there isn't any background thread to make a log checkpoint.....
[21 Jul 2014 15:03]
zhai weixiang
And if innodb force recovery is larger than 3, then srv_read_only_mode is set to 1. log_make_checkpoint_at is not triggered by the shutdown thread. quoted code from logs_empty_and_mark_files_at_shutdown: 2223 if (!srv_read_only_mode) { 2224 log_make_checkpoint_at(LSN_MAX, TRUE); 2225 } 2226 2227 log_mutex_enter(); 2228 2229 lsn = log_sys->lsn; 2230 2231 if (lsn != log_sys->last_checkpoint_lsn 2232 ) { 2233 2234 log_mutex_exit(); 2235 2236 goto loop; 2237 }
[22 Jul 2014 1:54]
zhai weixiang
I also make a clean shutdown without setting innodb-force-recovery ,and then restart the server with innodb-force-recovery=6, still hang on shutdown.. :(
[22 Jul 2014 5:57]
zhai weixiang
a simple fix with a test case. make a fast shutdown when innodb_force_recovery=6
Attachment: bug73341.diff (application/octet-stream, text), 3.22 KiB.
[22 Jul 2014 6:54]
MySQL Verification Team
Hello Zhai, Thank you for the report. Verified as described with 5.6.19. Thanks, Umesh
[22 Jul 2014 7:00]
MySQL Verification Team
// 5.6.19 1 sigwait,signal_hand,pfs_spawn_thread,start_thread,clone 1 select,os_thread_sleep,logs_empty_and_mark_files_at_shutdown,innobase_shutdown_for_mysql,innobase_end,ha_finalize_handlerton,plugin_deinitialize,reap_plugins,plugin_shutdown,clean_up,unireg_end,kill_server,optimized,pfs_spawn_thread,start_thread,clone 1 pthread_cond_wait@@GLIBC_2.3.2,inline_mysql_cond_wait,argv=0x2ba8b78),__libc_start_main,_start 1 ??,os_aio_linux_collect,message1=0x7f829f20ce58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7f828cadde58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7f828c2dce58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7f828badbe58,,fil_aio_wait,io_handler_thread,start_thread,clone 1
[22 Jul 2014 7:03]
MySQL Verification Team
// 5.6.21 1 sigwait,signal_hand,pfs_spawn_thread,start_thread,clone 1 select,os_thread_sleep,logs_empty_and_mark_files_at_shutdown,innobase_shutdown_for_mysql,innobase_end,ha_finalize_handlerton,plugin_deinitialize,at,plugin_shutdown,clean_up,unireg_end,kill_server,kill_server_thread,pfs_spawn_thread,start_thread,clone 1 pthread_cond_wait@@GLIBC_2.3.2,inline_mysql_cond_wait,argv=0x29cad28),__libc_start_main,_start 1 ??,os_aio_linux_collect,message1=0x7fe77ddb6e58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7fe77d5b5e58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7fe77cdb4e58,,fil_aio_wait,io_handler_thread,start_thread,clone 1 ??,os_aio_linux_collect,message1=0x7fe77c5b3e58,,fil_aio_wait,io_handler_thread,start_thread,clone 1
[22 Sep 2014 18:50]
Daniel Price
Fixed as of the upcoming 5.6.22, 5.7.6 release, and here's the changelog entry: Attempting to shut down the server after starting the server with "innodb_force_recovery=6" would result in a hang. Thank you for the bug report.
[10 Dec 2014 13:08]
Laurynas Biveinis
$ bzr log -r 6169 ------------------------------------------------------------ revno: 6169 committer: bin.x.su@oracle.com branch nick: mysql-5.6 timestamp: Mon 2014-09-22 14:19:36 +0800 message: Bug#19265668 CANT'T SHUTDOWN SERVER WITH --INNODB-FORCE-RECOVERY=6 When starting the server with innodb_force_recovery=6 the server would hang if we try to shut it down. The root cause is that server is in read-only mode with innodb_force_recovery=6 and the log_sys->lsn is 12 bytes(LOG_BLOCK_HDR_SIZE) after log_sys->last_checkpoint_lsn and these 2 will never change. We have to identify this case. rb#6280, approved by Jimmy.