Bug #109751 Server can't start up after crash during ALTER TABLESPACE mysql ENCRYPT=ON/OFF
Submitted: 24 Jan 2023 7:01 Modified: 25 Jan 2023 7:44
Reporter: Zsolt Parragi (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: DDL Severity:S3 (Non-critical)
Version:8.0 OS:Any
Assigned to: CPU Architecture:Any

[24 Jan 2023 7:01] Zsolt Parragi
Description:
If the server crashes while an ALTER TABLESPACE mysql ENCRYPT=ON/OFF statement is in progress and another session is executing DDL statements, there's a good chance it won't be able to start up after that.

Depending on the exact codepath alter tablespace reached, or what happened in the other session, or if it's encryption or decryption, multiple startup crashes are possible either during recovery (doublewrite recovery can't find the encryption key for the DD tablespace) or shortly after recovery (checksum errors when opening tables)

How to repeat:
Add additional debug_sync points to storage/innobase/fsp/fsp0fsp.cc:

@@ -4248,6 +4248,8 @@ static dberr_t encrypt_begin_persist(fil_space_t *space) {
     ut_d(ut_error);
   }
 
+  DEBUG_SYNC(current_thd, "alter_encrypt_tablespace_wait_after_page0");
+
   /* Write operation type and progress (0 now) on page 0 */
   fsp_header_write_encryption_progress(
       space->id, space->flags, 0, Encryption::ENCRYPT_IN_PROGRESS, true, &mtr);
@@ -4440,6 +4442,9 @@ static dberr_t decrypt_begin_persist(fil_space_t *space) {
   }
 
   mtr_commit(&mtr);
+
+  DEBUG_SYNC(current_thd, "alter_decrypt_tablespace_wait_after_page0");
+
   return err;
 }

And execute the following MTR testcase (needs keyring loaded in master.opt):

--source include/have_debug_sync.inc

SET debug_sync = 'alter_encrypt_tablespace_wait_after_page0 WAIT_FOR s1';
--send ALTER TABLESPACE mysql ENCRYPTION='Y'

--connect(con1, localhost, root,,)
--connection con1
CREATE TABLE t1(id INT UNSIGNED NOT NULL) ENGINE=InnoDB, ENCRYPTION='Y';
INSERT INTO t1 VALUES (1);
INSERT INTO t1 VALUES (2);
--sleep 3

--source include/kill_mysqld.inc

--connection default
--error CR_SERVER_LOST
--reap

# Will fail
--source include/start_mysqld.inc
[24 Jan 2023 7:02] Zsolt Parragi
Complete diff with debug sync / mtr testcase to reproduce the issue

Attachment: bug.diff (text/x-patch), 1.76 KiB.

[25 Jan 2023 7:44] MySQL Verification Team
Hello Zsolt Parragi,

Thank you for the report.

regards,
Umesh