Bug #109310 Fail to upgrade MySQL5.7 cluster to MySQL 8
Submitted: 8 Dec 2022 1:52 Modified: 9 Dec 2022 3:12
Reporter: Liang Cheng Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Cluster: Replication Severity:S2 (Serious)
Version:5.7.30 to 8.0.31 OS:Linux
Assigned to: MySQL Verification Team CPU Architecture:Any
Tags: upgrade fails

[8 Dec 2022 1:52] Liang Cheng
Description:
Hi,
We have one mysql5.7 innodb cluster (single-primary) environment on EKS. Today, we pick up the mysql8 image, and let mysql upgrade automatically.
The enviornment has one primary and two slaves. The result is that primary and one slave can be upgraded to mysql8 successfully. HOwever, one slave is failed with following error. What does this mean and could be the cause
2022-12-07T11:03:57.231962Z 4 [ERROR] [MY-012119] [InnoDB] Cannot rename './mysql/#sql-ib3584-1767583560.ibd' to './mysql/slave_relay_log_info.ibd' for space ID 2584 because the target file exists. Remove the target file and try again.
2022-12-07T11:03:57.231993Z 4 [ERROR] [MY-013183] [InnoDB] Assertion failure: handler0alter.cc:7009:error == DB_SUCCESS thread 140216232716032
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
2022-12-07T11:03:57Z UTC - mysqld got signal 6 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
BuildID[sha1]=cc730f8d2780f63e4aa3ca1a37aff8fd980c5bed
Thread pointer: 0x560abedc63e0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f86a2be1bd0 thread_stack 0x100000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x560ab93dc2fd]
/usr/sbin/mysqld(print_fatal_signal(int)+0x393) [0x560ab82be4a3]
/usr/sbin/mysqld(my_server_abort()+0x7e) [0x560ab82be5ee]
/usr/sbin/mysqld(my_abort()+0xa) [0x560ab93d64fa]
/usr/sbin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x336) [0x560ab96c1746]
/usr/sbin/mysqld(bool ha_innobase::commit_inplace_alter_table_impl<dd::Table>(TABLE*, Alter_inplace_info*, bool, dd::Table*)+0x23d4) [0x560ab950dc94]
/usr/sbin/mysqld(ha_innobase::commit_inplace_alter_table(TABLE*, Alter_inplace_info*, bool, dd::Table const*, dd::Table*)+0x9e) [0x560ab950ea6e]
/usr/sbin/mysqld(+0x105f80d) [0x560ab81f580d]
/usr/sbin/mysqld(mysql_alter_table(THD*, char const*, char const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*)+0x57b5) [0x560ab820c845]
/usr/sbin/mysqld(Sql_cmd_alter_table::execute(THD*)+0x4db) [0x560ab861eb6b]
/usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0xafe) [0x560ab81583ae]
/usr/sbin/mysqld(+0xfed587) [0x560ab8183587]
/usr/sbin/mysqld(Prepared_statement::execute_server_runnable(THD*, Server_runnable*)+0x212) [0x560ab8184f22]
/usr/sbin/mysqld(Ed_connection::execute_direct(Server_runnable*)+0xa3) [0x560ab8186623]
/usr/sbin/mysqld(Ed_connection::execute_direct(MYSQL_LEX_STRING)+0x3b) [0x560ab81867fb]
/usr/sbin/mysqld(+0x21f2aaf) [0x560ab9388aaf]
/usr/sbin/mysqld(dd::upgrade::upgrade_system_schemas(THD*)+0xde8) [0x560ab938d6d8]
/usr/sbin/mysqld(+0x11e8736) [0x560ab837e736]
/usr/sbin/mysqld(+0x29d29b5) [0x560ab9b689b5]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7) [0x7f86afa36ea7]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f86af625a2f]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f86002cb228): # Columns added to keep information about REQUIRE_TABLE_PRIMARY_KEY_CHECK replication field ALTER TABLE slave_relay_log_info ADD Require_table_primary_key_check ENUM('STREAM','ON','OFF') NOT NULL DEFAULT 'STREAM' COMMENT 'Indicates what is the channel policy regarding tables having primary keys on create and alter table queries' AFTER Require_row_format
Connection ID (thread ID): 4

How to repeat:
Not sure if it could be repeated
[8 Dec 2022 18:11] MySQL Verification Team
Hi,

I tried reproducing it without a problem. I upgraded from 5.7.30 to 8.0.31 without a problem, but in any way you should not be upgrading from 5.7.30 to 8.0.31. You must first upgrade to latest minor version, so first you have to upgrade 5.7.30 to 5.7.40 and only then upgrade 5.7.40 to 8.0.31. That is the safe way to upgrade.

Please upgrade to 5.7.40 first before you upgrade to 8.0.31
Thanks
[9 Dec 2022 3:12] Liang Cheng
Thanks for your suggestion. But the primary db and another slave with the same version could be upgraded to MysQL8 without problem.
I checked the environment, we have /mysql/slave_relay_log_info.ibd before upgrade.
So why did this error occured if there is always have one slave_relay_log_info.ibd?  

Cannot rename './mysql/#sql-ib3584-1767583560.ibd' to './mysql/slave_relay_log_info.ibd' for space ID 2584 because the target file exists. Remove the target file and try again