Bug #96637 Clone fails on just upgraded server from 5.7
Submitted: 23 Aug 2019 12:30 Modified: 9 Dec 2019 19:39
Reporter: Satya Bodapati (OCA) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:8.0.17 OS:Any
Assigned to: CPU Architecture:Any

[23 Aug 2019 12:30] Satya Bodapati
Description:
1. create 5.7 datadir with --innodb-undo-tablespaces=10
2. start 8.0.17 on the 5.7 datadir
3. INSTALL CLONE PLUGIN

mysql> CLONE LOCAL DATA DIRECTORY='/tmp/backup2';
ERROR 1016 (HY000): Can't open file: './undo008' (errno: 2 - No such file or directory)

Note that this doesn't happen after reboot

How to repeat:
see above

Suggested fix:
The root cause is because upgrade leaves 'orphan/zombie' in-memory tablespaces for the 5.7 undo tablespaces.

My fix:

cat undo_crash.diff 
diff --git a/storage/innobase/srv/srv0start.cc b/storage/innobase/srv/srv0start.cc
index b00476be959..c9325061e9a 100644
--- a/storage/innobase/srv/srv0start.cc
+++ b/storage/innobase/srv/srv0start.cc
@@ -1396,8 +1396,12 @@ dberr_t srv_undo_tablespaces_upgrade() {
 
     fil_space_close(undo_space.id());
 
-    os_file_delete_if_exists(innodb_data_file_key, undo_space.file_name(),
-                             NULL);
+    dberr_t err =
+        fil_delete_tablespace(undo_space.id(), BUF_REMOVE_ALL_NO_WRITE);
+    if (err != DB_SUCCESS) {
+      ib::info() << "Delete of 5.7 undo tablespace: " << undo_space.file_name()
+                 << " failed";
+    }
   }
 
   /* Remove the tracking of these undo tablespaces from TRX_SYS page and
[23 Aug 2019 12:41] Satya Bodapati
orphan 5.7 undo tablespace(in-mem)

(*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: orphan_57_undo.diff (text/x-patch), 819 bytes.

[23 Aug 2019 12:43] Satya Bodapati
if ib::info() without error code is not allowed, please add that ;)
And change to warn() based on your preference
[28 Aug 2019 12:05] Sinisa Milivojevic
Hello Satya, my dear friend,

How are you ????

I was analysing and testing this bug report and I do not see what method have you used to upgrade from 5.7 to 8.0. Please, describe that.

However, I like both your analysis and your patch. 

Hence, I am verifying your bug as reported.

Please, provide your answer regardless of this bug being verified. It will be useful to our developers. You should know that ......

Have a nice life !!!!
[9 Dec 2019 19:39] Daniel Price
Posted by developer:
 
Fixed as of the upcoming 8.0.20  release, and here's the changelog entry:

After upgrading from MySQL 5.7 to MySQL 8.0, a CLONE INSTANCE operation
failed with the following error: ERROR 3862 (HY000): Clone Donor Error:
1016 : Can't open file: './undo001'. The upgrade process left behind
orphaned in-memory undo tablespaces. 

Thanks to Satya Bodapati for the contribution.
[10 Dec 2019 13:30] Sinisa Milivojevic
Thank you, Daniel.
[18 Dec 2019 15:09] Daniel Price
Posted by developer:
 
Changelog entry moved to 8.0.19 release notes.
[18 Dec 2019 15:16] Sinisa Milivojevic
Thank you, Daniel.