MySQL Bugs: #101968: MySQL Shell 8.0.22 crashes every time when loading a dump into MySQL 5.7.27

Bug #101968	MySQL Shell 8.0.22 crashes every time when loading a dump into MySQL 5.7.27
Submitted:	11 Dec 2020 1:09	Modified:	17 Dec 2020 22:32
Reporter:	Yoseph Phillips	Email Updates:
Status:	Verified	Impact on me:	None
Category:	Shell General / Core Client	Severity:	S1 (Critical)
Version:	8.0.22	OS:	Windows
Assigned to:		CPU Architecture:	Any

Description:
MySQL Shell 8.0.22 crashes every time when loading a dump into MySQL 5.7.27.

For this particular db when we follow the how to repeat steps MySQL crashes every time we try this.

The Windows Application log shows:
Faulting application name: mysqlsh.exe, version: 8.0.22.0, time stamp: 0x5f76ed14
Faulting module name: mysqlsh.exe, version: 8.0.22.0, time stamp: 0x5f76ed14
Exception code: 0xc0000005
Fault offset: 0x0000000000fe3bb5
etc.

and 
Problem signature:
P1: mysqlsh.exe
P2: 8.0.22.0
P3: 5f76ed14
P4: mysqlsh.exe
P5: 8.0.22.0
P6: 5f76ed14
P7: c0000005
P8: 0000000000fe3bb5
P9: 
P10: 

Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER1E0.tmp.mdmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER23F.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER25F.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER26D.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER27E.tmp.txt
etc.

The dump directory for this db is about 120 MB.
We have managed to do this successfully for a smaller 30 MB dump directory.

We will try to store as much info about this as possible in case Oracle need further information regarding this.

When we tried using 8.0.21 it had issues with running out of file handles, which seems to have been fixed in 8.0.22 - so we are not worried about that issue, but maybe that is relevant information to have.

How to repeat:
Connect to MySQL 8.0.22.
Use:
util.dumpSchemas(["dbName"], "d:/temp/dumpDirectory", {ocimds: true, compatibility: ["strip_definers"]})

Dump completed successfully.

Connect to MySQL 5.7.27.
Use:
util.loadDump("d:/temp/dumpDirectory", {ignoreVersion: true})

MySQL Shell Crashed.

It appears that each time it completes the executing DDL phase, then it seems as soon as it starts the loading using 4 threads phase the it crashes. It appears from the load-progress....json file that it manages to do something before crashing, but then always crashes at exactly the same point.

Hello Yoseph,

Thank you for the report and feedback.
I tried to follow your steps but with sample schema and not seeing any issues at my end. Any chance you can provide the dump(d:/temp/dumpDirectory) to reproduce the issue at our end? If the data you need to attach is more than 3MB, you should create a compressed archive of the data and a README file that describes the data with a filename that includes the bug number (recommended filename: mysql-bug-data-101968.zip) and upload one to sftp.oracle.com. More details are in the "Files" section of this report. Thank you.

regards,
Umesh

Result file containing details of attempt made to reproduce

Attachment: 101968.results.txt (text/plain), 16.03 KiB.

Dump Directory

Attachment: dump.zip (application/x-zip-compressed, text), 7.93 KiB.

We have uploaded the dump directory.
We managed to make a very small test dump directory which reproduces the issue both on 8.0.22 and 5.7.27, so to keep things simple for testing only 8.0.22 needs to be used.

It can also be reproduced using:
util.dumpTables("db1Name", [], "d:/temp/dumpDirectory", {all: true})
util.loadDump("d:/temp/dumpDirectory", {ignoreVersion: true, schema: "db2Name"})

Thank you for the requested data and feedback.
With the provided dump, observed that mysqlsh crashes while loading the dump on 8.0.22 instance.

-
 bin/mysqlsh --log-level=debug3
MySQL Shell 8.0.22

Copyright (c) 2016, 2020, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Other names may be trademarks of their respective owners.

Type '\help' or '\?' for help; '\quit' to exit.
 MySQL  JS > \c root@localhost:3333
Creating a session to 'root@localhost:3333'
Please provide the password for 'root@localhost:3333':
Fetching schema names for autocompletion... Press ^C to stop.
Your MySQL connection id is 9
Server version: 8.0.22 MySQL Community Server - GPL
No default schema selected; type \use <schema> to set one.
 MySQL  localhost:3333 ssl  JS >
 MySQL  localhost:3333 ssl  JS > util.loadDump("/tmp/dump/", {ignoreVersion: true, schema: "bug"})
ERROR: The 'local_infile' global system variable must be set to ON in the target server, after the server is verified to be trusted.
Util.loadDump: local_infile disabled in server (RuntimeError)
 MySQL  localhost:3333 ssl  JS >
 MySQL  localhost:3333 ssl  JS > util.loadDump("/tmp/dump/", {ignoreVersion: true, schema: "bug"})
Loading DDL and Data from '/tmp/dump/' using 4 threads.
Opening dump...
Target is MySQL 8.0.22. Dump was produced from MySQL 8.0.22
Checking for pre-existing objects...
Executing common preamble SQL
[Worker002] Executing DDL script for `bug`.`test_table`
1 thds loading | 0% (0 bytes / 98.68 MB), 0.00 B/s, 0 / 1 tables doneSegmentation fault (core dumped)

-

(gdb) bt
#0  0x0000000000e90c74 in mysqlsh::import_table::Transaction_buffer::find_last_row_boundary_before(unsigned long) ()
#1  0x0000000000e90dae in mysqlsh::import_table::Transaction_buffer::read(char*, unsigned int) ()
#2  0x0000000000e91080 in mysqlsh::import_table::local_infile_read(void*, char*, unsigned int) ()
#3  0x0000000001147409 in handle_local_infile(MYSQL*, char const*) ()
#4  0x0000000001152613 in ?? ()
#5  0x0000000001154906 in mysql_real_query ()
#6  0x0000000000f45f59 in mysqlshdk::db::mysql::Session_impl::run_sql(char const*, unsigned long, bool) ()
#7  0x0000000000f4656d in mysqlshdk::db::mysql::Session_impl::query(char const*, unsigned long, bool) ()
#8  0x0000000000e928a1 in mysqlsh::import_table::Load_data_worker::execute(std::shared_ptr<mysqlshdk::db::mysql::Session> const&, std::unique_ptr<mysqlshdk::storage::IFile, std::default_delete<mysqlshdk::storage::IFile> >, unsigned long, std::vector<unsigned long, std::allocator<unsigned long> > const*) ()
#9  0x0000000000e73350 in mysqlsh::Dump_loader::Worker::Load_chunk_task::load(std::shared_ptr<mysqlshdk::db::mysql::Session> const&, mysqlsh::Dump_loader*) ()
#10 0x0000000000e767ea in mysqlsh::Dump_loader::Worker::Load_chunk_task::execute(std::shared_ptr<mysqlshdk::db::mysql::Session> const&, mysqlsh::Dump_loader::Worker*, mysqlsh::Dump_loader*) ()
#11 0x0000000000e77239 in mysqlsh::Dump_loader::Worker::run() ()
#12 0x0000000000e77420 in ?? ()
#13 0x0000000001ab91ef in ?? ()
#14 0x00007fb52eebfea5 in start_thread () from /lib64/libpthread.so.0
#15 0x00007fb52cff08dd in clone () from /lib64/libc.so.6
(gdb)

Can you try again after deleting all *.idx files in the dump?

We are not sure if you were asking us or the MySQL Verification Team to do that, but that causes a different error:
ERROR: [Worker003] While loading \\?...@@0.tsv.zst: Failed to get the file size of '\\?...@test_table@@0.tsv.zst.idx': No such file or directory

We are not sure what the point of this test was. Oracle should be able to do the same test as we have provided the dump directory.