Bug #95128 wrong returning value and assertion in Fil_shard::open_file which cause crash
Submitted: 25 Apr 2019 9:22 Modified: 1 Aug 2019 14:56
Reporter: ZHAO SONG Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S6 (Debug Builds)
Version:8.0.13, 8.0.15 OS:CentOS (7)
Assigned to: CPU Architecture:Any

[25 Apr 2019 9:22] ZHAO SONG
Description:
storage/innobase/fil/fil0fil.cc:2460L
----------------------------------------------------------------------------------
bool Fil_shard::open_file(fil_node_t *file, bool extend) {
  ... ...
  if (space->purpose == FIL_TYPE_LOG) {
    file->handle =
        os_file_create(innodb_log_file_key, file->name, OS_FILE_OPEN,
                       OS_FILE_AIO, OS_LOG_FILE, read_only_mode, &success);
  } else if (file->is_raw_disk) {
    file->handle =
        os_file_create(innodb_data_file_key, file->name, OS_FILE_OPEN_RAW,
                       OS_FILE_AIO, OS_DATA_FILE, read_only_mode, &success);
  } else {
    file->handle =
        os_file_create(innodb_data_file_key, file->name, OS_FILE_OPEN,
                       OS_FILE_AIO, OS_DATA_FILE, read_only_mode, &success);
  }

  > ut_a(success);

  /* The file is ready for IO. */

  file_opened(file);

  > return (true);
}
----------------------------------------------------------------------------------

At the end of Fil_shard::open_file, after calling os_file_create, there is an assertion "ut_a(success)" and "return (true)" to its caller. But os_file_create may fail (because of OS_FILE_DISK_FULL ...etc), returns not defined file handler and set 'success' to be false, in that case, Fil_shard::open_file will hit that assertion and return misleading value('true') to its caller.

How to repeat:
mysql> create database test;
Query OK, 1 row affected (0.03 sec)

mysql> create table test.t1 (a int);
Query OK, 0 rows affected (0.12 sec)

mysql> create table test.t2 (a int);
Query OK, 0 rows affected (0.10 sec)

RESTART SERVER

mysql> set global debug='+d,ib_create_table_fail_disk_full';
Query OK, 0 rows affected (0.00 sec)

mysql> select * from test.t1;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Suggested fix:
Remove that assertion, after calling os_file_create, check its output parameter(success) and do the continue things based on that.
[25 Apr 2019 9:42] MySQL Verification Team
Hello ZHAO SONG,

Thank you for the report and test case.
Verified as described with 8.0.15 debug build.

regards,
Umesh
[25 Apr 2019 9:50] ZHAO SONG
Change Category
[1 Aug 2019 14:56] Daniel Price
Posted by developer:
 
Fixed as of the upcoming 8.0.18 release, and here's the changelog entry:

 A test case that attempts to open a file in read-only mode while the
server is in a disk-full state caused a debug assertion failure. The
assertion was removed to permit the server to retry opening the file, and
to report an error if unsuccessful after a number of attempts.