Bug #53270 Corruption in .frm file (backed with innobackup) will shutdown the server
Submitted: 28 Apr 2010 21:17 Modified: 12 Oct 2011 18:32
Reporter: Hema Sridharan Email Updates:
Status: Won't fix Impact on me:
None 
Category:MySQL Enterprise Backup Severity:S3 (Non-critical)
Version:3.5 OS:Any
Assigned to: CPU Architecture:Any

[28 Apr 2010 21:17] Hema Sridharan
Description:
* Create databases(db1, db2), tables, and load some data
* Execute backup using innobackup
* Drop the databases db1, db2
* Execute innobackup with --apply-logs
* Now check the backup directory created by innobackup and make some changes in the mysql/host.frm file (that is manually corrupt the file)
* Shutdown mysql server and then do innobackup with --copy-back (and it passes)
* Now restart the server and we will notice that server shuts down because of host.frm file.

Ideally corrupted file can result in restore failure(that is innobakcup --copy-back operation can fail) but can't lead to shutdown of server. This is a negative test and the expected result should be "restore failure".

How to repeat:
mysql> CREATE DATABASE db1;
Query OK, 1 row affected (0.00 sec)

mysql> CREATE DATABASE db2;
Query OK, 1 row affected (0.00 sec)

mysql> CREATE TABLE db1.t1(id INT)ENGINE=INNODB;
CREATE TABLE db2.t2(a CHAR(10))ENGINE=INNODB;
Query OK, 0 rows affected (0.12 sec)

mysql> CREATE TABLE db2.t2(a CHAR(10))ENGINE=INNODB;
Query OK, 0 rows affected (0.11 sec)

mysql>
mysql> INSERT INTO db1.t1 VALUES(1),(2),(3),(4);
Query OK, 4 rows affected (0.00 sec)
Records: 4  Duplicates: 0  Warnings: 0

mysql> INSERT INTO db2.t2 VALUES('a'),('b'),('c'),('d');
Query OK, 4 rows affected (0.00 sec)
Records: 4  Duplicates: 0  Warnings: 0

mysql>
mysql> SELECT * FROM db1.t1;
SELECT * FROM db2.t2;
+------+
| id   |
+------+
|    1 |
|    2 |
|    3 |
|    4 |
+------+
4 rows in set (0.00 sec)

mysql> SELECT * FROM db2.t2;
+------+
| a    |
+------+
| a    |
| b    |
| c    |
| d    |
+------+
4 rows in set (0.00 sec)

[hs221732@fimafeng10]/export/home2/tmp/mysql-5.1-ihb:> perl innobackup hema-inno.cnf mysql-test/backup/ --socket=/export/home2/tmp/mysql-5.1-ihb/mysql-test/var/tmp/mysqld.1.sock --user=root --ibbackup=/export/home2/tmp/mysql-5.1-ihb/install/ibbackup

mysql> drop database db1;
Query OK, 1 row affected (0.04 sec)

mysql> drop database db2;
Query OK, 1 row affected (0.07 sec)

Now manually corrupt the host.frm file in 2010-04-28_22-27-02/mysql/ directory

perl innobackup --apply-log hema-inno.cnf /export/home2/tmp/mysql-5.1-ihb/mysql-test/backup/2010-04-28_22-27-02/ --ibbackup=/export/home2/tmp/mysql-5.1-ihb/ibbackup

shutdown the server

./innobackup --copy-back hema-inno.cnf /export/home2/tmp/mysql-5.1-ihb/mysql-test/backup/2010-04-28_22-27-02/

Restart the server

Server log is:
100428 23:31:06 [Warning] '--default-character-set' is deprecated and will be removed in a future release. Please use '--character-set-server' instead.
100428 23:31:06 [Note] Plugin 'FEDERATED' is disabled.
100428 23:31:06 [Note] Plugin 'ndbcluster' is disabled.
InnoDB: The log file was created by ibbackup --apply-log at
InnoDB: ibbackup 100428 22:28:08
InnoDB: NOTE: the following crash recovery is part of a normal restore.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100428 23:31:06  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
100428 23:31:06  InnoDB: Started; log sequence number 0 53772
100428 23:31:06 [ERROR] /export/home2/tmp/mysql-5.1-ihb/sql/mysqld: Incorrect information in file: './mysql/host.frm'
100428 23:31:06 [ERROR] Fatal error: Can't open and lock privilege tables: Incorrect information in file: './mysql/host.frm'
100428 23:31:06 [Note] Got signal 15 to shutdown mysqld

As we can notice that server shuts down and gets signal 15 which is really a bad scenario.

Suggested fix:
Ideally the server should not shutdown and instead result in "restore failure" when innobackup --apply-log or innobackup --copy-back operation is performed.
[29 Apr 2010 6:03] Sveta Smirnova
Thank you for the report.

Verified as described using IHB 3.5.

Although this can be OK and considered as feature request for IHB, but not OK for future MEB release.
[29 Apr 2010 14:20] Sveta Smirnova
Shane,

problem is not server shutdown, but EB does not fail on corrupted table.
[12 Oct 2011 18:32] Sveta Smirnova
this bug would not be fixed