Bug #72079 Upgrade from 5.6.10 to 5.6.16 crashes and leaves unausable DB
Submitted: 19 Mar 2014 17:49 Modified: 5 May 2014 17:38
Reporter: Dario Kampkaspar Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Installing Severity:S1 (Critical)
Version:5.6.16 OS:Windows (Server 2008 R2)
Assigned to: Bin Su CPU Architecture:Any

[19 Mar 2014 17:49] Dario Kampkaspar
Description:
I today tried updating our MySQL-Server from 5.6.10 to 5.6.16. The updating process went along without any error messages but got stuck saying "Install success" in the details.

The error log shows the problems started when the new server version was starting up (Extract only, find full error log attached):

2014-03-19 18:01:16 2204 [Note] InnoDB: Renamed aux table 'katalog/FTS_0000000000003838_DELETED_CACHE' to 'katalog/FTS_0000000000000efe_DELETED_CACHE'.
2014-03-19 18:01:16 10f8  InnoDB: Assertion failure in thread 4344 in file fts0fts.cc line 6087
InnoDB: Failing assertion: fts_table.suffix != NULL
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
2014-03-19 18:01:44 2280 [Note] Plugin 'FEDERATED' is disabled.
2014-03-19 18:01:44 2280 [Note] InnoDB: Using atomics to ref count buffer pool pages
2014-03-19 18:01:44 2280 [Note] InnoDB: The InnoDB memory heap is disabled
2014-03-19 18:01:44 2280 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
2014-03-19 18:01:44 2280 [Note] InnoDB: Compressed tables use zlib 1.2.3
2014-03-19 18:01:44 2280 [Note] InnoDB: Not using CPU crc32 instructions
2014-03-19 18:01:44 2280 [Note] InnoDB: Initializing buffer pool, size = 107.0M
2014-03-19 18:01:44 2280 [Note] InnoDB: Completed initialization of buffer pool
2014-03-19 18:01:44 2280 [Note] InnoDB: Highest supported file format is Barracuda.
2014-03-19 18:01:45 f0c  InnoDB: Operating system error number 2 in a file operation.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
2014-03-19 18:01:45 2280 [ERROR] InnoDB: Could not find a valid tablespace file for 'katalog/FTS_0000000000003838_BEING_DELETED'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2014-03-19 18:01:45 2280 [ERROR] InnoDB: Tablespace open failed for '"katalog"."FTS_0000000000003838_BEING_DELETED"', ignored.

How to repeat:
I have not tried to repeat this.
[19 Mar 2014 17:51] Dario Kampkaspar
Server Error log since last start

Attachment: mysql.err (application/octet-stream, text), 34.70 KiB.

[21 Apr 2014 3:17] Bin Su
Yes, this is a bug. I can reproduce it.

The server before 5.6.11 would have 3 obsolete FTS aux common tables, which
are removed after 5.6.11. But the rename doesn't get rid of these 3 tables.
That's the cause of the bug.
[25 Apr 2014 13:39] Peter Lelie
Same problem here during upgrade from 5.6.10 to 5.6.18:

FROM E:\mySQL\errlog\P15NT01_ERR.log:
    2014-04-25 15:22:05  7588  Note  Plugin 'FEDERATED' is disabled.
          2014-04-25 15:22:05 22e8 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
    2014-04-25 15:22:05  7588  Note  InnoDB: Using atomics to ref count buffer pool pages
    2014-04-25 15:22:05  7588  Note  InnoDB: The InnoDB memory heap is disabled
    2014-04-25 15:22:05  7588  Note  InnoDB: Mutexes and rw_locks use Windows interlocked functions
    2014-04-25 15:22:05  7588  Note  InnoDB: Compressed tables use zlib 1.2.3
    2014-04-25 15:22:05  7588  Note  InnoDB: Not using CPU crc32 instructions
    2014-04-25 15:22:05  7588  Note  InnoDB: Initializing buffer pool, size = 8.0G
    2014-04-25 15:22:06  7588  Note  InnoDB: Completed initialization of buffer pool
    2014-04-25 15:22:07  7588  Note  InnoDB: Highest supported file format is Barracuda.
          2014-04-25 15:22:08 22e8  InnoDB: Assertion failure in thread 8936 in file fts0fts.cc line 6087
          InnoDB: Failing assertion: fts_table.suffix != NULL
          InnoDB: We intentionally generate a memory trap.
          InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
          InnoDB: If you get repeated assertion failures or crashes, even
          InnoDB: immediately after the mysqld startup, there may be
          InnoDB: corruption in the InnoDB tablespace. Please refer to
          InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
          InnoDB: about forcing recovery.
[5 May 2014 17:38] Daniel Price
Fixed as of 5.6.19, 5.7.5, and here's the changelog entry:

After upgrading from 5.6.10 to MySQL versions up to and including MySQL
5.6.18, "InnoDB" would attempt to rename obsolete full-text search
auxiliary tables on server startup, resulting in an assertion failure.

The following entry has been added to the 5.6.11 through 5.6.18 Release Notes:

"If you have "InnoDB" tables with full-text search indexes and you are upgrading from MySQL 5.6.10 to a MySQL version up to and including MySQL 5.6.18, the server will fail to start after the upgrade (Bug#72079). This bug is fixed in MySQL 5.6.19. As a workaround, remove full-text search indexes prior to upgrading and rebuild full-text search indexes after the upgrade is
completed."

Thank you for the bug report.
[2 Jun 2014 14:29] Laurynas Biveinis
$ bzr log -r 5907
------------------------------------------------------------
revno: 5907
committer: bin.x.su@oracle.com
branch nick: mysql-5.6
timestamp: Tue 2014-04-29 18:06:44 +0800
message:
  Bug #18634201 - Upgrade from 5.6.10 to 5.6.16 crashes and leaves unausable DB
  
  When user tries to upgrade from 5.6.10 to latest 5.6, the rename of aux
  tables could make server crash, since we don't get rid of those obsolete
  tables, which were removed away since 5.6.11.
  
  The patch fixed 2 issues:
  1) We shouldn't try to rename those obsolete tables.
  2) We should try to drop those obsolete tables when upgrade, so that those
     obsolete tables would not be left in the data directory.
  
  rb#5202, approved by Jimmy.