MySQL Bugs: #88511: innodb assertion failure in table rename

Bug #88511	innodb assertion failure in table rename
Submitted:	16 Nov 2017 8:04	Modified:	16 Nov 2017 8:11
Reporter:	Wei Zhao (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.7.17	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	Contribution

Description:
If you do 'optimize table' to a table T while there are massive DML (in our case delete) statements against T, mysqld can crash because of this assertion 
"ut_a(error == DB_SUCCESS);"
in function commit_cache_rebuild(), where the code snippet is as below:

    error = dict_table_rename_in_cache(
        ctx->old_table, ctx->tmp_name, FALSE);
    ut_a(error == DB_SUCCESS);

    DEBUG_SYNC_C("commit_cache_rebuild_middle");

    error = dict_table_rename_in_cache(
        ctx->new_table, old_name, FALSE);
    ut_a(error == DB_SUCCESS);

The dict_table_rename_in_cache() call must succeed according to the assertion, but it really can fail --- the fil_rename_tabelspace() it calls can fail (and in our case this is exactly why dict_table_rename_in_cache() failed) after retrying 25000 times.

How to repeat:
as above

Suggested fix:
always retrying in above case, as my patch does.

this patch fixes the bug (*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: idb_retry_25k.diff (application/octet-stream, text), 1.81 KiB.

Hello Wei Zhao,

Thank you for the bug report and contribution.

Thanks,
Umesh

Any progress? I met this problem in the latest version.

The patch here cannot fix the problem but make the system hang for a long time and core finally after 900s.

Turn off the change buffer will avoid the problem(system hang and core).
innodb_change_buffering=none

Refer to new bug for more details: https://bugs.mysql.com/bug.php?id=108087