MySQL Bugs: #97312: Combination of gtid innodb-force-recovery can result in a "debug

Bug #97312	Combination of gtid innodb-force-recovery can result in a "debug_skip" assertion
Submitted:	21 Oct 2019 12:23	Modified:	18 Nov 2019 19:44
Reporter:	Hrvoje Matijakovic	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Replication	Severity:	S6 (Debug Builds)
Version:	8.0.18, 8.0.17	OS:	Linux
Assigned to:		CPU Architecture:	x86

Description:
Combination of gtid innodb-force-recoverand can result in "debug_skip" assertion.
I wasn't able to reproduce this on a release builds.

gdb from 8.0.18 debug:

Core was generated by `/sdc/MS151019-mysql-8.0.18-linux-x86_64-debug/bin/mysqld --no-defaults --basedi'.
Program terminated with signal 6, Aborted.
#0  0x00007f557607fa61 in __pthread_kill (threadid=<optimized out>, signo=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:59
59        val = INTERNAL_SYSCALL (tgkill, err, 3, THREAD_GETMEM (THREAD_SELF, pid),
(gdb) bt
+bt
#0  0x00007f557607fa61 in __pthread_kill (threadid=<optimized out>, signo=6) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:59
#1  0x00000000048f003d in my_write_core (sig=6) at /sdc/MS-8.0.18_dbg/mysys/stacktrace.cc:305
#2  0x00000000035e030e in handle_fatal_signal (sig=6) at /sdc/MS-8.0.18_dbg/sql/signal_handler.cc:169
#3  <signal handler called>
#4  0x00007f5573ec7337 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#5  0x00007f5573ec8a28 in __GI_abort () at abort.c:90
#6  0x0000000004d0e9eb in ut_dbg_assertion_failed (expr=0x68c4170 "debug_skip", file=0x68c3fd0 "/sdc/MS-8.0.18_dbg/storage/innobase/clone/clone0repl.cc", line=478) at /sdc/MS-8.0.18_dbg/storage/innobase/ut/ut0dbg.cc:98
#7  0x0000000004e1fe16 in Clone_persist_gtid::flush_gtids (this=0x7f555d087880, thd=0x7f5524410000) at /sdc/MS-8.0.18_dbg/storage/innobase/clone/clone0repl.cc:478
#8  0x0000000004e20038 in Clone_persist_gtid::periodic_write (this=0x7f555d087880) at /sdc/MS-8.0.18_dbg/storage/innobase/clone/clone0repl.cc:526
#9  0x0000000004e20446 in clone_gtid_thread (persist_gtid=0x7f555d087880) at /sdc/MS-8.0.18_dbg/storage/innobase/clone/clone0repl.cc:596
#10 0x0000000004e23ab6 in std::_Bind<void (*(Clone_persist_gtid*))(Clone_persist_gtid*)>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (this=0x7f55243febc0, __args=<unknown type in /mnt/sdc/MS151019-mysql-8.0.18-linux-x86_64-debug/bin/mysqld, CU 0xe9b82d8, DIE 0xea49d19>) at /opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/functional:1074
#11 0x0000000004e23964 in std::_Bind<void (*(Clone_persist_gtid*))(Clone_persist_gtid*)>::operator()<, void>() (this=0x7f55243febc0) at /opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/functional:1133
#12 0x0000000004e23831 in Runnable::operator()<void (*)(Clone_persist_gtid*), Clone_persist_gtid*>(void (*&&)(Clone_persist_gtid*), Clone_persist_gtid*&&) (this=0x7f555fe460b8, f=<unknown type in /mnt/sdc/MS151019-mysql-8.0.18-linux-x86_64-debug/bin/mysqld, CU 0xe9b82d8, DIE 0xea4978e>) at /sdc/MS-8.0.18_dbg/storage/innobase/include/os0thread-create.h:101
#13 0x0000000004e2375d in std::_Bind_simple<Runnable (void (*)(Clone_persist_gtid*), Clone_persist_gtid*)>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=0x7f555fe460a8) at /opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/functional:1531
#14 0x0000000004e23627 in std::_Bind_simple<Runnable (void (*)(Clone_persist_gtid*), Clone_persist_gtid*)>::operator()() (this=0x7f555fe460a8) at /opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/functional:1520
#15 0x0000000004e235c6 in std::thread::_Impl<std::_Bind_simple<Runnable (void (*)(Clone_persist_gtid*), Clone_persist_gtid*)> >::_M_run() (this=0x7f555fe46090) at /opt/rh/devtoolset-4/root/usr/include/c++/5.3.1/thread:115
#16 0x00000000053e0b40 in execute_native_thread_routine ()
#17 0x00007f557607ae65 in start_thread (arg=0x7f55243ff700) at pthread_create.c:307
#18 0x00007f5573f8f88d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) quit

How to repeat:
# mysqld options required for replay:  --gtid_mode=ON --enforce_gtid_consistency=ON --innodb-force-recovery=1
CREATE DATABASE test;

Hello Hrvoje Matijakovic,

Thank you for the report and test case.
Observed that 8.0.18 debug build is affected.

regards,
Umesh

Posted by developer:
 
Hi.

In the above described case, InnoDB makes, unilaterally, all tables read-only. 
Therefore, it is expected for the write operation on `mysql.gtid_executed` to fail.
In the light on this, may be the assertion should be replaced.

Anyway, chaning the category.

Thank you for your time.

Bug #96524 marked as duplicate of this one

Posted by developer:
 
Fixed as of the upcoming 8.0.19 release, and here's the changelog entry:

A GTID table update while innodb_force_recovery was enabled caused a
debug assertion failure.