MySQL Bugs: #97823: InnoDB: Failing assertion: UT_LIST_GET_LEN(rseg->update_undo

Bug #97823	InnoDB: Failing assertion: UT_LIST_GET_LEN(rseg->update_undo_list) == 0
Submitted:	28 Nov 2019 7:05	Modified:	6 Dec 2019 3:10
Reporter:	Anthony Qiu	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	5.7.27	OS:	CentOS (CentOS release 6.10 (Final))
Assigned to:		CPU Architecture:	x86 (64-bit)
Tags:	failing assertion, trx0rseg.cc, UT_LIST_GET_LEN

Description:
11.01 is MySQL Server is hung: https://bugs.mysql.com/bug.php?id=97452

disable undo truncate

11.28 repeat hung:
2019-11-28 14:17:22 0x2ac5dc64d300  InnoDB: Assertion failure in thread 47029294519040 in file trx0rseg.cc line 134
InnoDB: Failing assertion: UT_LIST_GET_LEN(rseg->update_undo_list) == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
06:17:22 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

key_buffer_size=8388608
read_buffer_size=16777216
max_used_connections=3
max_threads=3000
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 147504340 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/data/svr/mysql57/bin/mysqld(my_print_stacktrace+0x35)[0xf54815]
/data/svr/mysql57/bin/mysqld(handle_fatal_signal+0x4a4)[0x7d21b4]
/lib64/libpthread.so.0(+0xf7e0)[0x2ac5dacb47e0]
/lib64/libc.so.6(gsignal+0x35)[0x2ac5dc0e64f5]
/lib64/libc.so.6(abort+0x175)[0x2ac5dc0e7cd5]
/data/svr/mysql57/bin/mysqld(_Z18ut_print_timestampP8_IO_FILE+0x0)[0x7c157e]
/data/svr/mysql57/bin/mysqld[0x113dd09]
/data/svr/mysql57/bin/mysqld(_Z13trx_sys_closev+0x1c0)[0x11438c0]
/data/svr/mysql57/bin/mysqld(_Z27innobase_shutdown_for_mysqlv+0x2ea)[0x11104fa]
/data/svr/mysql57/bin/mysqld[0xfefe40]
/data/svr/mysql57/bin/mysqld(_Z22ha_finalize_handlertonP13st_plugin_int+0x2c)[0x819abc]
/data/svr/mysql57/bin/mysqld[0xd39ecb]
/data/svr/mysql57/bin/mysqld[0xd3ea59]
/data/svr/mysql57/bin/mysqld(_Z15plugin_shutdownv+0x238)[0xd42bf8]
/data/svr/mysql57/bin/mysqld[0x7c7828]
/data/svr/mysql57/bin/mysqld(_Z11mysqld_mainiPPc+0x14f9)[0x7cc859]
/lib64/libc.so.6(__libc_start_main+0x100)[0x2ac5dc0d2d20]
/data/svr/mysql57/bin/mysqld[0x7c22e9]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
OS WAIT ARRAY INFO: reservation count 14177180
--Thread 47246519834368 has waited at trx0rseg.ic line 48 for 9  seconds the semaphore:
X-lock on RW-latch at 0x2aee4eaf2088 created in file buf0buf.cc line 1460
a writer (thread id 47230099764992) has reserved it in mode  exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file not yet reserved line 0
Last time write locked in file /export/home/pb2/build/sb_0-34537258-1560179931.8/mysql-5.7.27/storage/innobase/include/trx0rseg.ic line 48
--Thread 47249773008640 has waited at trx0trx.cc line 1185 for 8  seconds the semaphore:
Mutex at 0x395311a0, Mutex REDO_RSEG created trx0rseg.cc:211, lock var 1

--Thread 47249762092800 has waited at trx0undo.ic line 190 for 3  seconds the semaphore:
S-lock on RW-latch at 0x2aefd062e270 created in file buf0buf.cc line 1460
a writer (thread id 47246519834368) has reserved it in mode  exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 190
Last time write locked in file /export/home/pb2/build/sb_0-34537258-1560179931.8/mysql-5.7.27/storage/innobase/include/trx0undo.ic line 171
--Thread 47231014987520 has waited at buf0flu.cc line 1209 for 2  seconds the semaphore:
SX-lock on RW-latch at 0x2aee4eaf2088 created in file buf0buf.cc line 1460
a writer (thread id 47230099764992) has reserved it in mode  exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file not yet reserved line 0
Last time write locked in file /export/home/pb2/build/sb_0-34537258-1560179931.8/mysql-5.7.27/storage/innobase/include/trx0rseg.ic line 48
--Thread 47249919485696 has waited at trx0trx.cc line 1185 for 7  seconds the semaphore:
Mutex at 0x395311a0, Mutex REDO_RSEG created trx0rseg.cc:211, lock var 1

--Thread 47230099764992 has waited at trx0undo.ic line 171 for 9  seconds the semaphore:
X-lock on RW-latch at 0x2aefd062e270 created in file buf0buf.cc line 1460
a writer (thread id 47246519834368) has reserved it in mode  exclusive
number of readers 0, waiters flag 1, lock_word: 0
Last time read locked in file trx0undo.ic line 190
Last time write locked in file /export/home/pb2/build/sb_0-34537258-1560179931.8/mysql-5.7.27/storage/innobase/include/trx0undo.ic line 171
OS WAIT ARRAY INFO: signal count 18648997
RW-shared spins 0, rounds 28297595, OS waits 3154631
RW-excl spins 0, rounds 437787183, OS waits 3481380
RW-sx spins 13993428, rounds 272675408, OS waits 3351237
Spin rounds per wait: 28297595.00 RW-shared, 437787183.00 RW-excl, 19.49 RW-sx

Hello Mr. Qiu,

Thank you for your bug report.

In order to be able to verify this bug, we must be able to repeat it.  In order to repeat it, we need a test case, which will always lead to the above failing assertion.

Hence, we need a set of SQL statements leading to that assertion.

We are waiting on your response.

Thank you Mr. Qiu,

However, we do not have all data that we require.

First of all, we have only CREATE TABLE statements, without any contents. Second, we do not need all the tables, but only those that are involved in the falling assertion.

Last, but not least, we need exact set of DML statements to run to produce the assertion that you are reporting.

Without all these data, there is not much that we can do.

Also , this bug is duplicate of the bug #79743. However, the reporter was not able to provide a test case that would reproduce the assert, so we have to wait for your full and detailed scenario.

Hi Mr. Qiu,

No, I can't find that info there .....

What are some ways to capture the sql information you need?

Hi,

There are several ways of doing it, all of which are described throughout our Reference Manual. You can add many options to the innodb status, constantly query processlist, etc, etc, etc .....