Bug #104708 [FATAL] InnoDB: Aborting because of a corrupt database page in the system tables
Submitted: 24 Aug 2021 11:23 Modified: 25 Aug 2021 12:15
Reporter: 军 董 Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:5.7.21 OS:Any
Assigned to: CPU Architecture:Any

[24 Aug 2021 11:23] 军 董
Description:
Database page corruption on disk or a failed file read of page

This is the message written into the log:

2021-08-24T07:12:00.530673+08:00 3765785 [Note] InnoDB: Uncompressed page, stored checksum in field1 102005234, calculated checksums for field1: crc32 3107068166/3831191631, innodb 2592556411, none 3735928559, stored checksum in field2 696080796, calculated checksums for field2: crc32 3107068166/3831191631, innodb 1628036572, none 3735928559,  page LSN 4083815198 3583232164, low 4 bytes of LSN at page end 200968131, page number (if stored to page already) 237139426, space id (if created with >= MySQL-4.1.1 and stored already) 720146225
2021-08-24T07:12:00.530682+08:00 3765785 [Note] InnoDB: It is also possible that your operating system has corrupted its own file cache and rebooting your computer removes the error. If the corrupt page is an index page. You can also try to fix the corruption by dumping, dropping, and reimporting the corrupt table. You can use CHECK TABLE to scan your table for corruption. Please refer to http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html for information about forcing recovery.
2021-08-24T07:12:00.530713+08:00 3765785 [ERROR] [FATAL] InnoDB: Aborting because of a corrupt database page in the system tablespace. Or,  there was a failure in tagging the tablespace  as corrupt.
2021-08-24 07:12:00 0x7f3b62397700  InnoDB: Assertion failure in thread 139893027731200 in file ut0ut.cc line 942
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
23:12:00 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=124
max_threads=2020
thread_count=43
connection_count=42
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1077609 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x1e22a000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
2021-08-24T07:12:00.538308+08:00 3771139 [ERROR] Got error 180 when reading table './kmgxxx/dz_task_xxx'
stack_bottom = 7f3b62396e00 thread_stack 0x40000
/export/servers/mysql/bin/mysqld(my_print_stacktrace+0x2c)[0xeb703c]
/export/servers/mysql/bin/mysqld(handle_fatal_signal+0x451)[0x7a4e91]
/lib64/libpthread.so.0(+0xf370)[0x7f3cb7f3e370]
/lib64/libc.so.6(gsignal+0x37)[0x7f3cb6d3f1d7]
/lib64/libc.so.6(abort+0x148)[0x7f3cb6d408c8]
/export/servers/mysql/bin/mysqld[0x77577d]
/export/servers/mysql/bin/mysqld(_ZN2ib5fatalD1Ev+0x4d)[0x107f93d]
/export/servers/mysql/bin/mysqld(_Z20buf_page_io_completeP10buf_page_tb+0x3c7)[0x10c09e7]
/export/servers/mysql/bin/mysqld(_Z13buf_read_pageRK9page_id_tRK11page_size_t+0x44a)[0x10ebd1a]
/export/servers/mysql/bin/mysqld(_Z16buf_page_get_genRK9page_id_tRK11page_size_tmP11buf_block_tmPKcmP5mtr_tb+0x3d4)[0x10bdbf4]
/export/servers/mysql/bin/mysqld(_Z27btr_cur_search_to_nth_levelP12dict_index_tmPK8dtuple_t15page_cur_mode_tmP9btr_cur_tmPKcmP5mtr_t+0x61d)[0x109dc9d]
/export/servers/mysql/bin/mysqld[0x10a469e]
/export/servers/mysql/bin/mysqld(_ZN11ha_innobase16records_in_rangeEjP12st_key_rangeS1_+0x492)[0xefa992]
/export/servers/mysql/bin/mysqld(_ZN7handler27multi_range_read_info_constEjP15st_range_seq_ifPvjPjS3_P13Cost_estimate+0xc6)[0x7ea9a6]
/export/servers/mysql/bin/mysqld(_ZN10DsMrr_impl16dsmrr_info_constEjP15st_range_seq_ifPvjPjS3_P13Cost_estimate+0x48)[0x7f5528]
/export/servers/mysql/bin/mysqld[0xdc6f0b]
/export/servers/mysql/bin/mysqld(_Z17test_quick_selectP3THD6BitmapILj64EEyybN8st_order10enum_orderEPK16QEP_shared_ownerP4ItemPS2_PP14QUICK_SELECT_I+0xe05)[0xdd9255]
/export/servers/mysql/bin/mysqld(_ZN4JOIN17estimate_rowcountEv+0x325)[0xc81515]
/export/servers/mysql/bin/mysqld(_ZN4JOIN14make_join_planEv+0x3ef)[0xc877af]
/export/servers/mysql/bin/mysqld(_ZN4JOIN8optimizeEv+0x54c)[0xc8909c]
/export/servers/mysql/bin/mysqld(_ZN13st_select_lex8optimizeEP3THD+0x6a5)[0xccd125]
/export/servers/mysql/bin/mysqld(_Z12handle_queryP3THDP3LEXP12Query_resultyy+0x155)[0xccd305]
/export/servers/mysql/bin/mysqld[0x76873c]
/export/servers/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x417a)[0xc92dba]
/export/servers/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x3bd)[0xc9553d]
/export/servers/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0xa70)[0xc96080]
/export/servers/mysql/bin/mysqld(_Z10do_commandP3THD+0x177)[0xc979d7]
/export/servers/mysql/bin/mysqld(handle_connection+0x270)[0xd52950]
/export/servers/mysql/bin/mysqld(pfs_spawn_thread+0x1b1)[0x121a081]
/lib64/libpthread.so.0(+0x7dc5)[0x7f3cb7f36dc5]
/lib64/libc.so.6(clone+0x6d)[0x7f3cb6e0176d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (1e616c30): SELECT COUNT( 1 ) FROM dz_task_record     WHERE (activity_id = 'bigChildf21ddd9c' AND actor_id = '1405300205105872898' AND task_type = 102 AND day_time = '20210824')
Connection ID (thread ID): 3765785
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2021-08-23T23:12:01.319219Z mysqld_safe Number of processes running now: 0
2021-08-23T23:12:01.332245Z mysqld_safe mysqld restarted

How to repeat:
we are not able to find the conditions to reproduce it.
[25 Aug 2021 12:15] MySQL Verification Team
Hi Mr. 军 董,

Thank you for your bug report.

However, it is not our bug.

Your stacktrace clearly denotes that data on your hard disk have got corrupted, for whatever the reason. It could be a lasting or passing error in some sector, in RAM or it could be loss of power or missing log files due to some crash or similar. The last set of circumstances is possible if you have not set your system to be fully ACID compliant, as described in our Reference Manual.

Not a bug.
[31 Aug 2021 9:24] Christos Chatzaras
I think I had same or similar issue with 5.7.35:

https://forums.mysql.com/read.php?22,698557,698558#msg-698558
[31 Aug 2021 12:24] MySQL Verification Team
Hi,

The error that you get is very rare and a clear indication of the malfunctioning hardware. Regardless of which software and which version do you use ......
[31 Aug 2021 15:03] Christos Chatzaras
I use:

innodb_flush_log_at_trx_commit = 2

Could this be related? But there was no power outage or other crash in mysqld.
[1 Sep 2021 12:08] MySQL Verification Team
Hi,

No, for malfunctioning hardware there is no workaround with changes in the configurational variables.
[22 Sep 2021 12:27] MySQL Verification Team
Hi,

Hardware problems can not be alleviated by changing the configuration. You can only make them more less frequent ....... That is all.