| Bug #81445 | InnoDB: Assertion failure in thread 140573014648576 in file buf0buf.cc line 3870 | ||
|---|---|---|---|
| Submitted: | 17 May 2016 1:41 | Modified: | 24 Nov 2016 13:21 |
| Reporter: | zhouchang Mike | Email Updates: | |
| Status: | No Feedback | Impact on me: | |
| Category: | MySQL Server: InnoDB storage engine | Severity: | S1 (Critical) |
| Version: | 5.7.12 | OS: | Red Hat (2.6.32-358.el6.x86_64) |
| Assigned to: | CPU Architecture: | Any | |
[17 May 2016 1:41]
zhouchang Mike
[18 May 2016 9:20]
zhouchang Mike
Thread pointer: 0x7fade95f9000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7fad5ef3bde8 thread_stack 0x40000 /home/mysql1/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xf23af5] /home/mysql1/mysql/bin/mysqld(handle_fatal_signal+0x4a4)[0x7b7db4] /lib64/libpthread.so.0[0x326ac0f500] /lib64/libc.so.6(gsignal+0x35)[0x326a8328a5] /lib64/libc.so.6(abort+0x175)[0x326a834085] /home/mysql1/mysql/bin/mysqld(_Z18ut_print_timestampP8_IO_FILE+0x0)[0x7a7128] /home/mysql1/mysql/bin/mysqld[0x11bcfdd] /home/mysql1/mysql/bin/mysqld(_Z24btr_search_guess_on_hashP12dict_index_tP12btr_search_tPK8dtuple_tmmP9btr_cur_tmP5mtr_t+0x7b7)[0x11b37d7] /home/mysql1/mysql/bin/mysqld(_Z27btr_cur_search_to_nth_levelP12dict_index_tmPK8dtuple_t15page_cur_mode_tmP9btr_cur_tmPKcmP5mtr_t+0xfe4)[0x11a89e4] /home/mysql1/mysql/bin/mysqld[0x11098eb] /home/mysql1/mysql/bin/mysqld(_Z15row_search_mvccPh15page_cur_mode_tP14row_prebuilt_tmm+0x253e)[0x11136de] /home/mysql1/mysql/bin/mysqld(_ZN11ha_innobase10index_readEPhPKhj16ha_rkey_function+0x33a)[0x1026aaa] /home/mysql1/mysql/bin/mysqld(_ZN7handler18index_read_idx_mapEPhjPKhm16ha_rkey_function+0x78)[0x7fd2e8] /home/mysql1/mysql/bin/mysqld(_ZN7handler21ha_index_read_idx_mapEPhjPKhm16ha_rkey_function+0xa0)[0x7ffac0] /home/mysql1/mysql/bin/mysqld[0xccb223] /home/mysql1/mysql/bin/mysqld(_Z21join_read_const_tableP8JOIN_TABP11st_position+0xd1)[0xccf3d1] /home/mysql1/mysql/bin/mysqld(_ZN4JOIN29extract_func_dependent_tablesEv+0x318)[0xce1358] /home/mysql1/mysql/bin/mysqld(_ZN4JOIN14make_join_planEv+0x95e)[0xceed3e] /home/mysql1/mysql/bin/mysqld(_ZN4JOIN8optimizeEv+0x606)[0xcf2d96] /home/mysql1/mysql/bin/mysqld(_ZN13st_select_lex8optimizeEP3THD+0x672)[0xd377c2] /home/mysql1/mysql/bin/mysqld(_Z12handle_queryP3THDP3LEXP12Query_resultyy+0x21f)[0xd37a5f] /home/mysql1/mysql/bin/mysqld[0xcf93c3] /home/mysql1/mysql/bin/mysqld(_Z21mysql_execute_commandP3THDb+0x376f)[0xcfcdff] /home/mysql1/mysql/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x3ad)[0xcfe8fd] /home/mysql1/mysql/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x115d)[0xcffabd] /home/mysql1/mysql/bin/mysqld(_Z10do_commandP3THD+0x194)[0xd005a4] /home/mysql1/mysql/bin/mysqld(handle_connection+0x29c)[0xdcd1ec] /home/mysql1/mysql/bin/mysqld(pfs_spawn_thread+0x171)[0xf9e1c1] /lib64/libpthread.so.0[0x326ac07851] /lib64/libc.so.6(clone+0x6d)[0x326a8e890d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7fade94b0030): is an invalid pointer Connection ID (thread ID): 54457 Status: NOT_KILLED
[18 May 2016 23:56]
zhai weixiang
bug#79378 may be related
[20 May 2016 7:20]
zhouchang Mike
the same case does not occur in mysql 5.6.30. Does there any change in buf0buf.cc for mysql 5.7.12 ?
[20 May 2016 7:27]
zhouchang Mike
this function "buf_block_from_ahi" was added by the mysql 5.7.30
and I can not find in mysql 5.7.30
buf_block_t*
buf_block_from_ahi(const byte* ptr)
{
buf_pool_chunk_map_t::iterator it;
buf_pool_chunk_map_t* chunk_map = buf_chunk_map_ref;
ut_ad(buf_chunk_map_ref == buf_chunk_map_reg);
ut_ad(!buf_pool_resizing);
const byte* bound = reinterpret_cast<uintptr_t>(ptr)
> srv_buf_pool_chunk_unit
? ptr - srv_buf_pool_chunk_unit : 0;
it = chunk_map->upper_bound(bound);
ut_a(it != chunk_map->end());
buf_chunk_t* chunk = it->second;
ulint offs = ptr - chunk->blocks->frame;
offs >>= UNIV_PAGE_SIZE_SHIFT;
ut_a(offs < chunk->size);
buf_block_t* block = &chunk->blocks[offs];
/* The function buf_chunk_init() invokes buf_block_init() so that
block[n].frame == block->frame + n * UNIV_PAGE_SIZE. Check it. */
ut_ad(block->frame == page_align(ptr));
/* Read the state of the block without holding a mutex.
A state transition from BUF_BLOCK_FILE_PAGE to
BUF_BLOCK_REMOVE_HASH is possible during this execution. */
ut_d(const buf_page_state state = buf_block_get_state(block));
ut_ad(state == BUF_BLOCK_FILE_PAGE || state == BUF_BLOCK_REMOVE_HASH);
return(block);
}
[21 May 2016 6:00]
zhouchang Mike
I think this may be a critical bug.
[4 Aug 2016 7:00]
ChunFeng Tang
the same case occur to me when load a big mount of data 2016-08-02T09:09:54.053706Z 0 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.7.13-enterprise-commercial-advanced' socket: '/data/mysql/mysql.sock' port: 3306 MySQL Enterprise Server - Advanced Edition (Commercial) 2016-08-02T09:09:54.546121Z 0 [Note] InnoDB: Buffer pool(s) load completed at 160802 17:09:54 2016-08-02 17:13:45 0x7fa6a81b0700 InnoDB: Assertion failure in thread 140353761642240 in file buf0buf.cc line 3870 InnoDB: Failing assertion: offs < chunk->size InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 09:13:45 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. key_buffer_size=8388608 read_buffer_size=131072 max_used_connections=1 max_threads=151 thread_count=1 connection_count=1 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68189 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7fa67c000ae0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7fa6a81afe70 thread_stack 0x40000 /usr/sbin/mysqld(my_print_stacktrace+0x3b)[0xf04b7b] /usr/sbin/mysqld(handle_fatal_signal+0x461)[0x7bec21] /lib64/libpthread.so.0(+0xf100)[0x7fa7045e4100] /lib64/libc.so.6(gsignal+0x37)[0x7fa702fd85f7] /lib64/libc.so.6(abort+0x148)[0x7fa702fd9ce8] /usr/sbin/mysqld[0x78f21e] /usr/sbin/mysqld[0x111327d] /usr/sbin/mysqld(_Z24btr_search_guess_on_hashP12dict_index_tP12btr_search_tPK8dtuple_tmmP9btr_cur_tmP5mtr_t+0x7a7)[0x11032e7] /usr/sbin/mysqld(_Z27btr_cur_search_to_nth_levelP12dict_index_tmPK8dtuple_t15page_cur_mode_tmP9btr_cur_tmPKcmP5mtr_t+0x289)[0x10f5409] /usr/sbin/mysqld(_Z29row_ins_clust_index_entry_lowmmP12dict_index_tmP8dtuple_tmP9que_thr_tb+0x252)[0x10135c2] /usr/sbin/mysqld(_Z25row_ins_clust_index_entryP12dict_index_tP8dtuple_tP9que_thr_tmb+0xa8)[0x1014a48] /usr/sbin/mysqld(_Z12row_ins_stepP9que_thr_t+0x376)[0x1014f46] /usr/sbin/mysqld[0x1026f58] /usr/sbin/mysqld(_ZN11ha_innobase9write_rowEPh+0x1d6)[0xf48806] /usr/sbin/mysqld(_ZN7handler12ha_write_rowEPh+0x10d)[0x816bed] /usr/sbin/mysqld(_Z12write_recordP3THDP5TABLEP9COPY_INFOS4_+0xb9)[0xe4f249] /usr/sbin/mysqld(_Z10mysql_loadP3THDP12sql_exchangeP10TABLE_LISTR4ListI4ItemES8_S8_15enum_duplicatesb+0x1a6b)[0xe5ad4b] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THDb+0x2e86)[0xcd7626] /usr/sbin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x3b5)[0xcdb055] /usr/sbin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x8d7)[0xcdb9a7] /usr/sbin/mysqld(_Z10do_commandP3THD+0x19f)[0xcdd33f] /usr/sbin/mysqld(handle_connection+0x290)[0xd9b240] /usr/sbin/mysqld(pfs_spawn_thread+0x1b4)[0x12748a4] /lib64/libpthread.so.0(+0x7dc5)[0x7fa7045dcdc5] /lib64/libc.so.6(clone+0x6d)[0x7fa70309921d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7fa67c0054c0): LOAD DATA LOCAL INFILE '/soft/data/T_HDS_5MIN_FINAL.txt' INTO TABLE jfdb.T_HDS_5MIN_FINAL FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' ignore 1 lines Connection ID (thread ID): 2 Status: NOT_KILLED
[13 Aug 2016 8:07]
MySQL Verification Team
Looks related to large-pages, filed internally as Bug 23593654 - CRASH IN BUF_BLOCK_FROM_AHI WHEN LARGE PAGES AND AHI ARE ENABLED
[24 Oct 2016 13:21]
MySQL Verification Team
Please try version 5.7.16. Thanks.
[25 Nov 2016 1:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[5 Oct 2023 14:39]
Zafar Malik
I recently upgraded mysql8.0.32 to mysql8.0.34 and getting same kind of issue as mysql is crashing, details are below- OS : CentOS7.9 Mysql : Mysql 8.0.34-enterprise-commercial ######################################## DEADLOCK of threads detected! --Thread 139617853323008 has waited at lock0lock.cc line 4963 for 14 seconds the semaphore: Mutex at 0x7efccc331e98, Mutex TRX_SYS created trx0sys.cc:565, locked by 139618000262912 --Thread 139618000262912 has waited at p_s.cc line 588 for 0 seconds the semaphore: X-lock on RW-latch at 0x7efccc37d750 created in file sync0sharded_rw.h line 81 a writer (thread id 139617853323008) has reserved it in mode exclusive number of readers 0, waiters flag 1, lock_word: 0 Last time read locked in file lock0lock.cc line 5741 Last time write locked in file /var/lib/pb2/sb_1-11875240-1687434162.58/rpm/BUILD/mysqlcom-8.0.34/mysqlcom-8.0.34/storage/innobase/srv/srv0srv.cc line 1414 --Thread 139618043270912 has waited at trx0sys.h line 598 for 21 seconds the semaphore: Mutex at 0x7efccc331e98, Mutex TRX_SYS created trx0sys.cc:565, locked by 139618000262912 2023-10-04T21:30:33.033125Z 0 [ERROR] [MY-012982] [InnoDB] [FATAL] ######################################## Deadlock Detected! 2023-10-04T21:30:33.033489Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: sync0arr.cc:678:ib::fatal triggered thread 139620223969024 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 2023-10-04T21:30:33Z UTC - mysqld got signal 6 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. BuildID[sha1]=5a189c5df4f369c3fc93ece79595e5b3c0b4a321 Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0 thread_stack 0x100000 /usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x211744d] /usr/sbin/mysqld(print_fatal_signal(int)+0x37f) [0xfd7eaf] /usr/sbin/mysqld(my_server_abort()+0x7e) [0xfd7ffe] /usr/sbin/mysqld(my_abort()+0xa) [0x211161a] /usr/sbin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x31f) [0x2331f7f] /usr/sbin/mysqld() [0x233493f] /usr/sbin/mysqld() [0x22f3028] /usr/sbin/mysqld(sync_array_detect_deadlock()+0x8a) [0x22f33fa] /usr/sbin/mysqld(srv_error_monitor_thread()+0x18e) [0x22d83de] /usr/sbin/mysqld(void Detached_thread::operator()<void (*)()>(void (*&&)())+0xc2) [0x2222a42] /usr/sbin/mysqld() [0x2adcc14] /lib64/libpthread.so.0(+0x7ea5) [0x7efce24d0ea5] /lib64/libc.so.6(clone+0x6d) [0x7efce088bb0d] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.
