Bug #74596 | crash in btr_cur_latch_leaves | ||
---|---|---|---|
Submitted: | 28 Oct 2014 2:27 | Modified: | 22 Dec 2014 13:20 |
Reporter: | Mark Callaghan | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: InnoDB storage engine | Severity: | S3 (Non-critical) |
Version: | 5.7.5, 5.7.6 | OS: | Any |
Assigned to: | CPU Architecture: | Any |
[28 Oct 2014 2:27]
Mark Callaghan
[28 Oct 2014 10:53]
Mark Callaghan
Thread stack when gdb is attached #0 0x00007fbf4ca95e0b in raise () from /usr/local/fbcode/gcc-4.7.1-glibc-2.14.1/lib/libc.so.6 #1 0x00007fbf4ca9775f in abort () from /usr/local/fbcode/gcc-4.7.1-glibc-2.14.1/lib/libc.so.6 #2 0x00000000006896ff in ut_dbg_assertion_failed (expr=expr@entry=0xf2f468 "page_is_comp(get_block->frame) == page_is_comp(page)", file=file@entry=0xf2f320 "/s/bld/orig575/storage/innobase/btr/btr0cur.cc", line=line@entry=307) at /s/bld/orig575/storage/innobase/ut/ut0dbg.cc:68 #3 0x0000000000cd7042 in btr_cur_latch_leaves (block=block@entry=0x7fa32b6e34f0, page_id=..., page_size=..., latch_mode=latch_mode@entry=33, cursor=cursor@entry=0x7fbf4e335b50, mtr=mtr@entry=0x7fbf4e336280) at /s/bld/orig575/storage/innobase/btr/btr0cur.cc:306 #4 0x0000000000ce0289 in btr_cur_search_to_nth_level (index=index@entry=0x7f9dfd45f280, level=level@entry=0, tuple=tuple@entry=0x7f9dfe460880, mode=mode@entry=4, latch_mode=latch_mode@entry=33, cursor=cursor@entry=0x7fbf4e335b50, has_search_latch=has_search_latch@entry=0, file=file@entry=0xf1da90 "/s/bld/orig575/storage/innobase/row/row0ins.cc", line=line@entry=2372, mtr=mtr@entry=0x7fbf4e336280) at /s/bld/orig575/storage/innobase/btr/btr0cur.cc:1288 #5 0x0000000000c18c65 in btr_pcur_open_low (mtr=0x7fbf4e336280, line=2372, cursor=0x7fbf4e335b50, latch_mode=33, mode=4, tuple=0x7f9dfe460880, index=0x7f9dfd45f280, level=<optimized out>, file=<optimized out>) at /s/bld/orig575/storage/innobase/include/btr0pcur.ic:455 #6 row_ins_clust_index_entry_low (flags=flags@entry=0, mode=mode@entry=33, index=index@entry=0x7f9dfd45f280, n_uniq=n_uniq@entry=3, entry=entry@entry=0x7f9dfe460880, n_ext=n_ext@entry=0, thr=thr@entry=0x7f9df9854ae0, dup_chk_only=dup_chk_only@entry=false) at /s/bld/orig575/storage/innobase/row/row0ins.cc:2372 #7 0x0000000000c1facf in row_ins_clust_index_entry (index=0x7f9dfd45f280, entry=0x7f9dfe460880, thr=thr@entry=0x7f9df9854ae0, n_ext=n_ext@entry=0, dup_chk_only=dup_chk_only@entry=false) at /s/bld/orig575/storage/innobase/row/row0ins.cc:3208 #8 0x0000000000c1fe31 in row_ins_index_entry (thr=0x7f9df9854ae0, entry=<optimized out>, index=<optimized out>) at /s/bld/orig575/storage/innobase/row/row0ins.cc:3306 #9 row_ins_index_entry_step (thr=0x7f9df9854ae0, node=0x7f9df9854700) at /s/bld/orig575/storage/innobase/row/row0ins.cc:3448 #10 row_ins (thr=0x7f9df9854ae0, node=0x7f9df9854700) at /s/bld/orig575/storage/innobase/row/row0ins.cc:3590 #11 row_ins_step (thr=thr@entry=0x7f9df9854ae0) at /s/bld/orig575/storage/innobase/row/row0ins.cc:3762 #12 0x0000000000c2fe04 in row_insert_for_mysql_using_ins_graph (prebuilt=<optimized out>, mysql_rec=0x7f9e81f60580 "3") at /s/bld/orig575/storage/innobase/row/row0mysql.cc:1579 #13 row_insert_for_mysql (mysql_rec=mysql_rec@entry=0x7f9dfe451820 "\t\340\214\065", prebuilt=<optimized out>) at /s/bld/orig575/storage/innobase/row/row0mysql.cc:1695 #14 0x0000000000b84838 in ha_innobase::write_row (this=0x7f9dfe451420, record=0x7f9dfe451820 "\t\340\214\065") at /s/bld/orig575/storage/innobase/handler/ha_innodb.cc:6755 #15 0x000000000071c40f in handler::ha_write_row (this=0x7f9dfe451420, buf=0x7f9dfe451820 "\t\340\214\065") at /s/bld/orig575/sql/handler.cc:7423 #16 0x0000000000a2d02b in write_record (thd=thd@entry=0x7f9dfe418000, table=table@entry=0x7f9df985fc10, info=info@entry=0x7fbf4e336ba0, update=update@entry=0x7fbf4e336c20) at /s/bld/orig575/sql/sql_insert.cc:1347 #17 0x0000000000a2e8a2 in mysql_insert (thd=thd@entry=0x7f9dfe418000, table_list=0x7f9dfe423928, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_UPDATE) at /s/bld/orig575/sql/sql_insert.cc:707 #18 0x000000000090c614 in mysql_execute_command (thd=thd@entry=0x7f9dfe418000) at /s/bld/orig575/sql/sql_parse.cc:3367 #19 0x000000000090fa80 in mysql_parse (parser_state=<optimized out>, thd=0x7f9dfe418000) at /s/bld/orig575/sql/sql_parse.cc:5427 #20 mysql_parse (thd=0x7f9dfe418000, parser_state=<optimized out>) at /s/bld/orig575/sql/sql_parse.cc:5294 #21 0x000000000091086c in dispatch_command (command=COM_QUERY, thd=thd@entry=0x7f9dfe418000, packet=0x7f9df0536011 "INSERT INTO linkdb.linktable(id1, id2, link_type, visibility, data, time, version) VALUES (898444944, 898444944, 123456789, 1, x'625b2a55563d2d5c', 1414276955157, 0),(898444947, 898444947, 123456789, "..., packet_length=90009) at /s/bld/orig575/sql/sql_parse.cc:1250 #22 0x00000000009114ca in do_command (thd=thd@entry=0x7f9dfe418000) at /s/bld/orig575/sql/sql_parse.cc:834 #23 0x00000000009a4f04 in handle_connection (arg=arg@entry=0x7f9ec13fffe0) at /s/bld/orig575/sql/conn_handler/connection_handler_per_thread.cc:298 #24 0x0000000000adf943 in pfs_spawn_thread (arg=0x7f9e819023d0) at /s/bld/orig575/storage/perfschema/pfs.cc:2137 #25 0x00007fbf4e03712d in start_thread () from /usr/local/fbcode/gcc-4.7.1-glibc-2.14.1/lib/libpthread.so.0 #26 0x00007fbf4cb4d13d in clone () from /usr/local/fbcode/gcc-4.7.1-glibc-2.14.1/lib/libc.so.6
[28 Oct 2014 20:30]
Mark Callaghan
This is not a debug build, but the assert is from debug (UNIV_BTR_DEBUG) code: #ifdef UNIV_BTR_DEBUG ut_a(page_is_comp(get_block->frame) == page_is_comp(page)); ut_a(btr_page_get_next(get_block->frame, mtr) == page_get_page_no(page)); #endif /* UNIV_BTR_DEBUG */
[30 Oct 2014 22:01]
Mark Callaghan
Crash reproduces with innodb_page_size=16k
[4 Nov 2014 16:27]
Mark Callaghan
This does not reproduce in 5.7.4
[28 Nov 2014 4:47]
MySQL Verification Team
Hello Mark, Thank you for the bug report and details to reproduce the issue. Observed this crash with 5.7.6 optimized build. Thanks, Umesh
[28 Nov 2014 4:48]
MySQL Verification Team
## Snippet from error log 2014-11-28T03:36:24.332133Z 0 [Note] /export/umesh/mysql-5.7.6/bin/mysqld: ready for connections. Version: '5.7.6-m16-enterprise-commercial-advanced' socket: '/export/umesh/mysql-5.7.6/data/mysql.sock' port: 15000 MySQL Enterprise Server - Advanced Edition (Commercial) 2014-11-28 05:02:04 0x7fc9e046c700 InnoDB: Assertion failure in thread 140505027888896 in file btr0cur.cc line 301 InnoDB: Failing assertion: page_is_comp(get_block->frame) == page_is_comp(page) // Core ls -lh data/core.18113 -rw------- 1 umshastr common 133G Nov 28 05:03 data/core.18113 gdb bin/mysqld data/core.18113 .. . (gdb) bt #0 0x00007fe8a9d0c771 in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000693a75 in handle_fatal_signal (sig=6) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/signal_handler.cc:219 #2 <signal handler called> #3 0x00007fe8a89125c9 in raise () from /lib64/libc.so.6 #4 0x00007fe8a8913cd8 in abort () from /lib64/libc.so.6 #5 0x000000000066c360 in ut_dbg_assertion_failed (expr=expr@entry=0x1053f40 "page_is_comp(get_block->frame) == page_is_comp(page)", file=file@entry=0x1053d80 "/export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/btr/btr0cur.cc", line=line@entry=301) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/ut/ut0dbg.cc:67 #6 0x0000000000d30996 in btr_cur_latch_leaves (block=block@entry=0x7fd0f50798b0, page_id=..., page_size=..., latch_mode=latch_mode@entry=33, cursor=cursor@entry=0x7fc9e0469450, mtr=mtr@entry=0x7fc9e0469b80) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/btr/btr0cur.cc:300 #7 0x0000000000d389db in btr_cur_search_to_nth_level (index=index@entry=0x7fc9b001b6c0, level=level@entry=0, tuple=tuple@entry=0x7fc9741caba0, mode=mode@entry=PAGE_CUR_LE, latch_mode=latch_mode@entry=33, cursor=cursor@entry=0x7fc9e0469450, has_search_latch=has_search_latch@entry=0, file=file@entry=0x1041e10 "/export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc", line=line@entry=2351, mtr=mtr@entry=0x7fc9e0469b80) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/btr/btr0cur.cc:1282 #8 0x0000000000c642e4 in btr_pcur_open_low (level=0, file=0x1041e10 "/export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc", mtr=0x7fc9e0469b80, line=2351, cursor=0x7fc9e0469450, latch_mode=33, mode=PAGE_CUR_LE, tuple=0x7fc9741caba0, index=0x7fc9b001b6c0) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/include/btr0pcur.ic:455 #9 row_ins_clust_index_entry_low (flags=flags@entry=0, mode=mode@entry=33, index=index@entry=0x7fc9b001b6c0, n_uniq=n_uniq@entry=3, entry=entry@entry=0x7fc9741caba0, n_ext=n_ext@entry=0, thr=thr@entry=0x7fc9741c9e90, dup_chk_only=dup_chk_only@entry=false) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:2351 #10 0x0000000000c6b0ad in row_ins_clust_index_entry (index=0x7fc9b001b6c0, entry=0x7fc9741caba0, thr=thr@entry=0x7fc9741c9e90, n_ext=n_ext@entry=0, dup_chk_only=dup_chk_only@entry=false) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:3181 #11 0x0000000000c6b70d in row_ins_index_entry (thr=0x7fc9741c9e90, entry=<optimized out>, index=<optimized out>) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:3285 #12 row_ins_index_entry_step (thr=0x7fc9741c9e90, node=0x7fc9741c9ab0) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:3426 #13 row_ins (thr=0x7fc9741c9e90, node=0x7fc9741c9ab0) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:3568 #14 row_ins_step (thr=thr@entry=0x7fc9741c9e90) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0ins.cc:3739 #15 0x0000000000c7ced1 in row_insert_for_mysql_using_ins_graph (mysql_rec=mysql_rec@entry=0x7fc9741c8780 "=\270=:", prebuilt=prebuilt@entry=0x7fc9741c9430) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0mysql.cc:1570 #16 0x0000000000c7d31a in row_insert_for_mysql (mysql_rec=mysql_rec@entry=0x7fc9741c8780 "=\270=:", prebuilt=prebuilt@entry=0x7fc9741c9430) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/row/row0mysql.cc:1681 #17 0x0000000000bc0529 in ha_innobase::write_row (this=0x7fc9741c8380, record=0x7fc9741c8780 "=\270=:") at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/innobase/handler/ha_innodb.cc:6697 #18 0x00000000006d98bf in handler::ha_write_row (this=0x7fc9741c8380, buf=0x7fc9741c8780 "=\270=:") at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/handler.cc:7351 #19 0x0000000000a534e8 in write_record (thd=thd@entry=0x7fc974000b90, table=table@entry=0x7fc9741c79b0, info=info@entry=0x7fc9e046a6f0, update=update@entry=0x7fc9e046a770) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_insert.cc:1352 #20 0x0000000000a54ccb in mysql_insert (thd=thd@entry=0x7fc974000b90, table_list=0x7fc974005ae8, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_UPDATE) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_insert.cc:712 #21 0x000000000092490b in mysql_execute_command (thd=thd@entry=0x7fc974000b90) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_parse.cc:3334 #22 0x0000000000929708 in mysql_parse (thd=0x7fc974000b90, parser_state=<optimized out>) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_parse.cc:5386 #23 0x000000000092a859 in dispatch_command (command=COM_QUERY, thd=thd@entry=0x7fc974000b90, packet=0x7fc9740158b1 "INSERT INTO linkdb.linktable(id1, id2, link_type, visibility, data, time, version) VALUES (977145813, 977145813, 123456789, 1, x'4b254a575d4642203d', 1415823267812, 0),(977145814, 977145814, 123456789"..., packet_length=88239) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_parse.cc:1249 #24 0x000000000092c694 in do_command (thd=thd@entry=0x7fc974000b90) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/sql_parse.cc:833 #25 0x00000000009c6900 in handle_connection (arg=arg@entry=0x1b1dd1e10) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/sql/conn_handler/connection_handler_per_thread.cc:298 #26 0x0000000000b150d3 in pfs_spawn_thread (arg=0x1b1e3c4d0) at /export/home/pb2/build/sb_0-13761755-1417089283.97/mysqlcom-pro-5.7.6-m16/storage/perfschema/pfs.cc:2137 #27 0x00007fe8a9d07df3 in start_thread () from /lib64/libpthread.so.0 #28 0x00007fe8a89d347d in clone () from /lib64/libc.so.6
[28 Nov 2014 4:49]
MySQL Verification Team
## Environment used cat /etc/*release Oracle Linux Server release 7.0 NAME="Oracle Linux Server" VERSION="7.0" ID="ol" VERSION_ID="7.0" PRETTY_NAME="Oracle Linux Server 7.0" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:oracle:oracle_linux:7.0:GA:server" HOME_URL="https://linux.oracle.com/" BUG_REPORT_URL="https://bugzilla.oracle.com/" ORACLE_BUGZILLA_PRODUCT="Oracle Linux 7" ORACLE_BUGZILLA_PRODUCT_VERSION=7.0 ORACLE_SUPPORT_PRODUCT="Oracle Linux" REDHAT_SUPPORT_PRODUCT_VERSION=7.0 Red Hat Enterprise Linux Server release 7.0 (Maipo) Oracle Linux Server release 7.0 ## Java java -version java version "1.7.0_65" OpenJDK Runtime Environment (rhel-2.5.1.2.0.1.el7_0-x86_64 u65-b17) OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) // MySQL Build used commit: 1982dd7ab813e3cdb28d3bb119cc7b706cf01b15 date: 2014-11-27 13:27:39 +0300 build-date: 2014-11-27 12:38:45 +0100 short: 1982dd7 branch: mysql-trunk MySQL source 5.7.6 mysql> show variables like '%version%'; +-------------------------+---------------------------------------------------------+ | Variable_name | Value | +-------------------------+---------------------------------------------------------+ | innodb_version | 5.7.6 | | protocol_version | 10 | | slave_type_conversions | | | version | 5.7.6-m16-enterprise-commercial-advanced | | version_comment | MySQL Enterprise Server - Advanced Edition (Commercial) | | version_compile_machine | x86_64 | | version_compile_os | Linux | +-------------------------+---------------------------------------------------------+ 7 rows in set (0.00 sec) // Apache Maven md5sum apache-maven-3.2.3-bin.tar.gz 2fcfdb327eb94b8595d97ee4181ef0a6 apache-maven-3.2.3-bin.tar.gz bin/mvn --version Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T22:58:10+02:00) Maven home: /export/umesh/apache-maven-3.2.3 Java version: 1.7.0_65, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.65-2.5.1.2.0.1.el7_0.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "3.8.13-44.1.1.el7uek.x86_64", arch: "amd64", family: "unix" // linkbench - Downloaded from https://github.com/facebook/linkbench md5sum master.zip 2dc2d8ef1448f3eeaacdcb7e3ba649c2 master.zip -- skipped all tests when building linkbench mvn clean package -DskipTests
[6 Dec 2014 15:25]
Mark Callaghan
may be similar to http://bugs.mysql.com/bug.php?id=75124
[7 Dec 2014 7:31]
MySQL Verification Team
Bug #75124 marked as duplicate of this
[22 Dec 2014 13:20]
Daniel Price
Posted by developer: Fixed as of the upcoming 5.7.6 release, and here's the changelog entry: A crash occurred in "btr_cur_latch_leaves" while performing a load operation. Checking the page state without latching the page caused an inconsistency. The page state should only be checked after the page is latched. Thank you for the bug report.