Bug #21173 MySQL crashed after "show table status" on some range-partitioned table
Submitted: 20 Jul 2006 4:48 Modified: 2 Sep 2006 4:41
Reporter: McEase Tu Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Partitions Severity:S1 (Critical)
Version:5.1.12-BK, 5.1.11 OS:Linux (GNU/Linux)
Assigned to: Mikael Ronström CPU Architecture:Any

[20 Jul 2006 4:48] McEase Tu
Description:
There is a partitioned table with RANGE partition type. I have another multi-threaded program which will insert into a lot of records into this table. When it is running, I terminate it. After that, I want to see the table's status through "show table status" command. However, mysql crashed when it receive this command.

The version of mysql is 5.1.11. I compiled it by myself. The OS is GNU/Linux 2.6.15-1-686-smp. The compiler and libraries are gcc (GCC) 4.0.3 (Debian 4.0.3-1), libc6(2.3.6-3), libg++2.8.1.3-glibc2.2.

I also repeated in another AMD-64 machine.

How to repeat:
1. create the table:

mysql> create table test.partest(name varchar(32), thread  int, tran int, stmt int, create_time datetime) engine=innodb partition by range (UNIX_TIMESTAMP(create_time)) (partition p0 values less than (UNIX_TIMESTAMP('2006-07-20 12:00:00')), partition p1 values less than (UNIX_TIMESTAMP('2006-07-27 12:00:00')));
Query OK, 0 rows affected (0.01 sec)

2. Run the java program I attached later:

$java Partition <mysql-host> <user> [<pwd>]
about 5% finished ...
about 10% finished ...
<...omitted...>

When it appeared "about n% finished ...", press "Ctrl-C" to terminate the program.

The JDBC I used here is "mysql-connector-java-5.0-nightly-20051222-bin.jar".

3. See the table's status:

mysql> use test
Database changed
mysql> show table status;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> 
Number of processes running now: 0
060720 12:43:13  mysqld restarted
[20 Jul 2006 4:49] McEase Tu
The java program used to repeat the bug

Attachment: Partition.java (text/java), 1.79 KiB.

[20 Jul 2006 5:05] McEase Tu
Following is the output of resolve_stack_dump:

Stack range sanity check OK, backtrace follows:
0x818d831 handle_segfault + 461
0xffffe420 _end + -140233520
0x700b8fc8 _end + 1739579512
0x826c7c5 _ZN12ha_partition18get_auto_incrementEv + 125
0x82719d3 _ZN12ha_partition4infoEj + 117
0x8296c7b _Z24get_schema_tables_recordP3THDP13st_table_listP8st_tablebPKcS6_ + 667
0x829d827 _Z14get_all_tablesP3THDP13st_table_listP4Item + 1939
0x82951b9 _Z24get_schema_tables_resultP4JOIN + 389
0x81fc69a _ZN4JOIN4execEv + 1860
0x81fec65 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 1551
0x81ff15d _Z13handle_selectP3THDP6st_lexP13select_resultm + 429
0x81a3aa2 _Z21mysql_execute_commandP3THD + 1814
0x81ab674 _Z11mysql_parseP3THDPcj + 376
0x81ac292 _Z16dispatch_command19enum_server_commandP3THDPcj + 2074
0x81ad4c0 _Z10do_commandP3THD + 512
0x81ae080 handle_one_connection + 2832
0xb7f10ced _end + -1349165667
0xb7d4ddde _end + -1351012722
New value of fp=(nil) failed sanity check, terminating stack trace!
[20 Jul 2006 13:10] Valeriy Kravchuk
Thank you for a bug report. Verified just as described with your Java test case:

C:\Program Files\Java\jdk1.5.0_07\bin>-java-3.1.13-bin.jar Partition suse root
about 5% finished ...
about 10% finished ...

Stopped with Ctrl-C. Then, on Linux server (with latest 5.1.12-BK):

mysql> show processlist;
+----+-----------------+-----------+------+---------+------+-----------+------------------+
| Id | User            | Host      | db   | Command | Time | State     | Info          |
+----+-----------------+-----------+------+---------+------+-----------+------------------+
|  1 | event_scheduler | localhost | NULL | Connect | 1584 | Suspended | NULL          |
| 20 | root            | localhost | test | Query   |    0 | NULL      | show processlist |
+----+-----------------+-----------+------+---------+------+-----------+------------------+
2 rows in set (0.00 sec)

mysql> show table status;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
Number of processes running now: 0
060720 14:11:37  mysqld restarted

Resolved stack trace:

openxs@suse:~/dbs/5.1> bin/resolve_stack_dump -s /tmp/mysqld.sym -n mysql51.sta
ck
0x82166ea handle_segfault + 412
0xffffe420 _end + -143830976
(nil)
0x401a5903 _end + 931644707
0x81ecd43 _ZN12Field_medium7val_intEv + 95
0x830034d _ZN5Field14val_int_offsetEj + 39
0x831a15c _ZN11ha_innobase31innobase_read_and_init_auto_incEPx + 634
0x831a217 _ZN11ha_innobase18get_auto_incrementEyyyPyS0_ + 57
0x83113a6 _ZN12ha_partition18get_auto_incrementEyyyPyS0_ + 202
0x830fb2c _ZN12ha_partition4infoEj + 140
0x8374a92 _Z24get_schema_tables_recordP3THDP13st_table_listP8st_tablebPKcS6_ + 6
14
0x8374266 _Z14get_all_tablesP3THDP13st_table_listP4Item + 2240
0x837d476 _Z24get_schema_tables_resultP4JOIN + 492
0x8274537 _ZN4JOIN4execEv + 1689
0x8275e45 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orde
rSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 635
0x8270d0d _Z13handle_selectP3THDP6st_lexP13select_resultm + 309
0x8237fb6 _Z21execute_sqlcom_selectP3THDP13st_table_list + 592
0x823122b _Z21mysql_execute_commandP3THD + 1249
0x82399d2 _Z11mysql_parseP3THDPcj + 484
0x822f61e _Z16dispatch_command19enum_server_commandP3THDPcj + 1918
0x822ee90 _Z10do_commandP3THD + 518
0x822e01d handle_one_connection + 803
0x40047aa7 _end + 930211527
0x4023ec2e _end + 932272206
[28 Jul 2006 2:04] McEase Tu
I found maybe this problem is mainly related with the partition type "RANGE".

I created a program which is multi-threaded with transaction. After it finished normally, "show table status like 'partest'" will cause the server crash.
I created another program which is single-threaded and no transaction. After it finished normally, "show table status like 'partest'" will also cause the server crash.
Both of the program just execute the same prepared INSERT statement for a few hundreds of times.
However, the positions where mysql crashed by the two program are different.

=== stack trace after crashed by "multi-threaded with transactions" program ===
Stack range sanity check OK, backtrace follows:
0x81917ac handle_segfault + 668
0xffffe420 _end + -140066672
(nil)
0x8271534 _ZN12ha_partition18get_auto_incrementEv + 84
0x8270cc3 _ZN12ha_partition4infoEj + 627
0x82a176e _Z24get_schema_tables_recordP3THDP13st_table_listP8st_tablebPKcS6_ + 670
0x829c295 _Z14get_all_tablesP3THDP13st_table_listP4Item + 1621
0x8296d14 _Z24get_schema_tables_resultP4JOIN + 356
0x8207927 _ZN4JOIN4execEv + 1927
0x8209b27 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 455
0x820a399 _Z13handle_selectP3THDP6st_lexP13select_resultm + 297
0x81acd6d _Z21mysql_execute_commandP3THD + 10573
0x81b5641 _Z11mysql_parseP3THDPcj + 593
0x81b5c64 _Z16dispatch_command19enum_server_commandP3THDPcj + 1252
0x81b79fc handle_one_connection + 2380
0xb7f1cced _end + -1348949667
0xb7d59dde _end + -1350796722
New value of fp=(nil) failed sanity check, terminating stack trace!

=== stack trace after crashed by "single-threaded no transactions" program ===
Stack range sanity check OK, backtrace follows:
0x81917ac handle_segfault + 668
0xffffe420 _end + -140066672
(nil)
0x8271534 _ZN12ha_partition18get_auto_incrementEv + 84
0x8270cc3 _ZN12ha_partition4infoEj + 627
0x82a176e _Z24get_schema_tables_recordP3THDP13st_table_listP8st_tablebPKcS6_ + 670
0x829c295 _Z14get_all_tablesP3THDP13st_table_listP4Item + 1621
Stack trace seems successful - bottom reached
[7 Aug 2006 21:34] Mikael Ronström
The minimal test case that crashes the MySQL Server
here is:
create table t1 (a int)
engine = innodb
partition by key (a)
insert into t1 values (0);
show table status;

Given that this means that show table status crashes the
server on any InnoDB-table that has at least one record
in it, I raise this to a showstopper status.

The problem code is related to autoincrement code in the
partition handler.
[14 Aug 2006 10:19] Andrei Elkin
This bug seems to second what bug#20866 reported.
[31 Aug 2006 20:37] Mikael Ronström
fixed in 5.1-kt tree
[1 Sep 2006 7:52] Mikael Ronström
Patch will appear in 5.1.12
[2 Sep 2006 4:41] Jon Stephens
Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented bugfix in 5.1.12 changelog.
[15 Oct 2006 15:44] [ name withheld ]
Well, I hope this is fixed for MyISAM tables, too. Using MySQL 5.1.11-beta Intel compiled, I get this same error on the following table structure:

CREATE TABLE `tempsegments` (
  `pt` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `sc` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `hs` varchar(36) NOT NULL DEFAULT '',
  `fp` smallint(5) unsigned NOT NULL DEFAULT '0',
  `fx` smallint(5) unsigned NOT NULL DEFAULT '0',
  `px` smallint(5) unsigned NOT NULL DEFAULT '0',
  `nm` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `tp` int(10) unsigned NOT NULL DEFAULT '0',
  `xg` smallint(5) unsigned NOT NULL DEFAULT '0',
  `sv` int(10) unsigned NOT NULL DEFAULT '0',
  `sn` smallint(5) unsigned NOT NULL DEFAULT '0',
  `kb` smallint(5) unsigned NOT NULL DEFAULT '0',
  `et` varchar(5) NOT NULL DEFAULT '',
  `dx` varchar(20) NOT NULL DEFAULT '',
  `sj` varchar(150) NOT NULL DEFAULT '',
  `md` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY (`pt`,`sc`,`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 MAX_ROWS=4294967295 AVG_ROW_LENGTH=350 PARTITION BY HASH (`pt`) PARTITIONS 256 ;

`pt` is an integer between 0 and 255 created from php's hexdec on the 1st 2 chars of a sha1 hash, `sc` is another integer created from the 3rd and 4th characters of that hash, and the autoincrement id provides uniqueness. `hs` is the rest of the hash, but is not indexed to save index space. 

Trying to view table status on this table causes a crash every time. Trying to view the database itself in phpmyadmin causes a crash (phpmyadmin also says there are no tables in the database when this occurs), but viewing table status on other tables in the database is okay. This is the only partitioned table in the database.

If I recreate the table without the AUTO_INCREMENT and use php to generate an incremented `id`, everything works fine.

Here is my error log (repeated errors removed; first the SHOW TABLE STATUS one, then the view the db one):

mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=5
max_connections=100
threads_connected=4
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 269935 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x87bcd88
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb, backtrace may not be correct.
Bogus stack limit or frame pointer, fp=0xb, stack_bottom=0xb4450000, thread_stack=196608, aborting backtrace.
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x891c350 = SHOW TABLE STATUS LIKE 'tempsegments'
thd->thread_id=231

mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=4
max_connections=100
threads_connected=3
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 269935 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x883f038
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xb, backtrace may not be correct.
Bogus stack limit or frame pointer, fp=0xb, stack_bottom=0xb43b0000, thread_stack=196608, aborting backtrace.
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x882d448 = SELECT *,
                    `TABLE_SCHEMA`       AS `Db`,
                    `TABLE_NAME`         AS `Name`,
                    `ENGINE`             AS `Engine`,
                    `ENGINE`             AS `Type`,
                    `VERSION`            AS `Version`,
                    `ROW_FORMAT`         AS `Row_format`,
                    `TABLE_ROWS`         AS `Rows`,
                    `AVG_ROW_LENGTH`     AS `Avg_row_length`,
                    `DATA_LENGTH`        AS `Data_length`,
                    `MAX_DATA_LENGTH`    AS `Max_data_length`,
                    `INDEX_LENGTH`       AS `Index_length`,
                    `DATA_FREE`          AS `Data_free`,
                    `AUTO_INCREMENT`     AS `Auto_increment`,
                    `CREATE_TIME`        AS `Create_time`,
                    `UPDATE_TIME`        AS `Update_time`,
                    `CHECK_TIME`         AS `Check_time`,
                    `TABLE_COLLATION`    AS `Collation`,
                    `CHECKSUM`           AS `
thd->thread_id=16