MySQL Bugs: #51868: crash with myisam_use_mmap and partitioned myisam tables

Bug #51868	crash with myisam_use_mmap and partitioned myisam tables
Submitted:	9 Mar 2010 13:58	Modified:	18 Jun 2010 2:10
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: MyISAM storage engine	Severity:	S1 (Critical)
Version:	5.1.37,5.1.44,5.5.2-m2	OS:	Any
Assigned to:	Sergey Vojtovich	CPU Architecture:	Any
Tags:	crash, myisam_use_mmap, regression

Description:
5.1.44 stack trace:
mysqld.exe!memcpy()[memcpy.asm:326]
mysqld.exe!mi_mmap_pread()[mi_dynrec.c:155]
mysqld.exe!_mi_read_static_record()[mi_statrec.c:182]
mysqld.exe!mi_rkey()[mi_rkey.c:173]
mysqld.exe!ha_myisam::index_read_map()[ha_myisam.cc:1652]
mysqld.exe!ha_partition::handle_unordered_scan_next_partition()[ha_partition.cc:4556]
mysqld.exe!ha_partition::common_index_read()[ha_partition.cc:4081]
mysqld.exe!handler::index_read_idx_map()[handler.cc:4313]
mysqld.exe!write_record()[sql_insert.cc:1450]
mysqld.exe!select_insert::send_data()[sql_insert.cc:3180]
mysqld.exe!end_send()[sql_select.cc:12150]
mysqld.exe!evaluate_join_record()[sql_select.cc:11374]
mysqld.exe!sub_select()[sql_select.cc:11253]
mysqld.exe!do_select()[sql_select.cc:11004]
mysqld.exe!JOIN::exec()[sql_select.cc:2271]
mysqld.exe!mysql_select()[sql_select.cc:2461]
mysqld.exe!st_select_lex_unit::exec()[sql_union.cc:603]
mysqld.exe!mysql_union()[sql_union.cc:35]
mysqld.exe!handle_select()[sql_select.cc:273]
mysqld.exe!mysql_execute_command()[sql_parse.cc:3245]
mysqld.exe!mysql_parse()[sql_parse.cc:5975]
mysqld.exe!dispatch_command()[sql_parse.cc:1235]
mysqld.exe!do_command()[sql_parse.cc:874]
mysqld.exe!handle_one_connection()[sql_connect.cc:1127]
mysqld.exe!pthread_start()[my_winthread.c:85]
mysqld.exe!_callthreadstart()[thread.c:295]
mysqld.exe!_threadstart()[thread.c:275]
kernel32.dll!BaseThreadStart()

How to repeat:
#start server with --myisam-use-mmap=1

set sql_mode='';
drop table if exists t24;
create table `t24` (`col1` date not null,unique (`col1`)) engine=myisam
partition by linear key (col1) partitions 1;
replace into t24 select '0000-00-00' union all select '0000-00-00';
flush tables;
truncate t24;
replace into t24 select '0000-00-00' union all select '0000-00-00';

100309 11:24:59  InnoDB: Started; log sequence number 0 44233
100309 11:25:00 [Note] Event Scheduler: Loaded 0 events
100309 11:25:00 [Note] dbs/5.1/libexec/mysqld: ready for connections.
Version: '5.1.46-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
100309 11:25:38 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338309 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x2bb2ed8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f5f855c3eb8 thread_stack 0x40000
dbs/5.1/libexec/mysqld(my_print_stacktrace+0x35)[0xb2644d]
dbs/5.1/libexec/mysqld(handle_segfault+0x288)[0x699541]
/lib64/libpthread.so.0[0x3d6ec0f0f0]
/lib64/libc.so.6(memcpy+0x35)[0x3d6e482f35]
dbs/5.1/libexec/mysqld(mi_mmap_pread+0xb5)[0xa0b1a4]
dbs/5.1/libexec/mysqld(_mi_read_static_record+0x8e)[0xa0609a]
dbs/5.1/libexec/mysqld(mi_rkey+0x700)[0x9fb758]
dbs/5.1/libexec/mysqld(_ZN9ha_myisam14index_read_mapEPhPKhm16ha_rkey_function+0x8d)[0x9f0eff]
dbs/5.1/libexec/mysqld(_ZN12ha_partition36handle_unordered_scan_next_partitionEPh+0x1db)[0x8018fb]
dbs/5.1/libexec/mysqld(_ZN12ha_partition17common_index_readEPhb+0x25f)[0x800873]
dbs/5.1/libexec/mysqld(_ZN12ha_partition14index_read_mapEPhPKhm16ha_rkey_function+0xdf)[0x8005fd]
dbs/5.1/libexec/mysqld(_ZN7handler18index_read_idx_mapEPhjPKhm16ha_rkey_function+0x7b)[0x7f45fb]
dbs/5.1/libexec/mysqld(_Z12write_recordP3THDP8st_tableP12st_copy_info+0x421)[0x74c032]
dbs/5.1/libexec/mysqld(_ZN13select_insert9send_dataER4ListI4ItemE+0x214)[0x75117a]
dbs/5.1/libexec/mysqld[0x738526]
dbs/5.1/libexec/mysqld[0x7365d8]
dbs/5.1/libexec/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x179)[0x73627e]
dbs/5.1/libexec/mysqld[0x735d89]
dbs/5.1/libexec/mysqld(_ZN4JOIN4execEv+0x26bb)[0x71ecf7]
dbs/5.1/libexec/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x343)[0x71f4b9]
dbs/5.1/libexec/mysqld(_ZN18st_select_lex_unit4execEv+0xa91)[0x876beb]
dbs/5.1/libexec/mysqld(_Z11mysql_unionP3THDP6st_lexP13select_resultP18st_select_lex_unitm+0x99)[0x874a8d]
dbs/5.1/libexec/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0xd9)[0x717117]
dbs/5.1/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x36c3)[0x6ae652]
dbs/5.1/libexec/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x2c2)[0x6b6b5b]
dbs/5.1/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xd52)[0x6a9199]
dbs/5.1/libexec/mysqld(_Z10do_commandP3THD+0x27e)[0x6a8150]
dbs/5.1/libexec/mysqld(handle_one_connection+0x14c)[0x6a6451]
/lib64/libpthread.so.0[0x3d6ec06a3a]
/lib64/libc.so.6(clone+0x6d)[0x3d6e4de67d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x7f5f74004b88 is an invalid pointer
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
[miguel@hegel ~]$

Thank you for the bug report.

miguel, that 5.1.23 version did 'The variable 'MYSQLlval'' for everything you do, so it's irrelevant :P  5.1.37 crashes, but 5.1.34 does not. therefore, a regression.

Crash repeatable with 5.1.36:

C:\bugs\mysql-5.1.23>bin\mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.36-community-debug MySQL Community Server - Debug (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> set sql_mode='';
Query OK, 0 rows affected (0.00 sec)

mysql> drop table if exists t24;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> create table `t24` (`col1` date not null,unique (`col1`)) engine=myisam
    -> partition by linear key (col1) partitions 1;
Query OK, 0 rows affected (0.14 sec)

mysql> replace into t24 select '0000-00-00' union all select '0000-00-00';
Query OK, 3 rows affected (0.02 sec)
Records: 2  Duplicates: 1  Warnings: 0

mysql> flush tables;
Query OK, 0 rows affected (0.00 sec)

mysql> truncate t24;
Query OK, 0 rows affected (0.00 sec)

mysql> replace into t24 select '0000-00-00' union all select '0000-00-00';
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/103722

3408 Sergey Vojtovich	2010-03-18
      BUG#51868 - crash with myisam_use_mmap and partitioned
                  myisam tables
      
      Queries following TRUNCATE of partitioned MyISAM table
      may crash server if myisam_use_mmap is true.
      
      Internally this is MyISAM bug, but limited to partitioned
      tables, because MyISAM doesn't use ::delete_all_rows()
      method for TRUNCATE, but goes via table recreate instead.
      
      The code was not durable to mmap() failure, that is state
      for non-mmaped read/write was not fully restored. Was not
      repeatable on linux before, likely because (quote from man
      mmap):
        SUSv3  specifies  that  mmap() should fail if length is 0.
        However, in kernels before 2.6.12, mmap() succeeded in
        this case: no mapping was created and the call returned
        addr. Since kernel 2.6.12, mmap() fails with the error
        EINVAL for this case.
     @ mysql-test/r/partition.result
        A test case for BUG#51868.
     @ mysql-test/t/partition.test
        A test case for BUG#51868.
     @ storage/myisam/mi_delete_all.c
        _mi_unmap_file() is compressed record format specific,
        which is read-only. As compressed MyISAM data files are
        read-only, we must never use _mi_unmap_file() in
        mi_delete_all_rows().
     @ storage/myisam/mi_dynrec.c
        Make myisam mmap code more durable to errors:
        - set file_read/file_write handlers if mmap succeeded;
        - reset file_read/file_write handlers on unmap.
     @ storage/myisam/mi_extra.c
        Moved file_read/file_write handlers initialization to
        mi_dynmap_file().
     @ storage/myisam/myisamdef.h
        Added mi_munmap_file() declaration.

Approved with suggestions. Please see email.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/103973

3408 Sergey Vojtovich	2010-03-22
      BUG#51868 - crash with myisam_use_mmap and partitioned
                  myisam tables
      
      Queries following TRUNCATE of partitioned MyISAM table
      may crash server if myisam_use_mmap is true.
      
      Internally this is MyISAM bug, but limited to partitioned
      tables, because MyISAM doesn't use ::delete_all_rows()
      method for TRUNCATE, but goes via table recreate instead.
      
      MyISAM didn't properly fall back to non-mmaped I/O after
      mmap() failure. Was not repeatable on linux before, likely
      because (quote from man mmap):
        SUSv3  specifies  that  mmap() should fail if length is 0.
        However, in kernels before 2.6.12, mmap() succeeded in
        this case: no mapping was created and the call returned
        addr. Since kernel 2.6.12, mmap() fails with the error
        EINVAL for this case.
     @ mysql-test/r/partition.result
        A test case for BUG#51868.
     @ mysql-test/t/partition.test
        A test case for BUG#51868.
     @ storage/myisam/mi_delete_all.c
        _mi_unmap_file() is compressed record format specific,
        which is read-only. As compressed MyISAM data files are
        read-only, we must never use _mi_unmap_file() in
        mi_delete_all_rows().
     @ storage/myisam/mi_dynrec.c
        Make myisam mmap code more durable to errors:
        - set file_read/file_write handlers if mmap succeeded;
        - reset file_read/file_write handlers on unmap.
     @ storage/myisam/mi_extra.c
        Moved file_read/file_write handlers initialization to
        mi_dynmap_file().
     @ storage/myisam/myisamdef.h
        Added mi_munmap_file() declaration.

Pushed into 5.1.46 (revid:sergey.glukhov@sun.com-20100405111026-7kz1p8qlzglqgfmu) (version source revid:svoj@sun.com-20100326102110-mamv34eqvztt2jx2) (merge vers: 5.1.46) (pib:16)

Noted in 5.1.46 changelog.

After TRUNCATE TABLE of a MyISAM table, subsequent queries could
crash the server if myisam_use_mmap was enabled. 

Setting report to Need Merge pending push to Celosia.

Pushed into mysql-next-mr (revid:alik@sun.com-20100524190136-egaq7e8zgkwb9aqi) (version source revid:alik@sun.com-20100422150750-vp0n37kp9ywq5ghf) (pib:16)

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100524190941-nuudpx60if25wsvx) (version source revid:alik@sun.com-20100422150658-fkhgnwwkyugtxrmu) (merge vers: 6.0.14-alpha) (pib:16)

Pushed into 5.5.5-m3 (revid:alik@sun.com-20100524185725-c8k5q7v60i5nix3t) (version source revid:alexey.kopytov@sun.com-20100403083753-yswcxwe63nyqeac4) (merge vers: 5.5.4-m3) (pib:16)

Noted in 5.5.5, 6.0.14 changelogs.

Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)

Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609211156-tsac5qhw951miwtt) (merge vers: 5.1.46-ndb-6.2.19) (pib:16)

Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)