Bug #56172 Server crashes in ha_partition::reset on REBUILD PARTITION under LOCK TABLE
Submitted: 22 Aug 2010 20:59 Modified: 17 Dec 2010 5:35
Reporter: Elena Stepanova Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Partitions Severity:S2 (Serious)
Version:5.5.6-m3, 5.6.1-m4, 5.6.99 OS:Any
Assigned to: Mattias Jonsson
Tags: regression
Triage: Triaged: D1 (Critical)

[22 Aug 2010 20:59] Elena Stepanova
Description:
Version: '5.5.6-m3-debug-log'  
100822 23:49:59  InnoDB: Error: table `celosia_features`.`t_celosia_ddl_partitions` /* Temporary Partition `p0` */ does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?

100822 23:49:59 - mysqld got signal 11 ;

#2  0x00000000005588e5 in handle_segfault (sig=11)
    at mysql-5.5.6-m3/sql/mysqld.cc:2505
#3  <signal handler called>
#4  0x00002ad81be02ae7 in memset () from /lib64/libc.so.6
#5  0x0000000000a44d41 in ha_partition::reset (this=0x19e0000)
    at mysql-5.5.6-m3/sql/ha_partition.cc:5813
#6  0x000000000074795e in handler::ha_reset (this=0x19e0000)
    at mysql-5.5.6-m3/sql/handler.cc:4705
#7  0x000000000059ec24 in mark_used_tables_as_free_for_reuse (thd=0x19565e0, table=0x19e0660)
    at mysql-5.5.6-m3/sql/sql_base.cc:1271
#8  0x00000000005a08c5 in close_thread_tables (thd=0x19565e0)
    at mysql-5.5.6-m3/sql/sql_base.cc:1462
#9  0x00000000005f1dd1 in mysql_execute_command (thd=0x19565e0)
    at mysql-5.5.6-m3/sql/sql_parse.cc:4485
#10 0x00000000005f2106 in mysql_parse (thd=0x19565e0, rawbuf=0x1a10620 "ALTER TABLE t_celosia_ddl_partitions REBUILD PARTITION p0",
    length=57, parser_state=0x4588b9e0) at mysql-5.5.6-m3/sql/sql_parse.cc:5594
#11 0x00000000005f2ce8 in dispatch_command (command=COM_QUERY, thd=0x19565e0,
    packet=0x1a13351 "ALTER TABLE t_celosia_ddl_partitions REBUILD PARTITION p0", packet_length=57)
    at mysql-5.5.6-m3/sql/sql_parse.cc:1139
#12 0x00000000005f4147 in do_command (thd=0x19565e0)
    at mysql-5.5.6-m3/sql/sql_parse.cc:811
#13 0x00000000006c1cc6 in do_handle_one_connection (thd_arg=0x19565e0)
    at mysql-5.5.6-m3/sql/sql_connect.cc:1191
#14 0x00000000006c1d8f in handle_one_connection (arg=0x19565e0)
    at mysql-5.5.6-m3/sql/sql_connect.cc:1130
#15 0x000000000092a5a9 in pfs_spawn_thread (arg=0x19c3e40)
    at mysql-5.5.6-m3/storage/perfschema/pfs.cc:1015
#16 0x00002ad81b7e0143 in start_thread () from /lib64/libpthread.so.0
#17 0x00002ad81be4f8cd in clone () from /lib64/libc.so.6
#18 0x0000000000000000 in ?? ()

thd->query at 0x1a10620 = ALTER TABLE t_celosia_ddl_partitions REBUILD PARTITION p0
thd->thread_id=2
thd->killed=NOT_KILLED

Reproducible on current 5.5 and 5.6 bugfixing trees, debug and non-debug binaries. 
Not reproducible on 5.5.5-m3, 5.6.0-m4, 5.1.50, 5.1-bugteam (hence regression tag).

How to repeat:
--source include/have_innodb.inc

CREATE DATABASE celosia_features;
USE celosia_features;

SET GLOBAL innodb_file_format = Barracuda,
  innodb_file_per_table = ON,
  innodb_strict_mode = ON;

CREATE TABLE celosia_features.t_celosia_ddl_partitions
(id INT NOT NULL PRIMARY KEY)
  ENGINE = InnoDB  KEY_BLOCK_SIZE = 4
  PARTITION BY HASH(id) PARTITIONS 2;

SET GLOBAL innodb_file_per_table = OFF;

LOCK TABLE t_celosia_ddl_partitions WRITE;
ALTER TABLE t_celosia_ddl_partitions REBUILD PARTITION p0;

UNLOCK TABLES;
DROP DATABASE celosia_features;

--exit
[21 Sep 2010 23:31] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/118766

3210 Mattias Jonsson	2010-09-22
      Bug#56172: Server crashes in ha_partition::reset on
                 REBUILD PARTITION under LOCK TABLE
      
      When ALTER PARTITION failed under LOCK TABLE it could crash the server,
      due to it internally altered the locked table directly,
      which was not reverted in case of failure, resulting in bad table definition.
      
      Solved by instead of altering the locked table object and
      its partition_info struct, creating an internal temporary
      intermediate table object used for altering,
      just like the non partitioned mysql_alter_table.
      So if an error occur before the alter operation is complete,
      the original table is not modified at all.
      But if the alter operation have succeeded so far that it
      must be completed as whole,
      the table is properly closed and reopened.
      (The completion on failure is done by the ddl_log.)
     @ mysql-test/suite/parts/inc/partition_fail.inc
        Added tests under LOCK TABLE
     @ mysql-test/suite/parts/r/partition_debug_innodb.result
        Updated results
     @ mysql-test/suite/parts/r/partition_debug_myisam.result
        Updated results
     @ mysql-test/suite/parts/r/partition_special_innodb.result
        updated result
     @ mysql-test/suite/parts/t/partition_debug_innodb-master.opt
        Allowing stack-trace and core-files since the use of _exit() instead of abort().
     @ mysql-test/suite/parts/t/partition_debug_myisam-master.opt
        Allowing stack-trace and core-files since the use of _exit() instead of abort().
     @ mysql-test/suite/parts/t/partition_special_innodb.test
        changing comment, since this patch also fixes this.
     @ sql/sql_partition.cc
        Simulating crash with _exit() instead of abort() to avoid core-files on expected crashes.
        Removed unused arguments to fast_end_partition.
        Opening a intermediate table in prep_alter_part_table, instead of altering
        (a possible locked) normally opened table.
        That way we do not have to do anything more than close
        the intermediate table on error,
        leaving the ordinary table opened and locked.
        Also making sure that the intermediate table are
        closed/destroyed on failure. If no error occur
        it is later destroyed in the end of fast_alter_partition_table.
        Added ha_external_lock to make sure MyISAM flushed the index file
        after copying the partitions.
        This also leads to removal of the special close and removal from
        the table cache for other instances of the table.
     @ sql/sql_partition.h
        Changed the arguments for prep_alter_part_table and
        fast_alter_partition_table to use an intermediate table
        instead of altering a (possibly locked) normal table.
     @ sql/sql_table.cc
        Using an intermediate table created in prep_alter_part_table
        to be used in fast_alter_partition_table, also closing/destroying
        it on failure.
[28 Sep 2010 10:22] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/119248

3211 Mattias Jonsson	2010-09-28
      Bug#56172: Server crashes in ha_partition::reset on REBUILD PARTITION under LOCK TABLE
      
      Small correction of error handling before review.
      
      In case ha_change_partitions failed, the error would
      have been overwritten by the return code of ha_external_lock.
[30 Sep 2010 10:45] Mattias Jonsson
Mikael approved it with some minor comments.
[1 Oct 2010 10:58] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/119624

3210 Mattias Jonsson	2010-10-01
      Bug#56172: Server crashes in ha_partition::reset on
                 REBUILD PARTITION under LOCK TABLE
      
      Collapsed patch including updates from the reviews.
      
      In case of failure in ALTER ... PARTITION under LOCK TABLE
      the server could crash, due to it had modified the locked
      table object, which was not reverted in case of failure,
      resulting in a bad table definition used after the failed
      command.
      
      Solved by instead of altering the locked table object and
      its partition_info struct, creating an internal temporary
      intermediate table object used for altering,
      just like the non partitioned mysql_alter_table.
      So if an error occur before the alter operation is complete,
      the original table is not modified at all.
      But if the alter operation have succeeded so far that it
      must be completed as whole,
      the table is properly closed and reopened.
      (The completion on failure is done by the ddl_log.)
     @ mysql-test/suite/parts/inc/partition_fail.inc
        Added tests under LOCK TABLE
     @ mysql-test/suite/parts/r/partition_debug_innodb.result
        Updated results
     @ mysql-test/suite/parts/r/partition_debug_myisam.result
        Updated results
     @ mysql-test/suite/parts/r/partition_special_innodb.result
        updated result
     @ mysql-test/suite/parts/t/partition_special_innodb.test
        changing comment, since this patch also fixes this.
     @ sql/sql_partition.cc
        Added TODO, to use DBUG_SUICIDE() instead of abort()
        to avoid core-files on expected crashes.
        Removed unused arguments to fast_end_partition.
        Opening a intermediate table in prep_alter_part_table, instead of altering
        (a possible locked) normally opened table.
        That way we do not have to do anything more than close
        the intermediate table on error,
        leaving the ordinary table opened and locked.
        Also making sure that the intermediate table are
        closed/destroyed on failure. If no error occur
        it is later destroyed in the end of fast_alter_partition_table.
        Added ha_external_lock to make sure MyISAM flushed the index file
        after copying the partitions.
        This also leads to removal of the special close and removal from
        the table cache for other instances of the table.
     @ sql/sql_partition.h
        Changed the arguments for prep_alter_part_table and
        fast_alter_partition_table to use an intermediate table
        instead of altering a (possibly locked) normal table.
     @ sql/sql_table.cc
        Using an intermediate table created in prep_alter_part_table
        to be used in fast_alter_partition_table, also closing/destroying
        it on failure.
[1 Oct 2010 11:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/119629

3210 Mattias Jonsson	2010-10-01
      Bug#56172: Server crashes in ha_partition::reset on
                 REBUILD PARTITION under LOCK TABLE
      
      Collapsed patch including updates from the reviews.
      
      In case of failure in ALTER ... PARTITION under LOCK TABLE
      the server could crash, due to it had modified the locked
      table object, which was not reverted in case of failure,
      resulting in a bad table definition used after the failed
      command.
      
      Solved by instead of altering the locked table object and
      its partition_info struct, creating an internal temporary
      intermediate table object used for altering,
      just like the non partitioned mysql_alter_table.
      So if an error occur before the alter operation is complete,
      the original table is not modified at all.
      But if the alter operation have succeeded so far that it
      must be completed as whole,
      the table is properly closed and reopened.
      (The completion on failure is done by the ddl_log.)
     @ mysql-test/suite/parts/inc/partition_fail.inc
        Added tests under LOCK TABLE
     @ mysql-test/suite/parts/r/partition_debug_innodb.result
        Updated results
     @ mysql-test/suite/parts/r/partition_debug_myisam.result
        Updated results
     @ mysql-test/suite/parts/r/partition_special_innodb.result
        updated result
     @ mysql-test/suite/parts/t/partition_special_innodb.test
        changing comment, since this patch also fixes this.
     @ sql/sql_partition.cc
        Added TODO, to use DBUG_SUICIDE() instead of abort()
        to avoid core-files on expected crashes.
        Removed unused arguments to fast_end_partition.
        Opening a intermediate table in prep_alter_part_table, instead of altering
        (a possible locked) normally opened table.
        That way we do not have to do anything more than close
        the intermediate table on error,
        leaving the ordinary table opened and locked.
        Also making sure that the intermediate table are
        closed/destroyed on failure. If no error occur
        it is later destroyed in the end of fast_alter_partition_table.
        Added ha_external_lock to make sure MyISAM flushed the index file
        after copying the partitions.
        This also leads to removal of the special close and removal from
        the table cache for other instances of the table.
     @ sql/sql_partition.h
        Changed the arguments for prep_alter_part_table and
        fast_alter_partition_table to use an intermediate table
        instead of altering a (possibly locked) normal table.
     @ sql/sql_table.cc
        Using an intermediate table created in prep_alter_part_table
        to be used in fast_alter_partition_table, also closing/destroying
        it on failure.
[1 Oct 2010 19:52] Mattias Jonsson
Pushed into mysql-5.5-bugteam and mysql-trunk-merge.
[4 Oct 2010 13:01] Jon Stephens
Still waiting for merge to 5.5-main, set back to Need Merge.
[9 Nov 2010 19:47] Bugs System
Pushed into mysql-5.5 5.5.7-rc (revid:sunanda.menon@sun.com-20101109182959-otkxq8vo2dcd13la) (version source revid:sunanda.menon@sun.com-20101109182959-otkxq8vo2dcd13la) (merge vers: 5.5.7-rc) (pib:21)
[13 Nov 2010 16:08] Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:alexander.nozdrin@oracle.com-20101113152450-2zzcm50e7i4j35v7) (merge vers: 5.6.1-m4) (pib:21)
[13 Nov 2010 16:37] Bugs System
Pushed into mysql-next-mr (revid:alexander.nozdrin@oracle.com-20101113160336-atmtmfb3mzm4pz4i) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (pib:21)
[16 Nov 2010 14:40] Jon Stephens
Documented in the 5.5.7 changelog as follows:

        An ALTER TABLE statement acting on table partitions that failed
        while the affected table was locked could cause the server to
        crash.

Closed.
[15 Dec 2010 5:52] Bugs System
Pushed into mysql-5.1 5.1.55 (revid:sunanda.menon@oracle.com-20101215054055-vgwki317xg1wphhh) (version source revid:sunanda.menon@oracle.com-20101215054055-vgwki317xg1wphhh) (merge vers: 5.1.55) (pib:23)
[16 Dec 2010 21:47] Bugs System
Pushed into mysql-trunk 5.6.1 (revid:alexander.nozdrin@oracle.com-20101216181820-7afubgk2fmuv9qsb) (version source revid:alexander.nozdrin@oracle.com-20101216173826-ze3y5h450sksotrh) (merge vers: 5.6.1) (pib:23)
[16 Dec 2010 22:32] Bugs System
Pushed into mysql-5.5 5.5.9 (revid:jonathan.perkin@oracle.com-20101216101358-fyzr1epq95a3yett) (version source revid:jonathan.perkin@oracle.com-20101216101358-fyzr1epq95a3yett) (merge vers: 5.5.9) (pib:24)
[17 Dec 2010 5:35] Jon Stephens
Changelog entry also added to 5.1.54 changelog.

No additional changelog entries required.

Closed.