Bug #44108 Assertion bitmap_is_set in ha_partition::end_bulk_insert on slave
Submitted: 6 Apr 2009 9:47 Modified: 28 Oct 2009 14:40
Reporter: Philip Stoev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Partitions Severity:S1 (Critical)
Version:6.0-bzr,5.4 OS:Any
Assigned to: Mattias Jonsson CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[6 Apr 2009 9:47] Philip Stoev
Description:
When executing a row-based replication workload containing partition, the slave crashes immediately with the following backtrace:

mysqld: ha_partition.cc:3342: virtual int ha_partition::end_bulk_insert(bool): Assertion `bitmap_is_set(&m_bulk_insert_started, m_tot_parts)' failed.

/lib64/libc.so.6(__assert_fail+0xe9) [0x315a42bec9]
/build/bzr/mysql-6.0/sql/mysqld(ha_partition::end_bulk_insert(bool)+0x66) [0x828898]
/build/bzr/mysql-6.0/sql/mysqld(handler::ha_end_bulk_insert(bool)+0x36) [0x7875ec]
/build/bzr/mysql-6.0/sql/mysqld(Write_rows_log_event::do_after_row_operations(Slave_reporting_capability const*, int)+0xd3) [0x7d0583]
/build/bzr/mysql-6.0/sql/mysqld(Rows_log_event::do_apply_event(Relay_log_info const*)+0xc9d) [0x7da903]
/build/bzr/mysql-6.0/sql/mysqld(Log_event::apply_event(Relay_log_info const*)+0x28) [0x88beea]
/build/bzr/mysql-6.0/sql/mysqld(apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*, bool)+0x238) [0x8847b4]
/build/bzr/mysql-6.0/sql/mysqld [0x885bc5]
/build/bzr/mysql-6.0/sql/mysqld(handle_slave_sql+0x6ea) [0x886870]
/lib64/libpthread.so.0 [0x315b0073da]
/lib64/libc.so.6(clone+0x6d) [0x315a4e627d]

How to repeat:
Clone the test-extra-6.0 tree. Then run:

$ perl runall.pl \
--basedir=/build/bzr/mysql-6.0 \
--gendata=conf/replication_single_engine.zz \
--grammar=conf/replication_simple.yy \
--rpl_mode=row

The test will fail and you can observe that the slave is gone. All storage engines appear to be affected.

Please let me know if a simplified test case is required and/or you need the core and/or the binary log from this test.
[7 Apr 2009 10:05] Philip Stoev
This assertion was also observed on a stand-alone server with no replication. The only thing required is an INSERT ... SELECT + partitions:

CREATE TABLE `table10_innodb_key_pk_parts_2_int_autoinc` (`int` int,
`int_key` int,
pk integer auto_increment,
`char_key` char (1),
`char` char (1),
        key (`int_key` ),
primary key (pk),
key (`char_key` )) ENGINE=innodb PARTITION BY key (pk) partitions 2;
INSERT IGNORE INTO table10_innodb_key_pk_parts_2_int_autoinc VALUES  ('1', '9', NULL, 'f', 'j') ,  ('3', NULL, NULL, 'if', 'now') ,  ('-347056898', '6', NULL, 'for', 'v') ,  ('0', NULL, NULL, 'why', 'l') ,  (NULL, NULL, NULL, 'x', 't') ,  ('927586281', '1300333575', NULL, 'c', 'my') ,  ('1567750016', '1413675217', NULL, 'u', 'g') ,  ('0', '732412360', NULL, 'o', 'q') ,  ('-378868175', NULL, NULL, 'r', 'h') ,  ('6', '1437206324', NULL, 'a', 's');
INSERT INTO `table10_innodb_key_pk_parts_2_int_autoinc` ( `char` ) SELECT * FROM `table10_innodb_key_pk_parts_2_int_autoinc` LIMIT 32;
[2 Jun 2009 14:36] Mattias Jonsson
This assert is wrong. As the example show, select_insert::abort can call ha_end_bulk_insert without ha_start_bulk_insert being called.

Solution is to update the code removing this assumption and remove the assert.
[2 Jun 2009 16:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75477

3340 Mattias Jonsson	2009-06-02
      Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
      
      The assumption that start_bulk_insert must be called before end_bulk_insert
      was wrong, which lead to crash on assert.
      
      Solution was to remove the assertion and update the code to handle
      the situation where end_bulk_insert is called without any start call.
     @ mysql-test/r/partition_error.result
        Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
        
        Updated result
     @ mysql-test/t/partition_error.test
        Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
        
        Added test case
     @ sql/ha_partition.cc
        Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
        
        Removed the assert since the assumption of start_bulk_insert always
        was called before end_bulk_insert was wrong.
        
        Adding initailization and handling when start was not called before
        end_bulk_insert
[4 Jun 2009 11:27] Mattias Jonsson
Approved by Mikael
[23 Jul 2009 15:37] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/79203

3477 Mikael Ronstrom	2009-07-23
            Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
            
            The assumption that start_bulk_insert must be called before end_bulk_insert
            was wrong, which lead to crash on assert.
            
            Solution was to remove the assertion and update the code to handle
            the situation where end_bulk_insert is called without any start call.
           @ mysql-test/r/partition_error.result
              Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
              
              Updated result
           @ mysql-test/t/partition_error.test
              Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
              
              Added test case
           @ sql/ha_partition.cc
              Bug#44108: Assertion bitmap_is_set in ha_partition::end_bulk_insert
              
              Removed the assert since the assumption of start_bulk_insert always
              was called before end_bulk_insert was wrong.
              
              Adding initailization and handling when start was not called before
              end_bulk_insert
      modified:
        mysql-test/r/partition_error.result
        mysql-test/t/partition_error.test
        sql/ha_partition.cc
[14 Sep 2009 16:05] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (version source revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (merge vers: 5.4.4-alpha) (pib:11)
[16 Sep 2009 7:56] Jon Stephens
Documented bugfix in the 5.4.4 changelog as follows:

        Attempting to replicate an INSERT ... SELECT statement on a
        partitioned table using row-based replication caused the slave
        to crash.

Closed.
[7 Oct 2009 1:41] Paul Dubois
The 5.4 fix has been pushed into 5.4.3.
[21 Oct 2009 13:34] Philip Stoev
The original test case for this bug still fails against mysql-azalea-wl4571 . Re-opening this bug so that developers can verify that the proper stuff has been pushed to the proper trees.
[21 Oct 2009 19:37] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/87693

2830 Mattias Jonsson	2009-10-21
      port of the fix for bug#44108 into mysql-azalea-wl4571
[28 Oct 2009 14:40] Mattias Jonsson
Setting back to closed, since already pushed and documented in the appropriate versions