Bug #47103 RBR slave crash in table_def::type when modifying a merge table
Submitted: 3 Sep 2009 15:57 Modified: 31 Oct 2009 17:05
Reporter: Matthias Leich Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Row Based Replication ( RBR ) Severity:S3 (Non-critical)
Version:5.4 OS:Any
Assigned to: Luis Soares
Tags: merge, RBR, regression
Triage: Triaged: D1 (Critical)

[3 Sep 2009 15:57] Matthias Leich
Description:
My script:
----------
--source include/master-slave.inc

--disable_warnings
DROP TABLE IF EXISTS t1,t2,t3;
--enable_warnings

CREATE TABLE t1 ( f1 integer , f2 int, primary key (f1)) ENGINE = MyISAM;
CREATE TABLE t2 ( f1 integer , f2 int, primary key (f1)) ENGINE = MyISAM;
CREATE TABLE t3 ( f1 integer , f2 int, primary key (f1)) ENGINE = MERGE UNION (t1,t2);
INSERT IGNORE INTO t2 VALUES  (1, 1);
# The UPDATE gets a crash with replication format mix and row
UPDATE t3  SET `f2` = 7 LIMIT 1;

SELECT * FROM t3;

BTW: It looks like the PRIMARY KEY is not needed.

My command line:
----------------
./mysql-test-run --mem --mysqld=--binlog-format=row \
    --no-check-testcases ml2301
or
./mysql-test-run --mem --mysqld=--binlog-format=mixed \
    --no-check-testcases ml2301

Backtrace from mysql-next-bugfixing 2009-09-03:
-----------------------------------------------
Thread 1 (process 1560):
#0  0x00007fdd40e2ece6 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000b57f50 in my_write_core (sig=11) at stacktrace.c:309
#2  0x00000000006efa81 in handle_segfault (sig=11) at mysqld.cc:2738
#3  <signal handler called>
#4  0x00000000008c48be in table_def::type (this=0x16e1c80, index=0) at rpl_utility.h:177
#5  0x00000000008c410f in table_def::compatible_with (this=0x16e1c80, rli_arg=0x15a2f40, table=0x169e478) at rpl_utility.cc:192
#6  0x000000000080e1ec in Rows_log_event::do_apply_event (this=0x16b2f38, rli=0x15a2f40) at log_event.cc:7323
#7  0x00000000008bcd2c in Log_event::apply_event (this=0x16b2f38, rli=0x15a2f40) at log_event.h:1080
#8  0x00000000008b45f3 in apply_event_and_update_pos (ev=0x16b2f38, thd=0x16d1ec8, rli=0x15a2f40, skip=true) at slave.cc:2339
#9  0x00000000008bb91b in exec_relay_log_event (thd=0x16d1ec8, rli=0x15a2f40) at slave.cc:2503
#10 0x00000000008bc411 in handle_slave_sql (arg=0x1538760) at slave.cc:3275
#11 0x00007fdd40e2a040 in start_thread () from /lib64/libpthread.so.0
#12 0x00007fdd3fdcb08d in clone () from /lib64/libc.so.6
#13 0x0000000000000000 in ?? ()

System reaction in mysql-5.1-bugteam late August 2009
-----------------------------------------------------
UPDATE t3  SET `f2` = 7 LIMIT 1;
+Warnings:
+Note   1592    Statement may not be safe to log in statement format.

I assume that the 5.4 server wants to give me the same
warning, but fails somehow during this.

How to repeat:
See above
[5 Sep 2009 10:10] Philip Stoev
There is another test case, with DELETE:

--source include/master-slave.inc
CREATE TABLE `table0_int_autoinc` ( `int` int, pk integer auto_increment, `int_key` int,        primary key (pk), key (`int_key` ));
CREATE TABLE `table10_int_autoinc` ( `int` int, pk integer auto_increment, `int_key` int,       primary key (pk), key (`int_key` ));
INSERT IGNORE INTO table10_int_autoinc VALUES  (NULL, NULL, '-474021888') ,  ('1', NULL, NULL) ,  ('1141047296', NULL, NULL) ,  (NULL, NULL, NULL) ,  (NULL, NULL, '1') ,  (NULL, NULL, '9') ,  ('0', NULL, '1225785344') ,  (NULL, NULL, '1574174720') ,  ('2', NULL, NULL) ,  ('6', NULL, '3');

CREATE TABLE IF NOT EXISTS t1_merge_1_A LIKE test . table0_int_autoinc;

ALTER TABLE t1_merge_1_A ENGINE = MERGE UNION ( table10_int_autoinc , table0_int_autoinc);

DELETE FROM t1_merge_1_A;
[21 Sep 2009 9:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/83860

2824 Luis Soares	2009-09-21
      BUG#47103: RBR slave crash in table_def::type when modifying a
      merge table
      
      When applying the table map event, a tables_to_lock list is
      updated on the rli internal state. This list extends the regular
      TABLE_LIST, by adding extra properties: a flag and a table
      definition reference. These fields are used later, when applying
      a rows event, to check whether the table definition on the slave
      is compatible with the one existing on the master.
      
      The problem is that when the slave thread opens the tables,
      defined on tables_to_lock, it passes the tables_to_lock as a
      parameter to the function that will actually perform the open
      tables (simple_open_n_lock_tables). Internally, this function
      appends to the list, references to the tables that are part of
      the merge table, whithout filling the table definition
      portion. Consequently, the slave crashes later when verifying the
      table compatibility because, it accesses the uninitialized table
      definition data.
      
      We fix this, by testing the flag m_tabledef_valid before
      actuallly checking whether tables are compatible or not. Tables
      that have valid table definitions (were filled when applying
      Table map events), will have this flag set.
     @ mysql-test/suite/rpl/t/rpl_row_merge_engine.test
        Test case.
     @ sql/log_event.cc
        Deployed a check to m_tabledef_valid before actually verifying
        table compatibility.
[30 Sep 2009 0:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85108

2824 Luis Soares	2009-09-30
      BUG#47103: RBR slave crash in table_def::type when modifying a
      merge table
      
      When applying the table map event, a tables_to_lock list is
      updated on the rli internal state. This list extends the regular
      TABLE_LIST, by adding extra properties: a flag and a table
      definition reference. These fields are used later, when applying
      a rows event, to check whether the table definition on the slave
      is compatible with the one existing on the master.
      
      The problem is that when the slave thread opens the tables,
      defined on tables_to_lock, it passes the tables_to_lock as a
      parameter to the function that will actually perform the open
      tables (simple_open_n_lock_tables). Internally, this function
      appends to the list, references to the tables that are part of
      the merge table, whithout filling the table definition
      portion. Consequently, the slave crashes later when verifying the
      table compatibility because, it accesses the uninitialized table
      definition data.
      
      We fix this, by checking only the table definitions that were set
      when processing the table map event. We do this by iterating over
      the table list until we reach the tables_to_lock_count
      variable. Additionally, we also deploy an assertion so that every
      table that is eligible to for table definition comparison, is
      asserted to have m_tabledef_valid set.
     @ mysql-test/suite/rpl/t/rpl_row_merge_engine.test
        Test case.
     @ sql/log_event.cc
        Extended loop so that its stop condition now considers the number
        of tables got in the Table_map_event. Additionally, a DBUG_ASSERT
        was deployed so that every table selected to the comparison procedure
        is asserted to have the m_tabledef_valid set.
[1 Oct 2009 13:36] Luis Soares
Checking, with a test case that combines both test from Matthias
and Philip, against the development trees, I have found that:

mysql-5.1-bugteam            => Test SUCCEEDS
mysql-next-mr                => Test SUCCEEDS
mysql-6.0-codebase-bugfixing => Test FAILS

So, bug is only repeatable in mysql-6.0-codebase-bugfixing.
[1 Oct 2009 14:55] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85398

2824 Luis Soares	2009-10-01
      BUG#47103: RBR slave crash in table_def::type when modifying a
      merge table
      
      When applying the table map event, a tables_to_lock list is
      updated on the rli internal state. This list extends the regular
      TABLE_LIST, by adding extra properties: a flag and a table
      definition reference. These fields are used later, when applying
      a rows event, to check whether the table definition on the slave
      is compatible with the one existing on the master.
      
      The problem is that when the slave thread opens the tables,
      defined on tables_to_lock, it passes the tables_to_lock as a
      parameter to the function that will actually perform the open
      tables (simple_open_n_lock_tables). Internally, this function
      appends to the list, references to the tables that are part of
      the merge table, whithout filling the table definition
      portion. Consequently, the slave crashes later when verifying the
      table compatibility because, it accesses the uninitialized table
      definition data.
      
      We fix this, by checking only the table definitions that were set
      when processing the table map event. We do this by iterating over
      the table list until we reach the tables_to_lock_count
      variable. Additionally, we also deploy an assertion so that every
      table that is eligible to for table definition comparison, is
      asserted to have m_tabledef_valid set.
     @ mysql-test/suite/rpl/t/rpl_row_merge_engine.test
        The two test cases from bug report as well as more robust test 
        case.
     @ sql/log_event.cc
        Extended loop so that its stop condition now considers the number
        of tables got in the Table_map_event. Additionally, a DBUG_ASSERT
        was deployed so that every table selected to the comparison procedure
        is asserted to have the m_tabledef_valid set.
[9 Oct 2009 9:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86321

3651 Luis Soares	2009-10-09 [merge]
      BUG#47103: automerge 6.0-cb-bf local bug branch --> 6.0-cb-bf up to date.
[9 Oct 2009 10:01] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86325

3653 Luis Soares	2009-10-09 [merge]
      BUG#47103: automerge 6.0-cb-bf local bug branch --> 6.0-cb-bf up to date.
[9 Oct 2009 10:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86326

3654 Luis Soares	2009-10-09 [merge]
      BUG#47103: automerge 6.0-cb-bf local bug branch --> 6.0-cb-bf up to date.
[9 Oct 2009 10:04] Luis Soares
Pushed to mysql-6.0-codebase-bugfixing.
[31 Oct 2009 8:17] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091031081410-qkxmjsdzjmj840aq) (version source revid:guilhem@mysql.com-20091010123356-qfouds7ucvkvbczt) (merge vers: 6.0.14-alpha) (pib:13)
[31 Oct 2009 17:05] Jon Stephens
Documented bugfix in the 6.0.14 changelog as follows:

        Updates to MERGE tables when using row-based replication could
        cause the slave to crash.

Closed.
[8 Jun 2011 19:54] Sveta Smirnova
Fix for this bug have not been backported to 5.5 series: see bug #61450 also
[21 Jul 2011 7:04] Shane Bester
see bug #61932