Bug #12326 rpl_row_basic_* (myisam & innodb) causes core in slave mysqld process
Submitted: 2 Aug 2005 20:20 Modified: 15 Sep 2005 15:52
Reporter: Jonathan Miller Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Replication Severity:S1 (Critical)
Version:5.0-wl1012 OS:Linux (Linux)
Assigned to: Lars Thalmann CPU Architecture:Any

[2 Aug 2005 20:20] Jonathan Miller
Description:
rpl_row_basic gets the following bt:

0x818a964 handle_segfault + 452
0x401855cd _end + 934034941
0x40297662 _end + 935157394
0x402968a0 _end + 935153872
0x84b6e24 my_malloc + 36
0x81822da _Z15prepare_pendingI21Delete_rows_log_eventEP14Rows_log_eventP3THDP8st_tablejPK9st_bitmapjjb + 90
0x81807d1 _ZN3THD10delete_rowEP8st_tablebPK9st_bitmapjPKc + 241
0x8235ace _ZN7handler13ha_delete_rowEPKc + 254
0x820b620 _ZN21Delete_rows_log_event11do_exec_rowEP8st_tableP17st_relay_log_info + 64
0x820a507 _ZN14Rows_log_event10exec_eventEP17st_relay_log_info + 1063
0x828484f _Z20exec_relay_log_eventP3THDP17st_relay_log_info + 575
0x82825a8 handle_slave_sql + 1048
0x4017f6de _end + 934010638
0x402fe6c7 _end + 935579383

How to repeat:
pull clone wl1012 and compile using compile-pentium-max.
run mysql-test-run as follows:
./mysql-test-run --mysqld=--binlog-format=row --do-test=rpl_row_basic_2myisam --force --skip-ndb 

or
./mysql-test-run --mysqld=--binlog-format=row --do-test=rpl_row_basic_3innodb --force --skip-ndb
[7 Sep 2005 11:37] Lars Thalmann
This is a duplicate of BUG#12134.
[7 Sep 2005 17:39] Jonathan Miller
Currently seems to still core on slave since update.

Back Trace as follows:

 0x00e05402 in __kernel_vsyscall ()
#1  0x00a2855f in pthread_kill () from /lib/libpthread.so.0
#2  0x082c46fb in write_core (sig=20252) at stacktrace.c:220
#3  0x0819c8b2 in handle_segfault (sig=11) at mysqld.cc:2062
#4  <signal handler called>
#5  0x084d8b5c in mi_scan (info=0x8ddf808, buf=0x8ddf7c8 "�\230\235")
    at mi_scan.c:45
#6  0x08263f3b in ha_myisam::rnd_next (this=0x8dfcb88,
    buf=0x8ddf7c8 "�\230\235") at ha_myisam.cc:1244
#7  0x08224782 in find_and_fetch_row (table=0x8e0e3f8,
    key=0x8ddf7d0 "�XX\003", record_buf=0x8ddf7c8 "�\230\235")
    at log_event.cc:5711
#8  0x08224a2a in Delete_rows_log_event::do_exec_row (this=0x8ddf7c8,
    table=0x8e0e3f8, rli=0x8dd0574) at log_event.cc:5866
#9  0x082240d8 in Rows_log_event::exec_event (this=0x8ddfa50, rli=0x8dd0574)
    at log_event.cc:5105
#10 0x082b9e6a in handle_slave_sql (arg=0x8dca390) at slave.cc:2960
#11 0x00a25b80 in start_thread () from /lib/libpthread.so.0
#12 0x0097d9ce in clone () from /lib/libc.so.6

Also get:
rpl_row_basic_2myisam           *** glibc detected *** /home/ndbdev/jmiller/mysql5.0/sql/mysqld: malloc(): memory corruption: 0x08ddf7e0 ***
[ fail ]

Thanks!
[7 Sep 2005 17:42] Jonathan Miller
rpl_row_basic_3innodb           *** glibc detected *** /home/ndbdev/jmiller/mysql5.0/sql/mysqld: malloc(): memory corruption: 0x0920e2a0 ***
======= Backtrace: =========
/lib/libc.so.6[0x1760ea]
/lib/libc.so.6(malloc+0x74)[0x177492]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(my_malloc+0x27)[0x8515087]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(_ZN3THD33binlog_prepare_pending_rows_eventI21Delete_rows_log_eventEEP14Rows_log_eventP8st_tablejPK9st_bitmapjjb+0x6f)[0x819379f]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(_ZN3THD17binlog_delete_rowEP8st_tablebPK9st_bitmapjPKc+0x15b)[0x8191d0b]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(_Z14binlog_log_rowI21Delete_rows_log_eventEbP8st_tablePKcPc+0xef)[0x825f99f]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(_ZN7handler13ha_delete_rowEPKc+0x3c)[0x825e17c]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(_ZN14Rows_log_event10exec_eventEP17st_relay_log_info+0x3e8)[0x82240d8]
/home/ndbdev/jmiller/mysql5.0/sql/mysqld(handle_slave_sql+0x6fa)[0x82b9e6a]
/lib/libpthread.so.0[0xa25b80]
/lib/libc.so.6(__clone+0x5e)[0x1dc9ce]
======= Memory map: ========
00111000-00112000 rwxp 00111000 00:00 0
00112000-00235000 r-xp 00000000 03:02 1926370    /lib/libc-2.3.5.so
00235000-00237000 r-xp 00123000 03:02 1926370    /lib/libc-2.3.5.so
00237000-00239000 rwxp 00125000 03:02 1926370    /lib/libc-2.3.5.so
00239000-0023c000 rwxp 00239000 00:00 0
00244000-00373000 rwxp 00244000 00:00 0
00376000-00377000 r-xp 00376000 00:00 0
00377000-004c7000 rwxp 00377000 00:00 0
004c7000-004c8000 --xp 004c7000 00:00 0
004c8000-004f8000 rwxp 004c8000 00:00 0
0050d000-0050e000 rwxp 0050d000 00:00 0
0050e000-0050f000 --xp 0050e000 00:00 0
0050f000-0053f000 rwxp 0050f000 00:00 0
0053f000-00540000 --xp 0053f000 00:00 0
00540000-00570000 rwxp 00540000 00:00 0
00570000-00571000 --xp 00570000 00:00 0
00571000-005a1000 rwxp 00571000 00:00 0
005ba000-005bb000 rwxp 005ba000 00:00 0
005bb000-005bc000 --xp 005bb000 00:00 0
005bc000-005ec000 rwxp 005bc000 00:00 0
0064a000-0074f000 rwxp 0064a000 00:00 0
00755000-007c1000 rwxp 00755000 00:00 0
007ed000-007ee000 rwxp 007ed000 00:00 0
00895000-008af000 r-xp 00000000 03:02 1924000    /lib/ld-2.3.5.so
008af000-008b0000 r-xp 00019000 03:02 1924000    /lib/ld-2.3.5.so
008b0000-008b1000 rwxp 0001a000 03:02 1924000    /lib/ld-2.3.5.so
008d9000-008da000 rwxp 008d9000 00:00 0
009de000-009e0000 r-xp 00000000 03:02 1926373    /lib/libdl-2.3.5.so
009e0000-009e1000 r-xp 00001000 03:02 1926373    /lib/libdl-2.3.5.so
009e1000-009e2000 rwxp 00002000 03:02 1926373    /lib/libdl-2.3.5.so
009e4000-00a07000 r-xp 00000000 03:02 1926374    /lib/libm-2.3.5.so
00a07000-00a08000 r-xp 00022000 03:02 1926374    /lib/libm-2.3.5.so
00a08000-00a09000 rwxp 00023000 03:02 1926374    /lib/libm-2.3.5.so
00a0b000-00a1d000 r-xp 00000000 03:02 985062     /usr/lib/libz.so.1.2.2.2
00a1d000-00a1e000 rwxp 00011000 03:02 985062     /usr/lib/libz.so.1.2.2.2
00a20000-00a2e000 r-xp 00000000 03:02 1926371    /lib/libpthread-2.3.5.so
00a2e000-00a2f000 r-xp 0000d000 03:02 1926371    /lib/libpthread-2.3.5.so
00a2f000-00a30000 rwxp 0000e000 03:02 1926371    /lib/libpthread-2.3.5.so
00a30000-00a32000 rwxp 00a30000 00:00 0
00a34000-00a43000 r-xp 00000000 03:02 1923938    /lib/libresolv-2.3.5.so
00a43000-00a44000 r-xp 0000e000 03:02 1923938    /lib/libresolv-2.3.5.so
00a44000-00a45000 rwxp 0000f000 03:02 1923938    /lib/libresolv-2.3.5.so
00a45000-00a47000 rwxp 00a45000 00:00 0
00a5c000-00a61000 r-xp 00000000 03:02 1926384    /lib/libcrypt-2.3.5.so
00a61000-00a62000 r-xp 00004000 03:02 1926384    /lib/libcrypt-2.3.5.so
00a62000-00a63000 rwxp 00005000 03:02 1926384    /lib/libcrypt-2.3.5.so
00a63000-00a8a000 rwxp 00a63000 00:00 0
00a8c000-00a8e000 r-xp 00000000 03:02 1926377    /lib/libcom_err.so.2.1
00a8e000-00a8f000 rwxp 00001000 03:02 1926377    /lib/libcom_err.so.2.1
00a9b000-00aa4000 r-xp 00000000 03:02 1926375    /lib/libgcc_s-4.0.1-20050727.so.1
00aa4000-00aa5000 rwxp 00009000 03:02 1926375    /lib/libgcc_s-4.0.1-20050727.so.1
00aa7000-00aa9000 r-xp 00000000 03:02 990591     /usr/lib/libkrb5support.so.0.0
00aa9000-00aaa000 rwxp 00001000 03:02 990591     /usr/lib/libkrb5support.so.0.0
00aac000-00abe000 r-xp 00000000 03:02 1924002    /lib/libnsl-2.3.5.so
00abe000-00abf000 r-xp 00011000 03:02 1924002    /lib/libnsl-2.3.5.so
00abf000-00ac0000 rwxp 00012000 03:02 1924002    /lib/libnsl-2.3.5.so
00ac0000-00ac2000 rwxp 00ac0000 00:00 0
00b48000-00b49000 --xp 00b48000 00:00 0
00b49000-00b79000 rwxp 00b49000 00:00 0
00b99000-00bb0000 r-xp 00000000 03:02 990594     /usr/lib/libgssapi_krb5.so.2.2
00bb0000-00bb1000 rwxp 00017000 03:02 990594     /usr/lib/libgssapi_krb5.so.2.2
00bb3000-00cab000 r-xp 00000000 03:02 1926378    /lib/libcrypto.so.0.9.7f
00cab000-00cbd000 rwxp 000f8000 03:02 1926378    /lib/libcrypto.so.0.9.7f
00cbd000-00cc0000 rwxp 00cbd000 00:00 0
00cc2000-00ce5000 r-xp 00000000 03:02 990592     /usr/lib/libk5crypto.so.3.0
00ce5000-00ce6000 rw[ fail ]
Errors are (from /home/ndbdev/jmiller/mysql5.0/mysql-test/var/log/mysqltest-time) :
/home/ndbdev/jmiller/mysql5.0/client/.libs/lt-mysqltest: line 118: failed in select master_pos_wait('master-bin.000001', 2716): 2013: Lost connection to MySQL server during query
-------back trace ------
#0  0x00376402 in __kernel_vsyscall ()
#1  0x00a2855f in pthread_kill () from /lib/libpthread.so.0
#2  0x082c46fb in write_core (sig=21302) at stacktrace.c:220
#3  0x0819c8b2 in handle_segfault (sig=6) at mysqld.cc:2062
#4  <signal handler called>
#5  0x00376402 in __kernel_vsyscall ()
#6  0x0013a118 in raise () from /lib/libc.so.6
#7  0x0013b888 in abort () from /lib/libc.so.6
#8  0x0016f22a in __libc_message () from /lib/libc.so.6
#9  0x001760ea in _int_malloc () from /lib/libc.so.6
#10 0x00177492 in malloc () from /lib/libc.so.6
#11 0x08515087 in my_malloc (size=124, my_flags=24) at my_malloc.c:35
#12 0x0819379f in THD::binlog_prepare_pending_rows_event<Delete_rows_log_event>
    (this=0x9238730, table=0x920d6a8, serv_id=1, cols=0x5eb1d0, colcnt=3,
    needed=9, is_transactional=true) at log_event.h:619
#13 0x08191d0b in THD::binlog_delete_row (this=0x9238730, table=0x920d6a8,
    is_trans=true, cols=0x5eb1d0, colcnt=0, record=0x920de70 "�XX\003")
    at sql_class.cc:2390
#14 0x0825f99f in binlog_log_row<Delete_rows_log_event> (table=0x920d6a8,
    before_record=0x6 <Address 0x6 out of bounds>, after_record=0x0)
    at log_event.h:2059
#15 0x0825e17c in handler::ha_delete_row (this=0x920dd48,
    buf=0x920de70 "�XX\003") at handler.cc:2557
---Type <return> to continue, or q <return> to quit---
#16 0x082240d8 in Rows_log_event::exec_event (this=0x920e208, rli=0x92007d4)
    at log_event.cc:5105
#17 0x082b9e6a in handle_slave_sql (arg=0x91fa5f0) at slave.cc:2960
#18 0x00a25b80 in start_thread () from /lib/libpthread.so.0
#19 0x001dc9ce in clone () from /lib/libc.so.6
[11 Sep 2005 23:16] Lars Thalmann
The described failure no longer happens.  
The reason why Innodb fails is captured in BUG#13090.
[12 Sep 2005 11:55] Jonathan Miller
still cause core on slave.
[15 Sep 2005 14:30] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/29913
[15 Sep 2005 15:52] Lars Thalmann
Fixed
[15 Sep 2005 15:57] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/29929