Bug #24919 InnoDB: lock_clust_rec_read_check_and_lock() leads to slave crash
Submitted: 8 Dec 2006 17:21 Modified: 3 Jul 2007 17:42
Reporter: Valeriy Kravchuk Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S2 (Serious)
Version:4.1.21, 4.1.23-BK OS:Linux (Linux)
Assigned to: Inaam Rana CPU Architecture:Any

[8 Dec 2006 17:21] Valeriy Kravchuk
Description:
I wonder whether the intention lock check in lock_clust_rec_read_check_and_lock() is really needed:

ut_ad(mode != LOCK_X
|| lock_table_has(thr_get_trx(thr), index->table, LOCK_IX));
ut_ad(mode != LOCK_S
|| lock_table_has(thr_get_trx(thr), index->table, LOCK_IS));

if (!page_rec_is_supremum(rec)) {

lock_rec_convert_impl_to_expl(rec, index);
}

If I enable innodb and read commited mode in the slave, mysql test rpl000001 crashes in debug builds of MySQL 4.1.21 and 4.1.23-BK:

openxs@suse:~/dbs/4.1/mysql-test> ./mysql-test-run rpl000001
Logging: ./mysql-test-run rpl000001
MySQL Version 4.1.23
...
Starting Tests in the 'main' suite

TEST RESULT TIME (ms)
-------------------------------------------------------

rpl000001 [ fail ]

Errors are (from /home/openxs/dbs/4.1/mysql-test/var/log/mysqltest-time) :
mysqltest: At line 97: Query failed while probing slave for stop: Lost connection to MySQL server during query
(the last lines may be the most important ones)
Result from queries before failure can be found in r/rpl000001.log

Aborting: rpl000001 failed in default mode. To continue, re-run with '--force'.
Stopping All Servers
openxs@suse:~/dbs/4.1/mysql-test> cat r/rpl000001.log
stop slave;
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
...
create table t2(id int);
insert into t2 values(connection_id());
create temporary table t3(n int);
insert into t3 select get_lock('crash_lock%20C', 1) from t2;
update t1 set n = n + get_lock('crash_lock%20C', 2);
select (@id := id) - id from t2;
(@id := id) - id
0
kill @id;
drop table t2;
ERROR 08S01: Server shutdown in progress

In the error log we have:

0x815cd56
0x833e84d
0x82976c3
0x829e46c
0x82016de
0x8201a18
0x81ef032
0x81aae12
0x81aa8cb
0x819e97d
0x819eee3
0x819a64e
0x8174701
0x8177b3c
0x81ceeed
0x82365b1
0x8237993
0x40050aa7
0x40247c2e
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow
instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8c590cd = insert into t3 select get_lock('crash_lock%20C', 1) from t2
thd->thread_id=10

Resolved stack trace is:

0x815cd56 handle_segfault + 412
0x833e84d lock_clust_rec_read_check_and_lock + 1410
0x82976c3 sel_set_rec_lock + 94
0x829e46c row_search_for_mysql + 3294
0x82016de _ZN11ha_innobase13general_fetchEPcjj + 386
0x8201a18 _ZN11ha_innobase8rnd_nextEPc + 136
0x81ef032 _Z13rr_sequentialP14st_read_record + 40
0x81aae12 _Z10sub_selectP4JOINP13st_join_tableb + 536
0x81aa8cb _Z9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure + 987
0x819e97d _ZN4JOIN4execEv + 6157
0x819eee3 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orde
rSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 575
0x819a64e _Z13handle_selectP3THDP6st_lexP13select_result + 270
0x8174701 _Z21mysql_execute_commandP3THD + 9281
0x8177b3c _Z11mysql_parseP3THDPcj + 330
0x81ceeed _ZN15Query_log_event10exec_eventEP17st_relay_log_info + 543
0x82365b1 _Z20exec_relay_log_eventP3THDP17st_relay_log_info + 769
0x8237993 handle_slave_sql + 1363
0x40050aa7 _end + 933620263
0x40247c2e _end + 935680942

How to repeat:
Make the following changes in mysql-test-run script (for slave server). Comment out these lines (near line 3547 in current 4.1.23-BK):

  if ( $type eq 'slave' )
  {

...

#    mtr_add_arg($args, "%s--skip-innodb", $prefix);
#    mtr_add_arg($args, "%s--skip-ndbcluster", $prefix);
#    mtr_add_arg($args, "%s--skip-slave-start", $prefix);

Then add two following lines after them:

    mtr_add_arg($args, "%s--default-storage-engine=innodb", $prefix);
    mtr_add_arg($args, "%s--transaction-isolation=READ-COMMITTED", $prefix);

Save script and run the rpl000001 test from MySQL test suite.

Suggested fix:
I don't know.
[9 Dec 2006 12:09] Valeriy Kravchuk
Bug #24910 was marked a a duplicate of this one.
[7 Jun 2007 13:57] Heikki Tuuri
Assigning this to Inaam.

Please find out if this still exists in 5.1.

I guess the assertion itself is too strong?

Please fix in 5.1 if this still is there.

--Heikki
[3 Jul 2007 17:42] Inaam Rana
Works fine in 5.0.
[10 Sep 2010 15:28] Vasil Dimov
Could be related to Bug#56716 mode != LOCK_X || lock_table_has(thr_get_trx(thr), index->table, LOCK_IX)