Bug #17084 Crash during execution of datadict test suite in stress mode (ICC compiler)
Submitted: 2 Feb 2006 22:18 Modified: 10 Aug 2006 21:10
Reporter: Alexey Stroganov Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S1 (Critical)
Version:5.0.17a-cert OS:Linux (RHAS4(IA64))
Assigned to: Assigned Account CPU Architecture:Any

[2 Feb 2006 22:18] Alexey Stroganov 
Description:
Server crashes during execution of set of test cases from datadict test suite  in stress mode. I would note that symptoms above was observed on RHAS4(IA64) box only with binary which was built with icc compiler.

During investiagtion  I found that datadict part of funcs_1 test suite cause a crash. Datadict test consists of 13 complex test cases and  I was able to reduced set of test cases from 13 to 3 but further reducing is pretty complex due to logic of remain test cases. 

I observed crashes of server in two points:

1) Crash happens in most of the cases in this place:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 2305843009356296832 (LWP 7448)]
row_upd_changes_some_index_ord_field_binary (table=0x1, update=0x0) at row0upd.c:1079
1079    in row0upd.c
(gdb) bt
#0  row_upd_changes_some_index_ord_field_binary (table=0x1, update=0x0) at row0upd.c:1079
#1  0x400000000073fc30 in pars_process_assign_list (node=0x1) at pars0pars.c:909
#2  0x400000000073b0f0 in pars_update_statement (node=0x1, cursor_sym=0x0, search_cond=0x20000000007ae0e8)
    at pars0pars.c:968
#3  0x400000000074cb30 in $$1$2_0$TAG$6a$0$1 () at pars0grm.y:357
#4  0x4000000000749e80 in pars_sql (str=0x0) at pars0pars.c:1760
#5  0x4000000000702f40 in row_rename_table_for_mysql (old_name=0x0,
    new_name=0x25794baf <Address 0x25794baf out of bounds>, trx=0x0) at row0mysql.c:3783
#6  0x400000000053bbb0 in ha_innobase::rename_table (this=0x0, from=0x60000000001d2e88 "/",
    to=0x2000000008f9a740 "db_datadict") at ha_innodb.cc:5207
#7  0x4000000000559d90 in mysql_rename_table (base=142588384, old_db=0x20 <Address 0x20 out of bounds>,
    old_name=0x4000000000afe9e0 "%s2-%lx-%lx", new_db=0x60000000001d2f08 "#sql",
    new_name=0x183d <Address 0x183d out of bounds>) at sql_table.cc:1877
#8  0x4000000000557070 in mysql_alter_table (thd=0x0, new_db=0x0, new_name=0x0, create_info=0x0,
    table_list=0x0, fields=@0x0, keys=@0x0, order_num=0, order=0x0, handle_duplicates=DUP_ERROR, ignore=false,
    alter_info=0x20000000087fbe10, do_send_ok=true) at sql_table.cc:3805
#9  0x40000000002e8c40 in mysql_create_index (thd=0x60000000001d7aa0, table_list=0x20000000087fbe50, keys=@0x0)
    at sql_parse.cc:7117
#10 0x40000000002dee90 in $$1$312_0$TAG$2$0$311 () at sql_parse.cc:2922
#11 0x40000000002edf40 in mysql_parse (thd=0x60000000001dd1f0, inBuf=0x0, length=258760) at sql_parse.cc:5628
#12 0x40000000002d52f0 in $$1$307_0$TAG$DEFAULT$0$306 () at sql_parse.cc:1715
#13 0x40000000002ce8b0 in handle_one_connection (arg=0x0) at sql_parse.cc:1511
#14 0x2000000000081d10 in start_thread () from /lib/tls/libpthread.so.0
#15 0x200000000057f310 in __clone2 () from /lib/tls/libc.so.6.1

2) I see crash in  this place only 2 or 3 time:

mysqld: lock.cc:944: bool wait_if_global_read_lock(THD *, bool, bool):
Assertion ! (&LOCK_open)->count || ! pthread_equal(pthread_self(),
(&LOCK_open)->thread)' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 2305843009336586880 (LWP 14252)]
0xa000000000010641 in __kernel_syscall_via_break ()
(gdb) bt
#0  0xa000000000010641 in __kernel_syscall_via_break ()
#1  0x20000000004373d0 in raise () from /lib/tls/libc.so.6.1
#2  0x2000000000439c90 in abort () from /lib/tls/libc.so.6.1
#3  0x2000000000427c40 in __assert_fail () from /lib/tls/libc.so.6.1
#4  0x4000000000293220 in wait_if_global_read_lock () at lock.cc:944
#5  0x4000000000540a40 in mysql_create_db (thd=0x0, db=0x3 <Address 0x3 out of bounds>, create_info=0x3ffffff,
    silent=true) at sql_db.cc:421
#6  0x40000000002e4c20 in $$1$312_0$TAG$DEFAULT$0$311 () at sql_parse.cc:3602
#7  0x40000000002edf40 in mysql_parse (thd=0x60000000001dd1f0, inBuf=0x0, length=258760) at sql_parse.cc:5628
#8  0x40000000002d52f0 in $$1$307_0$TAG$DEFAULT$0$306 () at sql_parse.cc:1715
#9  0x40000000002ce8b0 in handle_one_connection (arg=0x0) at sql_parse.cc:1511
#10 0x2000000000081d10 in start_thread () from /lib/tls/libpthread.so.0
#11 0x200000000057f310 in __clone2 () from /lib/tls/libc.so.6.1

How to repeat:
Ran test case against servers on RHAS4(IA64) box.
[2 Feb 2006 23:29] Heikki Tuuri 
Alexey,

this bug was introduced by Ingo to the debug version a couple of months ago, and I believe it is already fixed in 5.0.19:

mysqld: lock.cc:944: bool wait_if_global_read_lock(THD *, bool, bool):
Assertion ! (&LOCK_open)->count || ! pthread_equal(pthread_self(),
(&LOCK_open)->thread)' failed.

Regards,

Heikki
[2 Feb 2006 23:33] Heikki Tuuri 
Alexey,

I wonder if the debugger is showing this right:

#6  0x400000000053bbb0 in ha_innobase::rename_table (this=0x0,
from=0x60000000001d2e88 "/",
    to=0x2000000008f9a740 "db_datadict") at ha_innodb.cc:5207

The table names are nonsensical. Can you look inside the debugger what is the SQL statement that MySQL is executing?

Regards,

Heikki
[2 Feb 2006 23:36] Heikki Tuuri 
Alexey,

it would also be nice to know the value of str in InnoDB's internal SQL parser:

#4  0x4000000000749e80 in pars_sql (str=0x0) at pars0pars.c:1760
#5  0x4000000000702f40 in row_rename_table_for_mysql (old_name=0x0,
    new_name=0x25794baf <Address 0x25794baf out of bounds>, trx=0x0) at
row0mysql.c:3783

Regards,

Heikki
[27 Jul 2006 16:36] Heikki Tuuri 
Lowering the priority of this since there is no feedback.
[29 Jul 2006 5:43] Heikki Tuuri 
This might be an ICC compiler bug since this crash was only observed on Itanium with a binary built with that compiler.
[10 Aug 2006 21:10] Heikki Tuuri 
Putting this to the 'No feedback' state.