Bug #37598 Falcon deadlock with 4 sessions doing deletes to different rows
Submitted: 24 Jun 2008 1:21 Modified: 12 Sep 2008 20:24
Reporter: Mark Callaghan Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S3 (Non-critical)
Version:6.0.5alpha, 6.0.6 bzr OS:Any
Assigned to: Kevin Lewis CPU Architecture:Any
Tags: deadlock, falcon

[24 Jun 2008 1:21] Mark Callaghan
Description:
I tried to see if this is a duplicate, but there were 175 hits for '+falcon +deadlock'.

I built Falcon 6.0.5alpha from source using gcc 4.0.3 and glibc 2.3.6 with NPTL. My server is a 4 core AMD (2 dual core chips) with model name Dual-Core AMD Opteron(tm) Processor 8214 HE.

The test is sysbench run as:
sysbench --test oltp --mysql-host 127.0.0.1 --mysql-user root --mysql-password  --mysql-db test --oltp-table-size 1000000 --max-time 300 --max-requests 0 --mysql-table-engine falcon --db-ps-mode disable --mysql-engine-trx yes --oltp-table-size 1000000 --num-threads 4 run

Each session is stuck on a delete statement:
+----+------+-----------------+------+---------+------+----------+------------------------------------+
| Id | User | Host            | db   | Command | Time | State    | Info                               |
+----+------+-----------------+------+---------+------+----------+------------------------------------+
|  9 | root | localhost:41850 | test | Query   | 7692 | updating | DELETE from sbtest where id=429385 |
| 10 | root | localhost:41851 | test | Query   | 7692 | updating | DELETE from sbtest where id=495785 |
| 11 | root | localhost:41852 | test | Query   | 7692 | updating | DELETE from sbtest where id=499269 |
| 12 | root | localhost:41853 | test | Query   | 7692 | updating | DELETE from sbtest where id=495743 |

How to repeat:
See above
[24 Jun 2008 1:22] Mark Callaghan
Thread stacks:

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x083bc97c in SyncObject::wait ()
#4  0x083bcba8 in SyncObject::lock ()
#5  0x083bb9a1 in Sync::lock ()
#6  0x083c46ff in Table::validateAndInsert ()
#7  0x083c53a8 in Table::deleteRecord ()
#8  0x083b2258 in StorageDatabase::deleteRow ()
#9  0x083b6c5e in StorageTable::deleteRow ()
#10 0x083a9d3b in StorageInterface::delete_row ()
#11 0x08307a18 in handler::ha_delete_row ()
#12 0x082b3230 in mysql_delete ()
#13 0x08229a76 in mysql_execute_command ()
#14 0x0822cfbe in mysql_parse ()
#15 0x0822e022 in dispatch_command ()
#16 0x0822e437 in do_command ()
#17 0x0821dec6 in handle_one_connection ()
#18 0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#19 0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x083bc97c in SyncObject::wait ()
#4  0x083bcba8 in SyncObject::lock ()
#5  0x083bb9a1 in Sync::lock ()
#6  0x083c46ff in Table::validateAndInsert ()
#7  0x083c53a8 in Table::deleteRecord ()
#8  0x083b2258 in StorageDatabase::deleteRow ()
#9  0x083b6c5e in StorageTable::deleteRow ()
#10 0x083a9d3b in StorageInterface::delete_row ()
#11 0x08307a18 in handler::ha_delete_row ()
#12 0x082b3230 in mysql_delete ()
#13 0x08229a76 in mysql_execute_command ()
#14 0x0822cfbe in mysql_parse ()
#15 0x0822e022 in dispatch_command ()
#16 0x0822e437 in do_command ()
#17 0x0821dec6 in handle_one_connection ()
#18 0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#19 0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x083bc97c in SyncObject::wait ()
#4  0x083bcba8 in SyncObject::lock ()
#5  0x083bb9a1 in Sync::lock ()
#6  0x083c46ff in Table::validateAndInsert ()
#7  0x083c53a8 in Table::deleteRecord ()
#8  0x083b2258 in StorageDatabase::deleteRow ()
#9  0x083b6c5e in StorageTable::deleteRow ()
#10 0x083a9d3b in StorageInterface::delete_row ()
#11 0x08307a18 in handler::ha_delete_row ()
#12 0x082b3230 in mysql_delete ()
#13 0x08229a76 in mysql_execute_command ()
#14 0x0822cfbe in mysql_parse ()
#15 0x0822e022 in dispatch_command ()
#16 0x0822e437 in do_command ()
#17 0x0821dec6 in handle_one_connection ()
#18 0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#19 0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x083bc97c in SyncObject::wait ()
#4  0x083bcba8 in SyncObject::lock ()
#5  0x083bb9a1 in Sync::lock ()
#6  0x083c46ff in Table::validateAndInsert ()
#7  0x083c53a8 in Table::deleteRecord ()
#8  0x083b2258 in StorageDatabase::deleteRow ()
#9  0x083b6c5e in StorageTable::deleteRow ()
#10 0x083a9d3b in StorageInterface::delete_row ()
#11 0x08307a18 in handler::ha_delete_row ()
#12 0x082b3230 in mysql_delete ()
#13 0x08229a76 in mysql_execute_command ()
#14 0x0822cfbe in mysql_parse ()
#15 0x0822e022 in dispatch_command ()
#16 0x0822e437 in do_command ()
#17 0x0821dec6 in handle_one_connection ()
#18 0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#19 0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
[24 Jun 2008 1:26] Mark Callaghan
A few more thread stacks:
#1  0x4df1e82e in do_sigwait () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x4df1e8cf in sigwait () from /lib/tls/i686/cmov/libpthread.so.0
#3  0x08214ded in signal_hand ()
#4  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#5  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x084ffd20 in os_event_wait ()
#3  0x085303b7 in srv_master_thread ()
#4  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#5  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4de62171 in select () from /lib/tls/i686/cmov/libc.so.6
#2  0x0850088d in os_thread_sleep ()
#3  0x0852fd2f in srv_error_monitor_thread ()
#4  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#5  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4de62171 in select () from /lib/tls/i686/cmov/libc.so.6
#2  0x0850088d in os_thread_sleep ()
#3  0x0852f6a2 in srv_lock_timeout_and_monitor_thread ()
#4  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#5  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x084ffd20 in os_event_wait ()
#3  0x084fecc0 in os_aio_simulated_handle ()
#4  0x084cb74c in fil_aio_wait ()
#5  0x08531150 in io_handler_thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x084ffd20 in os_event_wait ()
#3  0x084fecc0 in os_aio_simulated_handle ()
#4  0x084cb74c in fil_aio_wait ()
#5  0x08531150 in io_handler_thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x084ffd20 in os_event_wait ()
#3  0x084fecc0 in os_aio_simulated_handle ()
#4  0x084cb74c in fil_aio_wait ()
#5  0x08531150 in io_handler_thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x084ffd20 in os_event_wait ()
#3  0x084fecc0 in os_aio_simulated_handle ()
#4  0x084cb74c in fil_aio_wait ()
#5  0x08531150 in io_handler_thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08441f6a in Scheduler::schedule ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x083bc97c in SyncObject::wait ()
#4  0x083bcba8 in SyncObject::lock ()
#5  0x083bb9a1 in Sync::lock ()
#6  0x084233c9 in RecordLeaf::retireRecords ()
#7  0x0847e530 in RecordGroup::retireRecords ()
#8  0x0847e530 in RecordGroup::retireRecords ()
#9  0x0847e530 in RecordGroup::retireRecords ()
#10 0x083bd805 in Table::retireRecords ()
#11 0x083ed451 in Database::retireRecords ()
#12 0x083ed9e8 in Database::scavenge ()
#13 0x08440b58 in Scavenger::execute ()
#14 0x08441ee4 in Scheduler::schedule ()
#15 0x083ca3d4 in Thread::thread ()
#16 0x083ca600 in Thread::thread ()
#17 0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#18 0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08473262 in Gopher::gopherThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08473262 in Gopher::gopherThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08473262 in Gopher::gopherThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6

#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08473262 in Gopher::gopherThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x08473262 in Gopher::gopherThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x0841f300 in PageWriter::writer ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1af6c in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463559 in Synchronize::sleep ()
#3  0x0846345a in Synchronize::sleep ()
#4  0x083eccb3 in Database::ticker ()
#5  0x083ca3d4 in Thread::thread ()
#6  0x083ca600 in Thread::thread ()
#7  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#8  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x0846ef2a in Cache::ioThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
#1  0x4df1acf6 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/tls/i686/cmov/libpthread.so.0
#2  0x08463413 in Synchronize::sleep ()
#3  0x0846ef2a in Cache::ioThread ()
#4  0x083ca3d4 in Thread::thread ()
#5  0x083ca600 in Thread::thread ()
#6  0x4df183a8 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#7  0x4de697fe in clone () from /lib/tls/i686/cmov/libc.so.6
[24 Jun 2008 1:29] Mark Callaghan
There is nothing special in the db error log.

show engine falcon status returns nothing. I assume this has not been implemented. That is an odd decision given that debugging features are sorely needed by Falcon (it is new code).

Innodb has a great feature where it dumps lock status info in the db error log when there is a long lock wait from an internal deadlock such as this. That would make debugging much easier.

show engine falcon mutex also appears to be unimplemented. Why make debugging that much harder?
[24 Jun 2008 1:42] Mark Callaghan
Non default my.cnf vars for Falcon
falcon_page_cache_size=400M
falcon_record_memory_max=400M
skip_falcon_consistent_read
[24 Jun 2008 1:45] Mark Callaghan
I corrected my bug search syntax -- there are only 8 bugs matching +falcon +deadlock. Possibly a duplicate of http://bugs.mysql.com/bug.php?id=36526
[24 Jun 2008 2:32] Mark Callaghan
Wow. I ran the test again and it is hung again.
[24 Jun 2008 13:42] Mark Callaghan
Output with falcon_debug_mask=2
Stalled threads
  Thread 0xb2e78ba8 (-755586128) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf40184a0 (-755385424) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ef0 (-755786832) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf401c608 (-143410256) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
Stalled synchronization objects:
  SyncObject f435d2b8: state 1, monitor 0, waiters 4
    Waiting thread b2e78ba8 (-755586128), type 1; Table::validateAndInsert
    Waiting thread f40184a0 (-755385424), type 1; Table::validateAndInsert
    Waiting thread b2e78ef0 (-755786832), type 1; Table::validateAndInsert
    Waiting thread f401c608 (-143410256), type 1; Table::validateAndInsert
------------------------------------
Stalled threads
  Thread 0xf4009608 (-738116688) sleep=0, grant=0, locks=1, who 0, parent=0xf3ffd890
    Pending RecordLeaf::retireRecords2 state 0 (1) syncObject 0xf435d680
Stalled synchronization objects:
  SyncObject f435d680: state 1, monitor 0, waiters 1
    Waiting thread f4009608 (-738116688), type 1; RecordLeaf::retireRecords2
------------------------------------
------------------------------------
Stalled threads
  Thread 0xf40184a0 (-755385424) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ba8 (-755586128) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ef0 (-755786832) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf401c608 (-143410256) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
Stalled synchronization objects:
  SyncObject f435d2b8: state 1, monitor 0, waiters 4
    Waiting thread b2e78ba8 (-755586128), type 1; Table::validateAndInsert
    Waiting thread f40184a0 (-755385424), type 1; Table::validateAndInsert
    Waiting thread b2e78ef0 (-755786832), type 1; Table::validateAndInsert
    Waiting thread f401c608 (-143410256), type 1; Table::validateAndInsert
------------------------------------
Stalled threads
  Thread 0xb2e78ef0 (-755786832) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ba8 (-755586128) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf40184a0 (-755385424) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf401c608 (-143410256) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
Stalled synchronization objects:
  SyncObject f435d2b8: state 1, monitor 0, waiters 4
    Waiting thread b2e78ba8 (-755586128), type 1; Table::validateAndInsert
    Waiting thread f40184a0 (-755385424), type 1; Table::validateAndInsert
    Waiting thread b2e78ef0 (-755786832), type 1; Table::validateAndInsert
    Waiting thread f401c608 (-143410256), type 1; Table::validateAndInsert
------------------------------------
Stalled threads
  Thread 0xf401c608 (-143410256) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ba8 (-755586128) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xf40184a0 (-755385424) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
  Thread 0xb2e78ef0 (-755786832) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf435d2b8
Stalled synchronization objects:
  SyncObject f435d2b8: state 1, monitor 0, waiters 4
    Waiting thread b2e78ba8 (-755586128), type 1; Table::validateAndInsert
    Waiting thread f40184a0 (-755385424), type 1; Table::validateAndInsert
    Waiting thread b2e78ef0 (-755786832), type 1; Table::validateAndInsert
    Waiting thread f401c608 (-143410256), type 1; Table::validateAndInsert
[24 Jun 2008 14:04] Mark Callaghan
After changing from skip_falcon_consistent_read to falcon_consistent_read I still get hangs every time.

------------------------------------
Stalled threads
  Thread 0xabfbbeb0 (-1143653456) sleep=0, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf453e2b8
  Thread 0xabfbc0f8 (-1143452752) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf453e2b8
  Thread 0xabfbc340 (-1143854160) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf453e2b8
  Thread 0xabfbc588 (-1144054864) sleep=1, grant=0, locks=1, who 0, parent=(nil)
    Pending Table::validateAndInsert state 0 (1) syncObject 0xf453e2b8
Stalled synchronization objects:
  SyncObject f453e2b8: state 1, monitor 0, waiters 4
    Waiting thread abfbc0f8 (-1143452752), type 1; Table::validateAndInsert
    Waiting thread abfbc340 (-1143854160), type 1; Table::validateAndInsert
    Waiting thread abfbbeb0 (-1143653456), type 1; Table::validateAndInsert
    Waiting thread abfbc588 (-1144054864), type 1; Table::validateAndInsert
[24 Jun 2008 18:20] Sveta Smirnova
Thank you for the report.

Verified as described. To repeat I increased max-time to 3000.
[26 Jun 2008 0:19] Kevin Lewis
This deadlock is actually very similar to Bug#37395
[1 Jul 2008 17:32] Mark Callaghan
I ran the test with 1, 2, 3 and 4 concurrent sessions. The hang does not occur for 1, 2 or 3.
[12 Sep 2008 20:24] Kevin Lewis
This bug is similar to Bug#37395 in that is refers to a deadlock involving Table::validateAndInsert.  Deadlocks with this function were happening from several sources a few months ago, but not any more.

Evaluating these call stacks in light of the information from the newly completed Deadlock Detection Tool (syncHandler.cpp) does not show any reason for a deadlock between the threads documented.  There have been numerous changes in the code where certain syncObjects are locked since this bug was reported.  This is true for the call stacks given in this bug.  So it seems that we should close this bug as "Can't Repeat" until a similar deadlock occurs again.

If it happens again, please provide a full call stack of all threads and the version of code you were running.