MySQL Bugs: #36330: Falcon DBT2 crash in Transaction::needToLock

Bug #36330	Falcon DBT2 crash in Transaction::needToLock
Submitted:	25 Apr 2008 3:42	Modified:	8 Jan 2009 10:01
Reporter:	Christopher Powers	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S2 (Serious)
Version:	6.0.5	OS:	Linux
Assigned to:	Christopher Powers	CPU Architecture:	Any

Description:
DBT2 crashes in Transaction::needToLock(). Likely due to lack of syncPrior protection surrounding scan of prior record version chain.

How to repeat:
Backtrac:

080424 23:10:35 [Note] Plugin 'InnoDB' disabled by command line option
080424 23:10:35 [Note] Event Scheduler: Loaded 0 events
080424 23:10:35 [Note] /data0/work/mysql/mysql-6.0-falcon/sql/mysqld: ready for connections.
Version: '6.0.6-alpha'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(print_stacktrace+0x27)[0x7805c7]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(handle_segfault+0x336)[0x638b46]
/lib64/libpthread.so.0[0x2ae416d0dc10]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN11Transaction10needToLockEP6Record+0x46)[0x7f25e6]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN5Table14fetchForUpdateEP11TransactionP6Recordb+0x48)[0x7e6068]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN15StorageDatabase11nextIndexedEP12StorageTablePvib+0x64)[0x7d6574]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN16StorageInterface10index_nextEPh+0xaf)[0x7d1b8f]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN16StorageInterface10index_readEPhPKhj16ha_rkey_function+0xd7)[0x7cfc57]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN7handler18index_read_idx_mapEPhjPKhm16ha_rkey_function+0x71)[0x7285a1]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld[0x6b1d94]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld[0x6b1fbe]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld[0x6b3980]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_ZN4JOIN8optimizeEv+0x531)[0x6b48a1]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x135)[0x6be4d5]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x172)[0x6beef2]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld[0x643caa]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z21mysql_execute_commandP3THD+0x1ab5)[0x648f25]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x32f)[0x64f1bf]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x8dc)[0x64faac]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(_Z10do_commandP3THD+0xc6)[0x6506b6]
/data0/work/mysql/mysql-6.0-falcon/sql/mysqld(handle_one_connection+0xf4)[0x641374]
/lib64/libpthread.so.0[0x2ae416d06143]
/lib64/libc.so.6(__clone+0x6d)[0x2ae41744b74d]
080424 23:35:24 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388608
read_buffer_size=1048576
max_used_connections=32
max_threads=1601
thread_count=32
connection_count=32
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4942811 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x12a6b40
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x12b2060 = SELECT d_tax, d_next_o_id
FROM district 
WHERE d_w_id = 14
  AND d_id = 2
FOR UPDATE
thd->thread_id=38
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
Fatal signal 6 while backtracing
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.
mysqld: my_new.cc:50: int __cxa_pure_virtual(): Assertion `"Pure virtual method called." == "Aborted"' failed.

Suggested fix:
Add syncPrior shared lock to Transaction::needToLock().

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45982

ChangeSet@1.2645, 2008-04-24 22:44:18-05:00, cpowers@xeno.mysql.com +4 -0
  Bug #36330: Falcon DBT2 crash in Transaction::needToLock
  - Added syncPrior protection to Transaction::needToLock() 
  - Removed RecordVersion::getSyncPrior()
  - Record::getSyncPrior() now returns syncObject

The crash is most likely caused by an unprotected scan of prior record versions. 

Traversals of prior record version lists must be protected by either a shared or exclusive lock on Table::syncPrior.

Patch looks good.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46057

ChangeSet@1.2650, 2008-04-25 18:50:01-05:00, cpowers@xeno.mysql.com +1 -0
  Bug #36330: Falcon DBT2 crash in Transaction::needToLock
  - Added syncPrior to RecordScavenge::inventoryRecord.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46067

ChangeSet@1.2651, 2008-04-26 22:13:23-05:00, cpowers@xeno.mysql.com +4 -0
  Bug#36330: Falcon DBT2 crash in Transaction::needToLock
  
  "Forced" scavenges are performed immediately free record cache memory, and
  should therefore be less constrained than the periodic scavenges peformed by
  the Scavenger thread.
  
  Streamlined forced record scavenging:
  - Don't call Table::inventoryRecords() for forced scavenges.
  - Ignore scavenge generation if scavenge is forced--just scavenge anyway.

This change is in the 6.0.5 release.

Sorry, it was pushed AFTER the 6.0.5 release. It is in 6.0.6.

Internal only. No documentation needed.