MySQL Bugs: #34990: Falcon: falcon_bug_34351_A & falcon_bug_34251

Bug #34990	Falcon: falcon_bug_34351_A & falcon_bug_34251_C fail periodically
Submitted:	3 Mar 2:41	Modified:	22 Aug 19:30
Reporter:	Kevin Lewis
Status:	Documenting
Category:	Server: Falcon	Severity:	S2 (Serious)
Version:	6.0.4	OS:	Any
Assigned to:	Kevin Lewis	Target Version:	6.0.5
Triage:	D2 (Serious) / R3 (Medium) / E3 (Medium)

Description:
New testcases falcon_bug_34351_A & falcon_bug_34251_C fail periodically.  It is not
because of any update conflict, which is the problem with 34351.  These tests create an
environment in which two threads wait on each other quite often in fetchForUpdate(). 
These tests fail nearly all the time in pushbuild, but they fail on my laptop after
several runs when I set mysql_test_run to run 10 in a row.

With falcon_bug_34251_C I have hit;
	ASSERT(IS_CONSISTENT_READ(transaction->isolationLevel));
in table.cpp. line 2455.  And I have have also hit;
	bool ret = insert(record, NULL, recordNumber);
	ASSERT(ret);
in table.cpp, line 2917.

In addition, when I change falcon_bug_34351_C so that key(t1_uuid) is not created, I have
regularly hit a lock wait timeout on Windows.  But the funny thing is that it does not
wait 50 seconds.  SyncObject::wait() calls thread->sleep (timeout) and returns with value
zero, but yet the thread->lockGranted is still false.  So then timedout(timeout) is called
when it really was not a 50 second timeout.  This situation is very likely to be the same
as bug Bug#34890 - falcon_bug_22150.test fails on Pushbuild.

How to repeat:
perl mysql-test-run.pl --force --suite=falcon --manual-debug falcon_bug_34351_C
falcon_bug_34351_C falcon_bug_34351_C falcon_bug_34351_C falcon_bug_34351_C
falcon_bug_34351_C falcon_bug_34351_C falcon_bug_34351_C falcon_bug_34351_C
falcon_bug_34351_C falcon_bug_34351_C

set to verified since I have consistently hit ASSERT(ret); at table.cpp, line 2917.

I reproduced the assert in Table::insert many times and found that the call to
Section::insertStub() was returning the same record Number to two different threads that
did an insert of a new record.  The second one looses.  

Section::insertStub() uses Table::syncInsert to protect a couple bitmaps it uses to decide
which record slot is available.  But it would release this SyncObject during the search of
the section page.  I made it hold the synObject for the whole function and the assert did
not happen anymore.  

In addition, I also could no longer reproduce the other assert in table.cpp. line 2455
   ASSERT(IS_CONSISTENT_READ(transaction->isolationLevel));

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43486

ChangeSet@1.2588, 2008-03-05 14:25:44-06:00, klewis@klewis-mysql. +12 -0
  Bug#34990 - Fixed the condition in which two different 
  threads could get the same record number for an insert.

Pushed into 6.0.4-alpha

Fix is in 6.0.5-alpha.