Bug #43049 Assertion failed in Statement.cpp
Submitted: 20 Feb 2009 9:01 Modified: 15 May 2009 14:47
Reporter: Alexander Nozdrin Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S3 (Non-critical)
Version:6.0-TRUNK OS:Any
Assigned to: Kevin Lewis CPU Architecture:Any
Tags: crash, F_STARTUP, pb2, pushbuild, sporadic, test failure, widespread

[20 Feb 2009 9:01] Alexander Nozdrin
Description:
Symptoms:
-------------------------------------------------------------
CURRENT_TEST: main.warnings
[Falcon] Error: assertion (contextId >= 0 && contextId < numberContexts) failed at line 508 in file Statement.cpp

mysqltest got signal 6
read_command_buf at 0x1a27f80 = 
conn->name at 0xaf270d8 = 
conn->cur_query at (nil) is an invalid pointer
Attempting backtrace...
stack_bottom = (nil) thread_stack 0x40000
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(my_print_stacktrace+0x43) [0x648bcb]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded [0x60aafd]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded [0x60ab5b]
/lib64/libpthread.so.0 [0x399cc0de70]
/lib64/libpthread.so.0(raise+0x2d) [0x399cc0dd4d]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Error::debugBreak()+0x1f) [0x6dec1b]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Error::error(char const*, ...)+0x199) [0x6dedc7]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Error::assertionFailed(char const*, char const*, int)+0x3e) [0x6deebe]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Nfs::Statement::getContext(int)+0x47) [0x114d70f]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(FsbInversion::open(Nfs::Statement*)+0x69) [0x11dccd5]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(FsbSieve::open(Nfs::Statement*)+0x41) [0x11dd0d1]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(NUpdate::evalStatement(Nfs::Statement*)+0x50c) [0x120a26c]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Nfs::Statement::start(NNode*)+0x16a) [0x115015e]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(PreparedStatement::executeUpdate()+0xec) [0x10c8ef8]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Database::updateCardinalities()+0x296) [0x106cc00]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Database::cardinalityThreadMain()+0x88) [0x106ceb2]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Database::cardinalityThreadMain(void*)+0x26) [0x106cf74]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Thread::thread()+0xa5) [0x6e8873]
/export/home/pb2/build/sb_0-None-1234873942.5/mysql-6.0-gcov/libmysqld/examples/mysqltest_embedded(Thread::thread(void*)+0x2e) [0x6e8c78]
/lib64/libpthread.so.0 [0x399cc062f7]
/lib64/libc.so.6(clone+0x6d) [0x399c0d1b6d]
Writing a core file...
-------------------------------------------------------------

How to repeat:
XREf2: http://tinyurl.com/b3dsj2
[26 Mar 2009 6:23] Alexander Nozdrin
The problem seems to be widespread (may affect various tests).
It also occurs in PB1.

XRef: http://tinyurl.com/ct5oc4

Assertion:
[Falcon] Error: assertion (contextId >= 0 && contextId < numberContexts) failed at line 508 in file Statement.cpp

Stack trace:
mysqltest got signal 6
read_command_buf at 0x8bb3c00 = 
conn->name at 0x9695928 = 
conn->cur_query at (nil) is an invalid pointer
Attempting backtrace...
stack_bottom = (nil) thread_stack 0x30c00
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(my_print_stacktrace+0x3a)[0x8224d9a]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded[0x81ce45e]
[0xffffe420]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN5Error10debugBreakEv+0x12)[0x86d38d2]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN5Error5errorEPKcz+0x66)[0x86d3946]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN5Error15assertionFailedEPKcS1_i+0x27)[0x86d39c7]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN3Nfs9Statement10getContextEi+0x2e)[0x872a70e]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN12FsbInversion5fetchEPN3Nfs9StatementE+0x21)[0x87e2041]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN8FsbSieve5fetchEPN3Nfs9StatementE+0x1f)[0x87e22ff]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN7NUpdate13evalStatementEPN3Nfs9StatementE+0x86)[0x87f65c6]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN3Nfs9Statement5startEP5NNode+0x89)[0x872ccf9]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN17PreparedStatement13executeUpdateEv+0x25)[0x86edfb5]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN8Database19updateCardinalitiesEv+0x11a)[0x86c184a]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN8Database21cardinalityThreadMainEv+0x48)[0x86c19a8]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN6Thread6threadEv+0x85)[0x860f765]
/data0/pushbuild/pb2/pb/bzr_mysql-6.0/143/mysql-6.0.11-alpha-pb143/libmysqld/examples/mysqltest_embedded(_ZN6Thread6threadEPv+0x12)[0x860f8d2]
/lib/tls/libpthread.so.0[0x401130bd]
/lib/tls/libc.so.6(__clone+0x5e)[0x4026101e]
[1 Apr 2009 5:05] Kevin Lewis
It looks like updateCardinalities is failing at startup.  There was a change to this on February 19 2009 1:23am with this changeset.
    http://lists.mysql.com/commits/66823
That same day was the last occurrance of this problem in mysql-6.0-falcon-team according to pushbuild xref.  Let's watch this failure in xref for a few more weeks to see if it occurs in any trees that have this patch.  As of 3/31/2009, this patch had still not been uploaded to main.
[1 Apr 2009 5:53] Kevin Lewis
I noticed that updateCardinalities ran during a recovery once on my system. It should not do this.  Another inspection of the code revealed that Database::cardinalityThreadMain(void) only waits for 1 second before starting the while (!thread->shutdownInProgress) loop.  This loop runs updateCardinalities() once and then sleeps to be signaled.  This first time run of updateCardinalities() by the updateCardinalities thread must wait for recovery to finish.  In that way, all other dependent data structures should also be set up before it runs.
[1 Apr 2009 6:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/70999

3091 Kevin Lewis	2009-04-01
      Bug#43049 - Assertion during updateCardinalities() at startup.  Make sure the cardinalities thread does not run updateCardinalities() the first time until after recovery has finished.
[1 Apr 2009 18:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71132

3092 Kevin Lewis	2009-04-01
      Bug#43049 - Assertion during updateCardinalities() at startup.  Make sure the cardinalities thread does not run updateCardinalities() the first time until after recovery has finished.  There is a small gap of time after the serialLog is created and before recovoery starts when serialLog->recovering == false.  But the first 1 second sleep should cover for that.
[15 Apr 2009 16:59] Bugs System
Pushed into 6.0.11-alpha (revid:hky@sun.com-20090415164923-9arx29ye5pzxd4zf) (version source revid:kevin.lewis@sun.com-20090401185104-pgbpsotxhrb7fys9) (merge vers: 6.0.11-alpha) (pib:6)
[15 May 2009 14:47] MC Brown
Internal/test fix. No changelog entry required.