Bug #34251 Crash on Falcon CREATE TABLE
Submitted: 2 Feb 2008 16:19 Modified: 1 Oct 2008 11:28
Reporter: Philip Stoev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Falcon storage engine Severity:S1 (Critical)
Version:6.0.4 OS:Linux (x86_64)
Assigned to: Vladislav Vaintroub CPU Architecture:Any
Tags: compiler_suspected

[2 Feb 2008 16:19] Philip Stoev
Description:
When trying to do a CREATE TABLE ENGINE=Falcon , a crash occurs with the following backtrace:

#0  Dbb::open (this=0x43884dd0, fileName=0x2aaab02013b8 "pXК", cacheSize=46912585599920, transId=2954892108) at Dbb.cpp:460
#1  0x0000000000845d23 in Database::openDatabase (this=0x2aaab0000bb0, filename=0x43884dd0 "/home/pstoev/6.0.4/mysql-test/var/master-data/falcon_master.fts")
    at Database.cpp:677
#2  0x0000000000842ffa in Connection::getDatabase (this=0x2aaab0201680, dbName=0x2aaab020134c "FALCON_MASTER",
    dbFileName=0x43884dd0 "/home/pstoev/6.0.4/mysql-test/var/master-data/falcon_master.fts", threads=0x2aaab02013b8) at Connection.cpp:1653
#3  0x0000000000840fbd in Connection::openDatabase (this=0x2aaab0201680, dbName=0x2aaab020134c "FALCON_MASTER", filename=0x2aaab0201528 "0^К",
    account=0xafed54 "mysql", password=0xafed54 "mysql", address=0x0, parent=0x2aaab02013b8) at Connection.cpp:931
#4  0x00000000008193ca in StorageDatabase::getOpenConnection (this=0x2aaab02011c8) at JString.h:141
#5  0x000000000081d1d5 in StorageHandler::initialize (this=0x2aaab0000048) at StorageHandler.cpp:899
#6  0x000000000081c801 in StorageHandler::createTable (this=0x2aaab0000048, pathname=0x43886700 "./test/insdel1_tbl", tableSpaceName=0x0, tempTable=false)
    at StorageHandler.cpp:591
#7  0x000000000080ec08 in StorageInterface::create (this=0x1cfb840, mySqlName=0x43886700 "./test/insdel1_tbl", form=0x43885ae0, info=0x43886e20)
    at ha_falcon.cpp:679
#8  0x000000000073787b in ha_create_table (thd=0x438858e0, path=0x43885ae0 "PU\210C", db=0x1cf72d8 "test", table_name=0x1cf72e0 "insdel1_tbl",
    create_info=0x43886e20, update_create_info=false) at handler.cc:2706
#9  0x00000000006ffad9 in rea_create_table (thd=0x1c8d590, path=0x43886700 "./test/insdel1_tbl", db=0x1cf72d8 "test", table_name=0x1cf72e0 "insdel1_tbl",
    create_info=0x43886e20, create_fields=@0x43886f38, keys=2954892584, key_info=0x2aaab0201528, file=0x1cf8090) at unireg.cc:473
#10 0x000000000074d5cb in mysql_create_table_no_lock (thd=0x1c8d590, db=0x1cf72d8 "test", table_name=0x1cf72e0 "insdel1_tbl", create_info=0x43886e20,
    alter_info=0x43886ef0, internal_tmp_table=false, select_field_count=2954892584) at sql_table.cc:3474
#11 0x000000000074d914 in mysql_create_table (thd=0x1c8d590, db=0x1cf72d8 "test", table_name=0x1cf72e0 "insdel1_tbl", create_info=0x43886e20,
    alter_info=0x43886ef0, internal_tmp_table=false, select_field_count=2954892584) at sql_table.cc:3578
#12 0x000000000065d225 in mysql_execute_command (thd=0x1c8d590) at sql_parse.cc:2229
#13 0x0000000000662bfe in mysql_parse (thd=0x1c8d590,
    inBuf=0x1cf7100 "CREATE TABLE test.insdel1_tbl(id MEDIUMINT NOT NULL AUTO_INCREMENT, dt TIMESTAMP, user CHAR(255), uuidf LONGBLOB, fkid MEDIUMINT, filler VARCHAR(255), PRIMARY KEY(id))ENGINE=Falcon", length=180, found_semicolon=0x43887580) at sql_parse.cc:5410
#14 0x000000000065b315 in dispatch_command (command=COM_QUERY, thd=0x1c8d590, packet=0x1cf71b4 "", packet_length=180) at sql_parse.cc:948
#15 0x000000000065ab12 in do_command (thd=0x1c8d590) at sql_parse.cc:697
#16 0x00000000006598cf in handle_one_connection (arg=0x2aaab0201928) at sql_connect.cc:1146
#17 0x00002abbc50a709e in start_thread () from /lib64/libpthread.so.0
#18 0x00002abbc610f4cd in clone () from /lib64/libc.so.6
#19 0x0000000000000000 in ?? ()

Notice that fileName argument to Dbb::open appears corrupt.

How to repeat:
Instructions will follow in a private comment.
[2 Feb 2008 16:33] Philip Stoev
Machine is openSUSE 10.2 (X86-64)
[2 Feb 2008 17:44] Philip Stoev
Works correctly on openSUSE 10.3 (X86-64) with same binary. Uname -a:

Linux xeno 2.6.22.16-0.1-default #1 SMP 2008/01/23 14:28:52 UTC x86_64 x86_64 x86_64 GNU/Linux
[4 Feb 2008 7:14] Philip Stoev
Those two lines are very suspicious:

#0  Dbb::open (this=0x43884dd0, fileName=0x2aaab02013b8 "pXК", cacheSize=46912585599920,
transId=2954892108) at Dbb.cpp:460
#1  0x0000000000845d23 in Database::openDatabase (this=0x2aaab0000bb0, filename=0x43884dd0
"/home/pstoev/6.0.4/mysql-test/var/master-data/falcon_master.fts")

Please observe that the address of the this argument on open() is equal to the address of the filename argument to openDatabase(). At the same time, the first portion of the "this" argument to openDatabase() is the same as the first portion of the "fileName" argument to open(). This looks like a endian issue to me.
[4 Feb 2008 12:07] Valeriy Kravchuk
I can not repeat this with latest 6.0-BK on 32-bit SuSE 9.3. So it looks like platfrom/distribution-specific bug.
[4 Feb 2008 12:09] Vladislav Vaintroub
It can't be endianess bug,all x86_64 machines are of the same little endianess.It looks more like debugger got confused, maybe as a result of too much optimization(-fomit-frame-pointer comes to mind as one of the evil options)
[4 Feb 2008 12:39] MySQL Verification Team
I can't repeat on Fedora Core 8 64-bit with latest source:

=======================================================

TEST                           RESULT         TIME (ms)
-------------------------------------------------------

main.partition_falcon          [ pass ]            388
-------------------------------------------------------
Stopping All Servers
All 1 tests were successful.
The servers were restarted 1 times
Spent 0.388 of 6 seconds executing testcases

[miguel@mira mysql-test]$ cat /etc/issue
Fedora release 8 (Werewolf)
Kernel \r on an \m

[miguel@mira mysql-test]$
[4 Feb 2008 13:34] Philip Stoev
I can repeat with a freshly installed openSUSE 10.2 (X86-64), uname:

Linux suse 2.6.18.2-34-default #1 SMP Mon Nov 27 11:46:27 UTC 2006 x86_64 x86_64 x86_64 GNU/Linux

Processor is Intel(R) Core(TM)2 Duo CPU     T5250  @ 1.50GHz, that is, the problem is repeatable on both AMD and Intel.

10.2 is the previous OpenSuse version, 10.3 is the current version.
[4 Feb 2008 16:34] Susanne Ebrecht
Ubuntu 64bit:
main.partition_falcon          [ pass ]            955

But I have had a skipped with an older bk tree on FreeBSD 64bit.

Will test it with newer bk tree on FreeBSD later this day (when I am back on FreeBSD).
[5 Feb 2008 1:51] Vladislav Vaintroub
I run mysqld on the same machine as Philip and the callstack I got
looks sane and a little bit different

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1133021504 (LWP 27483)]
0x0000000000ae28b4 in _Unwind_Resume ()
(gdb) bt
#0  0x0000000000ae28b4 in _Unwind_Resume ()
#1  0x000000000084e495 in Dbb::open (this=0x2aaab0201928,
    fileName=0x43884dd0 "/home/vvaintroub/vlad/master-data/falcon_master.fts", c
acheSize=4194304, transId=0) at Dbb.cpp:539
#2  0x0000000000845d23 in Database::openDatabase (this=0x2aaab0000bb0,
    filename=0x43884dd0 "/home/vvaintroub/vlad/master-data/falcon_master.fts")
    at Database.cpp:677
#3  0x0000000000842ffa in Connection::getDatabase (this=0x2aaab0201680,
    dbName=0x2aaab020134c "FALCON_MASTER",
    dbFileName=0x43884dd0 "/home/vvaintroub/vlad/master-data/falcon_master.fts",
 threads=0x2aaab02013b8) at Connection.cpp:1653
#4  0x0000000000840fbd in Connection::openDatabase (this=0x2aaab0201680,
....

Note, that _Unwind_Resume function in question is responsible for unwinding the stack on C++ exceptions or longjmp and comes from /lib64/libgcc_s.so.1

We do throw and catch C++ exceptions, and yet if stack unwinding function crashes with SIGSEV, this seems to be not our fault, can be an unfortunate combination of the newer compiler and older unwinding code in libgcc_so.1 that do not work together well.
The fact that after compiling on Suse 10.2 Philip could run mysqld without
problems  and the fact that Linux with higher version can run this mysqld without problems both speak for this theory.

I'd strongly suggest a real Linux specialist looks at the issue and makes his judgement. My guess is not very educated -  some googling and a little bit past experience with exception handler crashes on a very different OS.
[5 Feb 2008 11:00] Philip Stoev
Updating libgcc to libgcc42-4.2.3_20071030-3.6.x86_64.rpm  solved the problem. It appears that this package is not offered as part of SuSE automatic updates, one needs to go manually to their web site at 

http://software.opensuse.org

and search for "libgcc" for opensuse 10.2
[2 May 2008 1:20] Paul DuBois
Do we need a changelog entry for this report? Looks like a problem with a particular Linux distribution, not with MySQL. If we need a changelog entry, what should it say? Thanks.
[2 May 2008 16:28] Vladislav Vaintroub
Paul,
this was a problem of outdated compiler and OS libraries combination. (GCC 3.2.3 and OpenSuse 10.2 x86_64)
 
Meanwhile, build team switched to more recent compiler and the problem does not appear anymore.
[1 Oct 2008 11:28] Jon Stephens
Closed without further action per previous 2 comments.