Bug #17616 Crash in creation of "mysql" database
Submitted: 21 Feb 2006 14:48 Modified: 21 May 2006 14:24
Reporter: Francesco Riosa Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S3 (Non-critical)
Version:5.1.6_alpha OS:Linux (linux (custom))
Assigned to: CPU Architecture:Any

[21 Feb 2006 14:48] Francesco Riosa
Description:
As synopsis say, after building of mysql 5.1.6 from sources, it was impossible to create the "mysql" database.

Analyzing this bug keep in mind it has been build on a system with 
- gcc (GCC) 4.1.0-beta20060210  (prerelease)
- glibc 2.3.6-r2
- Linux localhost 2.6.16-rc2-mm1 #3 PREEMPT Fri Feb 17 20:24:06 CET 2006 i686 AMD Athlon(tm)  AuthenticAMD GNU/Linux

How to repeat:
After a failed try with "mysql_install_db", I've cleaned the "datadir" directory and run on two different consoles the server and the client, i/o from is prefixed by #s, the client i/o by #c.

#s env -i sudo gdb libexec/mysqld
#s 
#s set args --no-defaults \
#s --skip-innodb --skip-name-resolve --skip-networking  \
#s --server-id=51 \
#s --skip-grant-tables \
#s --log=/mnt/sda6/dest/var/mysqld.log \
#s --debug=d:t:i:o,/mnt/sda6/dest/var/mysqld.trace \
#s --one-thread \
#s --user=mysql \
#s --datadir=/mnt/sda6/dest/var/ \
#s --socket=/mnt/sda6/dest/var/mysqld-501.sock \
#s --pid-file=/mnt/sda6/dest/var/mysqld-501.pid
#s 
#s run
#s 
#s libexec/mysqld: Table 'mysql.general_log' doesn't exist
#s 060221 14:12:48 [ERROR] Failed to initialize log tables. Falling back to the old-fashioned logs
#s 060221 14:12:48 [Warning] Can't open and lock time zone table: Table 'mysql.time_zone_leap_second' doesn't exist trying to live without them
#s 060221 14:12:48 [Note] libexec/mysqld: ready for connections.
#s Version: '5.1.6-alpha-debug-log'  socket: '/mnt/sda6/dest/var/mysqld-501.sock'  port: 0  Source distribution
#s 
#s (gdb) run
#s Starting program: /mnt/sda6/dest/libexec/mysqld --no-defaults --skip-innodb --skip-name-resolve --skip-networking  --server-id=51 --skip-grant-tables --log=/mnt/sda6/dest/var/mysqld.log --debug=d:t:i:o,/mnt/sda6/dest/var/mysqld.trace --one-thread --user=mysql --datadir=/mnt/sda6/dest/var/ --socket=/mnt/sda6/dest/var/mysqld-501.sock --pid-file=/mnt/sda6/dest/var/mysqld-501.pid
#s [Thread debugging using libthread_db enabled]
#s [New Thread -1211180832 (LWP 30690)]
#s /mnt/sda6/dest/libexec/mysqld: Table 'mysql.general_log' doesn't exist
#s 060221 14:32:33 [ERROR] Failed to initialize log tables. Falling back to the old-fashioned logs
#s [New Thread -1219839056 (LWP 30693)]
#s 060221 14:32:33 [Warning] Can't open and lock time zone table: Table 'mysql.time_zone_leap_second' doesn't exist trying to live without them
#s 060221 14:32:33 [Note] /mnt/sda6/dest/libexec/mysqld: ready for connections.
#s Version: '5.1.6-alpha-debug-log'  socket: '/mnt/sda6/dest/var/mysqld-501.sock'  port: 0  Source distribution

#c # bin/mysql --socket=/mnt/sda6/dest/var/mysqld-501.sock
#c mysql> CREATE DATABASE mysql;
#c Query OK, 1 row affected (0.00 sec)
#c 
#c # bin/mysql --socket=/mnt/sda6/dest/var/mysqld-501.sock < ./mysql-test/lib/init_db.sql

#s safe_mutex: Trying to unlock mutex that wasn't locked at ha_myisam.cc, line 1379
#s 			Last used at , line: 0
#s 
#s Program received signal SIGABRT, Aborted.
#s [Switching to Thread -1211201312 (LWP 31298)]
#s 0xffffe410 in __kernel_vsyscall ()

#s (gdb) bt
#s #0  0xffffe410 in __kernel_vsyscall ()
#s #1  0xb7d135be in raise () from /lib/tls/libc.so.6
#s #2  0xb7e20ff4 in ?? () from /lib/tls/libc.so.6
#s #3  0xb7ce88e0 in ?? ()
#s #4  0xb7d14f2e in abort () from /lib/tls/libc.so.6
#s #5  0x00000000 in ?? ()

#s (gdb) cont
#s Continuing.
#s mysqld got signal 6;
#s This could be because you hit a bug. It is also possible that this binary
#s or one of the libraries it was linked against is corrupt, improperly built,
#s or misconfigured. This error can also be caused by malfunctioning hardware.
#s We will try our best to scrape up some info that will hopefully help diagnose
#s the problem, but since we have already crashed, something is definitely wrong
#s and this may fail.
#s 
#s key_buffer_size=8388572
#s read_buffer_size=131072
#s max_used_connections=0
#s max_connections=100
#s threads_connected=1
#s It is possible that mysqld could use up to
#s key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225788 K
#s bytes of memory
#s Hope that's ok; if not, decrease some variables in the equation.
#s 
#s thd=0x86d7838
#s Attempting backtrace. You can use the following information to find out
#s where mysqld died. If you see no messages after this, something went
#s terribly wrong...
#s Cannot determine thread, fp=0xbf8ecf68, backtrace may not be correct.
#s Stack range sanity check OK, backtrace follows:
#s 0x81a6853
#s 0xffffe420
#s 0x86f5fd0
#s 0x40
#s (nil)
#s New value of fp=(nil) failed sanity check, terminating stack trace!
#s Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved
#s stack trace is much more helpful in diagnosing the problem, so please do
#s resolve it
#s Trying to get some variables.
#s Some pointers may be invalid and cause the dump to abort...
#s thd->query at 0x86e2e28 = INSERT INTO time_zone_name (Name, Time_Zone_id) VALUES
#s   ('MET', 1), ('UTC', 2), ('Universal', 2),
#s   ('Europe/Moscow',3), ('leap/Europe/Moscow',4),
#s   ('Japan', 5)
#s thd->thread_id=2
#s The manual page at http://www.mysql.com/doc/en/Crashing.html contains
#s information that should help you find out what is causing the crash.
#s 
#s Program exited with code 01.

#c ERROR 2013 (HY000) at line 219: Lost connection to MySQL server during query
[21 Feb 2006 14:51] Francesco Riosa
mysqld.trace (compressed with bzip2)

Attachment: mysqld.trace.bz2 (application/x-bzip, text), 42.41 KiB.

[21 Feb 2006 14:53] Francesco Riosa
configure options below:

CHOST="i686-pc-linux-gnu" \
CC=gcc \
CFLAGS="-O2 -march=athlon-xp" \
CXX=gcc \
CXXFLAGS="-O2 -march=athlon-xp -felide-constructors \
   -fno-exceptions -fno-rtti" \
./configure --prefix=/mnt/sda6/dest/ \
--with-low-memory --enable-assembler --enable-local-infile \
--with-mysqld-user=mysql \
--enable-thread-safe-client \
--with-zlib-dir=/usr \
--without-readline \
--without-docs \
--enable-shared \
--enable-static \
--without-libwrap \
--with-openssl \
--with-debug=full \
--with-bench \
--with-server \
--with-extra-tools \
--with-innodb \
--without-raid \
--with-extra-charsets=all \
--without-berkeley-db \
--with-geometry \
--without-ndbcluster \
--without-big-tables \
--without-ndb-binlog \
--without-embedded-privilege-control \
--without-embedded-server \
--with-archive-storage-engine \
--with-csv-storage-engine \
--with-blackhole-storage-engine \
--with-federated-storage-engine \
--with-partition \
--with-max-indexes=128 \
--with-row-based-replication \
&& make && make install
[22 Feb 2006 11:47] Valeriy Kravchuk
Thank you for a problem report. Can you, please, try to execute standard mysql_install_db once again and send the error log's content for this attempt?
[22 Feb 2006 15:22] Francesco Riosa
cut and paste from the console, resolved trough "resolve_stack_dump"

Attachment: mysqld.stack.resolved (application/octet-stream, text), 4.75 KiB.

[22 Feb 2006 15:25] Francesco Riosa
debug trace of the server run by mysql_install_db

Attachment: mysqld.trace.bz2 (application/x-download, text), 8.40 KiB.

[22 Feb 2006 15:34] Francesco Riosa
the two previous attachments are the output of the following command:

./bin/mysql_install_db \
--no-defaults \
--basedir=/mnt/sda6/dest/ \
--user=mysql \
--skip-name-resolve \
--verbose \
--debug=d:t:i:o,/mnt/sda6/dest/var/mysqld.trace \
--log=/mnt/sda6/dest/var/mysqld.log \

the kernel version has changed and it's now:
Linux localhost 2.6.16-rc4-mm1 #1 PREEMPT Tue Feb 21 17:59:59 2006 i686 AMD Athlon(tm)  AuthenticAMD GNU/Linux

the last function enter this time is "find_type", just a quick test:

grep -c find_type mysqld.trace.*
mysqld.trace.1:116
mysqld.trace.2:116
mysqld.trace.3:116

show that _probably_ the path to the crash is always the same.
[22 Feb 2006 17:00] Francesco Riosa
also reproducible with different behaviour (mysqld.trace) on a more stable machine:

gcc (GCC) 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)
glibc-2.3.5
Linux francesco-riosa.com 2.6.13-hardened #1 Sat Oct 1 03:49:31 CEST 2005 i686 AMD Duron(tm) Processor AuthenticAMD GNU/Linux
[2 Mar 2006 14:19] Francesco Riosa
5.1.7 : ./bin/mysql_install_db (same options) trace

Attachment: mysqld.trace.bz2 (application/x-download, text), 11.31 KiB.

[2 Mar 2006 14:21] Francesco Riosa
diff between traces of 5.1.6 and 5.1.7 (removed first 12 chars of each row)

Attachment: mysqld.trace-5.1.6-5.1.7.diff.bz2 (application/x-download, text), 15.13 KiB.

[2 Mar 2006 18:01] Valeriy Kravchuk
I had not tried with your exact configure, but in 5.1.8-BK () mysql_install_db works just fine for me on SuSE 9.3:

openxs@suse:~/dbs/5.1> bin/mysql_install_db
Installing all prepared tables
Fill help tables

To start mysqld at boot time you have to copy support-files/mysql.server
to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/home/openxs/dbs/5.1/bin/mysqladmin -u root password 'new-password'
/home/openxs/dbs/5.1/bin/mysqladmin -u root -h suse password 'new-password'
See the manual for more instructions.

You can start the MySQL daemon with:
cd /home/openxs/dbs/5.1 ; /home/openxs/dbs/5.1/bin/mysqld_safe &

You can test the MySQL daemon with the benchmarks in the 'sql-bench' directory:
cd sql-bench ; perl run-all-tests

Please report any problems with the /home/openxs/dbs/5.1/bin/mysqlbug script!

The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at https://order.mysql.com
openxs@suse:~/dbs/5.1> bin/mysqld_safe &
[1] 12237
openxs@suse:~/dbs/5.1> Starting mysqld daemon with databases from /home/openxs/d
bs/5.1/var

openxs@suse:~/dbs/5.1> bin/mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 5.1.8-beta-debug

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| cluster            |
| mysql              |
| test               |
+--------------------+
4 rows in set (0.01 sec)

mysql> exit
Bye
openxs@suse:~/dbs/5.1> uname -a
Linux suse 2.6.11.4-20a-default #1 Wed Mar 23 21:52:37 UTC 2005 i686 i686 i386 GNU/Linux

So, the difference can be in kernel, CPU, glibc version etc. Please, check.

Doo you want me to try on a platfrom more similar to yours?
[2 Mar 2006 18:03] Valeriy Kravchuk
Sorry, missed some information. I had used ChangeSet@1.2205.1.2, 2006-03-02 08:44:11+01:00 and the following (simpler) configure:

./configure --prefix=/home/openxs/dbs/5.1 --with-extra-charsets=all \
--with-readline --with-innodb --with-ndbcluster \
--with-mysqld-ldflags=-all-static --enable-thread-safe-client \
--with-debug=full
[3 Mar 2006 14:51] Francesco Riosa
I cannot test bitkeeper sources, sorry. Still on 5.1.7 the smallest change needed to make the server work is to remove the following options:

--with-max-indexes=128 \
--enable-assembler \

Lowering severity from "Serious" to "Non-critical", may be still lower ?
Thanks for the time spent.
[21 Apr 2006 14:24] Valeriy Kravchuk
Please, try to repeat without

--enable-assembler

option only and inform about the results. Do you have links in /usr/include to kernel sources?
[21 May 2006 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".