Bug #18785 Mysql got signal 10 (at startup)
Submitted: 5 Apr 2006 0:20 Modified: 18 Jun 2006 18:35
Reporter: Oleg Krasnov Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:5.0.19 OS:FreeBSD (FreeBSD 4.10)
Assigned to: CPU Architecture:Any

[5 Apr 2006 0:20] Oleg Krasnov
Description:
Guys, i have a big problem with mysql through network.

In a words, i have some FreeBSD 4.10 server with mysql 5.0.19 on some of them.
Databases were simply binary copied from mysql 4.0.18 and all worked fine until i tried to connect to mysqld not as localhost:3306 but thought the network.

1 server with Apache 2.0.54 + php 4.3.11 built with libmysqlclient from mysql 5.0.19(pthreads) and mysql-server 5.0.19 (linuxthreads) work fine.

2 server with Apache 2.0.54 + php 4.3.11 built with libmysql from mysql 5.0.19(pthreads) and two mysql-servers 5.0.19 (linuxthreads). First mysqld works fine (cause of connections through localhost only). The second (got connects from other server) shows such a message at startup (about 4-5 times):

---------------------------------------------------------------------------
060405 03:15:28  mysqld started
060405  3:15:28 [Note] /usr/local/mysql/libexec/mysqld: ready for connections.
Version: '5.0.19-SMP-log'  socket: ''  port: 3308  HRN Hosting Group
mysqld got signal 10;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=131072
max_used_connections=1
max_connections=100
threads_connected=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 225791 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
---------------------------------------------------------------------------

all servers build with following configure parameters

---------------------------------------------------------------------------
#!/bin/sh

CFLAGS='-O3 -pipe -march=pentiumpro -D__USE_UNIX98 -D_REENTRANT -D_THREAD_SAFE -DHAVE_BROKEN_REALPATH -I/usr/local/includ
e/pthread/linuxthreads' \
CXX=gcc \
CC=gcc \
CXXFLAGS='-O3 -pipe -march=pentiumpro -D__USE_UNIX98 -D_REENTRANT -D_THREAD_SAFE -DHAVE_BROKEN_REALPATH -I/usr/local/incl
ude/pthread/linuxthreads -felide-constructors -fno-rtti -fno-exceptions' \
./configure --prefix=/usr/local/mysql \
        --with-mit-threads=no \
        --enable-assembler \
        --with-named-thread-libs='-DHAVE_GLIBC2_STYLE_GETHOSTBYNAME_R -D_THREAD_SAFE -DHAVE_BROKEN_REALPATH -I/usr/local/
include/pthread/linuxthreads -L/usr/local/lib -llthread -llgcc_r' \
        --enable-thread-safe-client \
        --with-server-suffix='-SMP' \
        --with-charset=cp1251 \
        --with-mysqld-user=mysql \
        --without-isam \
        --without-debug \
        --without-bench \
        --without-extra-tools \
        --without-docs \
        --without-man \
        --enable-local-infile \
        --with-readline \
        --with-vio \
        --with-libwrap \
        --with-innodb \
        --with-mysqld-ldflags=-all-static \
        --with-client-ldflags=-all-static

echo "HAVE_SYS_UN_H must be undef for mysql not to use /tmp/mysql.sock."
sed 's;^.*HAVE_SYS_UN_H.*$; #undef HAVE_SYS_UN_H;' < config.h  >config.h.new
mv config.h.new config.h
---------------------------------------------------------------------------

and another importaint thing, if i try too enlarge buffers like key_buffer or sort_buffer i get the river of "got signal 10" messages in log and bunch of broken tables.

PS: All configurations are exactly the same to previous mysql 4.0.18, on which they worked fine.

Looking forward your support.

How to repeat:
1) Build mysqld with linuxthreads of FreeBSD.
2) Build libmysqlclient with native FreeBSD pthreads.
3) Build libphp for Apache 2 with this library.

and try to connect from one comuter(2-3) to another running mysqld(1).
[5 Apr 2006 12:46] Valeriy Kravchuk
Thank you for a problem report. Can you, please, perform the upgrade as it is described in the manual, http://dev.mysql.com/doc/refman/5.0/en/upgrading-from-4-1.html. Run mysql_upgrade, at least. Then check if the problem still present.
[5 Apr 2006 14:57] Oleg Krasnov
mysql_upgrade was run on all described databases (include upgrade priviliges),
and fulltext indexes also were rebuilt.
[28 Apr 2006 18:09] Valeriy Kravchuk
Is that crash happening upon MySQL server startup or when you trying to connect from client based on pthreads to server based on linuxthreads?

What is the difference amond thoseserver where 5.0.19 works and those one where it crashes? my.cnf settings, ulimit values, hardware?

Can you try to repeat with a newer vertsion, 5.0.20a? With official MySQL binaries? Just to be sure where problem is.
[10 May 2006 9:16] Oleg Krasnov
All clients are built as pthread-ed applications (php with linuxthreads in FreeBSD environment is rather funny thing) and all servers are linuxthread-ed. We have got all versions of 4.0.x branch tested and production-used this way.
Signal 10 appears only when i start mysqld and we have waiting connections from clients (in my opinion), cause if i raise firewall to blck all connections but local everything is alright!

All servers are absolutely identical. But another tip is when i increase table, key, innodb or another memory-amount values at startup (i dont use my.cnf), i got never-ending story with "got signal 10" in log and many broken tables.

I ll try to reproduce it with 5.0.20a, but cause we removed 5.0.x from production quickly my reply with this section may be too late. So plese ask me details, and take into account what we've already had.
Cause of disabling unix socket and linuxthreads in build config (FreeBSD) we cannot use official binaries.
[12 May 2006 8:57] Valeriy Kravchuk
> But another tip is when i increase table, key, innodb or another 
> memory-amount values at startup (i dont use my.cnf), i got never-ending story
> with "got signal 10" in log and many broken tables.

What exact values do you used at startup? What results uname -a gives you? I remember that on FreeBSD processed are too limited in memory usage by default. Please, check.
[12 May 2006 16:52] Oleg Krasnov
For example:
        /usr/local/mysql/bin/mysqld_safe \
        --datadir=/mysql/db1/mysql \
        --log-slow-queries=/mysql/mysql1-slow-query.log \
        --myisam-recover=BACKUP,FORCE \
        --user=mysql \
        --pid-file=/tmp/mysqld1.pid \
        -O key_buffer=128M \
        -O tmp_table_size=128M \
        -O table_cache=256 \
        -O sort_buffer=4M \
        -O record_buffer=4M \
        -O max_connections=511 \
        -O max_connect_errors=511 \
        -O query_cache_size=16777216 \
        --innodb_data_file_path=ibdata1:256M:autoextend \
        --innodb_flush_log_at_trx_commit=1 \
        --innodb_log_file_size=128M \
        --port=3306 &

but if we use standart values, same thing but less errors.

uname -a:

FreeBSD h0.hrn.ru 4.10-RELEASE FreeBSD 4.10-RELEASE #2: Fri Feb  3 18:28:40 MSK 2006     ******@******.ru:/usr/src/sys/compile/H0  i386

Valeriy, all and everything is "unlimited" for mysql user.
The same options are stable on 4.0.26 and show the great perfomance.
[21 May 2006 10:13] Valeriy Kravchuk
I have two ideas about possible reaosns of this bug... To check them, please:

1. Try to recompile and rebuild with -O2 instead of -O3.
2. Try to dump all the data from 4.0.x and reload them on 5.0.21.

Check if same crashes will be repeatable after that.
[18 Jun 2006 18:35] Oleg Krasnov
Upgrading to Mysql version 5.0.22 seems remove this problem (we moved also from -O3 to -O2).

Thenk you for your support.