Bug #19436 mysqld got signal 11 after 'SHOW...' query
Submitted: 29 Apr 2006 13:08 Modified: 21 Jun 2006 10:40
Reporter: Sergei Batiukov Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:5.0.20-Debian_1-log OS:Linux (Debian GNU/Linux)
Assigned to: CPU Architecture:Any

[29 Apr 2006 13:08] Sergei Batiukov
Description:
mysql gets crashed with signal 11 after every 'SHOW..' query. We're running on Debian 3.1 linux with P4-2.4 GHz 1 Gb RAM. System is working under heavy load.

From /var/log/syslog:

Apr 29 13:25:19 host mysqld[4864]: bytes of memory
Apr 29 13:25:19 host mysqld[4864]: Hope that's ok; if not, decrease some variables in the equation.
Apr 29 13:25:19 host mysqld[4864]:
Apr 29 13:25:19 host mysqld[4864]: thd=0x4c910ee8
Apr 29 13:25:19 host mysqld[4864]: Attempting backtrace. You can use the following information to find out
Apr 29 13:25:19 host mysqld[4864]: where mysqld died. If you see no messages after this, something went
Apr 29 13:25:19 host mysqld[4864]: terribly wrong...
Apr 29 13:25:19 host mysqld[4864]: Cannot determine thread, fp=0x4c808f78, backtrace may not be correct.
Apr 29 13:25:19 host mysqld[4864]: Stack range sanity check OK, backtrace follows:
Apr 29 13:25:19 host mysqld[4864]: 0x81916fc
Apr 29 13:25:19 host mysqld[4864]: 0x4005cd26
Apr 29 13:25:19 host mysqld[4864]: 0x81cf75d
Apr 29 13:25:19 host mysqld[4864]: 0x81c86a1
Apr 29 13:25:19 host mysqld[4864]: 0x81ca024
Apr 29 13:25:19 host mysqld[4864]: 0x81ca671
Apr 29 13:25:19 host mysqld[4864]: 0x81ca787
Apr 29 13:25:19 host mysqld[4864]: 0x826c2c8
Apr 29 13:25:19 host mysqld[4864]: 0x8266b04
Apr 29 13:25:19 host mysqld[4864]: 0x81ed981
Apr 29 13:25:19 host mysqld[4864]: 0x81ef388
Apr 29 13:25:19 host mysqld[4864]: 0x81efbc3
Apr 29 13:25:19 host mysqld[4864]: 0x81a56c4
Apr 29 13:25:19 host mysqld[4864]: 0x81ab09c
Apr 29 13:25:19 host mysqld[4864]: 0x81ab626
Apr 29 13:25:19 host mysqld[4864]: 0x81ac6a8
Apr 29 13:25:19 host mysqld[4864]: 0x81acff4
Apr 29 13:25:19 host mysqld[4864]: 0x40056c37
Apr 29 13:25:19 host mysqld[4864]: 0x402ba2ba
Apr 29 13:25:19 host mysqld[4864]: New value of fp=(nil) failed sanity check, terminating stack trace!
Apr 29 13:25:19 host mysqld[4864]: Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow instruction
s on how to resolve the stack trace. Resolved
Apr 29 13:25:19 host mysqld[4864]: stack trace is much more helpful in diagnosing the problem, so please do
Apr 29 13:25:19 host mysqld[4864]: resolve it
Apr 29 13:25:19 host mysqld[4864]: Trying to get some variables.
Apr 29 13:25:19 host mysqld[4864]: Some pointers may be invalid and cause the dump to abort...
Apr 29 13:25:19 host mysqld[4864]: thd->query at 0x8c13d60 = SHOW TABLE STATUS FROM `usr_f1_1`
Apr 29 13:25:19 host mysqld[4864]: thd->thread_id=11792
Apr 29 13:25:19 host mysqld[4864]: The manual page at http://www.mysql.com/doc/en/Crashing.html contains
Apr 29 13:25:19 host mysqld[4864]: information that should help you find out what is causing the crash.

==============================
my.cnf:
==============================

[client]
port            = 3306
socket          = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket          = /var/run/mysqld/mysqld.sock
nice            = 0

[mysqld]
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
language        = /usr/share/mysql/english
max_connections = 5000
max_connect_errors=999999
default-character-set = cp1251
skip-external-locking
init-connect = "SET NAMES cp1251;"
old_passwords   = 1
bind-address            = 127.0.0.1
key_buffer              = 150M
max_allowed_packet      = 8M
thread_stack            = 512K
sort_buffer     = 15M
query_cache_limit       = 1048576
query_cache_size        = 16777216
query_cache_type        = 1
log-slow-queries        = /var/log/mysql/mysql-slow.log
log-bin                 = /var/log/mysql/mysql-bin.log
max_binlog_size         = 104857600
skip-bdb

[mysqldump]
quick
quote-names
max_allowed_packet      = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer              = 16M

======================
Resolved stack trace:
======================

0x81916fc handle_segfault + 668
0x4005cd26 _end + 933344886
0x81cf75d _Z7openfrmP3THDPKcS2_jjjP8st_table + 2573
0x81c86a1 _Z17open_unireg_entryP3THDP8st_tablePKcS4_S4_P13st_table_listP11st_mem_root + 145
0x81ca024 _Z10open_tableP3THDP13st_table_listP11st_mem_rootPbj + 1844
0x81ca671 _Z11open_tablesP3THDPP13st_table_listPjj + 721
0x81ca787 _Z30open_normal_and_derived_tablesP3THDP13st_table_listj + 39
0x826c2c8 _Z14get_all_tablesP3THDP13st_table_listP4Item + 1736
0x8266b04 _Z24get_schema_tables_resultP4JOIN + 356
0x81ed981 _ZN4JOIN4execEv + 2833
0x81ef388 _Z12mysql_selectP3THDPPP4ItemP13st_table_listjR4ListIS1_ES2_jP8st_orderSB_S2_SB_mP13select_resultP18st_select_lex_unitP13st_sel + 328
0x81efbc3 _Z13handle_selectP3THDP6st_lexP13select_resultm + 275
0x81a56c4 _Z21mysql_execute_commandP3THD + 5828
0x81ab09c _Z11mysql_parseP3THDPcj + 348
0x81ab626 _Z16dispatch_command19enum_server_commandP3THDPcj + 1190
0x81ac6a8 _Z10do_commandP3THD + 136
0x81acff4 handle_one_connection + 2100
0x40056c37 _end + 933320071
0x402ba2ba _end + 935824394

How to repeat:
Try to run 'show table status from' query when system is on load.
[12 May 2006 10:25] Valeriy Kravchuk
Thank you for a problem report. Are you running on AMD64 platform? What glibc version do you use?
[12 May 2006 11:02] Sergei Batiukov
I'm using 2.3.6-7 version of glibc running on P4 box.
Looks like disabling binary and slow logs solving this problem - had no crashes for last 4 days. But still this is an issue and what i've learned when was bugging with this:

CHECK queries are causing mysqld crash too (mysqlcheck execution crashes mysqld). 
Sometimes after mysqld got crashed with SIGSEGV(after SHOW or CHECK) it began crashing with signal 6 (i saw crashed SELECT queries).

Sometimes i saw following libc signs in syslog:
May  8 02:22:49 host mysqld[15537]: *** glibc detected *** malloc(): memory corruption: 0x08ca79b0 ***
[12 May 2006 11:24] Valeriy Kravchuk
OK. Then, please, try to repeat with 5.0.21 AND MySQL's binaries, not Debian.
[12 May 2006 11:53] Sergei Batiukov
I have no time for upgrading MySQL at the moment. I'll update to 0.21 with your binaries and report results ASAP.
[21 May 2006 10:40] Valeriy Kravchuk
Please, reopen this bug report when you'll have results from 5.0.21.
[21 Jun 2006 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".