Bug #57443 mysqld got signal 11
Submitted: 13 Oct 2010 19:44 Modified: 15 Oct 2010 13:59
Reporter: Reza Iskandar Achmad Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Server: General Severity:S1 (Critical)
Version:5.1.47-ndb-7.1.5-cluster-gpl OS:Linux
Assigned to: CPU Architecture:Any

[13 Oct 2010 19:44] Reza Iskandar Achmad
Description:
Version: '5.1.47-ndb-7.1.5-cluster-gpl'  socket: '/srv/mysql/data/mysql.sock'  port: 3306  MySQL Cluster Server (GPL)
101014  2:25:56 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=805306368
read_buffer_size=4194304
max_used_connections=1094
max_threads=100000
threads_connected=71
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2869017682 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x7fde30cd7980
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7fde7f2ca8b8 thread_stack 0x40000
/srv/mysql/bin/mysqld(my_print_stacktrace+0x33)[0x98dc63]
/srv/mysql/bin/mysqld(handle_segfault+0x324)[0x635f24]
/lib/libpthread.so.0(+0xf8f0)[0x7fdf53f7a8f0]
/srv/mysql/bin/mysqld(_ZN11Query_cache14get_free_blockEmcm+0x1b5)[0x768005]
/srv/mysql/bin/mysqld(_ZN11Query_cache14allocate_blockEmcm+0x47)[0x769bd7]
/srv/mysql/bin/mysqld(_ZN11Query_cache11store_queryEP3THDP10TABLE_LIST+0x478)[0x76c6c8]
/srv/mysql/bin/mysqld[0x6417a1]
/srv/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x402d)[0x649fad]
/srv/mysql/bin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x17c)[0x64e4dc]
/srv/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xe04)[0x64f3b4]
/srv/mysql/bin/mysqld(_Z10do_commandP3THD+0xde)[0x64fcbe]
/srv/mysql/bin/mysqld(handle_one_connection+0x1f0)[0x63fa10]
/lib/libpthread.so.0(+0x69ca)[0x7fdf53f719ca]
/lib/libc.so.6(clone+0x6d)[0x7fdf531ec6fd]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x7fde0975bb60 is an invalid pointer
thd->thread_id=9197
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
Bring cluster live with concurrent connection >1000
[13 Oct 2010 20:43] Sveta Smirnova
Thank you for the report.

Please provide more information about what kind of queries you were running and your configuration file.
[13 Oct 2010 22:01] Reza Iskandar Achmad
my.cnf

Attachment: my.cnf (application/octet-stream, text), 1.26 KiB.

[13 Oct 2010 22:04] Reza Iskandar Achmad
Actually we don't know what query causing the crash. It happens when MySQL running as VBulletin backend.

If you can, please suggest how to find out the query
[14 Oct 2010 6:00] MySQL Verification Team
probably not query specific... disable the query cache should be a workaround.
[14 Oct 2010 7:48] Reza Iskandar Achmad
disabling query cache would decrease performance of our database. I don't think we could not afford mysql without query cache.
[14 Oct 2010 9:21] Sveta Smirnova
Thank you for the feedback.

Stack trace looks very similar to  bug #39253. But this bug is about query cache, so turning it to off can be a workaround. Although this should be fixed in version you use.

Please add option --core to your mysqld configuration and update your OS settings so it allow mysqld to create cores, then in case of next crash send us core file and mysqld binary. Use mysqld-debug from the installation package.
[15 Oct 2010 13:59] Hartmut Holzgraefe
Duplicate of bug #56831