| Bug #32431 | MySQLd crashes after double free or corruption error | ||
|---|---|---|---|
| Submitted: | 16 Nov 2007 9:57 | Modified: | 20 Apr 2008 9:58 |
| Reporter: | Jari Ketola | Email Updates: | |
| Status: | No Feedback | Impact on me: | |
| Category: | MySQL Server: General | Severity: | S2 (Serious) |
| Version: | 5.0.22, 5.0.45 | OS: | Linux (2.6.18-8.1.15.el5 x86_64) |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | double free or corruption, glibc, signal 6, x86_64 | ||
[16 Nov 2007 10:31]
Valeriy Kravchuk
Thank you for a problem report. Please, try to repeat with a newer version, 5.0.45, and inform about the results.
[27 Nov 2007 8:05]
Jari Ketola
Running MySQL-server-community v5.0.45 now. I will update the bug after observing the server behavior for a week or so. Thanks.
[28 Dec 2007 0:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[8 Jan 2008 12:11]
Jari Ketola
Now running # mysqld --version mysqld Ver 5.0.45-community-log for unknown-linux-gnu on x86_64 (MySQL Community Edition (GPL)) After working fine for a while the server crashed again: *** glibc detected *** /usr/sbin/mysqld: corrupted double-linked list: 0x000000001a276060 *** 080103 13:32:55 - mysqld got signal 6; Still running kernel 2.6.18-8.1.15.el5 #1 SMP Mon Oct 22 08:32:28 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[17 Feb 2008 16:55]
Valeriy Kravchuk
Please, try to repeat with a newer version, 5.0.51a, and inform about the results.
[18 Mar 2008 0:02]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[20 Mar 2008 9:58]
Susanne Ebrecht
Please, let us know what exactly you did to get this error. Also we need to know how you installed MySQL and the version of your libc.
[20 Apr 2008 23:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
Description: MySQLd crashes unexpectedly at random intervals with the following error: *** glibc detected *** /usr/libexec/mysqld: double free or corruption (!prev): 0x000000001104c1e0 *** ======= Backtrace: ========= /lib64/libc.so.6[0x34d026e890] /lib64/libc.so.6(cfree+0x8c)[0x34d0271fac] /usr/libexec/mysqld(free_root+0x45)[0x808fb5] /usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x220)[0x560690] /usr/libexec/mysqld(_Z10do_commandP3THD+0x96)[0x561946] /usr/libexec/mysqld(handle_one_connection+0x990)[0x5623f0] /lib64/libpthread.so.0[0x34d0e061b5] /lib64/libc.so.6(clone+0x6d)[0x34d02cd36d] ======= Memory map: ======== mysqld got signal 6; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2093056 max_used_connections=112 max_connections=120 threads_connected=21 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 4210207 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. The server is running x86_64 Kernel 2.6.18-8.1.15.el5 (CentOS release 5). # uname -a Linux db2.afterdawn.net 2.6.18-8.1.15.el5 #1 SMP Mon Oct 22 08:32:28 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux # free -m total used free shared buffers cached Mem: 7987 7782 204 0 303 4728 -/+ buffers/cache: 2750 5236 Swap: 1992 0 1992 We are using the CentOS RPM for MySQL. # rpm -qi mysql-server Name : mysql-server Relocations: (not relocatable) Version : 5.0.22 Vendor: CentOS Release : 2.1.0.1 Build Date: Thu 30 Aug 2007 02:52:08 PM UTC Install Date: Wed 26 Sep 2007 06:58:11 AM UTC Build Host: builder6 Group : Applications/Databases Source RPM: mysql-5.0.22-2.1.0.1.src.rpm Size : 23629978 License: GPL Signature : DSA/SHA1, Fri 31 Aug 2007 12:36:32 AM UTC, Key ID a8a447dce8562897 The problem occurs on two of our database servers (both the master and slave), but is more prominent on the slave server which is running under more load (around 215 queries per second vs 85 qps, 1.5 load average vs. 0.25 load average). At times the crash only causes the slave instance to stop while the server continues responding to queries. We have had to build a monitoring system to automatically shut down the database, repair MyISAM tables and bring the server back up in case of a failure. It's also worth mentioning that the hardware on the server has been extensively tested (especially the memory), and no problems have been discovered. The server is running ECC memory and EDAC hasn't reported any parity errors on the system. How to repeat: Unknown.