MySQL Bugs: #19420: mysql crashed with "Assertion failure in thread"

Bug #19420	mysql crashed with "Assertion failure in thread"
Submitted:	28 Apr 2006 10:04	Modified:	15 May 2006 15:25
Reporter:	Eddie Duffy	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S2 (Serious)
Version:	mysql Ver 14.7 Distrib 4.1.11, for redh	OS:	Linux (Fedora 4 Core)
Assigned to:	Heikki Tuuri	CPU Architecture:	Any

Description:
Apologies if this one has been reported before. I've found reports of similar crashes, but none of them seem exactly the same as this one.

Last night we had a MySQL crash, with the following error:

=================================================
060427 21:57:15InnoDB: Assertion failure in thread 1147971936 in file buf0lru.c
line 824
InnoDB: Failing assertion: block->n_pointers == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=402653184
read_buffer_size=2093056
max_used_connections=4
max_connections=100
threads_connected=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 802415
 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Memory status:
Non-mmapped space allocated from system: 7430144
Number of free chunks:                   28
Number of fastbin blocks:                0
Number of mmapped regions:               20
Space in mmapped regions:                916320256
Maximum total allocated space:           0
Space available in freed fastbin blocks: 0
Total allocated space:                   7038464
Total free space:                        391680
Top-most, releasable space:              92048
Estimated memory (with thread stack):    923947008

Number of processes running now: 0
060427 21:57:16  mysqld restarted
=================================================

At present, we have six databases on this server, but are planning to add more over the next two weeks. After seeing this crash, we are now concerned about adding further databases. (Although, at the time of the crash, only one database would have been in use.)

The system is a Dell PowerEdge SC1420, with 2Gb memory.

Here's the output from uname -a and mysql> status:
Linux cds02m 2.6.11-1.1369_FC4smp #1 SMP Thu Jun 2 23:16:33 EDT 2005 x86_64 x86_
64 x86_64 GNU/Linux
mysql  Ver 14.7 Distrib 4.1.11, for redhat-linux-gnu (x86_64)

Please let me know if you need any further info.

Thanks,

Eddie.

How to repeat:
Unknown. (At present, we've only seen the problem once).

Thank you for the bug report. Your version is pretty older, could you
please the most recent version and if you still get the crash, enable
the the query log (--log) for to identify which query/tables are involved
and then try to provide a test case with that information.

Thanks in advance.

Hi!

I think a bug associated with this assertion failure was fixed about half a year ago.

Please test with a newer 4.1.xx.

Regards,

Heikki

Miguel/Heikki,

Thanks for your replies. It's a surprise that we are so far behind in MySQL versions, as we installed the version of MySQL that came with Fedora Core 4. Also, we have two other systems with identical software versions and a similar number of databases (25/26), and these two behave fine. We are still getting regular crashes on this one machine, one or two per night.

Regards,

Eddie.

Eddie,

running memtestx86 or a similar program could be warranted here.

Please test also a newer MySQL version.

I am putting this bug report to the 'Need feedback' state.

Regards,

Heikki

Thanks, Heikki.

We've now moved all the databases off that system to another identical server with identical software and guess what ... we've not seen a crash since!

So, we are planning to do a few things on the previously-failing system, such as run diagnostics, rebuild the O/S from scratch and then upgrade MySQL to 4.1.19 as you suggested. It now seems that it's either a hardware fault or some odd problem with the way the software was installed. And thanks for that memtestx86 suggestion: we'll factor that into our diagnostics tests.

I'll post more feedback in a couple of days, but it's not looking like a MySQL problem at the moment.

Regards,

Eddie.

Eddie,

good to know. Then it looks like a hardware problem, but you never know.

I put the bug report to the 'Can't repeat' status, but you can keep posting new information to the bug report.

Regards,

Heikki

We ran into a similiar problem on v4.1.14 in August '07 and again today Jan 24, 2008.

080124 21:50:32InnoDB: Assertion failure in thread 16505 in file buf0lru.c line 824
InnoDB: Failing assertion: block->n_pointers == 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/mysql/en/Forcing_recovery.html
InnoDB: about forcing recovery.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

MySQL rebooted itself. We did a normal shutdown/restart after that.