MySQL Bugs: #5538: InnoDB: Assertion failure using mysqldump -l

Bug #5538	InnoDB: Assertion failure using mysqldump -l
Submitted:	13 Sep 2004 10:20	Modified:	30 Oct 2004 10:59
Reporter:	Ronny Pettersen
Status:	Closed
Category:	Server: InnoDB	Severity:	S1 (Critical)
Version:	4.0.21	OS:	Linux (Linux 2.4.26(FC1))
Assigned to:	Heikki Tuuri	Target Version:

Description:
Whenever i run "mysqldump" with the locking option (-l) on InnoDB tables, the database
crashes with the error below.
If i leave out the "-l" option, mysqldump completes sucessfully.
This did not happen in any version prior to 4.0.21.
(I have been using mysqldump for daily backups since 4.0.18)
--
040913  9:33:37InnoDB: Assertion failure in thread 1105938 in file ha_innodb.cc line 4581
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. See section 6.1 of
InnoDB: http://www.innodb.com/ibman.php about forcing recovery.
mysqld got signal 11; 
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388600
read_buffer_size=33550336
max_used_connections=8
max_connections=100
threads_connected=9
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2367087 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x8892d38
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xbfd7eef8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8072614
0x826ac98
0x80d2564
0x8091db8
0x8091e80
0x80e0074
0x807e6d3
0x8081155
0x807c893
0x807c2de
0x807bb08
0x826844c
0x829df0a
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://www.mysql.com/doc/en/Using_stack_trace.html and follow instructions on
how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x88c2608 = show create table `ACL`
thd->thread_id=262
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
040913 09:33:37  mysqld restarted
040913  9:33:37 Warning: Asked for 196608 thread stack, but got 126976
040913  9:33:37  InnoDB: Database was not shut down normally.
InnoDB: Starting recovery from log files...
InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 0 484966296
InnoDB: Doing recovery: scanned up to log sequence number 0 484966296
040913  9:33:38  InnoDB: Flushing modified pages from the buffer pool...
040913  9:33:38  InnoDB: Started
/usr/sbin/mysqld: ready for connections.
Version: '4.0.21-standard'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Official
MySQL RPM

How to repeat:
mysqldump -u -p -l innodb_database

Thank you for this bug report. To properly diagnose the problem, we
need a backtrace to see what is happening behind the scenes. To
find out how to generate a backtrace, please read
http://www.mysql.com/doc/en/Making_trace_files.html

Once you have generated a backtrace, please submit it to this bug
report and change the status back to 'Open'. Thank you for helping
us make our products better.

Additional info:

Could you please send a complete test case, including the CREATE TABLE and INSERT
statements for populating the database? Was there a line
"InnoDB: Error: select_lock_type is %lu inside ::start_stmt()!"
immediately before the reported assertion failure? If yes, what number was displayed in
place of %lu?

I'm not running the 'debug' version of mysql. I'd have to find some other server to
install that one if necessary.
I'm using these Linux x86 RPMS from MySQL:
MySQL-server-4.0.21-0
MySQL-devel-4.0.21-0
MySQL-shared-compat-4.0.20-0
MySQL-client-4.0.21-0

There were one preceding line left out of the log:
InnoDB: Error: select_lock_type is 99999999 inside ::start_stmt()!

Complete steps to reproduce:
# mysql
USE test;
DROP TABLE IF EXISTS `IBTABLE`;
CREATE TABLE `IBTABLE` ( a int ) TYPE=InnoDB;
exit

# mysqldump -uroot -l test
-- MySQL dump 9.11
--
-- Host: localhost    Database: test
-- ------------------------------------------------------
-- Server version       4.0.21-standard
mysqldump: Can't get CREATE TABLE for table `IBTABLE` (Lost connection to MySQL server
during query)

Thanks for the details. I was able to repeat the crash on my system.

Hi!

I have now fixed this bug. The fix will appear in 4.0.22.

This bug did not affect 4.1.4.

The reason for the bug was that mysqldump -l uses LOCK TABLES ... READ LOCAL to lock the
tables. That READ LOCAL was internally TL_READ, which is normally used in ordinary
SELECTs. InnoDB was not prepared that inside LOCK TABLES , TL_READ could be used.

Workaround: do not use mysqldump with the -l option. Use the --single-transaction option.
Actually, contrary to what is said in the manual, the -l option does NOT guarantee a
consistent snapshot of all InnoDB tables is one database. You must use the
--single-transaction option to get a consistent snapshot.

Thank you,

Heikki

Thanks!
I did see "--single-transaction" mentioned while I was searching.
But I did not find this option in the manpage (in Linux RPMs).
(It's described in 'mysqldump -?' though)
Maybe someone could update the manpage as well?

The same bug appears on Windows 2000. Work around the same: add to the mysqldump command
line --single-transaction and --lock-tables=false

Hello

(answering to this report as #5834 was marked duplicate and has been closed)

The fix for this bug seems to be the following cvs commit:
http://lists.mysql.com/internals/16865
Is it save to apply this on the Debian package of MySQL which I maintain? Doing so would
be easier than telling people not do dump innodb databases without --single-transaction as
it's policy not to simply insert something into user edited config files like
/etc/mysql/my.cnf. A new minor versions won't come in the next 1-2 weeks, I guess?

bye,

-christian- <ch@debian.org>

Christian,

I think it is safe.

Regards,

Heikki

Fixed in 4.0.22.