MySQL Bugs: #30405: InnoDB tablespace free space info corruption

Bug #30405	InnoDB tablespace free space info corruption
Submitted:	14 Aug 2007 11:45	Modified:	25 Feb 2014 19:11
Reporter:	Axel Schwenke	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S4 (Feature request)
Version:	5.0.33	OS:	Linux
Assigned to:		CPU Architecture:	Any

Description:
The following InnoDB table space corruption was found on multiple different Linux systems (both x686 and x86_64):

InnoDB: Dump of the tablespace extent descriptor: len 40; hex 00000000000000020000000006b60000000004fe00000004feffffffffffffffbfffffffaffeffff; asc ;
InnoDB: Serious error! InnoDB is trying to free page 2119
InnoDB: though it is already marked as free in the tablespace!
InnoDB: The tablespace free space info is corrupt.
InnoDB: You may need to dump your InnoDB tables and recreate the whole
InnoDB: database!
InnoDB: Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
InnoDB: Assertion failure in thread 1162185056 in file fsp0fsp.c line 2980
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
InnoDB: Thread 1159522656 stopped in file row0sel.c line 2088
mysqld got signal 11;

We have 4 reported cases with always the same assertion failing (fsp0fsp.c line 2980) but for different InnoDB pages.

It is suspected that the problem comes from running multiple mysqld instances on the same InnoDB table space.

How to repeat:
Investigating. Could not yet reproduce.

Axel,

yes, that was the main suspect when Sunny and I studied this in June 2007. The .err logs contained in almost all cases complaints about two mysqld servers running on the same pid file.

--Heikki

Heikki,

thanks for confirming my guess. But how can this happen? AFAIK InnoDB puts an exclusive lock on the tablespace files (the error seems to be in the global table space, otherwise the customer runs with innodb-file-per-table). So normally InnoDB should not start up, but complain about not being able to lock the files. I remember to have seen those messages already.

So it seems the locking must be broken. Are there any usual suspects to check? The problem has been witnessed in the field but could not get reproduced for month now!

Calvin,

please look at this bug report.

My guess is that the users have been running two instances of mysqld on the same data files. The mystery is why InnoDB's data file advisory locking did not prevent that (advisory locking should work at least in Linux).

Look at the .err files and try to determine if mysqld complained about multiple instances listening on the same port.

Regards,

Heikki

Axel - could you please get the .err files from the customer(s)?

Thanks,
Calvin

Change it to feature request. According to the error log, it seems multiple instances could still be running corrently (do not know why). We may implement extra means to prevent it from happening.

This could be a duplicate of Bug #55284.

Similar internal bug was fixed in version 5.6.17, so I assume this one is closed too. Please wait next release, try it and if problem still exists feel free to reopen the report.