MySQL Bugs: #44781: InnoDB reporting Table Full rather than Disk Full when lacking disk space

Bug #44781	InnoDB reporting Table Full rather than Disk Full when lacking disk space
Submitted:	11 May 2009 11:10	Modified:	6 Jan 2011 15:47
Reporter:	Arjen Lentz	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Storage Engine API	Severity:	S3 (Non-critical)
Version:	4.1, 5.0, 5,1, 6.0 bzr	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	DB_OUT_OF_FILE_SPACE, ER_RECORD_FILE_FULL, HA_ERR_RECORD_FILE_FULL, innodb

Description:
Found by Peter Lieverdink (Open Query):

InnoDB appears to incorrectly map its out of file space error DB_OUT_OF_FILE_SPACE to A_ERR_RECORD_FILE_FULL (in ha_innodb.cc) which in turn gets reported as ER_RECORD_FILE_FULL
This means "Table Full", typically a condition of a MEMORY table reaching its max size, or a MyISAM table at its max pointer size, or a filesystem issue (2GB filesize limit for older filesystems).

However, a disk full situation should map to ER_DISK_FULL. Applications handle this error differently, as it's a different situation.

How to repeat:
In this particular case, innodb_file_per_table was enabled, and an import was trying to create a new table. The IBD file was partially created when the disk full situation occurred - the Table Full error was reported on "tablename.FRM" which also indicates that the errorcode is entirely wrong. The table create failed because the disk was really full (this was verified), not because a table or tablespace or whatever was full for any of the abovementioned reasons.

Suggested fix:
Correct the mapping. Thanks

Thank you for the report.

Verified as described.

Vasil, could you have a look?

The ER_DISK_FULL message is "Disk full (%s); waiting for someone to free some space..."

With the latest 5.1 I get this error if :

16:24:09 mysql> insert into t (select * from t);
ERROR 1114 (HY000): The table 't' is full

and InnoDB prints:

090522 16:26:03  InnoDB: Error: Write to file ./ibdata1 failed at offset 0 17825792.
InnoDB: 1048576 bytes should have been written, only -1 were written.
InnoDB: Operating system error number 28.
InnoDB: Check that your OS and file system support files of this size.
InnoDB: Check also that the disk is not full or a disk quota exceeded.
InnoDB: Error number 28 means 'No space left on device'.

But ER_DISK_FULL is not supposed to be used by the strage engines. The error codes available HA_ERR_* are in include/my_base.h and converted to text messages in sql/handler.cc:ha_init_errors(). There is no "disk full" error there!

Sorry, I hit submit before completing the sentence, it should read:

"With the latest 5.1 I get this error if there is no disk space left:"

Vasil:
Trigger a disk full on a MyISAM table, and see what it reports in errorlog (should be ER_DISK_FULL "Disk full (%s); waiting for someone to free some space...") then see in the MyISAM how it does it.

It may well have been hacked in, in the distant past. In that case the HA error array needs to be extended for you to fix the bug nicely, and a related bug filed on MyISAM so its code can be cleaned up.
I presume this also means that any other engine either a) does not check this condition or b) has the disk full error hacked in somehow, so would need cleaning up as well.
Interesting.

With MyISAM this is printed to the server log:

090525 12:29:22 [ERROR] ./libexec/mysqld: Disk is full writing './test/t.MYD' (Errcode: 28). Waiting for someone to free space... (Expect up to 60 secs delay for server to continue after freeing disk space)
090525 12:29:22 [ERROR] ./libexec/mysqld: Retry in 60 secs. Message reprinted in 600 secs
(this message repeats)

and an error is never returned to the client, the INSERT just hangs (at least for 15 mins).

With MyISAM:

In mysys/my_write.c:

 54     if ((my_errno == ENOSPC || my_errno == EDQUOT) &&
 55         (MyFlags & MY_WAIT_IF_FULL))
 56     {
 57       wait_for_free_space(my_filename(Filedes), errors);
 58       errors++;
 59       continue;
 60     }

if this condition is not true, and the codes does not wait for free space to be available, then this is printed to the server log:

090525 13:11:00 [ERROR] ./libexec/mysqld: Incorrect key file for table './test/t.MYI'; try to repair it

and the client gets:

13:10:57 mysql> insert into t select * from t;
ERROR 126 (HY000): Incorrect key file for table './test/t.MYI'; try to repair it
13:11:00 mysql>

A new error code HA_ERR_DISK_FULL should be added to my_base.h, changing category to "Storage Engines API".