MySQL Bugs: #60788: InnoDB crashes with an assertion failure when receiving a signal on pwrite()

Bug #60788	InnoDB crashes with an assertion failure when receiving a signal on pwrite()
Submitted:	7 Apr 2011 7:10	Modified:	17 Dec 2012 18:32
Reporter:	Alexey Kopytov	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S3 (Non-critical)
Version:		OS:	Any
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
According to POSIX write()/pwrite() return -1 with errno set to EINTR when those calls are interrupted by a signal before they write any data, or return the number of bytes written if a signal is received after successfully writing some data.

Neither of those cases are handled correctly by InnoDB. Here is an example of pwrite() returning -1 with errno set to EINTR:

InnoDB: Error: Write to file ./test/test.ibd failed at offset 0 5308416.
InnoDB: 983040 bytes should have been written, only -1 were written.
InnoDB: Operating system error number 4.
InnoDB: Check that your OS and file system support files of this size.
InnoDB: Check also that the disk is not full or a disk quota exceeded.
InnoDB: Error number 4 means 'Interrupted system call'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: Assertion failure in thread 1140169024 in file os/os0file.c line 4205
InnoDB: Failing assertion: ret
InnoDB: We intentionally generate a memory trap.

MyISAM is not affected by this problem because mysys handles those cases correctly by restarting the write.

I think all of the above applies to reads as well.

How to repeat:
Examine the os_file_pwrite()/os_file_pread() code. Compare with how EINTR is handled in my_pwrite()/my_pread().

Verified by code review. This is what we see in my_pwrite():

...
    if ((writenbytes= pwrite(Filedes, Buffer, Count,offset)) == Count)
      break;
    my_errno= errno;
...
    if ((writenbytes && writenbytes != (size_t) -1) || my_errno == EINTR)
      continue;                                 /* Retry */
...

Nothing like this in os_file_pwrite() implementation.

Duplicate/ will be fixed by bug#54160

This bug has been closed as a duplicate of bug 54160, which in turn
has been fixed in 5.7 presumably by [1].  That fix handles the
partial read/write case correctly, but not the EINTR case with -1
returned (i.e. interruption before the first byte transferred).  By
code review it looks like the following would happen for such
EINTR with -1 from syscall: writes would return failure without any
retries, and reads would be retried or failed depending on
srv_use_native_aio value (which here would apply to sync reads).

If this is correct, then this bug cannot be considered fully fixed.

[1]

mysql-server$ bzr log -r 4334
------------------------------------------------------------
revno: 4334
committer: Aditya A <aditya.a@oracle.com>
branch nick: mysql-trunk
timestamp: Mon 2012-08-27 10:57:55 +0530
message:
  BUG#11761646 - INNODB SHOULD RETRY ON FAILED READ OR WRITE, 
                 NOT IMMEDIATELY PANIC
  Related:
  http://bugs.mysql.com/bug.php?id=54430
  (innodb should retry partial reads/writes where errno was 0)
  
  Summary
  -------
  In the Linux flavours the read and write system calls can 
  do partial reads and writes.
  
  The man page of read says that " The return value is 
  is smaller than the number of bytes requested; this 
  may happen for example because fewer bytes are actually
  available right now (maybe because we were close to
  end-of-file,or  because  we  are  reading from  a pipe, 
  or from a terminal), or because read() was interrupted by
  a signal."         
  
  
  The Fix
  -------
  Initially InnoDB was not handling the partial read and 
  writes.With this fix innodb tries NUM_RETRIES_ON_PARTIAL_IO 
  (which by default is equal to 10)times to read or write the
  partial data .The fix also takes care of partial read and
  write in case of Linux native API's where read and write 
  are asynyncronus operations.
  
  rb:1158 [approved by inaam.rana]