Bug #61575 InnoDB: Error: io_setup() failed

Submitted: 21 Jun 2011 7:24 Modified: 15 May 2012 19:39
Reporter: Marko Mäkelä Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:5.5+ OS:Linux
Assigned to: Inaam Rana CPU Architecture:Any
Tags: aio, linux

[21 Jun 2011 7:24] Marko Mäkelä
Description:
This is from http://forums.mysql.com/read.php?11,406737,406737

MySQL Forums :: Install :: Install fails with error io_setup() failed seg fault

According to the Linux manual page for io_setup(), EAGAIN means that it is useless to retry with the same parameters: "The specified nr_events exceeds the user's limit of available events." InnoDB seems to think otherwise:

110208 11:56:51 InnoDB: Using Linux native AIO
110208 11:56:51 InnoDB: Warning: io_setup() failed with EAGAIN. Will make 5 attempts before giving up.
InnoDB: Warning: io_setup() attempt 1 failed.
InnoDB: Warning: io_setup() attempt 2 failed.
InnoDB: Warning: io_setup() attempt 3 failed.
InnoDB: Warning: io_setup() attempt 4 failed.
InnoDB: Warning: io_setup() attempt 5 failed.
110208 11:56:54 InnoDB: Error: io_setup() failed with EAGAIN after 5 attempts.
InnoDB: You can disable Linux Native AIO by setting innodb_native_aio = off in my.cnf
110208 11:56:54 InnoDB: Initializing buffer pool, size = 128.0M
110208 11:56:54 InnoDB: Completed initialization of buffer pool
110208 11:56:54 - mysqld got signal 11 ;

How to repeat:
Compile InnoDB on a Linux system that has libaio installed. Do not disable innodb_use_native_aio.

sudo sysctl fs.aio-max-nr=1
mysqld

This will lead to array==NULL being dereferenced in os_aio_linux_handle(), called by fil_aio_wait() in an io_handler_thread().

	/* Find the array and the local segment. */
	segment = os_aio_get_array_and_local_segment(&array, global_seg);
	n = array->n_slots / array->n_segments;

Suggested fix:
In innobase_start_or_create_for_mysql(), adjust the io_limit to whatever is reported by sysctl, instead of hard-coding it at

io_limit = 8 * SRV_N_PENDING_IOS_PER_THREAD;

Consider making this a configureable start-up parameter.

Refuse to start up if os_aio_init() fails.
[22 Jun 2011 12:24] Stewart Smith
note that we have hit this with the default setting for Ubuntu kernels and running mtr (or other tests). I can quite trivially hit this on laptop/desktop.
[15 May 2012 19:39] John Russell
Adding to the descriptions of innodb_read_io_threads and 
innodb_write_io_threads: 

On Linux systems, running multiple MySQL servers (typically more than 
12) with default settings for innodb_read_io_threads, 
innodb_write_io_threads, and the Linux aio-max-nr setting can exceed 
system limits. Ideally, increase the aio-max-nr setting; as a 
workaround, you might reduce the settings for one or both of the 
MySQL configuration options.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.039 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.