Bug #105982 hard-coded libaio API parameter may not helpful to performance
Submitted: 27 Dec 2021 8:11 Modified: 3 Jan 2022 8:02
Reporter: haochen he Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Storage Engines Severity:S5 (Performance)
Version:all, 5.7, 8.0 OS:Linux
Assigned to: CPU Architecture:Any
Tags: io, libaio, performance

[27 Dec 2021 8:11] haochen he
Description:
Today's devices usually have deep IO queues.
 
  "AHCI/SATA has one command queue with a depth of 32 commands, 
   NVMe is designed to have up to 65,535 queues with as many as
   65,536 commands per queue"

While I notice that the default IO engine (libaio) hard-codes the batch size to 1:

./mysql-8.0.20/storage/innobase/os/os0file.cc:2205:
In Function: LinuxAIOHandler::resubmit(Slot *slot)
	int ret = io_submit(m_array->io_ctx(m_segment), 1, &iocb);
							\___________________
									    |
./mysql-8.0.20/storage/innobase/os/os0file.cc:2540:  			    |
In Function: AIO::linux_dispatch(Slot *slot)				   \|/
	int ret = io_submit(m_aio_ctx[io_ctx_index], 1, &iocb);			   
						     \_______________\ Hard-Coded
								     /
./mysql-8.0.20/storage/innobase/os/os0file.cc:2322:		           /|\    
In Function: LinuxAIOHandler::collect()			    		    |
	auto ret = io_getevents(io_ctx, 1, m_n_slots, events, &timeout);    |
					\___________________________________|

I am doubtful if this may be helpful to performance. Frequently submit single IOs
May make the system frequently do context switch. It can be fruitful if MySQL can
make this configurable in case the strage device is not the bottlenect.

Experiments can prove the performance gain:

	https://github.com/kkourt/aio-test

	./aio-test -f file -w 100 -q 16
				      \_____
	./aio-test -f file -w 100 -q 1 	    \ compare performance of different depth
				      \_____/
	In a traditional HDD, there is no difference.
	In a NVMe SSD, there is 2x more gain!

How to repeat:
Read the code

Suggested fix:
Make the io-depth configurable.
[3 Jan 2022 8:02] MySQL Verification Team
Hi haochen he,

Thank you for the report and feedback.

regards,
Umesh