Bug #95491 The unused wake up in simulator AIO of function reserve_slot
Submitted: 23 May 2019 11:23 Modified: 2 Jun 2019 18:40
Reporter: Zongzhi Chen (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S5 (Performance)
Version:ALL OS:Any
Assigned to: CPU Architecture:Any

[23 May 2019 11:23] Zongzhi Chen
Description:
There is performance problem in the reserve_slot function.

when there is no empty slot in the local AIO array, then we need wake up the io_handler thread to execute the IO operation, and free some slots.
However, right now in the code InnoDB will wake up all the io_handler_threads(), not just the thread that has no empty slot in the AIO array.

The function is 

    if (!srv_use_native_aio) {
      /* If the handler threads are suspended,
      wake them so that we get more slots */
      os_aio_simulated_wake_handler_threads();
    }

In my test case, when we open the random_read_ahead = ON, and in the warmup phase, there is about 86.94% cpu waste in the raw_spin_lock, this is cause by the 
AIO::wake_simulated_handler_thread(ulint global_segment, ulint segment).
It has to call acquire(), and the implementation of mutex in linux is futex. The we will waste lots of cpu time.

I have change the code to wake up the specific thread like this way.

    if (!srv_use_native_aio) {
      /* If the handler threads are suspended,
      wake them so that we get more slots */

      if (type.is_read()) {
        wake_simulated_handler_thread(local_seg + 2);
      } else if (type.is_write()) {
        wake_simulated_handler_thread(local_seg + 2 + s_reads->m_n_segments);
      } else {
        os_aio_simulated_wake_handler_threads();
      }
    }

The the cpu time will decrease to  43.22% in my case. The reason is that we reduce the number of wake up times.

How to repeat:
open the  innodb_random_read_ahead = ON and use simulator AIO
And start the server, then there isn't any buffer pool in the memory and there would be lots of simulator AIO.
[23 May 2019 11:26] Zongzhi Chen
origin cpu usage

Attachment: origin.png (image/png, text), 684.56 KiB.

[23 May 2019 11:27] Zongzhi Chen
the optimized cpu usage

Attachment: Image.png (image/png, text), 1.12 MiB.

[27 May 2019 13:52] MySQL Verification Team
Hi Mr. Zongzhi,

Thank you for your bug report.

I agree with you that this is very nice performance enhancement !!!!

Verified as reported.
[2 Jun 2019 18:40] Zongzhi Chen
so what's the next step?
how can I summit the code?
[3 Jun 2019 12:51] MySQL Verification Team
Hi Mr. Zongzhi,

By verifying this bug, it will enter into scheduling process, about which we do not know anything, as there are other teams dealing with this task. The plans, when they are made, stay internal.

We do not understand your second question. What code would you like to submit ??? 

If it is a patch, just use the "Files" tab and upload it .....
[4 Jun 2019 18:05] Zongzhi Chen
The patch that solve this problem

(*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: wake_up.patch (application/octet-stream, text), 3.49 KiB.

[5 Jun 2019 11:48] MySQL Verification Team
Thank you very much Mr. Zongzhi.

Your contribution is very much appreciated.