MySQL Bugs: #48263: Allocate buffer pool in parallel on boxes with multiple CPUs/cores

Bug #48263	Allocate buffer pool in parallel on boxes with multiple CPUs/cores
Submitted:	23 Oct 2009 10:43	Modified:	13 May 2010 16:12
Reporter:	Valeriy Kravchuk	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S4 (Feature request)
Version:	5.1	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
InnoDB needs a lot of time to start up with huge buffer pool, even after clean shutdown. For 40G one may need to wait from 10 (Nehalem at 2.93GHz) to 23 seconds on some older AMDs, for example.

Now, when 128G and more of RAM becoming a commodity, as well as multiple CPUs/cores, it would be nice to speed up this allocation somehow using more than one CPU for this task.

Allocation is currently done using buf_chunk_init() function, as one big chunk it seems. Maybe we can get some benefit from allocating buffer pool "concurrently", as several smaller chunks (number of chunks can be related to number of CPUs/cores or innodb_thread_concurrency, for example).

How to repeat:
Try to start MySQL server with innodb_buffer_pool_size = 40G or more. Note that only one CPU is busy during this long enough process.

Suggested fix:
Allocate InnoDB buffer pool as a set of smaller chunks with total size more or less equals to innodb_buffer_pool. Try do do this in several cincurrent threads maybe.

In any case, we have to do something to speed up buffer pool initialization. Otherwise it is hard to create good HA solutions for servers with InnoDB storage engine as primary one.