Description:
On the page:
https://dev.mysql.com/doc/refman/8.0/en/large-page-support.html
The instruction to enable huge pages says the following:
---
As system root, open the file /etc/sysctl.conf in a text editor, and add the line shown here, where P is the number of large pages obtained in the previous step:
vm.nr_hugepages=P
Using the actual value obtained previously, the additional line should look like this:
vm.nr_huge_pages=64
---
These lines are inconsistent. As far as I know, the correct sysctl directive is vm.nr_hugepages, without the second underscore, as shown in the earlier line.
The confusion may arise because internally, there are kernel variables named nr_huge_pages/max_huge_pages and the overall feature is usually called "huge pages". It is also possible, as a later note suggests, that some OSes do use nr_huge_pages. But either way, it should be used consistently in the docs.
I also found that the number of huge pages was not quite enough, I needed:
vm.nr_hugepages=66
despite having the default innodb_buffer_pool_size of 128MB (134217728)
Perhaps there is some additional overhead not accounted for in buffer size.
How to repeat:
Read the page and notice that trying to set e.g.:
# sysctl -w vm.nr_huge_pages=64
sysctl: cannot stat /proc/sys/vm/nr_huge_pages: No such file or directory
and that if it does work that way, then it won't work setting vm.nr_hugepages=P as initially suggested.
Suggested fix:
* Adjust the documentation to consistently use vm.nr_hugepages
* Consider using 66 for the default number of huge pages rather than 64 if this does not actually allocate enough huge pages for the default setup, i.e. there is some error like "large_page_aligned_alloc mmap(138412032 bytes) failed" in the MySQL logs
* Perhaps also consider noting what is necessary on common OS distributions to enable locking large pages in memory, which may also restrict huge pages.
For what it's worth, on Debian with Ubuntu's package I needed to:
groupadd hugepages
gpasswd -a mysql hugepages
prlimit --pid 1 --memlock=unlimited
edit /etc/security/limits.conf to contain:
@hugepages soft memlock unlimited
@hugepages hard memlock unlimited
id mysql to get the hugepages group ID, e.g. 1001
sysctl -w vm.hugetlb_shm_group=1001
and add vm.hugetlb_shm_group=1001 to /etc/sysctl.conf
And also, when AppArmor was enabled - which it is by default - edit the file:
/etc/apparmor.d/usr.sbin.mysqld
so that the top line was
/usr/sbin/mysqld flags=(attach_disconnected) {
and there was a line before the }
owner / r,
and then run
apparmor_parser --replace /etc/apparmor.d/usr.sbin.mysqld
service mysql restart
...in addition to the nr_hugepages allocation. Failure to do the apparmor changes would result in log lines like:
kernel: audit: type=1400 audit(1716739783.915:14): apparmor="DENIED" operation="file_mmap" class="file" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="" pid=420487 comm="boot" requested_mask="r" denied_mask="r" fsuid=103 ouid=103
[Warning] [MY-012681] [InnoDB] large_page_aligned_alloc mmap(138412032 bytes) failed; errno 13
and without owner / r, it would instead have:
kernel: audit: type=1400 audit(1716741112.324:17): apparmor="DENIED" operation="file_mmap" class="file" profile="/usr/sbin/mysqld" name="/" pid=422377 comm="boot" requested_mask="r" denied_mask="r" fsuid=103 ouid=103
