| Bug #43606 | 4GB Limit on huge_pages shared memory set-up | ||
|---|---|---|---|
| Submitted: | 12 Mar 2009 17:02 | Modified: | 18 Dec 2009 20:45 |
| Reporter: | Cor Cornelisse (OCA) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: General | Severity: | S2 (Serious) |
| Version: | 5.1 | OS: | Linux (64-bit Intel/AMD) |
| Assigned to: | Alexey Kopytov | CPU Architecture: | Any |
| Tags: | Contribution, Huge_Pages Shared-Memory 4G Limit | ||
[23 Mar 2009 16:28]
Susanne Ebrecht
Many thanks for writing a bug report. Do you have a stack trace for us?
[24 Mar 2009 10:32]
Cor Cornelisse
I'm afraid I don't have a stack-trace available, the machines I patched this on are all back in production, so creating one would require a lot of work for an obvious bug if you ask me. May I suggest you compile and run the following to illustrate what I think is happening ;)
=========================
#include <stdio.h>
int main (int argc, char **argv)
{
unsigned int largePageSize = 2097152;
unsigned long iWasSupposedToBeBig = 42946967296 * 8;
printf("%lu, %lu \n", largePageSize, iWasSupposedToBeBig);
iWasSupposedToBeBig = ((iWasSupposedToBeBig - 1) &~ (largePageSize - 1)) + largePageSize;
printf("%lu, %lu \n", largePageSize, iWasSupposedToBeBig);
}
=========================
If you replace the line
iWasSupposedToBeBig = ((iWasSupposedToBeBig - 1) &~ (largePageSize - 1)) + largePageSize;
by
iWasSupposedToBeBig = ((iWasSupposedToBeBig - 1) &~ ((unsigned long) largePageSize - 1)) + largePageSize;
[26 May 2009 9:03]
Masood Mortazavi
Cor -- Since you seem interested in contributing to MySQL, may I suggest that you sign the Sun|MySQL contributor agreement, i.e. the SCA. The instructions are given here: http://forge.mysql.com/wiki/Sun_Contributor_Agreement This will facilitate the handling of your contributions -- this one, and others in the future.
[17 Aug 2009 10:06]
Susanne Ebrecht
This could be a duplicate of bug#39842
[17 Aug 2009 10:53]
Cor Cornelisse
Surely looks like it, same behavior I've noticed! Patch should fix it though.... btw: I've signed the SCA, it'll probably take a week or so before it's processed, so I'm not on the list yet.
[21 Aug 2009 5:01]
Susanne Ebrecht
Bug #39842 is set as duplicate of this bug here.
[21 Aug 2009 5:02]
Susanne Ebrecht
Bug #46849 is set as duplicate of this bug here.
[1 Sep 2009 14:50]
liz drachnik
Hi Cor - I have not received an SCA request - please make sure you go here The process is explained here: http://forge.mysql.com/wiki/Sun_Contributor_Agreement Thank you !
[1 Sep 2009 14:52]
liz drachnik
Hi Cor - I have not received an SCA request - please make sure you go here The process is explained here: http://forge.mysql.com/wiki/Sun_Contributor_Agreement Thank you !
[18 Sep 2009 7:19]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/83686 3105 Alexey Kopytov 2009-09-18 Bug #43606: 4GB Limit on huge_pages shared memory set-up Large pages allocator could not allocate more than 4 GB due to incorrect size alignment. @ mysys/my_largepage.c Large pages allocator could not allocate more than 4 GB due to incorrect size alignment.
[30 Sep 2009 8:16]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20090929093622-1mooerbh12e97zux) (version source revid:alexey.kopytov@sun.com-20090918075600-4ia2ieidbv6pxh3u) (merge vers: 6.0.14-alpha) (pib:11)
[1 Oct 2009 0:34]
Paul DuBois
Noted in 6.0.14 changelog. Attempts to enable large_pages with a shared memory segment larger than 4GB caused a server crash. Setting report to NDI pending push into 5.1.x, 5.4.x.
[6 Oct 2009 9:01]
Bugs System
Pushed into 5.1.40 (revid:joro@sun.com-20091006073316-lea2cpijh9r6on7c) (version source revid:alexey.kopytov@sun.com-20090918075438-mfl49t57w22l6cws) (merge vers: 5.1.39) (pib:11)
[7 Oct 2009 0:47]
Paul DuBois
Noted in 5.1.40 changelog. Setting report to NDI pending push into 5.4.x.
[22 Oct 2009 7:07]
Bugs System
Pushed into 5.5.0-beta (revid:alik@sun.com-20091022060553-znkmxm0g0gm6ckvw) (version source revid:alik@sun.com-20091013094238-g67x6tgdm9a7uik0) (merge vers: 5.5.0-beta) (pib:13)
[22 Oct 2009 19:54]
Paul DuBois
Noted in 5.5.0 changelog.
[18 Dec 2009 10:33]
Bugs System
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:48]
Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:03]
Bugs System
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:18]
Bugs System
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
Description: In an attempt to enable huge_pages with a shared memory segment of over 4GB (48GB in this case), MySQL would crash because it runs out of memory, though it only allocates at most 4GB of huge_pages, it exits at signal 11. A quick look at the source showed the computation done to calculate the size of the memory segment is stored in an int (32bit), thus limited to 4GB. Also IPC_CREAT appears to be necessary to allocate these amounts of memory correctly. See patch in suggested fix section, code speaks for itself. How to repeat: Configure large_pages in my.cnf Enable huge_page support in kernel Set ulimit to unlimited for mysql as well as the root user. Configure shmmax shmall and nr_hugepages according to your key_buffer memory requirements (more then 4GB to reproduce the bug) Load the indexes into ram, watch it die Suggested fix: --- my_largepage.c.orig 2009-03-12 17:27:55.000000000 +0100 +++ my_largepage.c 2009-03-12 17:39:08.000000000 +0100 @@ -121,9 +121,9 @@ DBUG_ENTER("my_large_malloc_int"); /* Align block size to my_large_page_size */ - size = ((size - 1) & ~(my_large_page_size - 1)) + my_large_page_size; + size = ((size - 1size_t) & ~( (size_t)my_large_page_size - 1size_t)) + (size_t)my_large_page_size; - shmid = shmget(IPC_PRIVATE, size, SHM_HUGETLB | SHM_R | SHM_W); + shmid = shmget(IPC_PRIVATE, size, SHM_HUGETLB | IPC_CREAT | SHM_R | SHM_W); if (shmid < 0) { if (my_flags & MY_WME)