Bug #35616 memory overrun on 64-bit linux on setting large values for keybuffer-size
Submitted: 27 Mar 2008 15:47 Modified: 16 Sep 2008 4:48
Reporter: Tatiana Azundris Nuernberg Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: General Severity:S3 (Non-critical)
Version:5.1.24-rc, 5.0 OS:Linux (64 bit)
Assigned to: Tatiana Azundris Nuernberg CPU Architecture:Any

[27 Mar 2008 15:47] Tatiana Azundris Nuernberg
Description:
setting really large values for keybuffer-size (2^63-1) causes memory overrun warnings on 64-bit.

How to repeat:
on 64-bit machine (e.g. blade09), try allocating a LOT of memory, e.g.

SET @@global.key_buffer_size = 9223372036854775807;

Then, check warnings. You should see overruns of the form

y allocated at my_largepage.c:64 was overrun, discovered at 'mf_keycache.c:441'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_base.cc:2978'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2019'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2032'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:169'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:169'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2057'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2058'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:845'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:130'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:989'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2057'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2058'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:1499'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:110'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:1631'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'mulalloc.c:50'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'lock.cc:843'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.h:190'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
[7 Apr 2008 2:25] Tatiana Azundris Nuernberg
my_static.h:
struct st_irem
{
  struct st_irem *next;		/* Linked list of structures	   */
  struct st_irem *prev;		/* Other link			   */
  char *filename;		/* File in which memory was new'ed */
  uint32 linenum;		/* Line number in above file	   */
! uint32 datasize;		/* Size requested		   */
  uint32 SpecialValue;		/* Underrun marker value	   */
};

safemalloc.c:_mymalloc() takes a size_t. Interestingly, we use that size_t for our pointers, so we write the end-of-block canary to the right place (64-bit safe), but then we irem->datasize=size, which obviously is lossy for size>UINT32_MAX.

Our checks use irem->datasize to calculate the end-of-block address, so if that value is wrong (because of above overflow), we won't find our canary there (which is at the correct address).

To wit, these checks can be disabled, but even if they are not, they just bring a warning, but do not result in further action!

Also, sf_malloc_cur_memory and friends are affected (we add the correct size on alloc, but deduct the overflown value), so our statistics are rubbish for chunks > UINT32_MAX.

Worst of all, _myrealloc() has (size is a size_t, irem->datasize a uint32):

  if ((data= _mymalloc(size,filename,lineno,MyFlags))) /* Allocate new area */
  {
    size=min(size, irem->datasize);		/* Move as much as possibly */
    memcpy((uchar*) data, ptr, (size_t) size);	/* Copy old data */

If we try to resize a segment that is already larger than UINT32_MAX, we might get the new block, but then only copy parts of the old data to the new buffer!

*** This might possibly result in DATA LOSS in the real world! ***

changing the struct to "size_t datasize" heals this.

Note to self: also uint32 in 5.0!
[7 Apr 2008 2:51] Tatiana Azundris Nuernberg
FWIW, same in 4.1
[28 Apr 2008 8:15] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46097

ChangeSet@1.2610, 2008-04-28 10:14:40+02:00, tnurnberg@noir.koehntopp.de +2 -0
  Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
  
  We could allocate chunks larger than 4GB, but did our size-accounting in 32-bit
  values. This could lead to spurious warnings, inaccurate accounting, and, in
  theory, data loss.
  
  Affected: 64-bit platforms. Debug-build (with safemalloc). At least one buffer
  larger than 4GB. For potential data loss, a re-alloc on such a buffer would be
  necessary.
[14 May 2008 15:19] Bugs System
Pushed into 5.0.62
[14 May 2008 15:21] Bugs System
Pushed into 5.1.25-rc
[22 May 2008 9:50] Bugs System
Pushed into 6.0.6-alpha
[16 Jun 2008 19:05] Paul DuBois
Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

On 64-bit systems, assigning values of 2^63 − 1 or larger to
key_buffer_size caused memory overruns.
[20 Jun 2008 19:54] Paul DuBois
I have pulled the changelog entry pending determination of which trees the fix ends up getting pushed to.
[19 Aug 2008 14:00] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51955

2709 Tatiana A. Nurnberg	2008-08-19
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      We could allocate chunks larger than 4GB, but did our
      size-accounting in 32-bit values. This could lead to
      spurious warnings, inaccurate accounting, and, in
      theory, data loss.
        
      Affected: 64-bit platforms. Debug-build (with safemalloc).
      At least one buffer larger than 4GB. For potential data
      loss, a re-alloc on such a buffer would be necessary.
[19 Aug 2008 14:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51957

2776 Tatiana A. Nurnberg	2008-08-19 [merge]
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      We could allocate chunks larger than 4GB, but did our
      size-accounting in 32-bit values. This could lead to
      spurious warnings, inaccurate accounting, and, in
      theory, data loss.
        
      Affected: 64-bit platforms. Debug-build (with safemalloc).
      At least one buffer larger than 4GB. For potential data
      loss, a re-alloc on such a buffer would be necessary.
[21 Aug 2008 8:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52189

2793 Tatiana A. Nurnberg	2008-08-21 [merge]
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      portability fixes
[21 Aug 2008 18:03] Bugs System
Pushed into 5.1.28  (revid:azundris@mysql.com-20080821050609-fepipt2c5x62fpd5) (version source revid:azundris@mysql.com-20080821050609-fepipt2c5x62fpd5) (pib:3)
[25 Aug 2008 18:01] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52476

2711 Tatiana A. Nurnberg	2008-08-21
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      additional fixes to eliminate warnings on some platforms
[25 Aug 2008 18:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52477

2711 Tatiana A. Nurnberg	2008-08-21
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      portability fixes / cleanup
[25 Aug 2008 20:17] Paul DuBois
Noted in 5.1.28 changelog.

On 64-bit systems, assigning values of 2^63 − 1 or larger to
key_buffer_size caused memory overruns.

Setting report to NDI pending push of fix to 6.0.x.
[13 Sep 2008 19:35] Bugs System
Pushed into 6.0.6-alpha  (revid:azundris@mysql.com-20080819141019-bu76qdr6hid2yj8x) (version source revid:sergefp@mysql.com-20080611231653-nmuqmw6dedjra79i) (pib:3)
[16 Sep 2008 4:48] Paul DuBois
Noted in 6.0.6 changelog.