MySQL Bugs: #35616: memory overrun on 64-bit linux on setting large values for keybuffer-size

Bug #35616	memory overrun on 64-bit linux on setting large values for keybuffer-size
Submitted:	27 Mar 2008 15:47	Modified:	16 Sep 2008 4:48
Reporter:	Tatiana Azundris Nuernberg	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: General	Severity:	S3 (Non-critical)
Version:	5.1.24-rc, 5.0	OS:	Linux (64 bit)
Assigned to:	Tatiana Azundris Nuernberg	CPU Architecture:	Any

Description:
setting really large values for keybuffer-size (2^63-1) causes memory overrun warnings on 64-bit.

How to repeat:
on 64-bit machine (e.g. blade09), try allocating a LOT of memory, e.g.

SET @@global.key_buffer_size = 9223372036854775807;

Then, check warnings. You should see overruns of the form

y allocated at my_largepage.c:64 was overrun, discovered at 'mf_keycache.c:441'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_base.cc:2978'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2019'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2032'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:169'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:169'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2057'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2058'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:845'
y allocated at my_largepage.c:64 was overrun, discovered at 'handler.cc:130'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:989'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2057'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:2058'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:1499'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:110'
y allocated at my_largepage.c:64 was overrun, discovered at 'table.cc:1631'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'transparent_file.cc
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_alloc.c:201'
y allocated at my_largepage.c:64 was overrun, discovered at 'mulalloc.c:50'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'my_open.c:151'
y allocated at my_largepage.c:64 was overrun, discovered at 'lock.cc:843'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.h:190'
y allocated at my_largepage.c:64 was overrun, discovered at 'sql_string.cc:51'

my_static.h:
struct st_irem
{
  struct st_irem *next;		/* Linked list of structures	   */
  struct st_irem *prev;		/* Other link			   */
  char *filename;		/* File in which memory was new'ed */
  uint32 linenum;		/* Line number in above file	   */
! uint32 datasize;		/* Size requested		   */
  uint32 SpecialValue;		/* Underrun marker value	   */
};

safemalloc.c:_mymalloc() takes a size_t. Interestingly, we use that size_t for our pointers, so we write the end-of-block canary to the right place (64-bit safe), but then we irem->datasize=size, which obviously is lossy for size>UINT32_MAX.

Our checks use irem->datasize to calculate the end-of-block address, so if that value is wrong (because of above overflow), we won't find our canary there (which is at the correct address).

To wit, these checks can be disabled, but even if they are not, they just bring a warning, but do not result in further action!

Also, sf_malloc_cur_memory and friends are affected (we add the correct size on alloc, but deduct the overflown value), so our statistics are rubbish for chunks > UINT32_MAX.

Worst of all, _myrealloc() has (size is a size_t, irem->datasize a uint32):

  if ((data= _mymalloc(size,filename,lineno,MyFlags))) /* Allocate new area */
  {
    size=min(size, irem->datasize);		/* Move as much as possibly */
    memcpy((uchar*) data, ptr, (size_t) size);	/* Copy old data */

If we try to resize a segment that is already larger than UINT32_MAX, we might get the new block, but then only copy parts of the old data to the new buffer!

*** This might possibly result in DATA LOSS in the real world! ***

changing the struct to "size_t datasize" heals this.

Note to self: also uint32 in 5.0!

FWIW, same in 4.1

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46097

ChangeSet@1.2610, 2008-04-28 10:14:40+02:00, tnurnberg@noir.koehntopp.de +2 -0
  Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
  
  We could allocate chunks larger than 4GB, but did our size-accounting in 32-bit
  values. This could lead to spurious warnings, inaccurate accounting, and, in
  theory, data loss.
  
  Affected: 64-bit platforms. Debug-build (with safemalloc). At least one buffer
  larger than 4GB. For potential data loss, a re-alloc on such a buffer would be
  necessary.

Pushed into 5.0.62

Pushed into 5.1.25-rc

Pushed into 6.0.6-alpha

Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

On 64-bit systems, assigning values of 2^63 − 1 or larger to
key_buffer_size caused memory overruns.

I have pulled the changelog entry pending determination of which trees the fix ends up getting pushed to.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51955

2709 Tatiana A. Nurnberg	2008-08-19
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      We could allocate chunks larger than 4GB, but did our
      size-accounting in 32-bit values. This could lead to
      spurious warnings, inaccurate accounting, and, in
      theory, data loss.
        
      Affected: 64-bit platforms. Debug-build (with safemalloc).
      At least one buffer larger than 4GB. For potential data
      loss, a re-alloc on such a buffer would be necessary.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/51957

2776 Tatiana A. Nurnberg	2008-08-19 [merge]
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      We could allocate chunks larger than 4GB, but did our
      size-accounting in 32-bit values. This could lead to
      spurious warnings, inaccurate accounting, and, in
      theory, data loss.
        
      Affected: 64-bit platforms. Debug-build (with safemalloc).
      At least one buffer larger than 4GB. For potential data
      loss, a re-alloc on such a buffer would be necessary.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52189

2793 Tatiana A. Nurnberg	2008-08-21 [merge]
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      portability fixes

Pushed into 5.1.28  (revid:azundris@mysql.com-20080821050609-fepipt2c5x62fpd5) (version source revid:azundris@mysql.com-20080821050609-fepipt2c5x62fpd5) (pib:3)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52476

2711 Tatiana A. Nurnberg	2008-08-21
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      additional fixes to eliminate warnings on some platforms

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/52477

2711 Tatiana A. Nurnberg	2008-08-21
      Bug#35616: memory overrun on 64-bit linux on setting large values for keybuffer-size
      
      portability fixes / cleanup

Noted in 5.1.28 changelog.

On 64-bit systems, assigning values of 2^63 − 1 or larger to
key_buffer_size caused memory overruns.

Setting report to NDI pending push of fix to 6.0.x.

Pushed into 6.0.6-alpha  (revid:azundris@mysql.com-20080819141019-bu76qdr6hid2yj8x) (version source revid:sergefp@mysql.com-20080611231653-nmuqmw6dedjra79i) (pib:3)

Noted in 6.0.6 changelog.