MySQL Bugs: #29419: Specifying a join_buffer > 4GB on 64 bit machines not possible.

Bug #29419	Specifying a join_buffer > 4GB on 64 bit machines not possible.
Submitted:	28 Jun 2007 12:58	Modified:	25 Oct 2007 18:18
Reporter:	Horst Hunger	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S2 (Serious)
Version:	5.1.16	OS:	Linux (64 bit)
Assigned to:	Ramil Kalimullin	CPU Architecture:	Any

Description:
A join_buffer_size of 3.5GB leads to a memory allocation of that value. If I set it to 5GB only about 1GB is allocated.  Seems to be a wrapping at the 4GB bound.

How to repeat:
on a machine with more that 4GB mem and 64bit processor:

Put the attached test program into "mysql-test/t", create a result file
and execute
./mysql-test-run.pl myisam_join_buffer_2

with join_buffer_size of 3.5GB and 5GB.

The allocated memory can be observed in the "top.log" generated by the test program in "mysql-test".

Verified using the myisam_join_buffer_2 test for bug #5731.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29938

ChangeSet@1.2405, 2007-06-29 15:37:36+04:00, kaa@polly.local +1 -0
  Bug #29419 "Specifying a join_buffer > 4GB on 64 bit machines not possible."
  
  Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4 GB.
  
  The test case for this bug is a part of the test suite for bug #5731.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31737

ChangeSet@1.2481, 2007-07-27 16:26:26-04:00, dkatz@damien-katzs-computer.local +1 -0
  Bug #29419 "Specifying a join_buffer > 4GB on 64 bit machines not possible."
  
  Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4 GB.
  
  The test case for this bug is a part of the test suite for bug #5731.

Patch ok to push into 5.1
Note that on 5.0 we should just limit the buffer to 4G via my_getopt()
This is becasue my_malloc() is not 64 bit clean in 5.0

We have also to limit the buffer to 4Gb in 5.0.

Pushed into 5.1.21-beta

Updating the status to "Patch queued" since the patch for this bug was pushed to 5.1.21

Pushed into 5.1.23-beta

Pushed into 5.0.52

Bug#5731, Bug#29419, Bug#29446 are related; this note appears in all three reports.

Noted in 5.0.52, 5.1.23 changelogs (and also 5.2.6 to note the changes explicitly because of discussion in these bug reports about work still to be done in 5.2).

Several buffer-size system variables were either being handled 
incorrectly for large values (for settings larger than 4GB, they were
truncated to values less than 4GB without a warning), or were limited
unnecessarily to 4GB even on 64-bit systems.

The following changes were made in 5.0.52: 

For key_buffer_size, values larger than 4GB are allowed on 64-bit 
platforms (except Windows, for which large values are truncated to
4GB with a warning). 

For join_buffer_size, sort_buffer_size, and myisam_sort_buffer_size,
values are limited to 4GB on all platforms. Larger values are
truncated to 4GB with a warning. 

The following changes were made in 5.1.23/5.2.6:

For key_buffer_size, values larger than 4GB are allowed on 64-bit 
platforms.  

For join_buffer_size, sort_buffer_size, and myisam_sort_buffer_size,
values larger than 4GB are allowed on 64-bit platforms (except
Windows, for which large values are truncated to 4GB with a warning). 

The following change was made in all versions:

Settings for read_buffer_size and read_rnd_buffer_size
are limited to 2GB on all platforms. Larger values are truncated to
2GB with a warning.

5.2.6 changes will appear in 6.0.3 instead.