Bug #29419 Specifying a join_buffer > 4GB on 64 bit machines not possible.
Submitted: 28 Jun 2007 12:58 Modified: 25 Oct 2007 18:18
Reporter: Horst Hunger Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:5.1.16 OS:Linux (64 bit)
Assigned to: Ramil Kalimullin

[28 Jun 2007 12:58] Horst Hunger
Description:
A join_buffer_size of 3.5GB leads to a memory allocation of that value. If I set it to 5GB only about 1GB is allocated.  Seems to be a wrapping at the 4GB bound.

How to repeat:
on a machine with more that 4GB mem and 64bit processor:

Put the attached test program into "mysql-test/t", create a result file
and execute
./mysql-test-run.pl myisam_join_buffer_2

with join_buffer_size of 3.5GB and 5GB.

The allocated memory can be observed in the "top.log" generated by the test program in "mysql-test".
[28 Jun 2007 14:10] Alexey Kopytov
Verified using the myisam_join_buffer_2 test for bug #5731.
[29 Jun 2007 11:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29938

ChangeSet@1.2405, 2007-06-29 15:37:36+04:00, kaa@polly.local +1 -0
  Bug #29419 "Specifying a join_buffer > 4GB on 64 bit machines not possible."
  
  Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4 GB.
  
  The test case for this bug is a part of the test suite for bug #5731.
[27 Jul 2007 20:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31737

ChangeSet@1.2481, 2007-07-27 16:26:26-04:00, dkatz@damien-katzs-computer.local +1 -0
  Bug #29419 "Specifying a join_buffer > 4GB on 64 bit machines not possible."
  
  Use size_t instead of uint when calculating join buffer size, because uint can be overflown on 64-bit platforms and join_buffer_size > 4 GB.
  
  The test case for this bug is a part of the test suite for bug #5731.
[30 Jul 2007 21:20] Michael Widenius
Patch ok to push into 5.1
Note that on 5.0 we should just limit the buffer to 4G via my_getopt()
This is becasue my_malloc() is not 64 bit clean in 5.0
[1 Aug 2007 5:13] Ramil Kalimullin
We have also to limit the buffer to 4Gb in 5.0.
[2 Aug 2007 19:12] Bugs System
Pushed into 5.1.21-beta
[29 Aug 2007 9:34] Alexey Kopytov
Updating the status to "Patch queued" since the patch for this bug was pushed to 5.1.21
[18 Oct 2007 21:34] Bugs System
Pushed into 5.1.23-beta
[18 Oct 2007 21:36] Bugs System
Pushed into 5.0.52
[25 Oct 2007 18:18] Paul Dubois
Bug#5731, Bug#29419, Bug#29446 are related; this note appears in all three reports.

Noted in 5.0.52, 5.1.23 changelogs (and also 5.2.6 to note the changes explicitly because of discussion in these bug reports about work still to be done in 5.2).

Several buffer-size system variables were either being handled 
incorrectly for large values (for settings larger than 4GB, they were
truncated to values less than 4GB without a warning), or were limited
unnecessarily to 4GB even on 64-bit systems.

The following changes were made in 5.0.52: 

For key_buffer_size, values larger than 4GB are allowed on 64-bit 
platforms (except Windows, for which large values are truncated to
4GB with a warning). 

For join_buffer_size, sort_buffer_size, and myisam_sort_buffer_size,
values are limited to 4GB on all platforms. Larger values are
truncated to 4GB with a warning. 

The following changes were made in 5.1.23/5.2.6:

For key_buffer_size, values larger than 4GB are allowed on 64-bit 
platforms.  

For join_buffer_size, sort_buffer_size, and myisam_sort_buffer_size,
values larger than 4GB are allowed on 64-bit platforms (except
Windows, for which large values are truncated to 4GB with a warning). 

The following change was made in all versions:

Settings for read_buffer_size and read_rnd_buffer_size
are limited to 2GB on all platforms. Larger values are truncated to
2GB with a warning.
[29 Oct 2007 17:38] Paul Dubois
5.2.6 changes will appear in 6.0.3 instead.