MySQL Bugs: #33583: Falcon memory parameters on 32bit system can be set

Bug #33583	Falcon memory parameters on 32bit system can be set > 4GB
Submitted:	30 Dec 2007 2:30	Modified:	15 May 2009 13:06
Reporter:	Alexey Stroganov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S3 (Non-critical)
Version:	6.0.4-alpa-p1	OS:	Linux (i686 (32bit only))
Assigned to:	Sergey Vojtovich	CPU Architecture:	Any
Tags:	F_MEMORY

Description:
On 32bit system it is possible to set falcon memory parameters bigger than 4Gb that cause server freeze, random crashes, etc.

mysql-6.0.4-alpha-p1-linux-i686-glibc23> ./bin/mysqld --falcon-page-cache-size=5G

mysqladmin var -uroot | grep falcon_page_cache_size

| falcon_page_cache_size              | 5368709120                                                            

If I am trying to do the same with InnoDB I'm getting error and engine is disabled.

mysql-6.0.4-alpha-p1-linux-i686-glibc23> ./bin/mysqld --innodb-buffer-pool=5G

3:15:47 [ERROR] innobase_buffer_pool_size can't be over 4GB on 32-bit systems
3:15:47 [ERROR] Plugin 'InnoDB' init function returned error.
3:15:47 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.

How to repeat:
Start i686 binary with --falcon-page-cache-size=5G and check value of this parameter with:

mysqladmin var -uroot | grep falcon_page_cache_size

Chris Powers suggested this fix to  
void Configuration::setRecordMemoryMax(uint64 value)
{
    uint64 maxAddress = (uint64(1) << sizeof(void *)*8) - 1;
    recordMemoryMax = MIN(value, maxAddress);
. . .
}
Jim says "A little conditional compilation in the parameter definition would let 
the server give a good diagnostic if the configuration file contains an 
unreasonable value and would eliminate the need for active code."
The problem is to find a macro that would indicate any and all current and future 64 bit systems.  If we don't have on, this code will have to be revisited for new ports.  On the other hand, this run-time switch is only run at system startup, so it is not a CPU burner.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/40551

ChangeSet@1.2770, 2008-01-03 23:44:16-06:00, cpowers@xeno.mysql.com +3 -0
  Bug#33538 "Falcon memory parameters on 32bit system can be set > 4GB"
  - Changed the maximum for falcon_record_memory_max from 0xFFFFFFFFFFFFFFFF to the
maximum addressable memory, i.e. 4GB for 32-bit systems, etc.

Commit has wrong bug number.

Falcon now sets the maximum record cache memory to the highest memory addressable by the system, e.g. 4GB for 32-bit systems, etc. Prior to this change, the maximum allowable value for the record cache was 0xFFFFFFFFFFFFFFFF.

Note that the server silently adjusts out-of-range plugin parameters, such that setting falcon_record_memory_max to 5GB on a 32-bit system will not result in an error, although the parameter will be capped at 4GB.

Also note that InnoDB makes a special case for the innobase_buffer_pool_size parameter by explicitly verifying its value and, as mentioned above, throws an error if the value exceeds the 4GB on a 32-bit system.

There is no compelling reason for Falcon to selectively verify configuration parameters, particularly if the min, max and default values are provided to the server via the plugin handler. Doing so would be contrary to the design of the storage interface, and would also result in inconsistent storage engine behavior.

Still need to assign system-specific limit for falcon_page_cache_size.

Sergey,  Could you take a crack at this?  Chris has too many other issues at the moment.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/61356

2933 Sergey Vojtovich	2008-12-11
      BUG#33583 - Falcon memory parameters on 32bit system can be set > 4GB
      
      Fixed that falcon_page_cache_size can be set to value bigger than
      maximum addressable memory.
      
      With this fix, if falcon_page_cache_size is set to bigger than
      maximum addressable memory, it's value is adjusted to maximum
      addressable memory and a warning is issued to the error log.

Looks good, patch approved

Pushed into 6.0.10-alpha (revid:alik@sun.com-20090211182317-uagkyj01fk30p1f8) (version source revid:svoj@mysql.com-20081211122815-4tfohm7lh6zqqqu0) (merge vers: 6.0.9-alpha) (pib:6)

An entry has been added to the 6.0.10 changelog: 

It was possible to set Falcon memory parameters to values larger than the maximum memory supported by the supported by the host environment.