Bug #4516 innodb buffer pool limit
Submitted: 12 Jul 2004 15:01 Modified: 12 Jul 2004 15:12
Reporter: Ethan Joffe Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:mysql-standard-4.0.20-pc-linux-i686 OS:Linux (Redhat 7.3)
Assigned to: CPU Architecture:Any

[12 Jul 2004 15:01] Ethan Joffe
Description:
I have 4GB of memory in the server, but I am not able to set the innodb_buffer_pool_size higher than around 1.9G. MySQL is pretty much the only thing running on the box so I should be able to use the suggested 80% of memory.

Here is the error file output:

040712 05:28:52  mysqld started
040712  5:28:52  Warning: Asked for 196608 thread stack, but got 126976
InnoDB: Fatal error: cannot allocate 2621456384 bytes of
InnoDB: memory with malloc! Total allocated memory
InnoDB: by InnoDB 24482660 bytes. Operating system errno: 12
InnoDB: Cannot continue operation!
InnoDB: Check if you should increase the swap file or
InnoDB: ulimits of your operating system.
InnoDB: On FreeBSD check you have compiled the OS with
InnoDB: a big enough maximum process size.
InnoDB: We now intentionally generate a seg fault so that
InnoDB: on Linux we get a stack trace.
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=402653184
read_buffer_size=2093056
max_used_connections=0
max_connections=100
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 802415 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x83f4800
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Bogus stack limit or frame pointer, fp=0xbfffed38, stack_bottom=0x10001, thread_stack=126976, aborting backtrace.
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x10001  is invalid pointer
thd->thread_id=1920103779
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
040712 05:28:52  mysqld ended

Here is my.cnf:

[mysqld]
port		= 3306
socket		= /tmp/mysql.sock
skip-locking
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
myisam_sort_buffer_size = 64M
thread_cache = 8
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8
....
innodb_data_home_dir = /usr/local/mysql/data/innodb
innodb_data_file_path = ibdata1:2000M;ibdata2:2000M;ibdata3:2000M
innodb_log_group_home_dir = /usr/local/mysql/data/logs/
innodb_log_arch_dir = /usr/local/mysql/data/logs/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 2500M
innodb_additional_mem_pool_size = 20M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 256M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 0
innodb_lock_wait_timeout = 50

Here is the output of ulimit -a:

core file size        (blocks, -c) 0
data seg size         (kbytes, -d) unlimited
file size             (blocks, -f) unlimited
max locked memory     (kbytes, -l) unlimited
max memory size       (kbytes, -m) unlimited
open files                    (-n) 1024
pipe size          (512 bytes, -p) 8
stack size            (kbytes, -s) 8192
cpu time             (seconds, -t) unlimited
max user processes            (-u) 7168
virtual memory        (kbytes, -v) unlimited

and the kernel version:
 2.4.18-3bigmem #1 SMP Thu Apr 18 07:17:10 EDT 2002 i686 unknown

How to repeat:

Just use an innodb_buffer_pool_size > 2000M
[12 Jul 2004 15:12] Heikki Tuuri
Hi!

32-bit operating systems typically limit the process memory to 2 GB.

In theory, InnoDB could use up to 4 GB in a 32-bit Linux computer, but that requires the right glibc and Linux versions to work.

Regards,

Heikki
[12 Jul 2004 15:20] Ethan Joffe
Well, I am using the bigmem kernel version and all 4G shows up, so presumably Linux is covered. Any idea what glibc to use (if there is even one that will work with RH7.3 that supports 4G), and then do I just compile MySQL from source?
[13 Jul 2004 10:14] Heikki Tuuri
Ethan,

sorry, I do not remember what glibc's should work with a 4 GB user space.

Since mysqld-max is dynamically linked, you do not need to recompile mysqld.

Actually, you should test mysqld-max. Maybe your computer already has a glibc version that works with 4 GB!

Regards,

Heikki