MySQL Bugs: #29175: InnoDB cannot allocate memory anymore after mysql crash

Bug #29175	InnoDB cannot allocate memory anymore after mysql crash
Submitted:	18 Jun 2007 11:44	Modified:	18 Jul 2007 13:29
Reporter:	Oli Sennhauser	Email Updates:
Status:	No Feedback	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S3 (Non-critical)
Version:	4.1.20	OS:	Linux (Linux 2.6.9-42.0.8.ELsmp #1 SMP Tue Jan 30 12:33:47 EST 2007 i686 i686 i386 GNU/Linux)
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
Customer says that InnoDB is not able to restart after a MySQL crash and therefore they have to reboot the whole server.

How to repeat:
Most times it starts fine; sometimes it won't without restarting the server. Here's a sample log:

061222 00:15:23  mysqld restarted
061222  0:15:27  InnoDB: Error: cannot allocate 524304384 bytes of
InnoDB: memory with malloc! Total allocated memory
InnoDB: by InnoDB 6319072 bytes. Operating system errno: 12
InnoDB: Check if you should increase the swap file or
InnoDB: ulimits of your operating system.
InnoDB: On FreeBSD check you have compiled the OS with
InnoDB: a big enough maximum process size.
InnoDB: We keep retrying the allocation for 60 seconds...
061222 00:17:05  mysqld ended

We have 2GB memory, and only use ~40% of it because we cannot allocate more than 500MB to InnoDB buffer pool.

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
pending signals                 (-i) 1024
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 32764
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Just a couple more details:

mysql -V
mysql  Ver 14.7 Distrib 4.1.20, for redhat-linux-gnu (i686) using readline 4.3

uname -a
Linux 2.6.9-42.0.8.ELsmp #1 SMP Tue Jan 30 12:33:47 EST 2007 i686 i686 i386 GNU/Linux

cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
innodb_buffer_pool_size=500M;

Suggested fix:
no idea.

Oli,

does the customer have other processes running that could exhaust the memory?

Regards,

Heikki

There is Asterisk running, which spawns several (~20) concurrent PHP script processes which live for 1 to 3 minutes. This, however, accounts for little memory. MySQL will not start even with Asterisk stopped and no script processes running.

Nelson,

hmm... you should print 'top' with a terminal window long enough to show all processes and their virtual and resident memory sizes.

If there is a memory leak in the OS, we may get a hint of that, too.

I do not know what kind of an InnoDB bug could be behind this.

Regards,

Heikki

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".