MySQL Bugs: #27322: failure to allocate transaction_prealloc

Bug #27322	failure to allocate transaction_prealloc_size causes crash
Submitted:	21 Mar 2007 11:03	Modified:	1 May 2007 19:30
Reporter:	Shane Bester
Status:	Closed
Category:	Server: General	Severity:	S1 (Critical)
Version:	5.0.37	OS:	Microsoft Windows (Windows, linux)
Assigned to:	Bugs System	Target Version:
Tags:	crash, Memory, DoS

Description:
looks like failure to allocate memory for transaction_prealloc_size goes unnoticed, and
causes a crash.

this is important because normal users can cause a crash of a server.
Stack on linux:

0x80df631 handle_segfault + 529
0x83d798d __pthread_sighandler + 173
0x8375cc4 reset_root_defaults + 94
0x80e70df _Z18fix_trans_mem_rootP3THD13enum_var_type + 49
0x80ea61c _ZN7set_var6updateEP3THD + 100
0x80ea2a2 _Z17sql_set_variablesP3THDP4ListI12set_var_baseE + 176
0x80f9e2d _Z21mysql_execute_commandP3THD + 13907
0x80ff09e _Z11mysql_parseP3THDPcj + 308
0x80f5124 _Z16dispatch_command19enum_server_commandP3THDPcj + 1920
0x80f4999 _Z10do_commandP3THD + 529
0x80f3aca handle_one_connection + 874
0x83d2861 pthread_start_thread + 225
0x842d6fa __clone + 106
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow
instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it

Stack on windows:
mysqld-debug.exe!reset_root_defaults()  Line 103 + 0xf bytes	C
mysqld-debug.exe!fix_trans_mem_root()  Line 1414 + 0x23 bytes	C++
mysqld-debug.exe!set_var::update()  Line 3266 + 0x16 bytes	C++
mysqld-debug.exe!sql_set_variables()  Line 3148 + 0x11 bytes	C++
mysqld-debug.exe!mysql_execute_command()  Line 3791 + 0x10 bytes	C++
mysqld-debug.exe!mysql_parse()  Line 5928 + 0x9 bytes	C++
mysqld-debug.exe!dispatch_command()  Line 1795 + 0x17 bytes	C++
mysqld-debug.exe!do_command()  Line 1577 + 0x31 bytes	C++
mysqld-debug.exe!handle_one_connection()  Line 1191 + 0x9 bytes	C++
mysqld-debug.exe!pthread_start()  Line 62 + 0x7 bytes	C
mysqld-debug.exe!_callthreadstart()  Line 293 + 0xf bytes	C
mysqld-debug.exe!_threadstart()  Line 277	C

How to repeat:
set session transaction_prealloc_size=1024*1024*1024*1;
show processlist;
set session transaction_prealloc_size=1024*1024*1024*2;
show processlist;
set session transaction_prealloc_size=1024*1024*1024*3;
show processlist;
set session transaction_prealloc_size=1024*1024*1024*4;
show processlist;
set session transaction_prealloc_size=1024*1024*1024*5;

Suggested fix:
make sure that all such globally settable variables cannot cause a crash.

Thank you for a problem report. Verified just as described with 5.0.37 on Windows XP
(mysqld-nt.exe also crashes).

I can not repeat the behaviour described with latest 5.0.40-BK on Linux (with non-debug
binaries), though. Can you repeat it with non-debug binaries?

yes, all binaries are affected.  please try on 32-bit binaries and make sure you don't
have alot of free memory.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/24255

ChangeSet@1.2437, 2007-04-11 03:32:49-04:00, dkatz@damien-katzs-computer.local +1 -0
  Bug #27322  	failure to allocate transaction_prealloc_size causes crash
  
  Fix for when memory pre-allocation fails.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/24309

ChangeSet@1.2437, 2007-04-11 14:44:51-04:00, dkatz@damien-katzs-computer.local +3 -0
  Bug #27322  	failure to allocate transaction_prealloc_size causes crash
  
  Fix for when memory pre-allocation fails.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/24436

ChangeSet@1.2437, 2007-04-12 18:03:26-04:00, dkatz@damien-katzs-computer.local +3 -0
  Bug #27322  	failure to allocate transaction_prealloc_size causes crash
  
  Fix for when memory pre-allocation fails.

Pushed into 5.0.42

Pushed into 5.1.18-beta

Noted in 5.0.42, 5.1.18 changelogs.

Failure to allocate memory associated with transaction_prealloc_size
could cause a server crash.