Bug #29158 crash on cleanup after out of memory scenerio
Submitted: 17 Jun 2007 7:46 Modified: 17 Jun 2007 14:35
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S1 (Critical)
Version:5.0.42 OS:Windows
Assigned to: CPU Architecture:Any
Tags: cleanup, crash

[17 Jun 2007 7:46] Shane Bester
Description:
while investigating bug #29104 I hit another crash.

Stack trace:

mysqld-debug.exe!JOIN::cleanup
mysqld-debug.exe!JOIN::destroy
mysqld-debug.exe!st_select_lex::cleanup
mysqld-debug.exe!st_select_lex_unit::cleanup
mysqld-debug.exe!st_select_lex::cleanup
mysqld-debug.exe!mysql_select
mysqld-debug.exe!handle_select
mysqld-debug.exe!mysql_execute_command
mysqld-debug.exe!mysql_parse
mysqld-debug.exe!dispatch_command
mysqld-debug.exe!do_command
mysqld-debug.exe!handle_one_connection
mysqld-debug.exe!pthread_start
mysqld-debug.exe!_callthreadstart
mysqld-debug.exe!_threadstart
kernel32.dll!FlsSetValue

Crash:

if (tables > const_tables) // Test for not-const tables
{
   free_io_cache(table[const_tables]);  <------------here
   filesort_free_buffers(table[const_tables],full);
}

-table	0x03a89680	st_table * *
-const_tables	0x03a89258	unsigned int

The crash happens shortly after this appeared in the error logs:

070617  9:40:05 [ERROR] mysqld-debug: Out of memory at line 201, '.\my_alloc.c'
070617  9:40:05 [ERROR] mysqld-debug: needed -1218034160 byte (3004818k), memory in use: 9735342 bytes (9508k)
070617  9:40:05 [ERROR] Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space

How to repeat:
will try get a testcase uploaded soon.

Suggested fix:
.
[17 Jun 2007 14:35] MySQL Verification Team
doesn't crash in 5.0.44-BK.  seems to be fixed already.