Bug #68025 Server crash by null pointer dereferencing in mem_heap_create_block()
Submitted: 4 Jan 2013 2:26 Modified: 14 Mar 2013 19:06
Reporter: Tianyin Xu Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S2 (Serious)
Version:5.5.28 OS:Linux
Assigned to: CPU Architecture:Any

[4 Jan 2013 2:26] Tianyin Xu
Description:
In mem_heap_create_block(), the code does not check the return pointer of malloc but directly manipulate it.  

If the malloc() fails and returns an empty value, mysqld server crashes due to null pointer dereferencing.

The code snippet is as follows (mem_area_alloc further calls malloc):

/*mysql-5.5.28/storage/innobase/mem/mem0mem.c*/
mem_heap_create_block(...) {
    ...
    block = mem_area_alloc(&len, mem_comm_pool);
    ...
    block->buf_block = buf_block;
    ...
}

How to repeat:
The simplest way to repeat is to set "innodb_log_file_size" to a large size, e.g., 50GB, to manually let malloc fail.

$ mysqld --innodb_log_buffer_size=50GB
130103 18:19:51 InnoDB: The InnoDB memory heap is disabled
130103 18:19:51 InnoDB: Mutexes and rw_locks use GCC atomic builtins
130103 18:19:51 InnoDB: Compressed tables use zlib 1.2.3.3
130103 18:19:51 InnoDB: Initializing buffer pool, size = 128.0M
130103 18:19:51 InnoDB: Completed initialization of buffer pool
02:19:51 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=0
max_threads=151
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338489 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
bin/mysqld(my_print_stacktrace+0x35)[0x872734]
bin/mysqld(handle_fatal_signal+0x3bf)[0x716a17]
/lib/libpthread.so.0(+0xf8f0)[0x7f3d032308f0]
bin/mysqld[0x9d138c]
bin/mysqld[0x9c6bc1]
bin/mysqld[0x9c6d26]
bin/mysqld[0x9c7b9e]
bin/mysqld[0x92ba5c]
bin/mysqld[0x8eb68b]
bin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x81)[0x717e8e]
bin/mysqld[0x5df297]
bin/mysqld(_Z11plugin_initPiPPci+0x5ad)[0x5dfb1d]
bin/mysqld[0x54a466]
bin/mysqld(_Z11mysqld_mainiPPc+0x43e)[0x54af03]
bin/mysqld(main+0x20)[0x545394]
/lib/libc.so.6(__libc_start_main+0xfd)[0x7f3d01eb2c4d]
bin/mysqld[0x5452b9]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Suggested fix:
Check the return pointer to avoid null pointer dereferencing and also print out a log message (the current error log doesn't show the root cause).
[4 Jan 2013 2:42] MySQL Verification Team
The value is capped on windows, so 32-bit binaries crash easier:

[Warning] option 'innodb-log-buffer-size': signed value 214748364800 adjusted to 2147482624

mysqld.exe!mem_heap_create_block()[mem0mem.c:366]
mysqld.exe!mem_heap_create_func()[mem0mem.ic:439]
mysqld.exe!mem_alloc_func()[mem0mem.ic:532]
mysqld.exe!log_init()[log0log.c:787]
mysqld.exe!innobase_start_or_create_for_mysql()[srv0start.c:1430]
mysqld.exe!innobase_init()[ha_innodb.cc:2590]
mysqld.exe!ha_initialize_handlerton()[handler.cc:543]
mysqld.exe!plugin_initialize()[sql_plugin.cc:1100]
mysqld.exe!plugin_init()[sql_plugin.cc:1386]
mysqld.exe!init_server_components()[mysqld.cc:3904]
mysqld.exe!win_main()[mysqld.cc:4483]
mysqld.exe!mysql_service()[mysqld.cc:4678]
mysqld.exe!mysqld_main()[mysqld.cc:4872]
mysqld.exe!__tmainCRTStartup()[crt0.c:266]

On linux, we get
Invalid write of size 8
at: mem_heap_create_block (mem0mem.cc:358)
by: log_init() (mem0mem.ic:434)
by: innobase_start_or_create_for_mysql() (srv0start.cc:1873)
by: innobase_init(void*) (ha_innodb.cc:3269)
by: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
by: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
by: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
by: init_server_components() (mysqld.cc:4753)
by: mysqld_main(int, char**) (mysqld.cc:5329)
by: (below main) (libc-start.c:226)
Address 0x70 is not stack'd, malloc'd or (recently) free'd
[14 Mar 2013 19:06] Bugs System
Added changelog entry for 5.5.32, 5.6.12, 5.7.2.

Starting "mysqld" with "--innodb_log_buffer_size=50GB" would fail to
allocate memory and return NULL. For non-debug builds there was no check
in place and a segmentation fault would occur. This fix adds a log message
stating that memory failed to be allocated, and adds an assertion.