Bug #29856 Very long prepared statement in SP can cause crash with certain options
Submitted: 17 Jul 2007 20:57 Modified: 3 Aug 2007 16:43
Reporter: Harrison Fisk Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Stored Routines Severity:S2 (Serious)
Version:5.0.44,5.0.48BK OS:Linux (Ubuntu 6.06)
Assigned to: Evgeny Potemkin CPU Architecture:Any
Tags: binary log, crash, prepared statement, query cache

[17 Jul 2007 20:57] Harrison Fisk 
Description:
If you startup MySQL with both query cache and the binary log enabled, then if you give a long parameter to a stored procedure with a prepared statement in it, you can crash the server.  I have gotten both signal 6 and signal 11.  mysqld will continue accepting connections after this occurs sometimes.

Removing either binary logging or query cache will cause it not to crash.

The backtrace given in the error log resolves to:

0x8189d49 handle_segfault + 417
0xffffe410 _end + -140473248
0xb7dd42b9 _end + -1350702327
0xb7e0687a _end + -1350496054
0xb7e0cfd4 _end + -1350469596
0xb7e0d34a _end + -1350468710
0x83ab10d free_root + 137
0x8287b68 _ZN7sp_head7executeEP3THD + 872
0x8288ceb _ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE + 719
0x81a34f0 _Z21mysql_execute_commandP3THD + 18840
0x81a561f _Z11mysql_parseP3THDPKcjPS2_ + 231
0x819d7ae _Z16dispatch_command19enum_server_commandP3THDPcj + 1166
0x819d2e4 _Z10do_commandP3THD + 144
0x819c976 handle_one_connection + 646
0xb7f42341 _end + -1349203055
0xb7e734ee _end + -1350050498

Running in Valgrind on 5.0.48BK I get the following output from the crash:

==12254== Thread 7:
==12254== Invalid write of size 4
==12254==    at 0x82F69AC: Query_cache::send_result_to_client(THD*, char*, unsigned) (sql_cache.cc:1137)
==12254==    by 0x8339E00: sp_instr_stmt::execute(THD*, unsigned*) (sp_head.cc:2527)
==12254==    by 0x833377E: sp_head::execute(THD*) (sp_head.cc:1076)
==12254==    by 0x833841D: sp_head::execute_procedure(THD*, List<Item>*) (sp_head.cc:1728)
==12254==    by 0x81DBD10: mysql_execute_command(THD*) (sql_parse.cc:4637)
==12254==    by 0x81DDA3B: mysql_parse(THD*, char const*, unsigned, char const**) (sql_parse.cc:6084)
==12254==    by 0x81DE565: dispatch_command(enum_server_command, THD*, char*, unsigned) (sql_parse.cc:1812)
==12254==    by 0x81DFAB6: do_command(THD*) (sql_parse.cc:1586)
==12254==    by 0x81DFFA3: handle_one_connection (sql_parse.cc:1197)
==12254==    by 0x4053340: start_thread (in /lib/tls/i686/cmov/libpthread-2.3.6.so)
==12254==    by 0x418E4ED: clone (in /lib/tls/i686/cmov/libc-2.3.6.so)
==12254==  Address 0x7D8CDB4 is 0 bytes after a block of size 16,452 alloc'd
==12254==    at 0x401C422: malloc (vg_replace_malloc.c:149)
==12254==    by 0x845AF37: _mymalloc (safemalloc.c:137)
==12254==    by 0x845A743: alloc_root (my_alloc.c:201)
==12254==    by 0x845ADBF: strmake_root (my_alloc.c:406)
==12254==    by 0x810FA13: Query_arena::strmake(char const*, unsigned) (sql_class.h:755)
==12254==    by 0x8339BFF: subst_spvars(THD*, sp_instr*, st_lex_string*) (sp_head.cc:891)
==12254==    by 0x8339DB7: sp_instr_stmt::execute(THD*, unsigned*) (sp_head.cc:2520)
==12254==    by 0x833377E: sp_head::execute(THD*) (sp_head.cc:1076)
==12254==    by 0x833841D: sp_head::execute_procedure(THD*, List<Item>*) (sp_head.cc:1728)
==12254==    by 0x81DBD10: mysql_execute_command(THD*) (sql_parse.cc:4637)
==12254==    by 0x81DDA3B: mysql_parse(THD*, char const*, unsigned, char const**) (sql_parse.cc:6084)
==12254==    by 0x81DE565: dispatch_command(enum_server_command, THD*, char*, unsigned) (sql_parse.cc:1812)
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'my_alloc.c:64'

And for thread 7:

Thread 7: status = VgTs_Runnable
==12254==    at 0x401C422: malloc (vg_replace_malloc.c:149)
==12254==    by 0x845AF37: _mymalloc (safemalloc.c:137)
==12254==    by 0x845A3C6: init_alloc_root (my_alloc.c:62)
==12254==    by 0x820C9D2: open_tables(THD*, TABLE_LIST**, unsigned*, unsigned) (sql_base.cc:2623)
==12254==    by 0x820D160: open_and_lock_tables(THD*, TABLE_LIST*) (sql_base.cc:3032)
==12254==    by 0x81D5416: mysql_execute_command(THD*) (sql_parse.cc:2631)
==12254==    by 0x8335B04: sp_instr_stmt::exec_core(THD*, unsigned*) (sp_head.cc:2577)
==12254==    by 0x83358D4: sp_lex_keeper::reset_lex_and_exec_core(THD*, unsigned*, bool, sp_instr*) (sp_head.cc:2430)
==12254==    by 0x8339E34: sp_instr_stmt::execute(THD*, unsigned*) (sp_head.cc:2530)
==12254==    by 0x833377E: sp_head::execute(THD*) (sp_head.cc:1076)
==12254==    by 0x833841D: sp_head::execute_procedure(THD*, List<Item>*) (sp_head.cc:1728)
==12254==    by 0x81DBD10: mysql_execute_command(THD*) (sql_parse.cc:4637)
==12254==    by 0x81DDA3B: mysql_parse(THD*, char const*, unsigned, char const**) (sql_parse.cc:6084)
==12254==    by 0x81DE565: dispatch_command(enum_server_command, THD*, char*, unsigned) (sql_parse.cc:1812)
==12254==    by 0x81DFAB6: do_command(THD*) (sql_parse.cc:1586)
==12254==    by 0x81DFFA3: handle_one_connection (sql_parse.cc:1197)
==12254==    by 0x4053340: start_thread (in /lib/tls/i686/cmov/libpthread-2.3.6.so)
==12254==    by 0x418E4ED: clone (in /lib/tls/i686/cmov/libc-2.3.6.so)

How to repeat:
1.  Create a my.cnf with the following in it:

[mysqld]
log-bin
query_cache_size=64M

2.  Run the script file attached to the issue.

Suggested fix:
Make mysqld not crash.
[17 Jul 2007 20:58] Harrison Fisk 
File to make mysqld crash

Attachment: testcase_29856.txt (text/plain), 16.34 KiB.
[23 Jul 2007 19:06] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31415

ChangeSet@1.2535, 2007-07-23 18:10:12+04:00, evgen@moonbone.local +3 -0
  Bug#29856: Insufficient buffer space led to a server crash.
  
  The subst_spvars function is used to create query string with SP variables 
  substituted with their values. This string is used later for the binary log
  and for the query cache. The problem is that the
  query_cache_send_result_to_client function requires some additional space
  after the query to store database name and query cache flags. This 
  space wasn't reserved by the subst_spvars function which led to a memory
  corruption and crash.
  
  Now the subst_spvars function reserves additional space for the query cache.
[24 Jul 2007 19:58] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31510

ChangeSet@1.2534, 2007-07-24 23:54:08+04:00, evgen@moonbone.local +3 -0
  Bug#29856: Insufficient buffer space led to a server crash.
  
  The subst_spvars function is used to create query string with SP variables 
  substituted with their values. This string is used later for the binary log
  and for the query cache. The problem is that the
  query_cache_send_result_to_client function requires some additional space
  after the query to store database name and query cache flags. This 
  space wasn't reserved by the subst_spvars function which led to a memory
  corruption and crash.
  
  Now the subst_spvars function reserves additional space for the query cache.
[25 Jul 2007 8:12] Sergey Petrunya 
Review feedback provided on IRC
[28 Jul 2007 11:06] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31762

ChangeSet@1.2534, 2007-07-28 15:01:29+04:00, evgen@moonbone.local +3 -0
  Bug#29856: Insufficient buffer space led to a server crash.
  
  The subst_spvars function is used to create query string with SP variables 
  substituted with their values. This string is used later for the binary log
  and for the query cache. The problem is that the
  query_cache_send_result_to_client function requires some additional space
  after the query to store database name and query cache flags. This 
  space wasn't reserved by the subst_spvars function which led to a memory
  corruption and crash.
  
  Now the subst_spvars function reserves additional space for the query cache.
[2 Aug 2007 19:13] Bugs System 
Pushed into 5.1.21-beta
[2 Aug 2007 19:15] Bugs System 
Pushed into 5.0.48
[3 Aug 2007 16:43] Paul DuBois 
Noted in 5.0.48, 5.1.21 changelogs.

Very long prepared statements in stored procedures could cause a 
server crash.