Bug #12532 sql_thread causing memory overrun
Submitted: 11 Aug 2005 18:13 Modified: 26 Aug 2005 19:36
Reporter: Matthew Lord Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Replication Severity:S1 (Critical)
Version:5.0.9-5.0.12-bk OS:Linux (linux x86_64)
Assigned to: Lars Thalmann CPU Architecture:Any

[11 Aug 2005 18:13] Matthew Lord
Description:
certain queries that execute fine in a normal thread are causing memory overruns when run by the sql_thread.  A common one is /*!40000 ALTER TABLE `pp2insert` DISABLE KEYS
*/.  We tested this with NPTL and LinuxThreads with the same results.   It does seem to
be config related because I could not repeat the problem using --no-defaults.  We've tested
this on RHEL 3, 4 and FC 4, all x86_64 machines.

Here's some helpful info:
(gdb) bt
#0 0x00ec9a31 in malloc_consolidate () from /lib/i686/libc.so.6
#1 0x00ecae2d in _int_malloc () from /lib/i686/libc.so.6
#2 0x00ecc731 in malloc () from /lib/i686/libc.so.6
#3 0x08409534 in _mymalloc (size24, filename=0x8484c94 "sql_base.cc", lineno88, MyFlags) at safemalloc.c:138
#4 0x081df963 in open_table (thd=0x8cbd418, table_list=0x8cc4868, mem_root=0x8cbd438, refresh=0x6bec1e3, flags=0)
at sql_base.cc:1188
#5 0x081dff28 in open_ltable (thd=0x8cbd418, table_list=0x8cc4868, lock_type=TL_WRITE_ALLOW_READ) at sql_base.cc:2093
#6 0x0829263d in mysql_alter_table (thd=0x8cbd418, new_db=0x8c88e14 "productcoredb", new_name=0x0, create_info=0x8cbd930, 
table_list=0x8cc4868, fields=@0x8cbd864, keys=@0x8cbd858, order_num=0, order=0x0, handle_duplicates=DUP_ERROR, ignore├║lse, 
alter_info=0x8cbda68, do_send_ok=true) at sql_table.cc:3076
#7 0x081b0ddd in mysql_execute_command (thd=0x8cbd418) at sql_parse.cc:3011
#8 0x081b7455 in mysql_parse (thd=0x8cbd418, inBuf=0x8c88e22 "/*!40000 ALTER TABLE `pp2insert` DISABLE KEYS */", lengthH)
at sql_parse.cc:5383
#9 0x0823337a in Query_log_event::exec_event (this=0x8c88ff0, rli=0x8c97764, 
query_arg=0x8c88e22 "/*!40000 ALTER TABLE `pp2insert` DISABLE KEYS */", q_len_argH) at log_event.cc:1619
#10 0x082337f4 in Query_log_event::exec_event (this=0x8c88ff0, rli=0x8c97764) at log_event.cc:1512
#11 0x082c9065 in exec_relay_log_event (thd=0x8cbd418, rli=0x8c97764) at slave.cc:3255
#12 0x082c99f9 in handle_slave_sql (arg=0x8c91508) at slave.cc:3803
#13 0x0027c99f in pthread_start_thread () from /lib/i686/libpthread.so.0
#14 0x0027cae9 in pthread_start_thread_event () from /lib/i686/libpthread.so.0
#15 0x00f2c46a in clone () from /lib/i686/libc.so.6

mysql> Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'sql_base.cc:1188'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:147'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:133'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:983'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:994'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:1013'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:1014'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:1013'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:1014'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:106'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:461'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:197'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:818'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:147'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mulalloc.c:51'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:147'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mulalloc.c:51'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mi_open.c:672'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'lock.cc:475'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'lock.cc:475'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'lock.cc:347'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'sql_lex.cc:183'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'sql_lex.cc:184'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'lock.cc:242'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mi_close.c:63'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:106'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mi_close.c:104'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_open.c:106'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'mi_close.c:111'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'table.cc:932'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'array.c:236'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:288'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:288'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:288'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:288'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:288'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:294'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'my_alloc.c:294'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'sql_base.cc:244'
Error: Memory allocated at log_event.cc:1333 was overrun, discovered at 'log_event.h:760'

How to repeat:
I have everything setup on hammer:~/mlord/mysql-5.0.12-beta-linux-x86_64/.  The bk build was done on 8/11 and it is a debug build.

From within this directory do this:
rm data/*.info

./bin/mysqld --defaults-file=/home/mysqldev/mlord/mysql-5.0.12-beta-linux-x86_64/my.cnf --skip-grant-tables --basedir=. --datadir=./data --replicate-same-server-id &

./bin/mysql

change master to master_host="localhost", relay_log_file="./hammer-relay-bin.000001", relay_log_pos=243421; /* can also use position 4 to see other queries that cause the problem */

start slave sql_thread;

Suggested fix:
Have to find out why log_event is causing a memory overrun.
[13 Aug 2005 0:13] Alexander Pachev
Is there a chance you could test it on a 32-bit x86 Linux?

Also, any chance you could try it with a different kernel?
[17 Aug 2005 15:36] Doug Thompson
The problem originally showed up on our 64 bit Opteron, Redhat Enterprise 4
servers, but the debugging was done on a 32 bit AMD box running Fedora Core 4
using a snapshot of 5.0.12 dated August 11th, Same result.
[19 Aug 2005 14:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/28533
[23 Aug 2005 23:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/28729
[23 Aug 2005 23:54] Lars Thalmann
Thanks for your bug report.
The patches for this will be included in release 5.0.12.
[24 Aug 2005 9:15] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/28743
[26 Aug 2005 19:36] Paul Dubois
Noted in 5.0.12 changelog.