Bug #30396 memory overrun during EXPLAIN EXTENDED
Submitted: 13 Aug 2007 22:56 Modified: 25 Aug 2007 15:58
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S1 (Critical)
Version:5.1.22,5.0.48 OS:Any
Assigned to: Igor Babaev CPU Architecture:Any
Tags: crash, explain, memory overrun

[13 Aug 2007 22:56] Shane Bester
Description:
I got a crash today:

070813 21:30:35  InnoDB: Started; log sequence number 0 1444508499
070813 21:30:35 [Note] /home/sbester/server/5.1/mysql-5.1.22-beta-linux-i686/bin/mysqld: ready for connections.
Version: '5.1.22-beta-debug'  socket: '/tmp/mysql.sock'  port: 3306  yes
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at ¨¤:150483528 was underrun, discovered at my_alloc.c:201
Error: Memory allocated at ¨¤:150483528 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at 070814  0:36:47 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=209715200
read_buffer_size=131072
max_used_connections=8
max_threads=151
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 534229 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x8fe71f8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x4ea2c2a4, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81ff55b handle_segfault + 541
0x4012c903 _end + 932837683
0x40106d6f _end + 932683167
0x84ceecc _checkchunk + 79
0x84cf05e _sanity + 119
0x84ce3a2 _mymalloc + 146
0x84cde57 alloc_root + 314
0x82d25e1 _ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyyb + 1175
0x825fc1e _Z22get_quick_record_countP3THDP10SQL_SELECTP8st_tablePK6BitmapILj64EEy + 158
0x82610e6 _Z20make_join_statisticsP4JOINP10TABLE_LISTP4ItemP16st_dynamic_array + 5126
0x825bc6f _ZN4JOIN8optimizeEv + 1889
0x825fa49 _Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select + 553
0x827bec7 _Z19mysql_explain_unionP3THDP18st_select_lex_unitP13select_result + 593
0x8213d2e _Z21execute_sqlcom_selectP3THDP10TABLE_LIST + 312
0x820cc18 _Z21mysql_execute_commandP3THD + 1648
0x8215928 _Z11mysql_parseP3THDPKcjPS2_ + 372
0x820b06d _Z16dispatch_command19enum_server_commandP3THDPcj + 2369
Stack trace seems successful - bottom reached
Please read http://dev.mysql.com/doc/refman/5.1/en/resolve-stack-dump.html
and follow instructions on how to resolve the stack trace.
Resolved stack trace is much more helpful in diagnosing the
problem, so please do resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...

How to repeat:
will simplify testcase later.

Suggested fix:
.
[13 Aug 2007 23:25] Shane Bester
run this sql to get a crash on 5.1

Attachment: bug30396_reduced_testcase.sql (application/octet-stream, text), 2.99 KiB.

[13 Aug 2007 23:29] Shane Bester
The memory overruns seem pretty severe.  Even the error messages themselves appear corrupt...

Version: '5.1.22-beta-debug'  socket: '/tmp/mysql.sock'  port: 3306  yes
Error: Memory allocated at ¨‘X‰ðâ:148766136 was underrun, discovered at my_alloc.c:201
Error: Memory allocated at ¨‘X‰ðâ:148766136 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at :-1515870811 was underrun, discovered at my_alloc.c:201
Error: Memory allocated at :-1515870811 was overrun, discovered at 'my_alloc.c:201'
Error: Memory allocated at (null):0 was underrun, discovered at my_alloc.c:201
Error: Memory allocated at (null):0 was overrun, discovered at 'my_alloc.c:201'
Error: Safemalloc link list destroyed, discovered at 'my_alloc.c:201'
root=0x8de99c0,count=197,irem=(nil)
Error: Memory allocated at ¨‘X‰ðâ:148766136 was underrun, discovered at my_alloc.c:355
Error: Memory allocated at ¨‘X‰ðâ:148766136 was overrun, discovered at 'my_alloc.c:355'
Error: Memory allocated at :-1515870811 was underrun, discovered at my_alloc.c:355
Error: Memory allocated at :-1515870811 was overrun, discovered at 'my_alloc.c:355'
Error: Memory allocated at (null):0 was underrun, discovered at my_alloc.c:355
Error: Memory allocated at (null):0 was overrun, discovered at 'my_alloc.c:355'
Error: Safemalloc link list destroyed, discovered at 'my_alloc.c:355'
[14 Aug 2007 8:32] Shane Bester
I tried to simplify the crashing query, but strange I could only get a crash with 6 or more joins :(
[15 Aug 2007 1:17] Igor Babaev
I succeeded to cause a memory corruption with 5.0 when running the following script:

DROP DATABASE IF EXISTS db_bug30396;
CREATE DATABASE db_bug30396
USE db_bug30396;

CREATE TABLE t1 (
  c1 int(11) NOT NULL AUTO_INCREMENT,
  c2 varchar(1000) DEFAULT NULL,
  c3 bigint(20) DEFAULT NULL,
  c4 bigint(20) DEFAULT NULL,
  PRIMARY KEY (c1)
);

EXPLAIN EXTENDED 

SELECT  join_2.c1  

FROM 
	t1 AS join_0, 
	t1 AS join_1, 
	t1 AS join_2, 
	t1 AS join_3, 
	t1 AS join_4, 
	t1 AS join_5, 
	t1 AS join_6, 
	t1 AS join_7
WHERE 
	join_0.c1=join_1.c1  AND 
	join_1.c1=join_2.c1  AND 
	join_2.c1=join_3.c1  AND 
	join_3.c1=join_4.c1  AND 
	join_4.c1=join_5.c1  AND 
	join_5.c1=join_6.c1  AND 
	join_6.c1=join_7.c1 
         OR 
	join_0.c2 < '?'  AND 
	join_1.c2 < '?'  AND
	join_2.c2 > '?'  AND
	join_2.c2 < '!'  AND
	join_3.c2 > '?'  AND 
	join_4.c2 = '?'  AND 
	join_5.c2 <> '?' AND
	join_6.c2 <> '?' AND 
	join_7.c2 >= '?' AND
        join_0.c1=join_1.c1  AND 
	join_1.c1=join_2.c1  AND 
        join_2.c1=join_3.c1  AND
	join_3.c1=join_4.c1  AND 
	join_4.c1=join_5.c1  AND 
	join_5.c1=join_6.c1  AND 
	join_6.c1=join_7.c1
GROUP BY 
	join_3.c1,
	join_2.c1,
	join_7.c1,
	join_1.c1,
	join_0.c1;
	
SHOW WARNINGS;

On the server side I had:

070814 18:17:43 [Note] /home/igor/mysql-5.0-opt/sql/mysqld: ready for connections.
Version: '5.0.48-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
[New Thread -1256752224 (LWP 23073)]
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'array.c:123'
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'array.c:123'
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'array.c:123'
Error: Memory allocated at my_alloc.c:201 was overrun, discovered at 'my_alloc.c:201'

...

Program received signal SIGABRT, Aborted.
[15 Aug 2007 17:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32584

ChangeSet@1.2495, 2007-08-15 10:24:18-07:00, igor@olga.mysql.com +7 -0
  Fixed bug #30396.
  The bug caused memory corruption for some queries with top OR level
  in the WHERE condition if they contained equality predicates and 
  other sargable predicates in disjunctive parts of the condition.
  
  The corruption happened because the upper bound of the memory
  allocated for KEY_FIELD and SARGABLE_PARAM internal structures
  containing info about potential lookup keys was calculated incorrectly
  in some cases. In particular it was calculated incorrectly when the
  WHERE condition was an OR formula with disjuncts being AND formulas
  including equalities and other sargable predicates.
[23 Aug 2007 16:51] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32981

ChangeSet@1.2569, 2007-08-23 22:24:31+05:00, gshchepa@gleb.loc +7 -0
  Fixed bug #30396.
  Recommit to 5.1.22.
  The bug caused memory corruption for some queries with top OR level
  in the WHERE condition if they contained equality predicates and 
  other sargable predicates in disjunctive parts of the condition.
  
  The corruption happened because the upper bound of the memory
  allocated for KEY_FIELD and SARGABLE_PARAM internal structures
  containing info about potential lookup keys was calculated incorrectly
  in some cases. In particular it was calculated incorrectly when the
  WHERE condition was an OR formula with disjuncts being AND formulas
  including equalities and other sargable predicates.
[23 Aug 2007 20:50] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32995

ChangeSet@1.2571, 2007-08-24 02:23:49+05:00, gshchepa@gleb.loc +7 -0
  Fixed bug #30396.
  Recommit to 5.1.22.
  The bug caused memory corruption for some queries with top OR level
  in the WHERE condition if they contained equality predicates and 
  other sargable predicates in disjunctive parts of the condition.
  
  The corruption happened because the upper bound of the memory
  allocated for KEY_FIELD and SARGABLE_PARAM internal structures
  containing info about potential lookup keys was calculated incorrectly
  in some cases. In particular it was calculated incorrectly when the
  WHERE condition was an OR formula with disjuncts being AND formulas
  including equalities and other sargable predicates.
[24 Aug 2007 7:20] Bugs System
Pushed into 5.1.22-beta
[24 Aug 2007 7:22] Bugs System
Pushed into 5.0.48
[25 Aug 2007 15:58] Paul Dubois
Noted in 5.0.48, 5.1.22 changelogs.

Memory corruption occurred for some queries with a top-level OR
operation in the WHERE condition if they contained equality
predicates and other sargable predicates in disjunctive parts of the
condition.
[4 Sep 2007 17:11] Bugs System
Pushed into 5.1.23-beta