Bug #32241 memory corruption due to large index map in 'Range checked for each record'
Submitted: 9 Nov 2007 18:16 Modified: 14 Jan 2008 20:11
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S2 (Serious)
Version:5.1.22, 6.0.3, 5.0.50 OS:Any
Assigned to: Alexey Kopytov CPU Architecture:Any
Tags: bfsm_2007_11_15
Triage: D1 (Critical)

[9 Nov 2007 18:16] Shane Bester
Description:
I found a buffer overflow warning when running mysqld in visual studio.
It says: Run-Time Check Failure #2 - Stack around the variable 'buf' was corrupted.

I tracked it down to running a certain EXPLAIN SELECT .. query. The cause is this piece of Extra information :

Range checked for each record (index map: 0x20004002090C)

In select_describe(), buf is a 9 byte buffer.
But, in this case there are 13 bytes written to it!

-----
char buf[MAX_KEY/8+1];
extra.append(STRING_WITH_LEN("; Range checked for each "
                          "record (index map: 0x"));
extra.append(tab->keys.print(buf));
extra.append(')');
-----

How to repeat:
will upload testcase later.

Suggested fix:
1) determine whether that index map is even valid.
2) if it is, make the buffer big enough.
[9 Nov 2007 18:53] Shane Bester
testcase: (note the length of the index map. overflows by 1 byte at least) More overflow with more indexes and columns.

drop table if exists `t1`;
create table `t1`(`c` int)engine=myisam;
insert into `t1` values (),();
drop table if exists `t2`;
create table `t2` (
`b` int,
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`),key(`b`),key(`b`),
key(`b`),key(`b`),key(`b`))engine=myisam;

insert into `t2` values (),(),();

explain
select 1
from
(select 1 from `t2`,`t1` where `b` < `c` group by 1 limit 1) as `d2`
;
[9 Nov 2007 18:55] Shane Bester
should be tested it under valgrind
[9 Nov 2007 18:58] Shane Bester
output of the testcase. notice the index map is longer than the buffer can hold

Attachment: bug32241_output.txt (text/plain), 3.07 KiB.

[16 Nov 2007 10:58] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/37938

ChangeSet@1.2549, 2007-11-16 13:58:09+03:00, kaa@polly.(none) +3 -0
  Fix for bug #32241: memory corruption due to large index map in 'Range
  checked for each record'
  
  The problem was in incorrectly calculated length of the buffer used to
  store a hexadecimal representation of an index map in
  select_describe(). This could result in buffer overrun and stack
  corruption under some circumstances.
  
  Fixed by correcting the calculation.
[14 Dec 2007 8:15] Bugs System
Pushed into 5.0.54
[14 Dec 2007 8:18] Bugs System
Pushed into 5.1.23-rc
[14 Dec 2007 8:21] Bugs System
Pushed into 6.0.5-alpha
[14 Jan 2008 20:11] Jon Stephens
Documented bugfix in the 5.0.54, 5.1.23, and 6.0.5 changelogs as follows:

        Memory corruption could occur due to large index map in
        Range checked for each record status reported
        by EXPLAIN SELECT. The problem was based in
        an incorrectly calculated length of the buffer used to
        store a hexadecimal representation of an index map, which could
        result in buffer overruns and stack corruption under some
        circumstances.