Bug #45989 memory leak after explain encounters an error in the query
Submitted: 7 Jul 2009 7:10 Modified: 20 Jun 2010 22:36
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S2 (Serious)
Version:5.1.37, 5.1.42, 5.4.4, 6.0.12 OS:Any
Assigned to: Georgi Kodinov CPU Architecture:Any
Tags: Leak, memory leak
Triage: Triaged: D2 (Serious)

[7 Jul 2009 7:10] Shane Bester
Description:
16 bytes in 1 blocks are definitely lost in loss record 1 of 3
at: malloc (vg_replace_malloc.c:149)
by: my_malloc (my_malloc.c:34)
by: String::real_alloc(unsigned) (sql_string.cc:51)
by: String::copy(String const&) (sql_string.h:202)
by: do_save_blob(Copy_field*) (field_conv.cc:296)
by: do_copy_null(Copy_field*) (field_conv.cc:207)
by: copy_fields(TMP_TABLE_PARAM*) (sql_select.cc:15059)
by: end_write_group(JOIN*, st_join_table*, bool) (sql_select.cc:12409)
by: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:11253)
by: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:11138)
by: do_select(JOIN*, List<Item>*, st_table*, Procedure*) (sql_select.cc:10894)
by: JOIN::exec() (sql_select.cc:1959)

How to repeat:
...
[14 Jul 2009 9:14] Shane Bester
--------------
testcase
--------------

drop table if exists `t1`;
create table `t1`(`a` longtext)engine=innodb charset=utf8;
insert into `t1` values (repeat('a',@@global.max_allowed_packet));
insert into `t1` values (repeat('b',@@global.max_allowed_packet));
explain extended select distinct 1 from `t1`,
(select distinctrow `a` as `away` from `t1` group by `a` with rollup) as `d1`  
where `t1`.`a` = `d1`.`a`;
 

The query will fail with error message:
ERROR 1054 (42S22): Unknown column 'd1.a' in 'where clause'

But it also leaks potentially alot of memory (5.1.37 stack trace):

 1,048,584 bytes in 1 blocks are definitely lost in loss record 3 of 3
 at: malloc (vg_replace_malloc.c:149)
 by: my_malloc (my_malloc.c:34)
 by: String::real_alloc(unsigned) (sql_string.cc:51)
 by: String::copy(String const&) (sql_string.h:202)
 by: do_save_blob(Copy_field*) (field_conv.cc:296)
 by: do_copy_null(Copy_field*) (field_conv.cc:207)
 by: copy_fields(TMP_TABLE_PARAM*) (sql_select.cc:15060)
 by: end_write_group(JOIN*, st_join_table*, bool) (sql_select.cc:12413)
 by: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:11257)
 by: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:11142)
 by: do_select(JOIN*, List<Item>*, st_table*, Procedure*) (sql_select.cc:10898)
 by: JOIN::exec() (sql_select.cc:1799)
[1 Sep 2009 7:44] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/82086

2795 Sergey Glukhov	2009-09-01
      Bug#45989 memory leak after explain encounters an error in the query
      Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
      The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
     @ mysql-test/r/explain.result
        test result
     @ mysql-test/t/explain.test
        test case
     @ sql/sql_select.cc
        Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
        The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
[4 Sep 2009 8:21] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/82418

2808 Sergey Glukhov	2009-09-04
      Bug#45989 memory leak after explain encounters an error in the query
      Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
      The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
     @ mysql-test/r/explain.result
        test result
     @ mysql-test/t/explain.test
        test case
     @ sql/sql_select.cc
        Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
        The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.
[14 Sep 2009 16:03] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (version source revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (merge vers: 5.4.4-alpha) (pib:11)
[18 Sep 2009 12:35] Tor Didriksen
tmp_join->tmp_join= 0;
-    tmp_table_param.copy_field=0;
+    tmp_table_param.cleanup();
     DBUG_RETURN(tmp_join->destroy());
   }

This code may call delete[] on something allocated in MEM_ROOT

The test main.subselect_notembedded consistently dumps core on me now.
I have verified, by tracing allocation
  new (thd->mem_root) Copy_field[field_count]
vs
  new Copy_field[field_count]
and deallocation
that this is indeed what happens.

$sql/mysqld --version
sql/mysqld  Ver 6.0.14-alpha-debug for pc-linux-gnu on i686 (Source distribution)
[22 Sep 2009 11:48] Tor Didriksen
Please disregard explanation in previous comment, I misunderstood what happens when we do 
  new (thd->mem_root) Copy_field[field_count]
vs
  new Copy_field[field_count]
and then delete[] the array later 
(array delete operator does TRASH, and actually does not free anything)

What *is* certain though, is that main.subselect_notembedded dumps core.

From the trace, I see that the same array of Copy_field objects is deleted twice
(which should be OK, since we actually don't delete anything)

Note that Copy_field::tmp (which is a string) is garbled in all objects, after
we have run ~Copy_field() on the array the first time.
This is done because the 'placement delete' will TRASH the entire array, since we have -DSAFEMALLOC on the command line.
So, the second time we delete this array, String::free dumps core.

Stack trace:
#4  <signal handler called>
#5  0x081e782f in String::free (this=0x69f4dfa0) at ../../sql/sql_string.h:196
#6  0x081e7893 in ~String (this=0x69f4dfa0) at ../../sql/sql_string.h:90
#7  0x0821d05a in ~Copy_field (this=0x69f4df74) at field.h:2155
#8  0x0821dc5c in TMP_TABLE_PARAM::cleanup (this=0x996cb70) at sql_class.h:2968
#9  0x08361bf4 in JOIN::cleanup (this=0x9967ee8, full=true) at sql_select.cc:10657
#10 0x0836f3a4 in JOIN::destroy (this=0x9967ee8) at sql_select.cc:2947
#11 0x0836f37c in JOIN::destroy (this=0x9948288) at sql_select.cc:2943
#12 0x0849db05 in st_select_lex::cleanup (this=0x9921008) at sql_union.cc:803
#13 0x0849d942 in st_select_lex_unit::cleanup (this=0x9915d10) at sql_union.cc:669
#14 0x0849db69 in st_select_lex::cleanup (this=0x99202d8) at sql_union.cc:810
#15 0x0849d942 in st_select_lex_unit::cleanup (this=0x9920470) at sql_union.cc:669
#16 0x0849db69 in st_select_lex::cleanup (this=0x991f5a8) at sql_union.cc:810
#17 0x0849d942 in st_select_lex_unit::cleanup (this=0x991f740) at sql_union.cc:669
#18 0x0849db69 in st_select_lex::cleanup (this=0x9881950) at sql_union.cc:810
#19 0x0849d942 in st_select_lex_unit::cleanup (this=0x9881ae8) at sql_union.cc:669
#20 0x0849db69 in st_select_lex::cleanup (this=0x9880c20) at sql_union.cc:810
#21 0x0849d942 in st_select_lex_unit::cleanup (this=0x9880db8) at sql_union.cc:669
#22 0x0849db69 in st_select_lex::cleanup (this=0x983e160) at sql_union.cc:810
#23 0x0849d942 in st_select_lex_unit::cleanup (this=0x983e2f8) at sql_union.cc:669
#24 0x0849db69 in st_select_lex::cleanup (this=0x983d430) at sql_union.cc:810
#25 0x0849d942 in st_select_lex_unit::cleanup (this=0x983d5c8) at sql_union.cc:669
#26 0x0849db69 in st_select_lex::cleanup (this=0x9841ea8) at sql_union.cc:810
#27 0x0837554b in mysql_select (thd=0x9840a48, rref_pointer_array=0x9841fac, tables=0x983cf60, wild_num=0, fields=@0x9841f3c, 
    conds=0x991c628, og_num=1, order=0x0, group=0x991c7b8, having=0x0, proc_param=0x0, select_options=2147764740, result=0x991c868, 
    unit=0x98419f0, select_lex=0x9841ea8) at sql_select.cc:3109
#28 0x0837594f in mysql_explain_union (thd=0x9840a48, unit=0x98419f0, result=0x991c868) at sql_select.cc:21878
#29 0x082da7ed in execute_sqlcom_select (thd=0x9840a48, all_tables=0x983cf60) at sql_parse.cc:4950
#30 0x082dc35b in mysql_execute_command (thd=0x9840a48) at sql_parse.cc:2121
#31 0x082e5225 in mysql_parse (thd=0x9840a48, 
    inBuf=0x983c8c0 "explain select sum(a),a from t1 where a> ( select sum(a) from t1 \nwhere a> ( select sum(a) from t1 \nwhere a> ( select sum(a) from t1 \nwhere a> ( select sum(a) from t1 \nwhere a> ( select sum(a) from t1"..., length=462, found_semicolon=0xb7498e68)
    at sql_parse.cc:5991
#32 0x082e5cb2 in dispatch_command (command=COM_QUERY, thd=0x9840a48, packet=0x9859769 "", packet_length=463) at sql_parse.cc:1074
#33 0x082e6fc6 in do_command (thd=0x9840a48) at sql_parse.cc:756
#34 0x082d3a36 in handle_one_connection (arg=0x9840a48) at sql_connect.cc:1164
[23 Sep 2009 9:59] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/84293

3608 Sergey Glukhov	2009-09-23 [merge]
      Bug#45989, the fix is reverted
      from mysql-pe as unnecessary.
     @ sql/sql_select.cc
        the fix is reverted
        from mysql-pe as unnecessary.
[25 Sep 2009 14:50] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/84675

3618 Alexander Nozdrin	2009-09-25
      Cherry-pick a patch from mysql-pe that rollbacks patch for Bug#45989.
[30 Sep 2009 8:16] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20090929093622-1mooerbh12e97zux) (version source revid:alik@sun.com-20090925144952-s1p9ib6f35rkdmvg) (merge vers: 6.0.14-alpha) (pib:11)
[6 Oct 2009 8:57] Bugs System
Pushed into 5.0.87 (revid:joro@sun.com-20091006073202-rj21ggvo2gw032ks) (version source revid:sergey.glukhov@sun.com-20090904072053-quchrnpayidj0eyx) (merge vers: 5.0.86) (pib:11)
[6 Oct 2009 8:59] Bugs System
Pushed into 5.1.40 (revid:joro@sun.com-20091006073316-lea2cpijh9r6on7c) (version source revid:joro@sun.com-20090923111148-sra3bjnpfe28wnmn) (merge vers: 5.1.40) (pib:11)
[8 Oct 2009 0:37] Paul Dubois
Noted in 5.0.87 changelog.

A memory leak occurred when EXPLAIN encountered a malformed query.

(Pushes into 5.1+ trees do not introduce changes.)
[19 Oct 2009 4:36] Shane Bester
In 5.1.41 the memory is now 'still reachable' instead of 'definitely lost'. Hence, there is still some lesser severe problems in 5.1 with whatever fix.
This shouldn't be shown:

1,048,612 bytes in 1 blocks are still reachable in loss record 13 of 13
at: malloc (vg_replace_malloc.c:195)
by: _mymalloc (safemalloc.c:137)
by: String::real_alloc(unsigned int) (sql_string.cc:51)
by: String::alloc(unsigned int) (sql_string.h:205)
by: String::copy(String const&) (sql_string.cc:200)
by: do_save_blob(Copy_field*) (field_conv.cc:296)
by: do_copy_null(Copy_field*) (field_conv.cc:207)
by: copy_fields(TMP_TABLE_PARAM*) (sql_select.cc:15088)
by: end_write_group(JOIN*, st_join_table*, bool) (sql_select.cc:12429)
by: evaluate_join_record(JOIN*, st_join_table*, int) (sql_select.cc:11273)
by: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:11158)
by: do_select(JOIN*, List<Item>*, st_table*, Procedure*) (sql_select.cc:10914)
[22 Oct 2009 6:34] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091022063126-l0qzirh9xyhp0bpc) (version source revid:alik@sun.com-20091019135554-s1pvptt6i750lfhv) (merge vers: 6.0.14-alpha) (pib:13)
[22 Oct 2009 7:06] Bugs System
Pushed into 5.5.0-beta (revid:alik@sun.com-20091022060553-znkmxm0g0gm6ckvw) (version source revid:alik@sun.com-20091013094238-g67x6tgdm9a7uik0) (merge vers: 5.5.0-beta) (pib:13)
[22 Oct 2009 19:41] Paul Dubois
Closing.

(Pushes into 5.1+ trees do not introduce changes.)
[28 Oct 2009 10:05] Tor Didriksen
Valgrind still reports 'possibly lost'
We leak memory for correct queries as well.
To repeat, try this, and watch the server grow in size:

CREATE TABLE t1(a LONGTEXT);
INSERT INTO t1 VALUES (repeat('a',@@global.max_allowed_packet));
INSERT INTO t1 VALUES (repeat('b',@@global.max_allowed_packet));

let $foo=1000;
while ($foo)
{
EXPLAIN SELECT DISTINCT 1 FROM t1,
        (SELECT DISTINCTROW a AS away FROM t1 GROUP BY a WITH ROLLUP) as d1;
dec $foo;
}

$../sql/mysqld --version
../sql/mysqld  Ver 6.0.14-alpha-valgrind-max-debug for pc-linux-gnu on i686 (Source distribution)
[28 Oct 2009 14:12] Tor Didriksen
also in next-mr-bugfixing
../sql/mysqld  Ver 5.5.0-beta-valgrind-max-debug for pc-linux-gnu on i686 (Source distribution)

==20697== 
==20697== 1,048,584,000 bytes in 1,000 blocks are possibly lost in loss record 6 of 6
==20697==    at 0x4006AEE: malloc (vg_replace_malloc.c:207)
==20697==    by 0x86DF824: my_malloc (my_malloc.c:34)
==20697==    by 0x827E2C2: String::real_alloc(unsigned) (sql_string.cc:51)
==20697==    by 0x81F5A9E: String::alloc(unsigned) (sql_string.h:208)
==20697==    by 0x827E894: String::copy(String const&) (sql_string.cc:200)
==20697==    by 0x83FFFAE: _ZL12do_save_blobP10Copy_field (field_conv.cc:296)
==20697==    by 0x83FE202: _ZL12do_copy_nullP10Copy_field (field_conv.cc:207)
==20697==    by 0x82FCAD1: copy_fields(TMP_TABLE_PARAM*) (sql_select.cc:15201)
==20697==    by 0x83095D1: _ZL15end_write_groupP4JOINP13st_join_tableb (sql_select.cc:12542)
==20697==    by 0x8305B26: _ZL20evaluate_join_recordP4JOINP13st_join_tablei (sql_select.cc:11382)
==20697==    by 0x8305D59: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:11268)
==20697==    by 0x831699D: _ZL9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure (sql_select.cc:11024)
==20697==    by 0x8325F3F: JOIN::exec() (sql_select.cc:1833)
==20697==    by 0x8322589: mysql_select(THD*, Item***, TABLE_LIST*, unsigned, List<Item>&, Item*, unsigned, st_order*, s
t_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:2439
)
==20697==    by 0x844AC8A: mysql_derived_filling(THD*, st_lex*, TABLE_LIST*) (sql_derived.cc:294)
==20697==    by 0x844AA32: mysql_handle_derived(st_lex*, bool (*)(THD*, st_lex*, TABLE_LIST*)) (sql_derived.cc:56)
[30 Oct 2009 10:20] Shane Bester
still seeing this bug in 5.1.40.  i'll try get the exact queries, if needed.

Attachment: 5.1.40_valgrind_output.txt (text/plain), 4.85 KiB.

[2 Dec 2009 16:51] Georgi Kodinov
None of the test cases in this bug causes any valgrind errors in 5.0-bugteam. Removing the 5.0 from the versions.
[2 Dec 2009 17:11] Georgi Kodinov
Shane,

Can you please specify exactly how you're testing it and using what version ? I've tried the attached test cases and wasn't able to get any valgrind warnings on my F12 using valgrind 3.5.
[2 Dec 2009 17:12] Georgi Kodinov
5.1-bugteam testcases

Attachment: testcases.zip (application/zip, text), 1.75 KiB.

[18 Dec 2009 10:29] Bugs System
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:45] Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:00] Bugs System
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:14] Bugs System
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
[3 Jan 2010 0:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[10 Jan 2010 11:58] Shane Bester
Hi!

Still repeatable using original testcase, on 5.1.42!  On 5.0.89 I *cannot* repeat it anymore.

==14640== 1,048,584 bytes in 1 blocks are definitely lost in loss record 7 of 7
==14640==    at 0x4C22889: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==14640==    by 0x898C31: my_malloc (my_malloc.c:34)
==14640==    by 0x5DA9A7: String::copy(String const&) (sql_string.cc:51)
==14640==    by 0x6F3DF3: _ZL12do_save_blobP10Copy_field (field_conv.cc:296)
==14640==    by 0x63AB6D: copy_fields(TMP_TABLE_PARAM*) (sql_select.cc:15197)
==14640==    by 0x64CBDB: _ZL15end_write_groupP4JOINP13st_join_tableb (sql_select.cc:12533)
==14640==    by 0x63BAEC: _ZL20evaluate_join_recordP4JOINP13st_join_tablei (sql_select.cc:11340)
==14640==    by 0x641381: sub_select(JOIN*, st_join_table*, bool) (sql_select.cc:11214)
==14640==    by 0x643C84: _ZL9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure (sql_select.cc:10970)
==14640==    by 0x659FED: JOIN::exec() (sql_select.cc:1844)
==14640==    by 0x6563B3: mysql_select(THD*, Item***, TABLE_LIST*, unsigned, List<Item>&, Item*, unsigned, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*) (sql_select.cc:2440)
==14640==    by 0x7261B6: mysql_derived_filling(THD*, st_lex*, TABLE_LIST*) (sql_derived.cc:294)
==14640== 
==14640== LEAK SUMMARY:
==14640==    definitely lost: 1,048,584 bytes in 1 blocks.
==14640==      possibly lost: 2,176 bytes in 8 blocks.
==14640==    still reachable: 1,552 bytes in 5 blocks.
==14640==         suppressed: 0 bytes in 0 blocks.

Precise instructions to repeat:

Get our binary: mysql-advanced-gpl-5.1.42-linux-x86_64-glibc23.tar.gz

tar zxvf mysql-advanced-gpl-5.1.42-linux-x86_64-glibc23.tar.gz

cd mysql-advanced-gpl-5.1.42-linux-x86_64-glibc23

./scripts/mysql_install_db --no-defaults

valgrind --tool=memcheck --leak-check=full -v --show-reachable=yes  ./bin/mysqld --no-defaults --skip-grant-tables --basedir=./ --datadir=./data

Login and paste the testcase in "how to repeat".
Then, mysqladmin --no-defaults -h127.0.0.1 -uroot shutdown.  See the leak.

I'm using: valgrind-3.2.2.SVN on openSUSE 10.2 (X86-64)
[14 Jan 2010 15:52] Georgi Kodinov
Wasn't able to repeat with 5.1-bugteam (the latest 5.1 source).

I've tried with the latest 5.1-bugteam using the attached test case. 
I've compiled with BUILD/compile-pentium-valgrind-max-no-ndb.
I've got the following output:
$ mtr --valgrind t/bug.test 
Logging: ./mysql-test-run.pl  --valgrind t/bug.test
100114 17:49:02 [Note] Plugin 'FEDERATED' is disabled.
MySQL Version 5.1.43
Turning on valgrind for all executables
Running valgrind with options " --show-reachable=yes --quiet "
Checking supported features...
 - skipping ndbcluster, mysqld not compiled with ndbcluster
 - SSL connections supported
 - binaries are debug compiled
Using "../libtool" when running valgrind or debugger
Collecting tests...
vardir: /home/kgeorge/mysql/work/B45989-5.1-bugteam/mysql-test/var
Checking leftover processes...
Removing old var directory...
Creating var directory '/home/kgeorge/mysql/work/B45989-5.1-bugteam/mysql-test/var'...
Installing system database...
Using server port 45842

==============================================================================

TEST                                      RESULT   TIME (ms)
------------------------------------------------------------

worker[1]  - 'localhost:13000' was not free
worker[1]  - 'localhost:13011' was not free
worker[1] Using MTR_BUILD_THREAD 302, with reserved ports 13020..13029
main.bug                                 [ pass ]   4247
------------------------------------------------------------
The servers were restarted 0 times
Spent 4.247 of 14 seconds executing testcases

All 1 tests were successful.
[14 Jan 2010 15:53] Georgi Kodinov
bug.test

Attachment: bug.test (application/octet-stream, text), 379 bytes.

[14 Jan 2010 15:55] Georgi Kodinov
bug.result

Attachment: bug.result (application/octet-stream, text), 526 bytes.

[15 Jan 2010 12:05] Tor Didriksen
The leak still exists, verified on:
mysqld  Ver 5.1.43-valgrind-max-debug for unknown-linux-gnu on x86_64 (Source distribution)

with parent
bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-5.1
and
bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-5.1-bugteam/

The reason it is hard to reproduce is most likely that mtr kills the server before valgrind has time to report.

Apply this patch to mtr:
=== modified file 'mysql-test/lib/My/SafeProcess/safe_process.cc'
--- mysql-test/lib/My/SafeProcess/safe_process.cc	2009-07-08 12:31:22 +0000
+++ mysql-test/lib/My/SafeProcess/safe_process.cc	2010-01-15 11:42:15 +0000
@@ -95,7 +95,8 @@
 
   message("Killing child: %d", child_pid);
   // Terminate whole process group
-  kill(-child_pid, SIGKILL);
+  // kill(-child_pid, SIGKILL);
+  kill(-child_pid, SIGTERM);
 
   pid_t ret_pid= waitpid(child_pid, &status, 0);
   if (ret_pid == child_pid)

You can also run this script, and watch the server grow in size 
by a few megabytes per second:

CREATE TABLE t1(a LONGTEXT);
INSERT INTO t1 VALUES (repeat('a',@@global.max_allowed_packet));
INSERT INTO t1 VALUES (repeat('b',@@global.max_allowed_packet));

let $foo=1000;
while ($foo)
{
EXPLAIN SELECT DISTINCT 1 FROM t1,
        (SELECT DISTINCTROW a AS away FROM t1 GROUP BY a WITH ROLLUP) as d1;
dec $foo;
}

drop table t1;
[18 Jan 2010 15:51] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/97283

3319 Georgi Kodinov	2010-01-18
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers after doing
      the conversion.
[19 Jan 2010 9:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/97334

3320 Georgi Kodinov	2010-01-18
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers after doing
      the conversion.
[19 Jan 2010 12:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/97369

3323 Georgi Kodinov	2010-01-19
      revert of the fix for bug #45989: pushed by mistake.
[27 Jan 2010 20:46] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/98398

2961 Alexander Nozdrin	2010-01-27 [merge]
      Null-merge the following changeset from mysql-trunk-merge:
      =========================================================
      revision-id: joro@sun.com-20100118155046-v6hvlyfmitcyep8p
      committer: Georgi Kodinov <joro@sun.com>
      branch nick: B45989-5.1-bugteam
      timestamp: Mon 2010-01-18 17:50:46 +0200
      message:
        Bug #45989 take 2 : memory leak after explain encounters an
        error in the query.
        
        Fixes a leak after materializing a GROUP BY subquery to a 
        temp table when the subquery has a blob column in the SELECT
        list.
        Fixed by correctly destructing temporary buffers after doing
        the conversion.
      =========================================================
[29 Jan 2010 10:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/98577

3320 Georgi Kodinov	2010-01-29
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers after doing
      the conversion.
[29 Jan 2010 10:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/98586

3340 Georgi Kodinov	2010-01-29
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers after doing
      the conversion.
[2 Feb 2010 16:30] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/98972

3340 Georgi Kodinov	2010-02-02
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers for re-usable
      queries
[4 Feb 2010 10:19] Bugs System
Pushed into 5.1.44 (revid:joro@sun.com-20100204101444-2j32mhqroo0iiio6) (version source revid:joro@sun.com-20100119124841-38vva51cuq3if7dc) (merge vers: 5.1.43) (pib:16)
[4 Feb 2010 15:20] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99297

3356 Georgi Kodinov	2010-02-02
      Bug #45989 take 2 : memory leak after explain encounters an
      error in the query.
      
      Fixes a leak after materializing a GROUP BY subquery to a 
      temp table when the subquery has a blob column in the SELECT
      list.
      Fixed by correctly destructing temporary buffers for re-usable
      queries
[4 Feb 2010 16:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99325

3359 Georgi Kodinov	2010-02-04
      Addendum to the fix for bug #45989
      
      Need to make sure the tmp join doesn't point to the structure already 
      freed by the cleanup() for the "base" join, as this can lead to 
      double free, because sometimes both tmp_join and join point to the 
      same tmp_table_params.copy_field array.
[5 Feb 2010 11:47] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100204063540-9czpdmpixi3iw2yb) (version source revid:alik@sun.com-20100127212738-qkp4xhr02szpiozy) (pib:16)
[5 Feb 2010 11:51] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100205113942-oqovjy0eoqbarn7i) (version source revid:alfranio.correia@sun.com-20100121210527-rbuheu5rnsmcakh1) (merge vers: 5.5.99-m3) (pib:16)
[5 Feb 2010 11:59] Bugs System
Pushed into 5.5.2-m2 (revid:alik@sun.com-20100203172258-1n5dsotny40yufxw) (version source revid:alexey.kopytov@sun.com-20100123210923-lx4o1ettww9fdkqk) (merge vers: 5.5.2-m2) (pib:16)
[10 Feb 2010 19:02] Paul Dubois
Noted in 5.5.2, 6.0.14 changelogs.

If EXPLAIN encountered an error in the query, a memory leak occurred.

Setting report to Need Merge pending push to 5.1, Celosia.
[1 Mar 2010 8:43] Bugs System
Pushed into 5.1.45 (revid:joro@sun.com-20100301083827-xnimmrjg6bh33o1o) (version source revid:joerg@mysql.com-20100212173307-ph563zr4wmoklgwd) (merge vers: 5.1.45) (pib:16)
[1 Mar 2010 17:12] Paul Dubois
Noted in 5.1.45 changelog.

Setting report to Need Merge pending push of Celosia to release tree.
[2 Mar 2010 14:33] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100302142746-u1gxdf5yk2bjrq3e) (version source revid:alik@sun.com-20100225090938-2j5ybqoau570mytu) (merge vers: 6.0.14-alpha) (pib:16)
[2 Mar 2010 14:39] Bugs System
Pushed into 5.5.3-m2 (revid:alik@sun.com-20100302072233-t3uqgjzdukt1pyhe) (version source revid:alexey.kopytov@sun.com-20100209075938-mmcnyf6w631ozc45) (merge vers: 5.5.2-m2) (pib:16)
[2 Mar 2010 14:44] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100302072432-k8xvfkgcggkwgi94) (version source revid:alik@sun.com-20100224135227-rcqs9pe9b2in80pf) (pib:16)
[2 Mar 2010 20:59] Paul Dubois
Setting report to Need Merge pending push of Celosia to release tree.
[6 Mar 2010 10:55] Bugs System
Pushed into 5.5.3-m3 (revid:alik@sun.com-20100306103849-hha31z2enhh7jwt3) (version source revid:alfranio.correia@sun.com-20100121210527-rbuheu5rnsmcakh1) (merge vers: 5.5.99-m3) (pib:16)
[7 Mar 2010 12:39] Paul Dubois
Already fixed in earlier 5.5.x.
[12 Mar 2010 14:10] Bugs System
Pushed into 5.1.44-ndb-7.0.14 (revid:jonas@mysql.com-20100312135944-t0z8s1da2orvl66x) (version source revid:jonas@mysql.com-20100312115609-woou0te4a6s4ae9y) (merge vers: 5.1.44-ndb-7.0.14) (pib:16)
[12 Mar 2010 14:26] Bugs System
Pushed into 5.1.44-ndb-6.2.19 (revid:jonas@mysql.com-20100312134846-tuqhd9w3tv4xgl3d) (version source revid:jonas@mysql.com-20100312060623-mx6407w2vx76h3by) (merge vers: 5.1.44-ndb-6.2.19) (pib:16)
[12 Mar 2010 14:40] Bugs System
Pushed into 5.1.44-ndb-6.3.33 (revid:jonas@mysql.com-20100312135724-xcw8vw2lu3mijrhn) (version source revid:jonas@mysql.com-20100312103652-snkltsd197l7q2yg) (merge vers: 5.1.44-ndb-6.3.33) (pib:16)
[17 Jun 2010 11:57] Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:36] Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609140708-52rvuyq4q500sxkq) (merge vers: 5.1.45-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:23] Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)