MySQL Bugs: #53236: Segfault in DTCollation::set(DTCollation&)

Bug #53236	Segfault in DTCollation::set(DTCollation&)
Submitted:	28 Apr 2010 7:50	Modified:	23 Nov 2010 3:28
Reporter:	Tor Didriksen	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Optimizer	Severity:	S3 (Non-critical)
Version:	Celosia, Dahlia, 6.0-codebase-bugfixing	OS:	Any
Assigned to:	Tor Didriksen	CPU Architecture:	Any

Description:
Discovered while working on Bug #52538

==7454== Invalid write of size 8
==7454==    at 0x6348CF: DTCollation::set(DTCollation&) (item.h:86)
==7454==    by 0x618C5C: change_cond_ref_to_const(THD*, I_List<COND_CMP>*, Item*, Item*, Item*, Item*) (sql_select.cc:12447)
==7454==    by 0x618AC6: change_cond_ref_to_const(THD*, I_List<COND_CMP>*, Item*, Item*, Item*, Item*) (sql_select.cc:12428)
==7454==    by 0x6194CB: propagate_cond_constants(THD*, I_List<COND_CMP>*, Item*, Item*) (sql_select.cc:12578)
==7454==    by 0x619156: propagate_cond_constants(THD*, I_List<COND_CMP>*, Item*, Item*) (sql_select.cc:12538)
==7454==    by 0x61B761: optimize_cond(JOIN*, Item*, List<TABLE_LIST>*, bool, Item::cond_result*) (sql_select.cc:13789)
==7454==    by 0x5FD477: JOIN::optimize() (sql_select.cc:1562)
==7454==    by 0x7DBC48: subselect_single_select_engine::exec() (item_subselect.cc:2253)
==7454==    by 0x7D5EBA: Item_subselect::exec() (item_subselect.cc:296)
==7454==    by 0x7D6037: Item_in_subselect::exec() (item_subselect.cc:346)
==7454==    by 0x7D7DDB: Item_in_subselect::val_bool() (item_subselect.cc:942)
==7454==    by 0x54F892: Item::val_bool_result() (item.h:855)
==7454==    by 0x7857ED: Item_in_optimizer::val_int() (item_cmpfunc.cc:1837)
==7454==    by 0x7A7C6C: eval_const_cond(Item*) (item_func.cc:78)
==7454==    by 0x61BD0A: internal_remove_eq_conds(THD*, Item*, Item::cond_result*) (sql_select.cc:13926)
==7454==    by 0x61B8C5: internal_remove_eq_conds(THD*, Item*, Item::cond_result*) (sql_select.cc:13836)

How to repeat:
#!/bin/bash
randgendir=/export/home/tmp/randgen

if test -d "$1"; then mysqlddir="$1"; fi

grammar=${randgendir}/conf/optimizer/optimizer_subquery.yy

cd ${randgendir}
perl ./runall-new.pl --basedir=${mysqlddir} \
--mtr-build-thread=21 --threads=1 --queries=7000 \
--Validator=MarkErrorLog \
--sqltrace \
--valgrind \
--gendata \
--grammar=${grammar}
exitval=$?

mysql_err=$mysqlddir/mysql-test/var/mysql.err
grep "Conditional jump or move depends" ${mysql_err} && exit 1
exit $exitval

This is the actual query which makes the server crash:

EXPLAIN EXTENDED SELECT    COUNT(  table1 . pk ) AS field1 FROM ( C AS table1 INNER JOIN ( ( CC AS table2 INNER JOIN CC AS table3 ON (table3 . pk = table2 . col_int_key  ) ) ) ON (table3 . col_varchar_key = table2 . col_varchar_key  ) ) WHERE (  ( 4, 6 )  IN ( SELECT   SUBQUERY1_t1 . pk AS SUBQUERY1_field1 , SUBQUERY1_t1 . pk AS SUBQUERY1_field2 FROM ( CC AS SUBQUERY1_t1 RIGHT  JOIN C AS SUBQUERY1_t2 ON (SUBQUERY1_t2 . col_varchar_nokey = SUBQUERY1_t1 . col_varchar_key  ) )    ) ) AND table1 . col_int_key = 249 AND ( table1 . col_int_key IS  NULL OR table1 . col_int_key = table2 . col_int_key )  HAVING field1 <= 8 ;

Suggested fix:
Don't call member functions when this == NULL.

Simplified test case for mtr:

CREATE TABLE t1 (
  pk INTEGER AUTO_INCREMENT,
  col_varchar VARCHAR(1),
  PRIMARY KEY (pk)
)
;

INSERT INTO t1 (col_varchar) 
VALUES
('w'),
('m')
;

SELECT  table1 . pk
FROM ( t1 AS table1 JOIN t1 AS table2 ON (table1 . col_varchar = table2 . col_varchar  ) ) 
WHERE ( 4, 6 ) IN ( SELECT SUBQUERY1_t1 . pk AS SUBQUERY1_field1 ,
                           SUBQUERY1_t1 . pk AS SUBQUERY1_field2
                    FROM ( t1 AS SUBQUERY1_t1 JOIN t1 AS SUBQUERY1_t2
                           ON (SUBQUERY1_t2 . col_varchar = SUBQUERY1_t1 . col_varchar  ) ) ) 
;

drop table t1;

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/106852

3844 Tor Didriksen	2010-04-28
      Bug#53236 Segfault in DTCollation::set(DTCollation&)
      
      Don't call member functions for a NIL pointer.
     @ mysql-test/r/subselect4.result
        Add test case.
     @ mysql-test/t/subselect4.test
        Add test case.
     @ sql/sql_select.cc
        If the (virtual) member function clone_item() returns NULL,
        there is no substitution to be made, and we don't need to set the collation.
        The test was invoking Item_cache::clone_item()

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/106900

3846 Tor Didriksen	2010-04-28
      Bug#53236 Segfault in DTCollation::set(DTCollation&)
      
      Don't call member functions for a NIL pointer.
     @ mysql-test/r/subselect4.result
        Add test case.
     @ mysql-test/t/subselect4.test
        Add test case.
     @ sql/sql_select.cc
        If the (virtual) member function clone_item() returns NULL,
        there is no substitution to be made, and we don't need to set the collation.
        The test was invoking Item_cache::clone_item()

Pushed to
bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-6.0-codebase-bugfixing/

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100507091908-vqyhpwf2km0aokno) (version source revid:alik@sun.com-20100507091737-12vceffs11elb25g) (merge vers: 6.0.14-alpha) (pib:16)

The server could crash when processing subqueries with empty results.

Noted in 6.0.14 changelog.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/108486

3180 Tor Didriksen	2010-05-18
      Bug#53236 Segfault in DTCollation::set(DTCollation&)
      
      Backport of tor.didriksen@sun.com-20100428145548-w2fk06r9bnlnqkzr
      Backported even though the test does not fail in this tree.
      
      Don't call member functions for a NIL pointer.
     @ mysql-test/r/subselect4.result
        Add test case.
     @ mysql-test/t/subselect4.test
        Add test case.
     @ sql/sql_select.cc
        If the (virtual) member function clone_item() returns NULL,
        there is no substitution to be made, and we don't need to set the collation.
        The test was invoking Item_cache::clone_item()

Verified that the bug exists in trunk-bugfixing,
and that the patch from 6.0 fixes it.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111853

3070 Tor Didriksen	2010-06-23
      Backport of Bug#53236 Segfault in DTCollation::set(DTCollation&)
      
      Don't call member functions for a NIL pointer.
     @ mysql-test/r/subselect4.result
        Add test case.
     @ mysql-test/t/subselect4.test
        Add test case.
     @ sql/sql_select.cc
        If the (virtual) member function clone_item() returns NULL,
        there is no substitution to be made, and we don't need to set the collation.
        The test was invoking Item_cache::clone_item()

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/111854

3266 Tor Didriksen	2010-06-23 [merge]
      Merge from trunk-bugfixing Backport of Bug#53236 Segfault in DTCollation::set(DTCollation&)

Backported and pushed to trunk-bugfixing.
Merged and pushed to next-mr-bugfixing.

Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100723121820-jryu2fuw3pc53q9w) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:18)

Pushed into mysql-next-mr (revid:alik@sun.com-20100723121929-90e9zemk3jkr2ocy) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (pib:18)

Noted in 5.5.6 changelog.

Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:18)

Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:vasil.dimov@oracle.com-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:18)

Bug does not appear in any released 5.6.x version.

Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)

Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)