Bug #36005 crash in subselect with single row (subselect_single_select_engine::exec)
Submitted: 11 Apr 2008 22:34 Modified: 13 May 2008 22:00
Reporter: Shane Bester
Status: Closed
Category:Server: Optimizer Severity:S1 (Critical)
Version:5.0.58, 5.1.25-bk, 6.0.5-bk OS:Any
Assigned to: Gleb Shchepa Target Version:5.0+
Triage: D1 (Critical)

[11 Apr 2008 22:34] Shane Bester 
Description:
stack trace:

mysqld.exe!subselect_single_select_engine::exec
mysqld.exe!Item_subselect::exec
mysqld.exe!Item_in_subselect::val_bool
mysqld.exe!Item_in_optimizer::val_int
mysqld.exe!Item::val_bool
mysqld.exe!Item_func_not::val_int
mysqld.exe!make_join_select
mysqld.exe!JOIN::optimize
mysqld.exe!mysql_select
mysqld.exe!mysql_explain_union
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart
kernel32.dll!BaseThreadStart

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` int,`b` int,key (`a`),key (`b`)) engine=myisam;
insert into `t1` values (2,4),(2,4),(2,4);
drop table if exists `t2`;
create table `t2` ( `a` int,`b` int,key (`a`),key (`b`)) engine=myisam;
insert into `t2` values (3,3),(3,3),(3,3);
select 1 from t1 where a not in ( select 1 from t1,t2 where 
t1.a=(select b from t2 limit 1) and  t1.a <= 1) and  t1.a= 1;
[11 Apr 2008 22:50] Sveta Smirnova 
Thank you for the report.

Verified as described.
[22 Apr 2008 10:31] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45803

ChangeSet@1.2610, 2008-04-22 13:30:22+05:00, gshchepa@host.loc +3 -0
  Fixed bug#36005: server crashes inside NOT IN clause subquery with
                   impossible WHERE/HAVING clause
                   (subselect_single_select_engine::exec).
  
  Allocation and initialization of joined table list t1, t2... of
  subqueries like:
  
      NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
  
  is optimized out, however server tries to traverse this list.
[22 Apr 2008 23:28] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45850

ChangeSet@1.2610, 2008-04-23 02:27:23+05:00, gshchepa@host.loc +3 -0
  Fixed bug#36005: server crashes inside NOT IN clause subquery with
                   impossible WHERE/HAVING clause
                   (subselect_single_select_engine::exec).
  
  Allocation and initialization of joined table list t1, t2... of
  subqueries like:
  
      NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
  
  is optimized out, however server tries to traverse this list.
[1 May 2008 8:16] Bugs System 
Pushed into 5.1.25-rc
[1 May 2008 8:19] Bugs System 
Pushed into 6.0.6-alpha
[6 May 2008 2:25] Bugs System 
Pushed into 5.0.62
[13 May 2008 22:00] Paul DuBois 
Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

The server crashed inside NOT IN subqueries with an impossible WHERE
or HAVING clause, such as NOT IN (SELECT ... FROM t1, t2, ... WHERE
0).