MySQL Bugs: #36005: crash in subselect with single row (subselect_single_select

Bug #36005	crash in subselect with single row (subselect_single_select_engine::exec)
Submitted:	11 Apr 2008 20:34	Modified:	13 May 2008 20:00
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Optimizer	Severity:	S1 (Critical)
Version:	5.0.58, 5.1.25-bk, 6.0.5-bk	OS:	Any
Assigned to:	Gleb Shchepa	CPU Architecture:	Any

Description:
stack trace:

mysqld.exe!subselect_single_select_engine::exec
mysqld.exe!Item_subselect::exec
mysqld.exe!Item_in_subselect::val_bool
mysqld.exe!Item_in_optimizer::val_int
mysqld.exe!Item::val_bool
mysqld.exe!Item_func_not::val_int
mysqld.exe!make_join_select
mysqld.exe!JOIN::optimize
mysqld.exe!mysql_select
mysqld.exe!mysql_explain_union
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart
kernel32.dll!BaseThreadStart

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` int,`b` int,key (`a`),key (`b`)) engine=myisam;
insert into `t1` values (2,4),(2,4),(2,4);
drop table if exists `t2`;
create table `t2` ( `a` int,`b` int,key (`a`),key (`b`)) engine=myisam;
insert into `t2` values (3,3),(3,3),(3,3);
select 1 from t1 where a not in ( select 1 from t1,t2 where 
t1.a=(select b from t2 limit 1) and  t1.a <= 1) and  t1.a= 1;

Thank you for the report.

Verified as described.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45803

ChangeSet@1.2610, 2008-04-22 13:30:22+05:00, gshchepa@host.loc +3 -0
  Fixed bug#36005: server crashes inside NOT IN clause subquery with
                   impossible WHERE/HAVING clause
                   (subselect_single_select_engine::exec).
  
  Allocation and initialization of joined table list t1, t2... of
  subqueries like:
  
      NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
  
  is optimized out, however server tries to traverse this list.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/45850

ChangeSet@1.2610, 2008-04-23 02:27:23+05:00, gshchepa@host.loc +3 -0
  Fixed bug#36005: server crashes inside NOT IN clause subquery with
                   impossible WHERE/HAVING clause
                   (subselect_single_select_engine::exec).
  
  Allocation and initialization of joined table list t1, t2... of
  subqueries like:
  
      NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
  
  is optimized out, however server tries to traverse this list.

Pushed into 5.1.25-rc

Pushed into 6.0.6-alpha

Pushed into 5.0.62

Noted in 5.0.62, 5.1.25, 6.0.6 changelogs.

The server crashed inside NOT IN subqueries with an impossible WHERE
or HAVING clause, such as NOT IN (SELECT ... FROM t1, t2, ... WHERE
0).