Bug #31758 inet_ntoa, oct, crashes server with null + filesort
Submitted: 22 Oct 2007 16:06 Modified: 29 Nov 2007 0:42
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S2 (Serious)
Version:5.1.23, 5.0.50? OS:Any
Assigned to: Alexey Botchkov CPU Architecture:Any
Tags: assertion, filesort, inet_ntoa, oct

[22 Oct 2007 16:06] Shane Bester 
Description:
I branched this off bug #31160 (MAKETIME() crashes server when returning NULL in ORDER BY using filesort)

filesort crashes when deleting from table and ordering by an expression that returned a null value due to inet_ntoa being called on invalid data.

stack trace from 5.1.23BK

mysqld.exe!_NMSG_WRITE
mysqld.exe!abort
mysqld.exe!_wassert
mysqld.exe!make_sortkey
mysqld.exe!find_all_keys
mysqld.exe!filesort
mysqld.exe!mysql_delete
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart
mysqld.exe!_threadstart
kernel32.dll!BaseThreadStart

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` bigint not null)engine=myisam;
insert into `t1` set `a` = 1024*1024*1024*4;
delete from `t1` order by (inet_ntoa(`a`)) desc limit 10;
[23 Oct 2007 11:29] MySQL Verification Team 
OCT() also has a problem:

drop table if exists `t1`;
create table `t1` (`a` char(36) not null)engine=myisam;
insert ignore into `t1` set `a` = ' ';
insert ignore into `t1` set `a` = ' ';
select * from `t1` order by (oct(`a`));
[30 Oct 2007 9:40] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/36621

ChangeSet@1.2688, 2007-10-30 12:35:03+04:00, holyfoot@mysql.com +3 -0
  Bug #31758 inet_ntoa, oct crashes server with null+filesort
  
  Item_func_inet_ntoa and Item_func_conv inherit 'maybe_null' flag from an
  argument, which is wrong.
  Both can be NULL with notnull arguments, so that's fixed.
[16 Nov 2007 9:28] Bugs System 
Pushed into 4.1.24
[16 Nov 2007 9:31] Bugs System 
Pushed into 5.0.52
[16 Nov 2007 9:33] Bugs System 
Pushed into 5.1.23-rc
[16 Nov 2007 9:35] Bugs System 
Pushed into 6.0.4-alpha
[29 Nov 2007 0:42] Paul DuBois 
Noted in 4.1.24, 5.0.52, 5.1.23, 6.0.4 changelogs.

The server could crash during filesort for ORDER BY based on
expressions with INET_NTOA() or OCT() if those functions returned
NULL.