MySQL Bugs: #52168: decimal casting catastrophes: crashes and valgrind errors on simple casts

Bug #52168	decimal casting catastrophes: crashes and valgrind errors on simple casts
Submitted:	18 Mar 2010 9:17	Modified:	7 Jul 2010 19:11
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Data Types	Severity:	S1 (Critical)
Version:	5.6.99-m4-debug, 5.5.3-m3	OS:	Any
Assigned to:	Øystein Grøvlen	CPU Architecture:	Any
Tags:	crash, decimal, regression, valgrind

Description:
Version: '5.6.99-m4-debug'  socket: ''  port: 3306  Source distribution
100318 11:12:19 - mysqld got exception 0xc0000005 ;

mysqld.exe!decimal_round()[decimal.c:1508]
mysqld.exe!my_decimal2int()[my_decimal.h:303]
mysqld.exe!Item_cache_decimal::val_int()[item.cc:7518]
mysqld.exe!Item_func_bit_or::val_int()[item_cmpfunc.cc:4025]
mysqld.exe!Item_func_trig_cond::val_int()[item_cmpfunc.h:434]
mysqld.exe!evaluate_join_record()[sql_select.cc:11522]
mysqld.exe!sub_select()[sql_select.cc:11473]
mysqld.exe!do_select()[sql_select.cc:11225]
mysqld.exe!JOIN::exec()[sql_select.cc:1899]
mysqld.exe!mysql_select()[sql_select.cc:2508]
mysqld.exe!handle_select()[sql_select.cc:271]
mysqld.exe!execute_sqlcom_select()[sql_parse.cc:4703]
mysqld.exe!mysql_execute_command()[sql_parse.cc:2191]
mysqld.exe!mysql_parse()[sql_parse.cc:5735]
mysqld.exe!dispatch_command()[sql_parse.cc:1024]
mysqld.exe!do_command()[sql_parse.cc:710]
mysqld.exe!do_handle_one_connection()[sql_connect.cc:1174]
mysqld.exe!handle_one_connection()[sql_connect.cc:1113]
mysqld.exe!pthread_start()[my_winthread.c:61]
mysqld.exe!_callthreadstartex()[threadex.c:348]
mysqld.exe!_threadstartex()[threadex.c:331]
kernel32.dll!FlsSetValue()

How to repeat:
#run under valgrind if no crash happens.
#valgrind --tool=memcheck --read-var-info=yes --track-origins=yes 
#--leak-check=full --db-attach=no --num-callers=50 -v --show-reachable=yes

set names utf8;
flush tables;
drop table if exists `t1`,`t2`;
create table `t1` (`a` int) engine=myisam;
insert into `t1` values ();
create table `t2`(`b` int)engine=myisam;
insert into t2 values (),();
select 1 from t1 left join `t2` on 1;
select max(`t22222`.`b`)
from `t1` left join `t2` as `t22222`
on convert(`t1`.`a`,decimal)|convert(`t22222`.`b` using cp932)
order by 1
;

full outputs from valgrind

Attachment: bug52168_5.6.99-m4-valgrind-output.txt (text/plain), 33.45 KiB.

Thank you for the report.

Verified as described. Version 5.1 is not affected.

regression source:
epotemkin@mysql.com-20091201212551-amf9utmatlvsbbar
(was fix for Bug#33546: Slowdown on re-evaluation of constant expressions)

See also 52122 which might be a duplicate.

The reduced test case below, gives valgrind errors on 64-bit Linux, but no crash.
(No valgrind errors on my 32-bit Ubuntu laptop for some reason).

create table t1 (a int);
insert into t1 values ();
create table t2(b int);
insert into t2 values (),();
select b from t1 join t2 where convert(a, decimal)|convert(b, decimal);
drop table t1, t2;

The problem is that if a NULL is stored in an Item_cache_decimal object,
the associated decimal object is not initialized.  However, it is still
accessed when val_* methods are called.  The following diff fixes the valgrind
issue:

=== modified file 'sql/item.cc'
--- sql/item.cc 2010-03-01 10:06:11 +0000
+++ sql/item.cc 2010-04-12 13:03:28 +0000
@@ -7513,7 +7513,7 @@ longlong Item_cache_decimal::val_int()
 {
   DBUG_ASSERT(fixed);
   longlong res;
-  if (!value_cached && !cache_value())
+  if ((!value_cached && !cache_value()) || null_value)
     return 0;
   my_decimal2int(E_DEC_FATAL_ERROR, &decimal_value, unsigned_flag, &res);
   return res;

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/105466

3010 oystein.grovlen@sun.com	2010-04-13
      Bug#52168 decimal casting catastrophes: crashes and valgrind errors on simple casts
      
      The problem is that if a NULL is stored in an Item_cache_decimal object,
      the associated my_decimal object is not initialized.  However, it is still
      accessed when val_int() is called. The fix is to check for null_value
      within val_int(), and return without accessing the my_decimal object when
      the cached value is NULL.
      
      Bug#52122 reports the same issue for val_real(), and this patch also includes
      fixes for val_real() and val_str() and corresponding test cases from that
      bug report.
     @ mysql-test/r/type_decimal.result
        Updated result file with test cases for Bug#52168 and Bug#52122.
     @ mysql-test/t/type_decimal.test
        Added test cases for Bug#52168 and Bug#52122.
     @ sql/item.cc
        In Item_cache_decimal::val_* methods, check for null_value and return early
        to avoid accessing an uninitialized my_decimal object.

This is bug is a regression on trunk (compared to 5.1).
Since it causes server crash on some platforms, I think this should be tagged
as SRMR03.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/105735

3019 oystein.grovlen@sun.com	2010-04-15
      Bug#52168 decimal casting catastrophes: crashes and valgrind errors on simple casts
      
      The problem is that if a NULL is stored in an Item_cache_decimal object,
      the associated my_decimal object is not initialized.  However, it is still
      accessed when val_int() is called. The fix is to check for null_value
      within val_int(), and return without accessing the my_decimal object when
      the cached value is NULL.
      
      Bug#52122 reports the same issue for val_real(), and this patch also includes
      fixes for val_real() and val_str() and corresponding test cases from that
      bug report.  
      
      Also, NULL is returned from val_decimal() when value is null. This will
      avoid that callers access an uninitialized my_decimal object.
     @ mysql-test/r/type_decimal.result
        Updated result file with test cases for Bug#52168 and Bug#52122.
     @ mysql-test/t/type_decimal.test
        Added test cases for Bug#52168 and Bug#52122.
     @ sql/item.cc
        In Item_cache_decimal::val_* methods, check for null_value and return early
        to avoid accessing an uninitialized my_decimal object.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/109503

3071 oystein.grovlen@sun.com	2010-05-28
      Bug#52168 decimal casting catastrophes: crashes and valgrind errors on simple casts
      
      The problem is that if a NULL is stored in an Item_cache_decimal object,
      the associated my_decimal object is not initialized.  However, it is still
      accessed when val_int() is called. The fix is to check for null_value
      within val_int(), and return without accessing the my_decimal object when
      the cached value is NULL.
      
      Bug#52122 reports the same issue for val_real(), and this patch also includes
      fixes for val_real() and val_str() and corresponding test cases from that
      bug report.  
      
      Also, NULL is returned from val_decimal() when value is null. This will
      avoid that callers access an uninitialized my_decimal object.
      
      Made similar changes to all other Item_cache classes.  Now all val_*
      methods should return a well defined value when actual value is NULL.
     @ mysql-test/r/type_decimal.result
        Updated result file with test cases for Bug#52168 and Bug#52122.
     @ mysql-test/t/type_decimal.test
        Added test cases for Bug#52168 and Bug#52122.
     @ sql/item.cc
        In Item_cache_*::val_* methods, return a well defined value
        when actual value is NULL.
        
        This is especially important for Item_cache_decimal since
        otherwise one risk accessing an uninitialized my_decimal object.
     @ sql/item.h
        Added method Item_cache::has_value() which returns TRUE if cache 
        object contains a non-null value.

Pushed into mysql-trunk-bugfixing with revid 
oystein.grovlen@sun.com-20100528153039-cbj04cckk9zy4q6t

Pushed into 5.5.5-m3 (revid:alik@sun.com-20100615080459-smuswd9ooeywcxuc) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (merge vers: 5.1.47) (pib:16)

Pushed into mysql-next-mr (revid:alik@sun.com-20100615080558-cw01bzdqr1bdmmec) (version source revid:mmakela@bk-internal.mysql.com-20100415070122-1nxji8ym4mao13ao) (pib:16)

Noted in 5.5.5 changelog.

Cast operations on NULL DECIMAL values could cause server crashes or
Valgrind warnings.