MySQL Bugs: #31227: memory overrun with decimal (6,6) and zerofill and group

Bug #31227	memory overrun with decimal (6,6) and zerofill and group_concat..
Submitted:	27 Sep 2007 10:48	Modified:	25 Oct 2007 0:10
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: General	Severity:	S1 (Critical)
Version:	5.1.23, 5.0.50	OS:	Any
Assigned to:	Tatiana Azundris Nuernberg	CPU Architecture:	Any

Description:
This doesn't crash the server often, but prints out these errors in logs:

070927 12:17:56 mysqld_safe Starting mysqld daemon with databases from /home/sbester/server/5.1/mysql-5.1.23-beta-linux-i686/data
InnoDB: The first specified data file ./ibdata1 did not exist:
InnoDB: a new database to be created!
070927 12:17:56  InnoDB: Setting file ./ibdata1 size to 10 MB
InnoDB: Database physically writes the file full: wait...
070927 12:17:56  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 5 MB
InnoDB: Database physically writes the file full: wait...
070927 12:17:57  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 5 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
070927 12:17:57  InnoDB: Started; log sequence number 0 0
070927 12:17:57 [Note] /home/sbester/server/5.1/mysql-5.1.23-beta-linux-i686/bin/mysqld: ready for connections.
Version: '5.1.23-beta-debug'  socket: '/tmp/mysql.sock'  port: 3306  yes
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.h:190'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.h:190'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'
Error: Memory allocated at sql_string.cc:51 was overrun, discovered at 'sql_string.cc:74'

How to repeat:
run debug build of server, or run it under valgrind and check:

drop table if exists t1;
create table `t1` (`col000` decimal (6,6) zerofill not null )engine=myisam;
insert into t1 values (),(),(),(),(),(),(),(),(),(),(),(),(),(),(),(),(),();
select group_concat(`t1`.`col000`) from t1;

This is a pathological case.
DECIMAL(a,b) == a digits altogether, of which b are fractional.
DECIMAL(5,3)    123.45, or -123.45(!)
This requires a characters, plus one extra for sign, plus one for period if b>0, *plus* one extra for a leading '0' if a==b:
DECIMAL(6,6)    -0.123456: (sign) (zero) (period) (6 decimal places) (\0)

This leading '0' is not factored in:
my_decimal.cc:88

int my_decimal2string(uint mask, const my_decimal *d,
                      uint fixed_prec, uint fixed_dec,
                      char filler, String *str)
{
  int length= (fixed_prec ? (fixed_prec + 1) : my_decimal_string_length(d));

Obvious fix:
  int length= (fixed_prec ? (fixed_prec + ((fixed_prec == fixed_dec) ? 1 : 0) 1) : my_decimal_string_length(d));

This is unwieldy though, and approaching a point where it likely offers little advantage in terms of speed over
  int length= my_decimal_string_length(d);

bt at alloc time
#0  String::real_alloc (this=0xb72ade70, arg_length=8) at sql_string.cc:50
#1  0x081b3f80 in String::alloc (this=0xb72ade70, arg_length=7) at sql_string.h:198
#2  0x083b9ae6 in my_decimal2string (mask=30, d=0xb72adda8, fixed_prec=6, fixed_dec=6, filler=48 '0', str=0xb72ade70) at my_decimal.cc:90
#3  0x08200f6c in Field_new_decimal::val_str (this=0x88601a8, val_buffer=0xb72ade70, val_ptr=0xb72ade70) at field.cc:2552
#4  0x0818b31a in Field::val_str (this=0x88601a8, str=0xb72ade70) at field.h:105
#5  0x0819b574 in Field::val_str (this=0x88601a8, str=0xb72ade70, new_ptr=0x88602e1 "\200") at field.h:285
#6  0x081949f7 in dump_leaf_key (key=0x88602e1 "\200", count=1, item=0x885af40) at item_sum.cc:2973
#7  0x08194ce1 in Item_func_group_concat::add (this=0x885af40) at item_sum.cc:3189
#8  0x082801ab in update_sum_func (func_ptr=0x885c258) at sql_select.cc:14501
#9  0x0828761a in end_send_group (join=0x885b408, join_tab=0x885c668, end_of_records=false) at sql_select.cc:11578
#10 0x08287b98 in evaluate_join_record (join=0x885b408, join_tab=0x885c4e8, error=0, report_error=0xb6de4d74 "") at sql_select.cc:10706
#11 0x08287dca in sub_select (join=0x885b408, join_tab=0x885c4e8, end_of_records=false) at sql_select.cc:10596
#12 0x0828fe09 in do_select (join=0x885b408, fields=0x885c190, table=0x0, procedure=0x0) at sql_select.cc:10353
#13 0x082a601e in JOIN::exec (this=0x885b408) at sql_select.cc:2101
#14 0x082a1f57 in mysql_select (thd=0xb6de4aa8, rref_pointer_array=0xb6de5b70, tables=0x885b1e8, wild_num=0, fields=@0xb6de5ae0, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2156153344, result=0x885b3f8, unit=0xb6de5814, select_lex=0xb6de5a4c) at sql_select.cc:2279
#15 0x082a6332 in handle_select (thd=0xb6de4aa8, lex=0xb6de57bc, result=0x885b3f8, setup_tables_done_option=0) at sql_select.cc:256
#16 0x0823b298 in mysql_execute_command (thd=0xb6de4aa8) at sql_parse.cc:2666

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/34952

ChangeSet@1.2526, 2007-10-05 09:38:57+02:00, tnurnberg@sin.intern.azundris.com +3 -0
  Bug#31227: memory overrun with decimal (6,6) and zerofill and group_concat
  
  Reserve the space for the leading 0 (before the decimal point) in DECIMAL(a,a) ZEROFILL.

pushed to 5.0.52, 5.1.23 maint.

Hi!  Will this patch fix this crash too?

drop table if exists `t1`;
create table `t1` (`a` decimal (1,0) zerofill ,`b` decimal (1,0) zerofill)engine=heap;
insert into `t1`(`a`,`b`) values (0,0);
create temporary table t select (ifnull(`a`,`b`)) from `t1`;

Shane,

this is most probably related to the problem described in bug #18984. Can you please report it as a separate bug and mention the two bug numbers there ?

Hi! I opened bug #31471 for this last testcase.

Pushed into 5.1.23-beta

Pushed into 5.0.52

Noted in 5.0.52, 5.1.23 changelogs.

Use of DECIMAL(n, n) ZEROFILL in GROUP_CONCAT() could cause a server crash.