MySQL Bugs: #55436: buffer overflow in debug binary of dbug_buff in Field_new_decimal::store

Bug #55436	buffer overflow in debug binary of dbug_buff in Field_new_decimal::store_value
Submitted:	21 Jul 2010 11:16	Modified:	9 May 2011 19:31
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Data Types	Severity:	S1 (Critical)
Version:	5.1.48-debug, 5.1.49-debug	OS:	Windows
Assigned to:	Tatiana Azundris Nuernberg	CPU Architecture:	Any

Description:
5.1.48-debug on windows gave these errors:

Error:Run-Time Check Failure #2 - Stack around the variable 'dbug_buff' was corrupted. At g:\mysql-5.1.48-winbuild\mysql-advanced-gpl-debug-5.1.48-build\sql\field.cc:2620

How to repeat:
.

testcase for 5.1.48-debug on windows:
---------------------------
set sql_mode='';
drop table if exists t1;
create table t1(`a` decimal(44,24))engine=myisam;
insert into `t1` set `a` = -64878E-85;

caught it in visual studio with 5.1.49-debug...
but the same code on linux under valgrind shows no problem.
not sure if dbug_buff is overflowing, or something else.

http://en.wikipedia.org/wiki/Valgrind#Limitations_of_Memcheck

Thank you for the bug report.

100721 19:54:39 [Note] Event Scheduler: Loaded 0 events
100721 19:54:39 [Note] C:\DBS\5.1\bin\mysqld: ready for connections.
Version: '5.1.49-Win X64-debug-log'  socket: ''  port: 3306  Source distribution
Error:Run-Time Check Failure #2 - Stack around the variable 'dbug_buff' was corrupted. At :0

Can confirm in debugger on OS X. Both first and second DBUG_PRINT() exceed buffer at end (pre-buffer canary unharmed).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/120342

3519 Tatiana A. Nurnberg	2010-10-08
      Bug#55436: buffer overflow in debug binary of dbug_buff in Field_new_decimal::store_value
      
      There were some misunderstandings about parameters pertaining to buffer-size.
      
      Patches fixes the reported off by one and several related ones,
      clarifies the documentation, and updates some formulae just to
      be on the safe side.
     @ mysql-test/r/type_newdecimal.result
        add test
     @ mysql-test/t/type_newdecimal.test
        add test
     @ sql/field.cc
        adjust buffer size by one to account for terminator.
     @ sql/log_event.cc
        adjust buffer size by one to account for terminator.
     @ sql/my_decimal.cc
        adjust buffer size by one to account for terminator.
        correct size formula, clarify needs in comments.
     @ sql/my_decimal.h
        clarify buffer-size needs to prevent future off-by-one bugs.
     @ sql/protocol.cc
        adjust buffer size by one to account for terminator.
     @ sql/sql_analyse.cc
        adjust buffer size by one to account for terminator.
     @ strings/decimal.c
        clarify buffer-size needs and parameters to prevent future off-by-one bugs

Noted in 5.6.3 changelogs.

In debug builds, Field_new_decimal::store_value() was subject to
buffer overflows. 

CHANGESET - http://lists.mysql.com/commits/136734

Noted in 5.1.58, 5.5.15 changelogs.