Bug #55436 buffer overflow in debug binary of dbug_buff in Field_new_decimal::store_value
Submitted: 21 Jul 2010 11:16 Modified: 9 May 2011 19:31
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Data Types Severity:S1 (Critical)
Version:5.1.48-debug, 5.1.49-debug OS:Microsoft Windows
Assigned to: Tatiana Azundris Nuernberg CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[21 Jul 2010 11:16] Shane Bester
Description:
5.1.48-debug on windows gave these errors:

Error:Run-Time Check Failure #2 - Stack around the variable 'dbug_buff' was corrupted. At g:\mysql-5.1.48-winbuild\mysql-advanced-gpl-debug-5.1.48-build\sql\field.cc:2620

How to repeat:
.
[21 Jul 2010 11:35] Shane Bester
testcase for 5.1.48-debug on windows:
---------------------------
set sql_mode='';
drop table if exists t1;
create table t1(`a` decimal(44,24))engine=myisam;
insert into `t1` set `a` = -64878E-85;
[21 Jul 2010 13:09] Shane Bester
caught it in visual studio with 5.1.49-debug...
but the same code on linux under valgrind shows no problem.
not sure if dbug_buff is overflowing, or something else.
[21 Jul 2010 13:25] Davi Arnaut
http://en.wikipedia.org/wiki/Valgrind#Limitations_of_Memcheck
[21 Jul 2010 22:58] Miguel Solorzano
Thank you for the bug report.

100721 19:54:39 [Note] Event Scheduler: Loaded 0 events
100721 19:54:39 [Note] C:\DBS\5.1\bin\mysqld: ready for connections.
Version: '5.1.49-Win X64-debug-log'  socket: ''  port: 3306  Source distribution
Error:Run-Time Check Failure #2 - Stack around the variable 'dbug_buff' was corrupted. At :0
[29 Sep 2010 3:04] Tatiana Azundris Nuernberg
Can confirm in debugger on OS X. Both first and second DBUG_PRINT() exceed buffer at end (pre-buffer canary unharmed).
[8 Oct 2010 7:39] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/120342

3519 Tatiana A. Nurnberg	2010-10-08
      Bug#55436: buffer overflow in debug binary of dbug_buff in Field_new_decimal::store_value
      
      There were some misunderstandings about parameters pertaining to buffer-size.
      
      Patches fixes the reported off by one and several related ones,
      clarifies the documentation, and updates some formulae just to
      be on the safe side.
     @ mysql-test/r/type_newdecimal.result
        add test
     @ mysql-test/t/type_newdecimal.test
        add test
     @ sql/field.cc
        adjust buffer size by one to account for terminator.
     @ sql/log_event.cc
        adjust buffer size by one to account for terminator.
     @ sql/my_decimal.cc
        adjust buffer size by one to account for terminator.
        correct size formula, clarify needs in comments.
     @ sql/my_decimal.h
        clarify buffer-size needs to prevent future off-by-one bugs.
     @ sql/protocol.cc
        adjust buffer size by one to account for terminator.
     @ sql/sql_analyse.cc
        adjust buffer size by one to account for terminator.
     @ strings/decimal.c
        clarify buffer-size needs and parameters to prevent future off-by-one bugs
[9 May 2011 19:31] Paul Dubois
Noted in 5.6.3 changelogs.

In debug builds, Field_new_decimal::store_value() was subject to
buffer overflows. 

CHANGESET - http://lists.mysql.com/commits/136734
[12 May 2011 14:33] Paul Dubois
Noted in 5.1.58, 5.5.15 changelogs.