MySQL Bugs: #28361: Buffer overflow in DECIMAL code on Windows

Bug #28361	Buffer overflow in DECIMAL code on Windows
Submitted:	10 May 2007 18:41	Modified:	10 Jun 2007 20:30
Reporter:	Mark Matthews
Status:	Closed
Category:	Server	Severity:	S1 (Critical)
Version:	5.1+, 5.0	OS:	Microsoft Windows (XP/2003)
Assigned to:	Alexey Botchkov	Target Version:

Description:
When running the JDBC compliance testsuite against 5.1.17 (or 5.2.x) on Windows, the
following sequence often crashes the MySQL server with the message 

"Microsoft Visual C++ Runtime Library : Buffer overrun detected!

Program: ...Files\MySQL\MySQL Server 5.1\bin\mysqld-nt.exe

A buffer overrun has been detected which has corrupted the program's
internal state.  The program cannot safely continue execution and must
now be terminated."

DROP TABLE IF EXISTS `Decimal_Tab`;
CREATE TABLE `Decimal_Tab` ( `MAX_VAL` decimal(32,15) default NULL, `MIN_VAL`
decimal(32,15) default NULL, `NULL_VAL` decimal(32,15) default NULL) TYPE=InnoDB;
delete from Decimal_Tab;
insert into Decimal_Tab values(9999999999,0.0000000001, null);

(I was not able to reproduce this from the command-line, only from the JDBC driver, but
jbalint was able to reproduce it on his machine with the same testsuite, and that's how
we have the stack trace):

WARNING: Stack unwind information not available. Following frames may be wrong.
00 ntdll!DbgBreakPoint
01 mysqld!_RTC_StackFailure(void * retaddr = 0x004289cf, char * varname = 0x00428a65
"_db_framep_")+0x10c
02 mysqld!_RTC_CheckStackVars(void * frame = 0xb3aebe2c, struct _RTC_framedesc * v =
0x0ae3d2db)+0x45
03 mysqld!Field_new_decimal::store_value(class my_decimal * decimal_value =
0x02222d98)+0x28f
04 mysqld!Field_new_decimal::store_decimal(class my_decimal * decimal_value =
0x02222d98)+0x75
05 mysqld!Item_decimal::save_in_field(class Field * field = 0x02242668, int
no_conversions = 0)+0x2f
06 mysqld!fill_record(class THD * thd = 0x021fb820, class List<Item> * fields =
0x021fc928, class List<Item> * values = 0x021fcb40, int ignore_errors = 0)+0x175
07 mysqld!fill_record_n_invoke_before_triggers(class THD * thd = 0x021fb820, class
List<Item> * fields = 0x021fc928, class List<Item> * values = 0x021fcb40, int
ignore_errors = 0, class Table_triggers_list * triggers = 0x00000000, trg_event_type
event = TRG_EVENT_UPDATE (1))+0x20
08 mysqld!mysql_update(class THD * thd = 0x021fb820, struct st_table_list * table_list =
0x02222a70, class List<Item> * fields = 0x021fc928, class List<Item> * values =
0x021fcb40, class Item * conds = 0x00000000, unsigned int order_num = 0, struct st_order
* order = 0x00000000, unsigned long limit = 0xffffffff, enum_duplicates handle_duplicates
= DUP_ERROR (0), int ignore = 0)+0x1216
09 mysqld!mysql_execute_command(class THD * thd = 0x021fb820)+0x1de3
0a mysqld!mysql_parse(class THD * thd = 0x021fb820, char * inBuf = 0x02222980 "update
Decimal_Tab set
NULL_VAL=0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875",
unsigned int length = 0x78)+0x166
0b mysqld!dispatch_command(enum_server_command command = COM_QUERY (3), class THD * thd =
0x021fb820, char * packet = 0x0221e8d1 "update Decimal_Tab set
NULL_VAL=0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875",
unsigned int packet_length = 0x79)+0x7eb
0c mysqld!do_command(class THD * thd = 0x021fb820)+0x215
0d mysqld!handle_one_connection(void * arg = 0x021fb820)+0x154
0e mysqld!pthread_start(void * param = 0x021dc3b8)+0x80
0f MSVCR80D!beginthread+0x221
10 MSVCR80D!beginthread+0x1bc
11 kernel32!GetModuleFileNameA+0xeb

How to repeat:
It's related to the following statements, but you _might_ have to get the JDBC compliance
testsuite running, if so, ask me:

DROP TABLE IF EXISTS `Decimal_Tab`;
CREATE TABLE `Decimal_Tab` ( `MAX_VAL` decimal(32,15) default NULL, `MIN_VAL`
decimal(32,15) default NULL, `NULL_VAL` decimal(32,15) default NULL) TYPE=InnoDB;
delete from Decimal_Tab;
insert into Decimal_Tab values(9999999999,0.0000000001, null);

Just checked, this is an issue in 5.1.18 (from the to-be-released builds from the
engineering team) as well.

To trigger this bug, the following statement should be sent:
update Decimal_Tab set NULL_VAL=
0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875;

The smallest string to cause stack corruption is:
0.000000000100000000000000003643219731549774157916554706559963960899

Starting in:
Field_new_decimal::store_value()->dbug_decimal_as_string()->decimal2string()

Looks to be somewhere in decimal2string() that its overwritten (strings/decimal.c)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27046

ChangeSet@1.2491, 2007-05-20 18:12:30+05:00, holyfoot@mysql.com +3 -0
  bug #28361 Buffer overflow in DECIMAL code on Windows
  
  my_decimal in a good occasion can contain more decimal digits than
  is officially supported (DECIMAL_MAX_PRECISION), so we need to
  prepare bigger buffer for the resulting string.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27050

ChangeSet@1.2491, 2007-05-20 21:22:57+05:00, holyfoot@mysql.com +3 -0
  bug #28361 Buffer overflow in DECIMAL code on Windows
  
  my_decimal in some cases can contain more decimal digits than
  is officially supported (DECIMAL_MAX_PRECISION), so we need to
  prepare bigger buffer for the resulting string.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27060

ChangeSet@1.2492, 2007-05-21 13:02:03+05:00, holyfoot@mysql.com +5 -0
  Bug #28361 Buffer overflow in DECIMAL code on Windows 
  result max length changed for the 'decimal' fields
  so test results have to be fixed

A copy of the 5.1 version of this fix 
is included in the sources used to build 5.1.19-beta.

Pushed into 5.1.20-beta

Pushed into 5.0.44

Thank you for your bug report. This issue has been committed to our source repository of
that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available
version, including the bug fix. More information about accessing the source trees is
available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented fix in 5.0.44 and 5.1.19 changelogs.