MySQL Bugs: #28361: Buffer overflow in DECIMAL code on Windows

Bug #28361	Buffer overflow in DECIMAL code on Windows
Submitted:	10 May 2007 16:41	Modified:	10 Jun 2007 18:30
Reporter:	Mark Matthews	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S1 (Critical)
Version:	5.1+, 5.0	OS:	Windows (XP/2003)
Assigned to:	Alexey Botchkov	CPU Architecture:	Any

Description:
When running the JDBC compliance testsuite against 5.1.17 (or 5.2.x) on Windows, the following sequence often crashes the MySQL server with the message 

"Microsoft Visual C++ Runtime Library : Buffer overrun detected!

Program: ...Files\MySQL\MySQL Server 5.1\bin\mysqld-nt.exe

A buffer overrun has been detected which has corrupted the program's
internal state.  The program cannot safely continue execution and must
now be terminated."

DROP TABLE IF EXISTS `Decimal_Tab`;
CREATE TABLE `Decimal_Tab` ( `MAX_VAL` decimal(32,15) default NULL, `MIN_VAL` decimal(32,15) default NULL, `NULL_VAL` decimal(32,15) default NULL) TYPE=InnoDB;
delete from Decimal_Tab;
insert into Decimal_Tab values(9999999999,0.0000000001, null);

(I was not able to reproduce this from the command-line, only from the JDBC driver, but jbalint was able to reproduce it on his machine with the same testsuite, and that's how we have the stack trace):

WARNING: Stack unwind information not available. Following frames may be wrong.
00 ntdll!DbgBreakPoint
01 mysqld!_RTC_StackFailure(void * retaddr = 0x004289cf, char * varname = 0x00428a65 "_db_framep_")+0x10c
02 mysqld!_RTC_CheckStackVars(void * frame = 0xb3aebe2c, struct _RTC_framedesc * v = 0x0ae3d2db)+0x45
03 mysqld!Field_new_decimal::store_value(class my_decimal * decimal_value = 0x02222d98)+0x28f
04 mysqld!Field_new_decimal::store_decimal(class my_decimal * decimal_value = 0x02222d98)+0x75
05 mysqld!Item_decimal::save_in_field(class Field * field = 0x02242668, int no_conversions = 0)+0x2f
06 mysqld!fill_record(class THD * thd = 0x021fb820, class List<Item> * fields = 0x021fc928, class List<Item> * values = 0x021fcb40, int ignore_errors = 0)+0x175
07 mysqld!fill_record_n_invoke_before_triggers(class THD * thd = 0x021fb820, class List<Item> * fields = 0x021fc928, class List<Item> * values = 0x021fcb40, int ignore_errors = 0, class Table_triggers_list * triggers = 0x00000000, trg_event_type event = TRG_EVENT_UPDATE (1))+0x20
08 mysqld!mysql_update(class THD * thd = 0x021fb820, struct st_table_list * table_list = 0x02222a70, class List<Item> * fields = 0x021fc928, class List<Item> * values = 0x021fcb40, class Item * conds = 0x00000000, unsigned int order_num = 0, struct st_order * order = 0x00000000, unsigned long limit = 0xffffffff, enum_duplicates handle_duplicates = DUP_ERROR (0), int ignore = 0)+0x1216
09 mysqld!mysql_execute_command(class THD * thd = 0x021fb820)+0x1de3
0a mysqld!mysql_parse(class THD * thd = 0x021fb820, char * inBuf = 0x02222980 "update Decimal_Tab set NULL_VAL=0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875", unsigned int length = 0x78)+0x166
0b mysqld!dispatch_command(enum_server_command command = COM_QUERY (3), class THD * thd = 0x021fb820, char * packet = 0x0221e8d1 "update Decimal_Tab set NULL_VAL=0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875", unsigned int packet_length = 0x79)+0x7eb
0c mysqld!do_command(class THD * thd = 0x021fb820)+0x215
0d mysqld!handle_one_connection(void * arg = 0x021fb820)+0x154
0e mysqld!pthread_start(void * param = 0x021dc3b8)+0x80
0f MSVCR80D!beginthread+0x221
10 MSVCR80D!beginthread+0x1bc
11 kernel32!GetModuleFileNameA+0xeb

How to repeat:
It's related to the following statements, but you _might_ have to get the JDBC compliance testsuite running, if so, ask me:

DROP TABLE IF EXISTS `Decimal_Tab`;
CREATE TABLE `Decimal_Tab` ( `MAX_VAL` decimal(32,15) default NULL, `MIN_VAL` decimal(32,15) default NULL, `NULL_VAL` decimal(32,15) default NULL) TYPE=InnoDB;
delete from Decimal_Tab;
insert into Decimal_Tab values(9999999999,0.0000000001, null);

Just checked, this is an issue in 5.1.18 (from the to-be-released builds from the engineering team) as well.

To trigger this bug, the following statement should be sent:
update Decimal_Tab set NULL_VAL=
0.00000000010000000000000000364321973154977415791655470655996396089904010295867919921875;

The smallest string to cause stack corruption is:
0.000000000100000000000000003643219731549774157916554706559963960899

Starting in:
Field_new_decimal::store_value()->dbug_decimal_as_string()->decimal2string()

Looks to be somewhere in decimal2string() that its overwritten (strings/decimal.c)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27046

ChangeSet@1.2491, 2007-05-20 18:12:30+05:00, holyfoot@mysql.com +3 -0
  bug #28361 Buffer overflow in DECIMAL code on Windows
  
  my_decimal in a good occasion can contain more decimal digits than
  is officially supported (DECIMAL_MAX_PRECISION), so we need to
  prepare bigger buffer for the resulting string.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27050

ChangeSet@1.2491, 2007-05-20 21:22:57+05:00, holyfoot@mysql.com +3 -0
  bug #28361 Buffer overflow in DECIMAL code on Windows
  
  my_decimal in some cases can contain more decimal digits than
  is officially supported (DECIMAL_MAX_PRECISION), so we need to
  prepare bigger buffer for the resulting string.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27060

ChangeSet@1.2492, 2007-05-21 13:02:03+05:00, holyfoot@mysql.com +5 -0
  Bug #28361 Buffer overflow in DECIMAL code on Windows 
  result max length changed for the 'decimal' fields
  so test results have to be fixed

A copy of the 5.1 version of this fix 
is included in the sources used to build 5.1.19-beta.

Pushed into 5.1.20-beta

Pushed into 5.0.44

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented fix in 5.0.44 and 5.1.19 changelogs.