MySQL Bugs: #28725: Falcon: crash with decimal

Bug #28725	Falcon: crash with decimal
Submitted:	28 May 2007 13:39	Modified:	2 Jul 2007 4:51
Reporter:	Peter Gulutzan	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Falcon storage engine	Severity:	S2 (Serious)
Version:	6.0.1-alpha-debug	OS:	Linux (SUSE 10 64-bit)
Assigned to:	Christopher Powers	CPU Architecture:	Any

Description:
I create a Falcon table with a decimal(19,19) column.
I insert a value.
I try to select.
Crash.

ChangeSet@1.2517, 2007-05-27 01:49:36+02:00

How to repeat:
mysql> create table td (s1 decimal(19,19)) engine=falcon;
Query OK, 0 rows affected (0.01 sec)

mysql> insert into td values (0.123456789012345678901234567890);
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> select * from td;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Thank you for the bug report. Verified as described on FC 6.0 32-bit.

Program received signal SIGFPE, Arithmetic exception.
[Switching to Thread -1308865648 (LWP 1298)]
0x0882f037 in __divdi3 ()
(gdb) bt full
#0  0x0882f037 in __divdi3 ()
No symbol table info available.
#1  0x084cc80f in ScaledBinary::putBinaryDecimal (number=1234567890123456789, ptr=0xad19b11 "\207[Í\025", precision=19, scale=19) at ScaledBinary.cpp:133
        absNumber = 1234567890123456789
        p = 0xad19b11 "\207[Í\025"
#2  0x084c76c6 in StorageInterface::decodeRecord (this=0xad19998, buf=0xad19b10 "ý\207[Í\025") at ha_falcon.cpp:2157
        value = 1234567890123456789
        precision = 19
        scale = 19
        field = (class Field *) 0xad19b38
        n = 0
        dataStream = (class EncodedDataStream *) 0xb7e852c4
        ptrDiff = 0
        old_map = (my_bitmap_map *) 0xad19b9c
        _db_func_ = 0x890050d "StorageInterface::rnd_next"
        _db_file_ = 0x88ffac6 "ha_falcon.cpp"
        _db_level_ = 9
        _db_framep_ = (char **) 0xb7b80894
<cut>

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27492

ChangeSet@1.2519, 2007-05-28 19:17:58+02:00, hakank@lu0011.wdf.sap.corp +3 -0
  Added test case for Bug#28725.

Problem is division by zero:

Program received signal SIGFPE, Arithmetic exception.
0x000000000085d917 in ScaledBinary::putBinaryDecimal (
    number=1234567890123456789, ptr=0x107c991 "\207[�025", precision=19, 
    scale=19) at ScaledBinary.cpp:133
133             putBinaryNumber(absNumber / powersOfTen[scale], precision - scale, &p, false);
(gdb) p absNumber
$1 = 1234567890123456789
(gdb) p powersOfTen
$2 = {1, 10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000, 
  1000000000, 10000000000, 100000000000, 1000000000000, 10000000000000, 
  100000000000000, 1000000000000000, 10000000000000000, 100000000000000000, 
  1000000000000000000}
(gdb) p scale
$3 = 19
(gdb) p powersOfTen[scale]
$4 = 0

This bug will be addressed by WL#3923, Arbitrary Precision Division for Falcon BigInt.

Still crashes:

CREATE TABLE t1 (a decimal(19,19)) Engine Falcon;
INSERT INTO t1 VALUES (0.123456789012345678901234567890);
SELECT * FROM t1;

Problem is division by zero

Program received signal SIGFPE, Arithmetic exception.
0x000000000079d3a1 in ScaledBinary::putBinaryDecimal (
    number=1234567890123456789, ptr=0xfb1f01 "\207[Í\025", 
    precision=<value optimized out>, scale=19) at ScaledBinary.cpp:135
135             putBinaryNumber(absNumber / powersOfTen[scale], precision - scal
e, &p, false);
(gdb) p absNumber
$1 = 1234567890123456789
(gdb) p powersOfTen[scale]

This is a pre-existing problem that was exposed when support for decimals having > 18 digits was introduced.

Numbers having a precision > 18 are assigned to the BigInt storage class. If the number is small enough, then BigInt operates upon the number as a binary int64 rather than an array of uint32 words.

When BigInt numbers are retrieved, the ScaledBinary class converts them from their binary Falcon representation into decimal format before returning them to the server. 

During the conversion, ScaledBinary descales BigInts by dividing them by a power of 10 scaling factor. The max scaling factor for int64 is 10^18, which is fine for BigInts that can be stored within 64 bits.

The problem occurs for BigInt numbers that fit within an int64 but have a scale  >= 19.  In this case, ScaledBinary tried to divide a BigInt by 10^19, which exceeds the range of the powersOfTen[] table, causing a divide by zero fault.

The solution is to store BigInt numbers having a scale > 18 as true BigInts (i.e. an array of 32-bit words) rather than int64, even if the number will fit withn an int64 field.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29144

ChangeSet@1.2580, 2007-06-19 21:43:12-05:00, chris@terrazzo.site +3 -0
  Bug#28725 "Falcon: crash with decimal"
  - Always store BigInt numbers having a scale > 18 as an array of uint32.
  
  Bug#29201 "decimal(20,20) has rounding error
  - Added 64-bit casts to BigInt::divide to avoid bitshift overflows.

falcon_bug_28725.test passes now.

A note has been added to the 6.0.1 changelog: 

Accessing data within DECIMAL columns wider than 18 digits would cause a crash. (Bug#28725)

It is repeatable with non-debug binary only. The problem seems in decoding code. I uses non-debug build:

mysql> INSERT INTO t1 VALUES (0.123456789012345678901234567890);
Query OK, 1 row affected, 1 warning (4 min 11.11 sec)

mysql> SELECT * FROM t1;
+-----------------------+
| a                     |
+-----------------------+
| 0.8231274640474997013 |
+-----------------------+
1 row in set (4.13 sec)

Then replaced with a debug build:

mysql> use test
Database changed
mysql> SELECT * FROM t1;
+-----------------------+
| a                     |
+-----------------------+
| 0.1234567890123456789 |
+-----------------------+
1 row in set (0.84 sec)

In this case, the value was inserted with the non-debug build. I did not insert the new value using debug.

The failure on Windows is due to a bug in the version of the Microsoft compiler used on Pushbuild.

The error occurs in BigInt::divide. Prior to division, both the dividend and divisor are normalized in order to guarantee that the high bit is 1:

   for (i = n - 1; i > 0; i--)
      vn[i] = (v[i] << s) | (uint32)(((uint64)v[i-1]) >> (MPBASE-s));

Here, 's' is the number of leading 0 bits and MPBASE is 32. The testcase produces an edge condition such that s == 0, resulting in a right shift of 32 bits. According to the MSDN, 

"[R]esults are undefined if the right operand of a shift expression is negative or if the right operand is greater than or equal to the number of bits in the (promoted) left operand."

The 64-bit cast should allow for a 32-bit right shift, which it does on Linux but not on Windows. This is a known problem that has since been corrected in Visual Studio 2005, Service Pack 1. Pushbuild uses an older version of Visual Studio to build Windows binaries.

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106414

Added a Windows workaround to BigInt::divide, where the shift amount is always <= 32. Macro SHR() shifts in stages if the shift amount is 32:

#define SHR(x, y) ((y) < 32 ? (x) >> (y) : ((x) >> 31 >> 1))

This macro should be removed when Pushbuild is upgraded to  Visual Studio 2005 SP1 or greater.

This bug should be documented for MySQL developers, and the remaining code should be analyzed for similar vulnerabilities.