Bug #29201 decimal(20, 20) has rounding error
Submitted: 19 Jun 2007 10:45 Modified: 2 Jul 2007 6:48
Reporter: Hakan Kuecuekyilmaz
Status: Closed
Category:Server: Falcon Severity:S2 (Serious)
Version:6.0.1-alpha OS:Any
Assigned to: Christopher Powers Target Version:

[19 Jun 2007 10:45] Hakan Kuecuekyilmaz 
Description:
Selecting a decimal value shows rounding errors with Falcon.

How to repeat:
[10:44] root@test>CREATE TABLE t1 (a decimal(20, 20)) Engine Falcon;
Query OK, 0 rows affected (0.01 sec)

[10:44] root@test>INSERT INTO t1 VALUES (0.123456789012345678901234567890);
Query OK, 1 row affected, 1 warning (0.00 sec)

[10:44] root@test>INSERT INTO t1 VALUES (0.123456789);
Query OK, 1 row affected (0.01 sec)

[10:44] root@test>SELECT a FROM t1;
+------------------------+
| a                      |
+------------------------+
| 0.00000000000000000000 |
| 0.00000000000000000000 |
+------------------------+
2 rows in set (0.00 sec)

6.0.1-alpha-debug
[19 Jun 2007 10:47] Hakan Kuecuekyilmaz 
Test case is falcon_bug_29201.test
[19 Jun 2007 20:10] Miguel Solorzano 
Thank you for the bug report. Verified as described.
[20 Jun 2007 4:42] Christopher Powers 
This was caused by bitshift overflows in BigInt::divide(). The overflows are now prevented
by casting the intermediate results of the bitshift operations to a 64-bit integer.
[20 Jun 2007 6:20] Hakan Kuecuekyilmaz 
falcon_bug_29201.test passes now.
[2 Jul 2007 6:48] MC Brown 
A note has been added to the 6.0.1 changelog:

Wide DECIMAL columns would show rounding errors during SELECT. (Bug#29201)