Bug #6889 rounding problem in V4.1.7

• Submitted: 30 Nov 2004 10:24
• Modified: 30 Nov 2004 21:34
• Reporter: Bruce Leister
• Status: Not a Bug
• Category: MySQL Server: MyISAM storage engine
• Severity: S2 (Serious)
• Version: 4.1.7
• OS: Linux (Linux Redhat 9)
• CPU Architecture: Any

[30 Nov 2004 10:24] Bruce Leister

Description:
create table test_table (hrs decimal(7.1), rate decimal(7,2), amt decimal(15,3));

insert into test_table (hrs,rate) values(17.2,145.6);

update test_table set amt=hrs*rate;

select * from test_table where amt != hrs*rate;

Why is the row returned ?

also
select * from test_table where format(amt,8) != format(hrs*rate,8);
returns no row, as expected!, but
select * from test_table where format(amt,15) != format(hrs*rate,15);
again returns the row ??

How to repeat:
mysql> select * from test_table where hrs*rate != amt;
+------+--------+----------+
| hrs  | rate   | amt      |
+------+--------+----------+
| 17.2 | 145.60 | 2504.320 |
+------+--------+----------+
1 row in set (0.01 sec)

mysql> select version();
+--------------------+
| version()          |
+--------------------+
| 4.1.7-standard-log |
+--------------------+
1 row in set (0.00 sec)

[30 Nov 2004 21:34] Alexander Keremidarski

Thank you for taking the time to write to us, but this is not
a bug. Please double-check the documentation available at
http://www.mysql.com/documentation/ and the instructions on
how to report a bug at http://bugs.mysql.com/how-to-report.php

Additional info:

Floating point calculation problems are addressed in the manual.