Bug #9708 calculation error
Submitted: 7 Apr 2005 9:13 Modified: 7 Apr 2005 12:12
Reporter: Oliver Peters Email Updates:
Status: Not a Bug Impact on me:
None 
Category:MySQL Server Severity:S3 (Non-critical)
Version:4.1.11 OS:Microsoft Windows (Windows2000)
Assigned to: Sergei Golubchik CPU Architecture:Any

[7 Apr 2005 9:13] Oliver Peters
Description:
SELECT 2.1/0.3*1.000000000000000
-> 7.000000000000001

SELECT 2.1/0.3*1.0000000000000000
-> 7.0000000000000009

In my opinion this is an error. 2.1/0.3 has to be exactly 7. I really rely on this believe

How to repeat:
see description

Suggested fix:
I only know a little awk and perl - so I'm almost helpless. In this case "totally helpless" ;-)
[7 Apr 2005 12:12] Sergei Golubchik
Well, as you know Perl, you can see that problem is intrinsic deficiency of floating point numbers - they are not precise:

$ perl -e 'printf "%.15f\n", 2.1/0.3'
7.000000000000001
$ perl -e 'printf "%.16f\n", 2.1/0.3'
7.0000000000000009

But MySQL 5.0.3 uses a precision math library that performs arithmetic calculatiions exactly, without precision loss.
[7 Apr 2005 12:41] Oliver Peters
thank you very much - now I understand and can sleep better

Oliver Peters