Bug #59193 Test "gis-precise" fails because of (numeric) precision differences

• Submitted: 28 Dec 2010 13:30
• Modified: 30 Jun 2011 15:03
• Reporter: Joerg Bruehe
• Status: Unsupported
• Category: Tests: Server
• Severity: S3 (Non-critical)
• Version: 5.6.1-m5
• OS: Other (HP-UX + Linux, IA64,)
• CPU Architecture: Any

[28 Dec 2010 13:30] Joerg Bruehe

Description:
This difference occurred during the tests runs in the 5.6.1-m5 build:

=====
main.gis-precise                         [ retry-fail ]
        Test ended at YYYY-MM-DD HH:MM:SS

CURRENT_TEST: main.gis-precise
--- /PATH/mysql-test/r/gis-precise.result
+++ /PATH/mysql-test/r/gis-precise.reject
@@ -122,10 +122,10 @@
 0.8944271909999159
 select astext(ST_Intersection(GeomFromText('POLYGON((0 0, 50 45, 40 50, 0 0))'), GeomFromText('POLYGON((50 5, 55 10, 0 45, 50 5))')));
 astext(ST_Intersection(GeomFromText('POLYGON((0 0, 50 45, 40 50, 0 0))'), GeomFromText('POLYGON((50 5, 55 10, 0 45, 50 5))')))
-POLYGON((26.470___ 23.823___,21.951___ 27.439___,23.855421686746986 29.819___,29.289___ 26.360___,26.470___ 23.823___))
+POLYGON((26.470___ 23.823___,21.951___ 27.439___,23.85542168674699  29.819___,29.289___ 26.360___,26.470___ 23.823___))
 select astext(ST_Intersection(GeomFromText('LINESTRING(0 0, 50 45, 40 50, 0 0)'), GeomFromText('LINESTRING(50 5, 55 10, 0 45, 50 5)')));
 astext(ST_Intersection(GeomFromText('LINESTRING(0 0, 50 45, 40 50, 0 0)'), GeomFromText('LINESTRING(50 5, 55 10, 0 45, 50 5)')))
-MULTIPOINT(26.470___ 23.823___,29.289___ 26.360___,21.951___ 27.439___,23.855421686746986 29.819___)
+MULTIPOINT(26.470___ 23.823___,29.289___ 26.360___,21.951___ 27.439___,23.85542168674699  29.819___)
 select astext(ST_Intersection(GeomFromText('LINESTRING(0 0, 50 45, 40 50)'), GeomFromText('LINESTRING(50 5, 55 10, 0 45)')));
 astext(ST_Intersection(GeomFromText('LINESTRING(0 0, 50 45, 40 50)'), GeomFromText('LINESTRING(50 5, 55 10, 0 45)')))
 POINT(29.289940828402365 26.360946745562128)
@@ -158,17 +158,17 @@
 insert into t1 values (geomfromtext('POLYGON((1 1, 10 10, 0 8, 1 1))'));
 select astext(geom), area(geom),area(ST_buffer(geom,2)) from t1;
 astext(geom)   area(geom)  area(ST_buffer(geom,2))
-POLYGON((0 0,10 10,0 8,0 0))   40  101.67150840374886
-POLYGON((1 1,10 10,0 8,1 1))   36  108.55539589266459
+POLYGON((0 0,10 10,0 8,0 0))   40  101.67150840374887
+POLYGON((1 1,10 10,0 8,1 1))   36  108.55539589266462
 select astext(ST_buffer(geom,2)) from t1;
 astext(ST_buffer(geom,2))
...
(more differences)
=====

(The double underscores above are my editing, "result" and "reject" file had the same digit sequence in those places.)

The failure occurs on HP-UX/IA64 (HP compiler) and Linux/IA64 (gcc), but not with the "specific" (for SLES and RHEL) IA64 RPMs for which we use icc.

How to repeat:
Found by running the test suite.

Suggested fix:
No idea.

It is no real surprise that numeric precision differs (unless special precautions are taken).