Bug #43552 Maria returned wrong rows with range access on float

Description:
CREATE TABLE t1(c1 FLOAT(10,5) UNSIGNED NOT NULL, c2 FLOAT(10,5) SIGNED NULL, c3 FLOAT, c4 INT, UNIQUE INDEX idx(c1,c2));

SELECT * FROM t1;
c1      c2      c3      c4
0.00000 -0.10000        -0.1    13
0.00000 0.00000 0       7
1.00000 1.00000 1       10
100.05000       -100.05000      100.05  1
100000.00000    -100000.00000   100000  5
100000.00000    100000.00000    111111000       19
12345.00000     NULL    NULL    NULL

With other engines:
----------------------
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2;
c1      c2      c3      c4
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 LIMIT 2;
c1      c2      c3      c4
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 DESC;
c1      c2      c3      c4
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 DESC LIMIT 2;
c1      c2      c3      c4

With Maria:
-------------
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2;
c1      c2      c3      c4
100000.00000    100000.00000    111111000       19
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 LIMIT 2;
c1      c2      c3      c4
100000.00000    100000.00000    111111000       19
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 DESC;
c1      c2      c3      c4
100000.00000    100000.00000    111111000       19
SELECT * FROM t1 WHERE c2 >= '-99999.99999' AND c2 < '0.0' AND c1 = '99999.99999' ORDER BY c1,c2 DESC LIMIT 2;
c1      c2      c3      c4
100000.00000    100000.00000    111111000       19

How to repeat:
Attaching the test case.

Test Case

Attachment: insert_decimal_bug_maria_43552.test (application/octet-stream, text), 1.08 KiB.

Thank you for the report.

Verified as described.

I tried to reproduce today. Results are the same on Maria, MyISAM and InnoDB.

So I cannot repeat.

I am using 6.0-maria:
revision-id:guilhem@mysql.com-20090305214216-6ibcfd78y20gttex
and I can repeat, running insert_decimal_bug_maria_43552, I see the difference between MyISAM and Maria.

goes away when removing HA_DO_INDEX_COND_PUSHDOWN from ha_maria.h

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69152

2726 Guilhem Bichot	2009-03-13
      Fix for multiple symptoms sharing the same cause:
      BUG#42297 Maria: crash in multi-range-read code
      BUG#42298 Maria: SELECT with join returns no rows
      BUG#42299 Maria: SELECT using cp1251-table returns no rows
      BUG#42681 Maria returns duplicate rows with range access on 'date type
      BUG#42683 Maria returns wrong results for <= NULL and <> NULL 
      BUG#43527 Maria returns no rows on multi range access with limit clause
      BUG#43530 Maria has Issues with range select <>, < with -ve range values on signed index
      BUG#43552 Maria returned wrong rows with range access on float
      BUG#43620 Maria throws 'Got error 176 from storage engine' on a range query
      BUG#43623 Maria returns no rows with date index on range access >, >=, BETWEEN
     @ mysql-test/suite/maria/r/maria.result
        after fixing the bug, we can see one more row in the result. Ah, if we had paid attention
        to maria.result when we added this straight_join test, we would have caught the bug immediately.
     @ mysql-test/suite/maria/r/maria4.result
        result
     @ mysql-test/suite/maria/t/maria4.test
        test for fixed bugs. All its pieces would fail (errno 176, missing rows, too many rows) without the entire
        bugfix of ma_rkey.c
     @ storage/maria/ma_rkey.c
        Because of missing (), icp_res was inverted compared to the result of
        ma_check_index_cond(), which wasn't desired (0==0 -> 1, 1==0 -> 0). We would go
        to "err:" wrongly and thus pick up the value of my_errno which was left from previous
        functions (for example, 176 left by the ha_tina CSV log write at start of statement!);
        sometimes the errno would be returned to client, sometimes it would just cause
        a matching row to be missed.
        This fixed BUG#42297 BUG#42298. But was not enough for BUG#43552:
        - icp_res==2 was not converted to "key not found", causing non-matching rows to be returned.
        Now the usage of icp_res is closer to ma_rnext.c and ma_rnext_same.c.

Pushed into 6.0.11-alpha (revid:guilhem@mysql.com-20090402210815-lu17n4kj8c73cfe8) (version source revid:guilhem@mysql.com-20090313141043-73a6mr7hsrqm3djc) (merge vers: 6.0.11-alpha) (pib:6)

A note has been added to the 6.0.11 changelog: 

Running a SELECT using a range query on FLOAT on a Maria table could return invalid result sets.