Bug #43530 Maria has Issues with range select <>, < with -ve range values on signed index

Description:
With other engines:
SELECT * FROM t3 WHERE c2 <> -128 ORDER BY c2,c7 DESC LIMIT 2;
c1      c2      c3      c4      c5      c6      c7
122     -123    124     125     126     127     128
115     -116    117     118     119     120     121

With Maria:
SELECT * FROM t3 WHERE c2 <> -128 ORDER BY c2,c7 DESC LIMIT 2;
c1      c2      c3      c4      c5      c6      c7
0       -128    1       2       3       4       5
122     -123    124     125     126     127     128

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
With other engines:
SELECT * FROM t3 WHERE c2 < -128 ORDER BY c2,c7;
c1      c2      c3      c4      c5      c6      c7

With Maria:
SELECT * FROM t3 WHERE c2 < -128 ORDER BY c2,c7;
c1      c2      c3      c4      c5      c6      c7
0       -128    1       2       3       4       5

How to repeat:
Attached please find the test case.

Test Case

Attachment: insert_number_bug_maria_43530.test (application/octet-stream, text), 1.35 KiB.

Thank you for the report.

Verified as described.

test case without unnecessary SELECTs

Attachment: bug43530.test (application/octet-stream, text), 893 bytes.

Same with all int types.

I cannot repeat this either.

I can repeat it. Take the file bug43530.test, remove engine=maria from it.
Run it, save the result. Rerun with --mysqld=--default-storage-engine=maria, see the difference:
@@ -17,10 +17,12 @@
 INSERT INTO t3 VALUES(0,-128,1,2,3,4,5),(255,127,6,7,8,9,10);
 SELECT * FROM t3 WHERE c2 <> -128 ORDER BY c2,c7 DESC LIMIT 2;
 c1     c2      c3      c4      c5      c6      c7
+0      -128    1       2       3       4       5
 122    -123    124     125     126     127     128
-115    -116    117     118     119     120     121
 SELECT * FROM t3 WHERE c2 < -128 ORDER BY c2,c7;
 c1     c2      c3      c4      c5      c6      c7
+0      -128    1       2       3       4       5
 SELECT * FROM t3 WHERE c2 < -128 ORDER BY c2,c7 LIMIT 2;
 c1     c2      c3      c4      c5      c6      c7
+0      -128    1       2       3       4       5
 drop table t3;

I am using 6.0-maria:
revision-id:guilhem@mysql.com-20090305214216-6ibcfd78y20gttex

goes away when removing HA_DO_INDEX_COND_PUSHDOWN from ha_maria.h

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69152

2726 Guilhem Bichot	2009-03-13
      Fix for multiple symptoms sharing the same cause:
      BUG#42297 Maria: crash in multi-range-read code
      BUG#42298 Maria: SELECT with join returns no rows
      BUG#42299 Maria: SELECT using cp1251-table returns no rows
      BUG#42681 Maria returns duplicate rows with range access on 'date type
      BUG#42683 Maria returns wrong results for <= NULL and <> NULL 
      BUG#43527 Maria returns no rows on multi range access with limit clause
      BUG#43530 Maria has Issues with range select <>, < with -ve range values on signed index
      BUG#43552 Maria returned wrong rows with range access on float
      BUG#43620 Maria throws 'Got error 176 from storage engine' on a range query
      BUG#43623 Maria returns no rows with date index on range access >, >=, BETWEEN
     @ mysql-test/suite/maria/r/maria.result
        after fixing the bug, we can see one more row in the result. Ah, if we had paid attention
        to maria.result when we added this straight_join test, we would have caught the bug immediately.
     @ mysql-test/suite/maria/r/maria4.result
        result
     @ mysql-test/suite/maria/t/maria4.test
        test for fixed bugs. All its pieces would fail (errno 176, missing rows, too many rows) without the entire
        bugfix of ma_rkey.c
     @ storage/maria/ma_rkey.c
        Because of missing (), icp_res was inverted compared to the result of
        ma_check_index_cond(), which wasn't desired (0==0 -> 1, 1==0 -> 0). We would go
        to "err:" wrongly and thus pick up the value of my_errno which was left from previous
        functions (for example, 176 left by the ha_tina CSV log write at start of statement!);
        sometimes the errno would be returned to client, sometimes it would just cause
        a matching row to be missed.
        This fixed BUG#42297 BUG#42298. But was not enough for BUG#43552:
        - icp_res==2 was not converted to "key not found", causing non-matching rows to be returned.
        Now the usage of icp_res is closer to ma_rnext.c and ma_rnext_same.c.

Pushed into 6.0.11-alpha (revid:guilhem@mysql.com-20090402210815-lu17n4kj8c73cfe8) (version source revid:guilhem@mysql.com-20090313141043-73a6mr7hsrqm3djc) (merge vers: 6.0.11-alpha) (pib:6)

A note has been added to the 6.0.11 changelog: 

Running a SELECT using a range query on with <> or < with a negative values on a Maria table could return invalid result sets.