Bug #118980 queries have different results using and without index
Submitted: 10 Sep 2:52 Modified: 11 Sep 13:04
Reporter: leo song Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S2 (Serious)
Version:MySQL8.0.41 OS:Any
Assigned to: MySQL Verification Team CPU Architecture:Any
Tags: char

[10 Sep 2:52] leo song
Description:
MySQL char type has different result when using and without index

for example
CREATE TABLE `insp` (
  `a` char(6) DEFAULT NULL,
  KEY `idx_a` (`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
insert into insp values('a\r');
insert into insp values('a\n');
insert into insp values('a\t');

mysql>select a,hex(a),length(a)  from insp FORCE INDEX(idx_a) where a <'a ';
Empty set (0.00 sec)

mysql>select a,hex(a),length(a)  from insp IGNORE INDEX(idx_a) where a <'a ';

+------+--------+-----------+
| a    | hex(a) | length(a) |
+------+--------+-----------+
   | 610D   |         2 |
| a
   | 610A   |         2 |
| a       | 6109   |         2 |
+------+--------+-----------+

The storage layer remove the trailing spaces when comparing the data。
Some effective results are filtered out.
for exmple 'a\r', '\t' are smaller than 'a   ', but they are bigger than 'a' after removed the trailing spaces.

How to repeat:
CREATE TABLE `insp` (
  `a` char(6) DEFAULT NULL,
  KEY `idx_a` (`a`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;
insert into insp values('a\r');
insert into insp values('a\n');
insert into insp values('a\t');

mysql>select a,hex(a),length(a)  from insp FORCE INDEX(idx_a) where a <'a ';
Empty set (0.00 sec)

mysql>select a,hex(a),length(a)  from insp IGNORE INDEX(idx_a) where a <'a ';

+------+--------+-----------+
| a    | hex(a) | length(a) |
+------+--------+-----------+
   | 610D   |         2 |
| a
   | 610A   |         2 |
| a       | 6109   |         2 |
+------+--------+-----------+
[11 Sep 13:04] MySQL Verification Team
Thank you for the report and test case.