Bug #83311 unexpected and inconsistent results with virtual column when meets index
Submitted: 9 Oct 2016 7:08 Modified: 18 Nov 2016 7:44
Reporter: 帅 Bang Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: DML Severity:S3 (Non-critical)
Version:5.7, 5.7.15 OS:Linux
Assigned to: CPU Architecture:Any

[9 Oct 2016 7:08] 帅 Bang
Description:
mysql> use test;
Database changed

mysql> set @@collation_connection ='utf8_bin';
Query OK, 0 rows affected (0.00 sec)

mysql> drop table if exists t5;
Query OK, 0 rows affected (0.01 sec)

mysql> create table t5(c1 varchar(10) CHARACTER SET utf8 DEFAULT NULL,c2 int(11) GENERATED ALWAYS AS ((`c1` > 'a')) VIRTUAL,c3 varchar(10) COLLATE utf8_bin DEFAULT NULL);
Query OK, 0 rows affected (0.01 sec)

mysql> insert into t5(c1,c3) values('C', 'C');
Query OK, 1 row affected (0.00 sec)

mysql> alter table t5 add column c4 int as ('a'='A');
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> set @@collation_connection ='utf8_general_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> alter table t5 add column c5 int as ('a'='A');
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> set @@collation_connection ='utf8_bin';
Query OK, 0 rows affected (0.00 sec)

mysql> alter table t5 add column c6 int as ('a'='A');
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> create index idx_t5 on t5(c4);
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> set @@collation_connection ='utf8_general_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> alter table t5 add column c7 int as ('a'='A');
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> select * from t5;
+------+------+------+------+------+------+------+
| c1   | c2   | c3   | c4 /*notice here, please */   | c5   | c6   | c7   |
+------+------+------+------+------+------+------+
| C    |    1 | C    |    1  |    1 |    1 |    1 |
+------+------+------+------+------+------+------+
1 row in set (0.00 sec)

mysql> select c4 from t5;
+------+
| c4   |
+------+
|    0 |  /*notice here, please */
+------+
1 row in set (0.00 sec)

With the help of explain, you will know that index is used when it is performed:

mysql> explain select c4 from t5;
+----+-------------+-------+------------+-------+---------------+--------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key    | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+--------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | t5    | NULL       | index | NULL          | idx_t5 | 5       | NULL |    1 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+--------+---------+------+------+----------+-------------+

So, 'select c4 from t5;' and 'select * from t5' return inconsistent results in terms of column c4.

How to repeat:
set @@collation_connection ='utf8_bin';
drop table if exists t5;
create table t5(c1 varchar(10) CHARACTER SET utf8 DEFAULT NULL,c2 int(11) GENERATED ALWAYS AS ((`c1` > 'a')) VIRTUAL,c3 varchar(10) COLLATE utf8_bin DEFAULT NULL);
insert into t5(c1,c3) values('C', 'C');
alter table t5 add column c4 int as ('a'='A');
set @@collation_connection ='utf8_general_ci';
alter table t5 add column c5 int as ('a'='A');
set @@collation_connection ='utf8_bin';
alter table t5 add column c6 int as ('a'='A');
create index idx_t5 on t5(c4);
set @@collation_connection ='utf8_general_ci';
alter table t5 add column c7 int as ('a'='A');
select * from t5;
select c4 from t5;

Suggested fix:
We must take the collation_connection into consideration, 

so both of the results in c4 should be 1.
[10 Oct 2016 6:16] MySQL Verification Team
Hello Bang,

Thank you for the report and test case.
Observed with 5.7.15 build.

Thanks,
Umesh
[18 Nov 2016 7:44] Erlend Dahl
Fixed in the upcoming 8.0.1 release under the heading of

Bug#22991924 GCOLS: UTF16 STRING OPERATIONS GIVE DOUBLED \0\0 STRINGS ON EACH TABLE REBUILD
[11 Jul 2017 11:48] Erlend Dahl
Fixed in 8.0.1 under the heading of

Bug#22991924 	GCOLS: UTF16 STRING OPERATIONS GIVE DOUBLED \0\0 STRINGS ON EACH TABLE REBUILD