Bug #71273 Optimizer trace error on range analysis of index on a binary column
Submitted: 2 Jan 2014 12:35 Modified: 18 Feb 2014 16:51
Reporter: Øystein Grøvlen Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S3 (Non-critical)
Version:5.6,5.7 OS:Any
Assigned to: CPU Architecture:Any

[2 Jan 2014 12:35] Øystein Grøvlen
Description:
When getting the trace for a query which involves a predicate on an indexed binary column, I get a warning:

+---------+------+--------------------------------------------------------------------------+
| Level   | Code | Message                                                                  |
+---------+------+--------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xD9[\x943j\x99...' for column 'TRACE' at row 1 |
+---------+------+--------------------------------------------------------------------------+

The trace output is truncated with the last part being:

                 "analyzing_range_alternatives": {
                    "range_scan_alternatives": [
                      {
                        "index": "i_b",
                        "ranges": [
                          " |

How to repeat:
create table t(i int primary key, b binary(16), key i_b(b));
insert into t values (1, x'D95B94336A9946A39CF5B58CFE772D8C');
set optimizer_trace='enabled=on';
explain select * from t where b in (0xD95B94336A9946A39CF5B58CFE772D8C);
select trace  from information_schema.optimizer_trace;
drop table t;
[2 Jan 2014 12:39] MySQL Verification Team
verified on latest trunk
mysql> show warnings;
+---------+------+--------------------------------------------------------------------------+
| Level   | Code | Message                                                                  |
+---------+------+--------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xD9[\x943j\x99...' for column 'TRACE' at row 1 |
+---------+------+--------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> select version();
+-----------------+
| version()       |
+-----------------+
| 5.7.4-m14-debug |
+-----------------+
1 row in set (0.00 sec)
[18 Feb 2014 16:51] Paul DuBois
Noted in 5.7.4 changelog.

Optimizer trace output from the range optimizer could include raw
binary data and generate unprintable characters. Now binary data is
printed in hex format.