MySQL Bugs: #32859: Character sets: no warning with non-fitting chariot wheel

Bug #32859	Character sets: no warning with non-fitting chariot wheel
Submitted:	29 Nov 2007 22:20	Modified:	18 Jan 2018 13:10
Reporter:	Peter Gulutzan	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Charsets	Severity:	S3 (Non-critical)
Version:	5.5, 6.0.4-alpha-debug	OS:	Linux (SUSE 10 64-bit)
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
I have a table with two columns: utf32 and latin1.
I put 0xc581 in the utf32 column.
(This is L With Stroke. BMP. Won't fit in latin1.)
I try to copy the utf32 column to the latin1 column.
I see a warning "Incorrect string value". Good.
I put 0x100cc in the utf32 column.
(This is Linear B Chariot Wheel. Non-BMP. Won't fit in latin1.)
I don't see a warning.

How to repeat:
mysql> create table t (utf32 char(5) character set utf32, latin1 char(5) character set latin1);
Query OK, 0 rows affected (0.00 sec)

mysql> insert into t (utf32) values (0xc581);
Query OK, 1 row affected (0.00 sec)

mysql> update t set latin1 = utf32;
Query OK, 1 row affected, 1 warning (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> show warnings;
+---------+------+-------------------------------------------------------------------------+
| Level   | Code | Message                                                                 |
+---------+------+-------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\x00\x00\xC5\x81' for column 'latin1' at row 1 |
+---------+------+-------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> delete from t;
Query OK, 1 row affected (0.00 sec)

mysql> insert into t (utf32) values (0x100cc);
Query OK, 1 row affected (0.00 sec)

mysql> update t set latin1 = utf32;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

Thank you for the bug report.

use test
drop table if exists t;
create table t (utf32 char(5) character set utf32, latin1 char(5) character set
latin1);
insert into t (utf32) values (0xc581);
update t set latin1 = utf32;
show warnings;
delete from t;
insert into t (utf32) values (0x100cc);
update t set latin1 = utf32;

c:\dev>6.0\bin\mysql -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 6.0.4-alpha-nt Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use test
Database changed
mysql> drop table if exists t;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> create table t (utf32 char(5) character set utf32, latin1 char(5) character set
    -> latin1);
Query OK, 0 rows affected (0.17 sec)

mysql> insert into t (utf32) values (0xc581);
Query OK, 1 row affected (0.00 sec)

mysql> update t set latin1 = utf32;
Query OK, 1 row affected, 1 warning (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> show warnings;
+---------+------+-------------------------------------------------------------------------+
| Level   | Code | Message                                                                 |
+---------+------+-------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\x00\x00\xC5\x81' for column 'latin1' at row 1 |
+---------+------+-------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> delete from t;
Query OK, 1 row affected (0.00 sec)

mysql> insert into t (utf32) values (0x100cc);
Query OK, 1 row affected (0.00 sec)

mysql> update t set latin1 = utf32;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql>

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114658

3093 Alexander Barkov	2010-07-29
      Bug#32859 Character sets: no warning with non-fitting chariot wheel
      Problem: Unicode-to-Latin1 conversion routine did not reject
      supplementary characters, instead it erroneously mapped them
      into Basic Multilingual Plane.
      Fix: adding range check: reject supplementary characters (U+10000 and higher).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/115324

3196 Alexander Barkov	2010-08-09
      Bug#32859 Character sets: no warning with non-fitting chariot wheel
      Problem: Unicode-to-Latin1 conversion routine did not reject
      supplementary characters, instead it erroneously mapped them
      into Basic Multilingual Plane.
      Fix: adding range check: reject supplementary characters (U+10000 and higher).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/115325

3197 Alexander Barkov	2010-08-09
      Bug#32859 Character sets: no warning with non-fitting chariot wheel
      Problem: Unicode-to-Latin1 conversion routine did not reject
      supplementary characters, instead it erroneously mapped them
      into Basic Multilingual Plane.
      Fix: adding range check: reject supplementary characters (U+10000 and higher).

Pushed into mysql-next-mr-bugfixing (currently marked as 5.6.99-m5)

Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)

Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)

Bug does not appear in any released 5.6.x version. No changelog entry needed.

(However, if this gets pushed to 5.5.x, please set back to Documenting)

Changing to 'Closed' was premature.
The bug still exists in 5.5.

What I don't understand is:
How did it get into "Documenting" state in the first place?
The progress entry is:
"
[13 Nov 17:20] Bugs System
Action: Update via API.
-Status:               Patch queued
+Status:               Documenting
"
Well, Ms API was wrong, and if she
did this often, we may have a problem
with many prematurely-closed bug reports.

Fixed in 5.6.