Bug #45263 utf32_general_ci, bad effects around CREATE TABLE AS SELECT
Submitted: 2 Jun 2009 12:00 Modified: 2 Sep 2010 16:01
Reporter: Matthias Leich Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Charsets Severity:S3 (Non-critical)
Version:5.5, 6.0 OS:Any
Assigned to: Alexander Barkov
Triage: Triaged: D1 (Critical)

[2 Jun 2009 12:00] Matthias Leich
Description:
SET COLLATION_CONNECTION=utf32_general_ci;
CREATE TEMPORARY TABLE t1 AS
SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;

mysqltest: At line 5: query 'CREATE TEMPORARY TABLE t1 AS
SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col' failed: 2013: Lost connection to MySQL server during query

Thread 1 (process 28021):
#0  0x00007f3159f02ce6 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000b4d650 in my_write_core (sig=6) at stacktrace.c:309
#2  0x00000000006ec5cf in handle_segfault (sig=6) at mysqld.cc:2711
#3  <signal handler called>
#4  0x00007f3158dfe5c5 in raise () from /lib64/libc.so.6
#5  0x00007f3158dffbb3 in abort () from /lib64/libc.so.6
#6  0x00007f3158df71e9 in __assert_fail () from /lib64/libc.so.6
#7  0x0000000000bc0e0a in my_well_formed_len_utf32 (cs=0x122d840, b=0x152a448 "00", e=0x152a44a "", nchars=2, error=0x416adb7c) at ctype-ucs2.c:2456
#8  0x00000000006e3fe8 in well_formed_copy_nchars (to_cs=0x122d840, to=0x152a132 "", to_length=8, from_cs=0x122d840, from=0x152a448 "00", from_length=2, nchars=2, well_formed_error_pos=0x416adc38,
    cannot_convert_error_pos=0x416adc30, from_end_pos=0x416adc28) at sql_string.cc:1016
#9  0x00000000006c07b0 in Field_varstring::store (this=0x152a160, from=0x152a448 "00", length=2, cs=0x122d840) at field.cc:6874
#10 0x00000000006188e7 in Item::save_in_field (this=0x1604bf0, field=0x152a160, no_conversions=false) at item.cc:5115
#11 0x000000000074a438 in fill_record (thd=0x1668578, ptr=0x152a158, values=@0x166a448, ignore_errors=true) at sql_base.cc:7466
#12 0x000000000074c328 in fill_record_n_invoke_before_triggers (thd=0x1668578, ptr=0x152a150, values=@0x166a448, ignore_errors=true, triggers=0x0, event=TRG_EVENT_INSERT) at sql_base.cc:7509
#13 0x00000000007a416d in select_create::store_values (this=0x1604ce0, values=@0x166a448) at sql_insert.cc:3774
#14 0x00000000007a8af9 in select_insert::send_data (this=0x1604ce0, values=@0x166a448) at sql_insert.cc:3164
#15 0x000000000079ee9d in JOIN::exec (this=0x1690990) at sql_select.cc:2357
#16 0x000000000079b8ee in mysql_select (thd=0x1668578, rref_pointer_array=0x166a528, tables=0x0, wild_num=0, fields=@0x166a448, conds=0x0, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0,
    select_options=2416200192, result=0x1604ce0, unit=0x1669ed8, select_lex=0x166a340) at sql_select.cc:3067
#17 0x00000000007a1149 in handle_select (thd=0x1668578, lex=0x1669e38, result=0x1604ce0, setup_tables_done_option=0) at sql_select.cc:310
#18 0x00000000006ff253 in mysql_execute_command (thd=0x1668578) at sql_parse.cc:2730
#19 0x000000000070644b in mysql_parse (thd=0x1668578, inBuf=0x1604050 "CREATE TEMPORARY TABLE t1 AS\nSELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col", length=83, found_semicolon=0x416aff30) at sql_parse.cc:5979
#20 0x0000000000707035 in dispatch_command (command=COM_QUERY, thd=0x1668578, packet=0x16c9129 "CREATE TEMPORARY TABLE t1 AS\nSELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col", packet_length=83) at sql_parse.cc:1064
#21 0x00000000007084f9 in do_command (thd=0x1668578) at sql_parse.cc:746
#22 0x00000000006f5c0d in handle_one_connection (arg=0x1668578) at sql_connect.cc:1146
#23 0x00007f3159efe040 in start_thread () from /lib64/libpthread.so.0
#24 0x00007f3158e9f08d in clone () from /lib64/libc.so.6
#25 0x0000000000000000 in ?? ()

There is no crash if I
- use the default COLLATION_CONNECTION "latin1_swedish_ci"
or
- use a server compiled without "debug"
  Here I get
  CREATE TEMPORARY TABLE t1 AS
  SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
  Warnings:
  Warning 1366 Incorrect string value: '\x30\x30'
          for column 'my_col' at row 1
  which is also suspicious because a
  SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
  gets a
  my_col
  00

My environment:
- mysql-azalea May 2009
  compile-pentium64-max, compile-pentium64-debug-max
- Linux OpenSuSE 11.0 64 Bit

How to repeat:
SET COLLATION_CONNECTION=utf32_general_ci;
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings
let $my_select= SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
eval $my_select;
eval CREATE TEMPORARY TABLE t1 AS $my_select;
DROP TEMPORARY TABLE t1;

Suggested fix:
Please achieve the following
1. Consistent behavior for
   CREATE TEMPORARY TABLE t1 AS
   SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
   and
   SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
   like both get similar errors/warnings/success.
2. No crash for server compiled with "debug"
[2 Jun 2009 12:21] Matthias Leich
I get the same bad effects for "utf32_unicode_ci".
[2 Jun 2009 12:37] Miguel Solorzano
Thank you for the bug report. Verified as described:

c:\mysql>bin\mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.4.4-alpha-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SET COLLATION_CONNECTION=utf32_general_ci;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE TEMPORARY TABLE t1 AS
    -> SELECT HEX(WEIGHT_STRING('abc', 1, 2, 0xC0)) AS my_col;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
[12 Aug 2010 9:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/115550

3197 Alexander Barkov	2010-08-12
      Bug#45263 utf32_general_ci, bad effects around CREATE TABLE AS SELECT
      Problem: Item_func_hex::val_str() returned data in ASCII format,
      which did not match collation.collation pointing to my_charset_utf32_general_ci.
      Fix: changing parent class of Item_func_hex to Item_str_ascii_func,
      as val_str() implementation is heavily ASCII-oriented.
      
        mysql-test/r/ctype_utf32.result
        mysql-test/t/ctype_utf32.test
        Adding test case
      
      
        sql/item_strfunc.cc
        sql/item_strfunc.h
        - Changing parent class to Item_str_ascii_func
        - Clean-up in Item_func_hex::fix_length_and_dec()
          Using fix_char_length() instead of setting max_length directly.
[18 Aug 2010 12:15] Alexander Barkov
The same crash happens in 5.5 without using WEIGHT_STRING() function.

SET collation_connection=utf32_general_ci;
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 AS SELECT hex(0x00);
[18 Aug 2010 12:22] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/116096

3187 Alexander Barkov	2010-08-18
      Bug#45263 utf32_general_ci, bad effects around CREATE TABLE AS SELECT
      
      Problem: Item_func_hex::val_str() returned data in ASCII format,
      which did not match collation.collation pointing to my_charset_utf32_general_ci.
      Fix: changing parent class of Item_func_hex to Item_str_ascii_func,
      as val_str() implementation is heavily ASCII-oriented.
      
        mysql-test/r/ctype_utf32.result
        mysql-test/t/ctype_utf32.test
        Adding test case
      
      
        sql/item_strfunc.cc
        sql/item_strfunc.h
        - Changing parent class to Item_str_ascii_func
        - Clean-up in Item_func_hex::fix_length_and_dec()
          Using fix_char_length() instead of setting max_length directly.
[18 Aug 2010 13:50] Alexander Barkov
Pushed into mysql-5.5-bugfixing (5.5.6-m3)
Pushed into mysql-trunk-bugfixing (5.6.1-m4)
Pushed into mysql-next-mr-bugfixing (5.6.99-m5)
[25 Aug 2010 9:22] Bugs System
Pushed into mysql-5.5 5.5.6-m3 (revid:alik@ibmvm-20100825092002-2yvkb3iwu43ycpnm) (version source revid:alik@ibmvm-20100825092002-2yvkb3iwu43ycpnm) (merge vers: 5.5.6-m3) (pib:20)
[30 Aug 2010 8:30] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@sun.com-20100830082732-n2eyijnv86exc5ci) (version source revid:alik@sun.com-20100830082732-n2eyijnv86exc5ci) (merge vers: 5.6.1-m4) (pib:21)
[30 Aug 2010 8:34] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100830082745-n6sh01wlwh3itasv) (version source revid:alik@sun.com-20100830082745-n6sh01wlwh3itasv) (pib:21)
[2 Sep 2010 16:01] Paul Dubois
Noted in 5.5.6, 5.6.1 changelogs.

With character_set_connection set to utf16 or utf32, CREATE TABLE t1
AS SELECT HEX() ... caused a server crash.