MySQL Bugs: #47756: Setting 2byte collation ID with 'set names' crashes the server

Bug #47756	Setting 2byte collation ID with 'set names' crashes the server
Submitted:	1 Oct 2009 7:45	Modified:	7 Mar 2010 1:38
Reporter:	Nidhi Shrotriya	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Charsets	Severity:	S3 (Non-critical)
Version:	6.0, 5.5.0-beta	OS:	Linux (ubuntu)
Assigned to:	Alexander Barkov	CPU Architecture:	Any
Tags:	5.5.0-beta, 6.0.14-alpha

Description:
Any command run after set names command with 2 byte collation id crashes the server.
Trace from server:
-------------------------
091001 12:59:28 [Note] /home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld: ready for connections.
Version: '6.0.14-alpha'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
091001 13:00:57 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337851 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0xa39cdc0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xad7fbf98 thread_stack 0x30c00
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(my_print_stacktrace+0x2d) [0x856f5cd]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(handle_segfault+0x3ff) [0x82456cf]
[0xb7ef5400]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(MYSQLlex(void*, void*)+0x62) [0x817a112]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(MYSQLparse(void*)+0x740) [0x8274af0]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x22a) [0x825dd9a]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x131b) [0x825f70b]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(do_command(THD*)+0xc3) [0x825ffc3]
/home/nidhi/mysql-builds/mysql-6.0-codebase/bin/mysqld(handle_one_connection+0x25e) [0x825049e]
/lib/tls/i686/cmov/libpthread.so.0 [0xb7ec64ff]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xb7ccc49e]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xa3ae7e0 = show variables like '%collation%'
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
Started 6.0 server as-

bin/mysqld_safe --no-defaults --character-sets-dir=$PWD/mysql-test/std_data to use 'Index.xml' file for 2 byte collation IDs. 

mysql> show collation like 'utf8_phone_ci';
+---------------+---------+-----+---------+----------+---------+
| Collation     | Charset | Id  | Default | Compiled | Sortlen |
+---------------+---------+-----+---------+----------+---------+
| utf8_phone_ci | utf8    | 352 |         |          |       8 |
+---------------+---------+-----+---------+----------+---------+
1 row in set (0.00 sec)

mysql> set names 'utf8' collate 'utf8_phone_ci';
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like '%collation%';
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> 091001 13:04:55 mysqld_safe Number of processes running now: 0
091001 13:04:55 mysqld_safe mysqld restarted

Thank you for the report.

Verified as described.

#0  0x0000003429e0b002 in pthread_kill () from /lib64/libpthread.so.0
#0  0x0000003429e0b002 in pthread_kill () from /lib64/libpthread.so.0
#1  0x0000000000b72e64 in my_write_core (sig=11) at stacktrace.c:309
#2  0x00000000006c74b8 in handle_segfault (sig=11) at mysqld.cc:2754
#3  <signal handler called>
#4  0x00000000005c7a0f in lex_one_token (arg=0x40a7fd00, yythd=0x1f23b058) at sql_lex.cc:883
#5  0x00000000005c7884 in MYSQLlex (arg=0x40a7fd00, yythd=0x1f23b058) at sql_lex.cc:823
#6  0x00000000006fbfcb in MYSQLparse (yythd=0x1f23b058) at sql_yacc.cc:16773
#7  0x00000000006e8803 in parse_sql (thd=0x1f23b058, parser_state=0x40a815c0, creation_ctx=0x0) at sql_parse.cc:8006
#8  0x00000000006e4250 in mysql_parse (thd=0x1f23b058, inBuf=0x1f299e80 "show variables like '%collation%'", length=33, found_semicolon=0x40a81ef0) at sql_parse.cc:5948
#9  0x00000000006d6f84 in dispatch_command (command=COM_QUERY, thd=0x1f23b058, packet=0x1f2efba9 "show variables like '%collation%'", packet_length=33) at sql_parse.cc:1074
#10 0x00000000006d61ee in do_command (thd=0x1f23b058) at sql_parse.cc:756
#11 0x00000000006d48d0 in handle_one_connection (arg=0x1f23b058) at sql_connect.cc:1164
#12 0x0000003429e061b5 in start_thread () from /lib64/libpthread.so.0
#13 0x00000034292cd39d in clone () from /lib64/libc.so.6
#14 0x0000000000000000 in ?? ()

Present in betony (5.5.0-beta).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/92267

2909 Alexander Barkov	2009-12-01
      Bug#47756 Setting 2byte collation ID with 'set names' crashes the server
      
      The problem is not actually related to 2byte collation IDs.
      The same crash happens if you change the collation ID in
      mysql-test/str_data/Index.xml to a value smaller than 256.
      
      Crash happened in SQL parser, because the "ident_map" and "state_map"
      arrays were not initialized in loadable utf8 collations.
      
      Fix: adding proper initialization of the "ident_map" and "state_map"
      members for loadable utf8 collations.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93048

2918 Alexander Barkov	2009-12-07
      Bug#47756 Setting 2byte collation ID with 'set names' crashes the server
      
      The problem is not actually related to 2byte collation IDs.
      The same crash happens if you change the collation ID in
      mysql-test/str_data/Index.xml to a value smaller than 256.
      
      Crash happened in SQL parser, because the "ident_map" and "state_map"
      arrays were not initialized in loadable utf8 collations.
      
      Fix: adding proper initialization of the "ident_map" and "state_map"
      members for loadable utf8 collations.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/93060

3756 Alexander Barkov	2009-12-07 [merge]
      Merging Bug#47756 from mysql-next-mr-bugfixing.

Pushed into:
mysql-next-mr-bugfixing (marked as 5.6.0)
mysql-6.0-codebase-bugfixing (marked as 6.0.14)

Pushed into 6.0.14-alpha (revid:alik@sun.com-20091211055901-yp18b3c7xuhl87rf) (version source revid:alik@sun.com-20091211055401-43rjwq7gjed6ds83) (merge vers: 6.0.14-alpha) (pib:13)

Pushed into 5.6.0-beta (revid:alik@sun.com-20091211055628-ltr7fero363uev7r) (version source revid:alik@sun.com-20091211055453-717czhtezc74u8db) (merge vers: 5.6.0-beta) (pib:13)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/94268

2916 Alexander Nozdrin	2009-12-15
      Backporting patch for Bug#47756
      from mysql-next-mr-bugfixing into mysql-trunk-bugfixing.
      
      NOTE: the "utf8_phone_ci" collation does not exist in mysql-trunk yet,
      so another collation with 2-byte collation ID is used: "utf8_test_ci".
      
      This patch will be null-merged to mysql-next-mr-bugfixing.
      
      Original revision:
      ------------------------------------------------------------
      revision-id: bar@mysql.com-20091207121153-hs3bqbmr0719ws21
      committer: Alexander Barkov <bar@mysql.com>
      branch nick: mysql-next-mr.b47756
      timestamp: Mon 2009-12-07 16:11:53 +0400
      message:
        Bug#47756 Setting 2byte collation ID with 'set names' crashes the server
        
        The problem is not actually related to 2byte collation IDs.
        The same crash happens if you change the collation ID in
        mysql-test/str_data/Index.xml to a value smaller than 256.
        
        Crash happened in SQL parser, because the "ident_map" and "state_map"
        arrays were not initialized in loadable utf8 collations.
        
        Fix: adding proper initialization of the "ident_map" and "state_map"
        members for loadable utf8 collations.
      ------------------------------------------------------------

Noted in 5.6.0, 6.0.14 changelogs.

User-defined collations with an ID less then 256 were not initialized
correctly when loaded and caused a server crash.

Pushed into 6.0.14-alpha (revid:alik@sun.com-20091221153807-80nxoli1tw1z9bxn) (version source revid:alik@sun.com-20091215205224-v4xxef6r433ha367) (merge vers: 6.0.14-alpha) (pib:15)

Pushed into 5.5.1-m2 (revid:alik@sun.com-20091221153538-ifi3mxf3y10ozxin) (version source revid:alik@sun.com-20091215202053-3uyr9gee2fsfguji) (merge vers: 5.5.0-beta) (pib:15)

Pushed into mysql-next-mr (revid:alik@sun.com-20091221153659-d7bt0fh6mhhclxuf) (version source revid:alik@sun.com-20091215204708-2ttqmc4r279i26a2) (pib:15)

Moved 5.6.0 changelog entry to 5.5.1.

Pushed into 5.5.3-m3 (revid:alik@sun.com-20100306103849-hha31z2enhh7jwt3) (version source revid:vvaintroub@mysql.com-20091221173845-8mqvlwp1bfp2w24g) (merge vers: 5.5.99) (pib:16)

Already fixed in earlier 5.5.x.