MySQL Bugs: #20404: SHOW CREATE TABLE fails with Turkish I

Bug #20404	SHOW CREATE TABLE fails with Turkish I
Submitted:	12 Jun 2006 16:11	Modified:	17 Nov 2006 18:23
Reporter:	Peter Gulutzan	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	5.1.12-beta-debug	OS:	Linux (SUSE 10.0)
Assigned to:	Alexander Barkov	CPU Architecture:	Any

Description:
I start mysqld with --lower-case-table-names=1.
I create tables whose names contain the Turkish Capital Letter I With Dot ( 'İ ').
The name  İ is okay.
The name  İİ is not okay. If I say SHOW CREATE TABLE İİ, I see
CREATE TABLE `iiİ` ...

How to repeat:
Start mysqld with --lower-case-table-names.

mysql> SET NAMES utf8;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE DATABASE db5;
Query OK, 1 row affected (0.00 sec)

mysql> USE db5;
Database changed
mysql> CREATE TABLE İ (s1 int);
Query OK, 0 rows affected (0.01 sec)

mysql> SHOW CREATE TABLE İ;
+-------+----------------------------------------------------------------------------------------+
| Table | Create Table                                                                           |
+-------+----------------------------------------------------------------------------------------+
| İ    | CREATE TABLE `i` (
  `s1` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 |
+-------+----------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

mysql> CREATE TABLE İİ (s1 int);
Query OK, 0 rows affected (0.01 sec)

mysql> SHOW CREATE TABLE İİ;
+-------+------------------------------------------------------------------------------------------+
| Table | Create Table                                                                             |
+-------+------------------------------------------------------------------------------------------+
| İİ  | CREATE TABLE `iiİ` (
  `s1` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1 |
+-------+------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

Thank you for a problem report. Please, send the results of:

SHOW VARIABLES LIKE 'char%';

from the mysql command line client, and results of

echo $LANG

from the shell.

Good questions. I should have said these things in my original posting.

The shell says:
pgulutzan@linux:~> echo $LANG
en_US.UTF-8

SHOW VARIABLES LIKE 'char%' says:
mysql> SHOW VARIABLES LIKE 'char%';
+--------------------------+----------------------------------------+
| Variable_name            | Value                                  |
+--------------------------+----------------------------------------+
| character_set_client     | utf8                                   |
| character_set_connection | utf8                                   |
| character_set_database   | latin1                                 |
| character_set_filesystem | binary                                 |
| character_set_results    | utf8                                   |
| character_set_server     | latin1                                 |
| character_set_system     | utf8                                   |
| character_sets_dir       | /usr/local/mysql/share/mysql/charsets/ |
+--------------------------+----------------------------------------+
8 rows in set (0.00 sec)

It doesn't matter that character-set-server=latin1. If I start mysqld with
--default-character-set=utf8, I still get the problem.

Verified just as described with 5.1.12-BK on Linux:

suse:/home/openxs/dbs/5.1 # bin/mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2 to server version: 5.1.12-beta-debug

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> drop database db5;
Query OK, 2 rows affected (0.05 sec)

mysql> show variables like 'lower%';
+------------------------+-------+
| Variable_name          | Value |
+------------------------+-------+
| lower_case_file_system | OFF   |
| lower_case_table_names | 1     |
+------------------------+-------+
2 rows in set (0.00 sec)

mysql> set names utf8;
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like 'char%';
+--------------------------+--------------------------------------------+
| Variable_name            | Value                                      |
+--------------------------+--------------------------------------------+
| character_set_client     | utf8                                       |
| character_set_connection | utf8                                       |
| character_set_database   | latin1                                     |
| character_set_filesystem | binary                                     |
| character_set_results    | utf8                                       |
| character_set_server     | latin1                                     |
| character_set_system     | utf8                                       |
| character_sets_dir       | /home/openxs/dbs/5.1/share/mysql/charsets/ |
+--------------------------+--------------------------------------------+
8 rows in set (0.01 sec)

mysql> CREATE TABLE İ (s1 int);
Query OK, 0 rows affected (0.03 sec)

mysql> SHOW CREATE TABLE İ\G
*************************** 1. row ***************************
       Table: İ
Create Table: CREATE TABLE `i` (
  `s1` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.01 sec)

mysql> CREATE TABLE İİ (s1 int);
Query OK, 0 rows affected (0.03 sec)

mysql> SHOW CREATE TABLE İİ\G
*************************** 1. row ***************************
       Table: İİ
Create Table: CREATE TABLE `iiİ` (
  `s1` int(11) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.01 sec)

mysql>

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/11675

ChangeSet@1.2234, 2006-09-11 13:54:41+05:00, bar@mysql.com +9 -0
  Bug#20404: SHOW CREATE TABLE fails with Turkish I
  
  Problem: SHOW CREATE TABLE printed garbage in table
  name for tables having TURKISH I
  (i.e. LATIN CAPITABLE LETTER I WITH DOT ABOVE)
  when lower-case-table-name=1.
  
  Reason: In some cases during lower/upper conversion in utf8,
  the result string can be shorter the original string
  (including the above letter). Old implementation of caseup_str()
  and casedn_str() didn't handle the result length properly,
  assuming that length cannot change.
  
  This fix changes the result type of cs->cset->casedn_str()
  and cs->cset->caseup_str() from VOID to UINT, to return
  the result length, as well as put '\0' terminator on a 
  proper place.
  
  Also, my_caseup_str_utf8() and my_casedn_str_utf8() were 
  rewritten not to use strlen() for performance purposes.
  It was done with help of adding of new functions - my_utf8_uni_no_range()
  and my_uni_utf8_no_range() - for null terminated strings.

Setting to "Patch approved", as got "ok" from two developers.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/14544

ChangeSet@1.2292, 2006-10-30 14:40:15+04:00, bar@mysql.com +9 -0
   Bug#20404: SHOW CREATE TABLE fails with Turkish I
    
    Problem: SHOW CREATE TABLE printed garbage in table
    name for tables having TURKISH I
    (i.e. LATIN CAPITABLE LETTER I WITH DOT ABOVE)
    when lower-case-table-name=1.
    
    Reason: In some cases during lower/upper conversion in utf8,
    the result string can be shorter the original string
    (including the above letter). Old implementation of caseup_str()
    and casedn_str() didn't handle the result length properly,
    assuming that length cannot change.
    
    This fix changes the result type of cs->cset->casedn_str()
    and cs->cset->caseup_str() from VOID to UINT, to return
    the result length, as well as put '\0' terminator on a 
    proper place.
    
    Also, my_caseup_str_utf8() and my_casedn_str_utf8() were 
    rewritten not to use strlen() for performance purposes.
    It was done with help of adding of new functions - my_utf8_uni_no_range()
    and my_uni_utf8_no_range() - for null terminated strings.

Pushed into 5.0-rpl, marked as 5.0.27

Merged into 5.1.13-rpl

Appeared in 5.0.32 common
Appeared in 5.1.14 common

Noted in 5.0.32, 5.1.14 changelogs.

With lower_case_table_names set to 1, SHOW CREATE TABLE printed
incorrect output for table names containing Turkish I (IN CAPITAL 
LETTER I WITH DOT ABOVE).