Bug #23097 mysql can't insert korean on mysql prompt.
Submitted: 9 Oct 2006 5:31 Modified: 27 Mar 2008 19:22
Reporter: JongSe Park Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Command-line Clients Severity:S3 (Non-critical)
Version:5.0.22, 5.0.44 RPM OS:Linux (linux and solaris)
Assigned to: Alexander Barkov CPU Architecture:Any
Tags: backslash, bfsm_2007_10_25, bfsm_2007_11_01, libedit, readline
Triage: D2 (Serious) / R1 (None/Negligible) / E3 (Medium)

[9 Oct 2006 5:31] JongSe Park
Description:
I install mysql 5.0.22(binary) on linux and solaris.
but, I can't insert korean on mysql prompt.

I use secureCRT tool to connect mysql server.
and I log in mysql server. but I can't insert any korean. like this;

mysql> select * from address where name = '';

I can't insert korean like this "select * from address where name = '길동'".
Is this a bug of mysql?
or what do I have to do to solve this problem? 

How to repeat:
I install mysql 5.0.22(binary) on linux or solaris.
I use secureCRT tool to connect mysql server.
and I log in mysql server. but I can't insert any korean. like this;

mysql> select * from address where name = '';

I can't insert korean like this "select * from address where name = '길동'".
[9 Oct 2006 5:57] Valeriy Kravchuk
Thank you for a problem report. Can you insert Korean characters in the command line window you call "mysql ..." command line client? Please, send the results of:

echo $LANG

from te command prompt, and

SHOW VARIABLES LIKE 'char%';

from mysql command line client. In general, there are no (serious and known) problems with Korean support in MySQL 5.0.x.
[9 Oct 2006 9:25] JongSe Park
echo $LANG
ko_KR.eucKR

mysql> SHOW VARIABLES LIKE 'char%';
+--------------------------+--------------------------------------------------------------+
| Variable_name            | Value                                                        |
+--------------------------+--------------------------------------------------------------+
| character_set_client     | euckr                                                        | 
| character_set_connection | euckr                                                        | 
| character_set_database   | euckr                                                        | 
| character_set_filesystem | euckr                                                        | 
| character_set_results    | euckr                                                        | 
| character_set_server     | euckr                                                        | 
| character_set_system     | utf8                                                         | 
| character_sets_dir       | /usr/local/mysql-pro-5.0.22-linux-i686/share/mysql/charsets/ | 
+--------------------------+--------------------------------------------------------------+
8 rows in set (0.00 sec)
[9 Oct 2006 9:45] Valeriy Kravchuk
Korean should be supported in your case, both by SecureCRT and by MySQL server. Please, send the results of

mysql --version

command.
[9 Oct 2006 23:51] JongSe Park
mysql version : 5.0.22
secureCRT version: 5.1.4
[10 Oct 2006 8:07] Valeriy Kravchuk
Please, send the results of:

mysql --version

command. Just copy and paste. I want to check, if readline library was used to build this binary.
[10 Oct 2006 9:27] JongSe Park
mysql version: 5.0.22
secureCRT version:5.1.4
[10 Oct 2006 9:28] JongSe Park
./mysql --version
./mysql  Ver 14.12 Distrib 5.0.22, for pc-linux-gnu (i686) using  EditLine wrapper
[26 Oct 2006 19:37] Sveta Smirnova
Can you use korean symbols in your Linux/Solaris command prompt?

Please, provide output of `echo 길동` command.
[26 Oct 2006 19:46] Sveta Smirnova
Thank you for the report.

I can not repeat it using current BK sources neither on Linux, nor on Solaris.
[27 Oct 2006 9:08] Sveta Smirnova
Both Linux and Solaris binaries were compiled with EditLine.

Solaris:
$~/mysql-5.0/bin/mysql --version
/export/home/sveta/mysql-5.0/bin/mysql  Ver 14.12 Distrib 5.0.27, for pc-solaris2.10 (i386) using  EditLine wrapper

Linux:
$bin/mysql --version
bin/mysql  Ver 14.12 Distrib 5.0.29, for pc-linux-gnu (i686) using  EditLine wrapper
[30 Oct 2006 4:09] JongSe Park
root@MySQL/>echo '길동'
길동
[12 Jul 2007 10:12] Sveta Smirnova
Thank you for the feedback.

Bug is present in current Enterprise RPM packages.

Workaround: use client from *.tar.gz packages.

Output from Enterprise RPM:

$./usr/bin/mysql --socket=/tmp/mysql_ssmirnova50.sock  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 997
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> select '\352\270\270\353\217\231';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

mysql> \q
Bye

Same from Enterprise *.tar.gz:

$~/mysql-enterprise-gpl-5.0.44-linux-i686/bin/mysql --socket=/tmp/mysql_ssmirnova50.sock  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 995
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.
mysql> select '길동';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

mysql> \q
Bye
[13 Jul 2007 10:30] Sveta Smirnova
If one compiles MySQL using configure `$./configure --with-extra-charsets=complex` she get broken mysql client too:

$./client/.libs/mysql --socket=/tmp/mysql_ssmirnova50.sock -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 172
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> select '\321\202\320\265\321\201\321\202';
+----------+
| тест |
+----------+
| тест | 
+----------+
1 row in set (0.00 sec)

$strings ./client/.libs/mysql | grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline

$strings ./client/.libs/mysql | grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln
[20 Jul 2007 5:36] Sveta Smirnova
Workaround if you met problem same as in previous comment: compile with readline like: ./configure --with-extra-charsets=complex --with-readline
[17 Oct 2007 2:06] JongSe Park
I installed mysql-enterprise-5.0.46-linux-i686.tar.gz
It occured same problem.

Could you tell me how to solve this problem in detail.
Is this still bug?
I want to install mysql with binary files(*.tar.gz).

Thanks
[17 Oct 2007 4:14] duke kim
[root@localhost libedit]# /usr/local/bin/mysql  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.46-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show variables like 'collation_%';
+----------------------+-----------------+
| Variable_name        | Value           |
+----------------------+-----------------+
| collation_connection | euckr_korean_ci | 
| collation_database   | euckr_korean_ci | 
| collation_server     | euckr_korean_ci | 
+----------------------+-----------------+
3 rows in set (0.00 sec)

mysql> show variables like 'character_set%';
+--------------------------+----------------------------------+
| Variable_name            | Value                            |
+--------------------------+----------------------------------+
| character_set_client     | euckr                            | 
| character_set_connection | euckr                            | 
| character_set_database   | euckr                            | 
| character_set_filesystem | binary                           | 
| character_set_results    | euckr                            | 
| character_set_server     | euckr                            | 
| character_set_system     | utf8                             | 
| character_sets_dir       | /usr/local/share/mysql/charsets/ | 
+--------------------------+----------------------------------+
8 rows in set (0.00 sec)

A-Type : This followed your reports.
B-Type : There are my result followed your reports.

A-Type : 
mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.

B-Type : 
mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.

A-Type : 
mysql> select '\352\270\270\353\217\231';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

B-Type : 
mysql> select '\352\270\270\353\217\231';
+--------------------+
| 352270270353217231 |
+--------------------+
| 352270270353217231 | 
+--------------------+
1 row in set (0.00 sec)

A-Type : 
mysql> select '길동';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

B-Type : 
mysql> select '';
+--+
|  |
+--+
|  | 
+--+
1 row in set (0.00 sec)

A-Type :
mysql> select '\321\202\320\265\321\201\321\202';
+----------+
| тест |
+----------+
| тест | 
+----------+
1 row in set (0.00 sec)
B-Type :
mysql> select '\321\202\320\265\321\201\321\202';
+--------------------------+
| 321202320265321201321202 |
+--------------------------+
| 321202320265321201321202 | 
+--------------------------+
1 row in set (0.00 sec)

A-Type :
$strings ./client/.libs/mysql | grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline

$strings ./client/.libs/mysql | grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln

B-Type :
[root@localhost libedit]# strings /usr/local/bin/mysql |grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline
[root@localhost libedit]# strings /usr/local/bin/mysql |grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln

thanks~
[17 Oct 2007 8:10] duke kim
The installation was a success with MySQL community 5.0.45-0 version 
but was failed with MySQL Enterprise 5.0.46 version(commerical).
I don’t see why. 
What reason?

I used the option given below.

shell> ./configure --with-readline 

Thanks~
[21 Feb 2008 2:16] jongse park
This bug wasn't fixed yet.
I think that Commercial version is not good for using korean character.
So almost Korean wouldn't use commercial version. if MySQL AB couldn't fix this bug. 
Please try it again. Binary files (*.tar.gz) don't support to write korean character on mysql client prompt.
[3 Mar 2008 22:34] Chad MILLER
I can't easily fix the mapping of bytes into backslash-octal strings.  That's a bug for another time.  See both of these
  protected void re_fastaddc(EditLine *el)
  private   void re_addc(EditLine *el, int c)
in cmd-line-utils/libedit/refresh.c , where it uses ASCII-centric  isprint()  to decide what to emit.
[4 Mar 2008 12:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43370

ChangeSet@1.2578, 2008-03-04 16:13:08+04:00, bar@mysql.com +5 -0
  Bug#23097 mysql can't insert korean on mysql prompt.
  Problem: libedit is a very pure-ASCII oriented library,
  and it is not aware of extended (0x80..0xFF) or even multi-byte
  characters. It considered such characters as non-printable
  and didn't allow to input them.
  Fix: make libedit think that all bytes >= 0x80 are printable.
[4 Mar 2008 12:41] Sergey Glukhov
ok to push
[5 Mar 2008 14:10] Alexander Barkov
Pushed into 5.0-engines (currently marked as 5.0.58)
[27 Mar 2008 11:20] Bugs System
Pushed into 5.0.60
[27 Mar 2008 11:21] Bugs System
Pushed into 5.1.24-rc
[27 Mar 2008 17:53] Bugs System
Pushed into 6.0.5-alpha
[27 Mar 2008 19:22] Paul Dubois
Noted in 5.0.60, 5.1.24, 6.0.5 changelogs.

For distributions compiled with the bundled libedit library, there
were difficulties using the mysql client to enter input for non-ASCII
or multi-byte characters.
[30 Mar 2008 0:04] Jon Stephens
Fix also noted for 5.1.23-ndb-6.3.11.