MySQL Bugs: #23097: mysql can't insert korean on mysql prompt.

Bug #23097	mysql can't insert korean on mysql prompt.
Submitted:	9 Oct 2006 5:31	Modified:	27 Mar 2008 19:22
Reporter:	JongSe Park	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Command-line Clients	Severity:	S3 (Non-critical)
Version:	5.0.22, 5.0.44 RPM	OS:	Linux (linux and solaris)
Assigned to:	Alexander Barkov	CPU Architecture:	Any
Tags:	backslash, bfsm_2007_10_25, bfsm_2007_11_01, libedit, readline

Description:
I install mysql 5.0.22(binary) on linux and solaris.
but, I can't insert korean on mysql prompt.

I use secureCRT tool to connect mysql server.
and I log in mysql server. but I can't insert any korean. like this;

mysql> select * from address where name = '';

I can't insert korean like this "select * from address where name = '길동'".
Is this a bug of mysql?
or what do I have to do to solve this problem? 

How to repeat:
I install mysql 5.0.22(binary) on linux or solaris.
I use secureCRT tool to connect mysql server.
and I log in mysql server. but I can't insert any korean. like this;

mysql> select * from address where name = '';

I can't insert korean like this "select * from address where name = '길동'".

Thank you for a problem report. Can you insert Korean characters in the command line window you call "mysql ..." command line client? Please, send the results of:

echo $LANG

from te command prompt, and

SHOW VARIABLES LIKE 'char%';

from mysql command line client. In general, there are no (serious and known) problems with Korean support in MySQL 5.0.x.

echo $LANG
ko_KR.eucKR

mysql> SHOW VARIABLES LIKE 'char%';
+--------------------------+--------------------------------------------------------------+
| Variable_name            | Value                                                        |
+--------------------------+--------------------------------------------------------------+
| character_set_client     | euckr                                                        | 
| character_set_connection | euckr                                                        | 
| character_set_database   | euckr                                                        | 
| character_set_filesystem | euckr                                                        | 
| character_set_results    | euckr                                                        | 
| character_set_server     | euckr                                                        | 
| character_set_system     | utf8                                                         | 
| character_sets_dir       | /usr/local/mysql-pro-5.0.22-linux-i686/share/mysql/charsets/ | 
+--------------------------+--------------------------------------------------------------+
8 rows in set (0.00 sec)

Korean should be supported in your case, both by SecureCRT and by MySQL server. Please, send the results of

mysql --version

command.

mysql version : 5.0.22
secureCRT version: 5.1.4

Please, send the results of:

mysql --version

command. Just copy and paste. I want to check, if readline library was used to build this binary.

mysql version: 5.0.22
secureCRT version:5.1.4

./mysql --version
./mysql  Ver 14.12 Distrib 5.0.22, for pc-linux-gnu (i686) using  EditLine wrapper

Can you use korean symbols in your Linux/Solaris command prompt?

Please, provide output of `echo 길동` command.

Thank you for the report.

I can not repeat it using current BK sources neither on Linux, nor on Solaris.

Both Linux and Solaris binaries were compiled with EditLine.

Solaris:
$~/mysql-5.0/bin/mysql --version
/export/home/sveta/mysql-5.0/bin/mysql  Ver 14.12 Distrib 5.0.27, for pc-solaris2.10 (i386) using  EditLine wrapper

Linux:
$bin/mysql --version
bin/mysql  Ver 14.12 Distrib 5.0.29, for pc-linux-gnu (i686) using  EditLine wrapper

root@MySQL/>echo '길동'
길동

Thank you for the feedback.

Bug is present in current Enterprise RPM packages.

Workaround: use client from *.tar.gz packages.

Output from Enterprise RPM:

$./usr/bin/mysql --socket=/tmp/mysql_ssmirnova50.sock  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 997
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> select '\352\270\270\353\217\231';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

mysql> \q
Bye

Same from Enterprise *.tar.gz:

$~/mysql-enterprise-gpl-5.0.44-linux-i686/bin/mysql --socket=/tmp/mysql_ssmirnova50.sock  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 995
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.
mysql> select '길동';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

mysql> \q
Bye

If one compiles MySQL using configure `$./configure --with-extra-charsets=complex` she get broken mysql client too:

$./client/.libs/mysql --socket=/tmp/mysql_ssmirnova50.sock -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 172
Server version: 5.0.46-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> select '\321\202\320\265\321\201\321\202';
+----------+
| тест |
+----------+
| тест | 
+----------+
1 row in set (0.00 sec)

$strings ./client/.libs/mysql | grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline

$strings ./client/.libs/mysql | grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln

Workaround if you met problem same as in previous comment: compile with readline like: ./configure --with-extra-charsets=complex --with-readline

I installed mysql-enterprise-5.0.46-linux-i686.tar.gz
It occured same problem.

Could you tell me how to solve this problem in detail.
Is this still bug?
I want to install mysql with binary files(*.tar.gz).

Thanks

[root@localhost libedit]# /usr/local/bin/mysql  -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.46-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show variables like 'collation_%';
+----------------------+-----------------+
| Variable_name        | Value           |
+----------------------+-----------------+
| collation_connection | euckr_korean_ci | 
| collation_database   | euckr_korean_ci | 
| collation_server     | euckr_korean_ci | 
+----------------------+-----------------+
3 rows in set (0.00 sec)

mysql> show variables like 'character_set%';
+--------------------------+----------------------------------+
| Variable_name            | Value                            |
+--------------------------+----------------------------------+
| character_set_client     | euckr                            | 
| character_set_connection | euckr                            | 
| character_set_database   | euckr                            | 
| character_set_filesystem | binary                           | 
| character_set_results    | euckr                            | 
| character_set_server     | euckr                            | 
| character_set_system     | utf8                             | 
| character_sets_dir       | /usr/local/share/mysql/charsets/ | 
+--------------------------+----------------------------------+
8 rows in set (0.00 sec)

A-Type : This followed your reports.
B-Type : There are my result followed your reports.

A-Type : 
mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.

B-Type : 
mysql> fname\040varchar(40)\040null\c
ERROR: 
Unknown command '\0'.
ERROR: 
Unknown command '\0'.

A-Type : 
mysql> select '\352\270\270\353\217\231';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

B-Type : 
mysql> select '\352\270\270\353\217\231';
+--------------------+
| 352270270353217231 |
+--------------------+
| 352270270353217231 | 
+--------------------+
1 row in set (0.00 sec)

A-Type : 
mysql> select '길동';
+--------+
| 길동 |
+--------+
| 길동 | 
+--------+
1 row in set (0.00 sec)

B-Type : 
mysql> select '';
+--+
|  |
+--+
|  | 
+--+
1 row in set (0.00 sec)

A-Type :
mysql> select '\321\202\320\265\321\201\321\202';
+----------+
| тест |
+----------+
| тест | 
+----------+
1 row in set (0.00 sec)
B-Type :
mysql> select '\321\202\320\265\321\201\321\202';
+--------------------------+
| 321202320265321201321202 |
+--------------------------+
| 321202320265321201321202 | 
+--------------------------+
1 row in set (0.00 sec)

A-Type :
$strings ./client/.libs/mysql | grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline

$strings ./client/.libs/mysql | grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln

B-Type :
[root@localhost libedit]# strings /usr/local/bin/mysql |grep readline
readline
_Z14batch_readlineP14st_line_buffer
rl_readline_name
_Z19batch_readline_initmP8_IO_FILE
_Z18batch_readline_endP14st_line_buffer
_Z22batch_readline_commandP14st_line_bufferPc
Can't initialize batch_readline
[root@localhost libedit]# strings /usr/local/bin/mysql |grep libedit
libedit_strlcpy
libedit_strlcat
libedit_fgetln

thanks~

The installation was a success with MySQL community 5.0.45-0 version 
but was failed with MySQL Enterprise 5.0.46 version(commerical).
I don’t see why. 
What reason?

I used the option given below.

shell> ./configure --with-readline 

Thanks~

This bug wasn't fixed yet.
I think that Commercial version is not good for using korean character.
So almost Korean wouldn't use commercial version. if MySQL AB couldn't fix this bug. 
Please try it again. Binary files (*.tar.gz) don't support to write korean character on mysql client prompt.

I can't easily fix the mapping of bytes into backslash-octal strings.  That's a bug for another time.  See both of these
  protected void re_fastaddc(EditLine *el)
  private   void re_addc(EditLine *el, int c)
in cmd-line-utils/libedit/refresh.c , where it uses ASCII-centric  isprint()  to decide what to emit.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43370

ChangeSet@1.2578, 2008-03-04 16:13:08+04:00, bar@mysql.com +5 -0
  Bug#23097 mysql can't insert korean on mysql prompt.
  Problem: libedit is a very pure-ASCII oriented library,
  and it is not aware of extended (0x80..0xFF) or even multi-byte
  characters. It considered such characters as non-printable
  and didn't allow to input them.
  Fix: make libedit think that all bytes >= 0x80 are printable.

ok to push

Pushed into 5.0-engines (currently marked as 5.0.58)

Pushed into 5.0.60

Pushed into 5.1.24-rc

Pushed into 6.0.5-alpha

Noted in 5.0.60, 5.1.24, 6.0.5 changelogs.

For distributions compiled with the bundled libedit library, there
were difficulties using the mysql client to enter input for non-ASCII
or multi-byte characters.

Fix also noted for 5.1.23-ndb-6.3.11.