Bug #29333 myisam corruption with character set cp932 collate cp932_japanese_ci
Submitted: 25 Jun 2007 10:52 Modified: 23 Jul 2007 1:36
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Charsets Severity:S1 (Critical)
Version:5.0.44, 5.1.20 OS:Any
Assigned to: Alexander Barkov
Tags: corruption, myisam

[25 Jun 2007 10:52] Shane Bester
Description:
inserts into a table defined as:

CREATE TABLE `t1` (   `c1` mediumtext character set cp932 collate cp932_japanese_ci,   KEY `c1` (`c1`(120)) ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

results in:

mysqld-debug: Table '.\test\t1' is marked as crashed and should be repaired

This bug looks alot like bug #29261 but it a different charset, so I'm opening a new bug report.

How to repeat:
import attached file into mysql

Suggested fix:
.
[25 Jun 2007 10:54] Shane Bester
mysql -uroot test <bug29333.sql

Attachment: bug29333.sql.bz2 (application/octet-stream, text), 104.18 KiB.

[30 Jun 2007 14:19] Shane Bester
uploading a smaller, simpler testcase:

mysql> create table tbl_121 (a varchar(2) character set cp932 collate cp932_japanese_ci,key(a))engine=myisam;
Query OK, 0 rows affected (0.08 sec)

mysql> load data local infile '/tmp/dump.txt' replace into table tbl_121 (@h) set a=UNHEX(@h);
Query OK, 65536 rows affected, 17775 warnings (0.72 sec)
Records: 65536  Deleted: 0  Skipped: 0  Warnings: 0

mysql> check table tbl_121 extended;
+-----------+-------+----------+-----------------------------------------------+
| Table     | Op    | Msg_type | Msg_text                                      |
+-----------+-------+----------+-----------------------------------------------+
| f.tbl_121 | check | error    | Record at: 1320  Can't find key for index:  1 |
| f.tbl_121 | check | error    | Corrupt                                       |
+-----------+-------+----------+-----------------------------------------------+
2 rows in set (0.06 sec)
[30 Jun 2007 14:20] Shane Bester
unzip and load data infile into the above mentioned table.

Attachment: dump.txt.bz2 (application/octet-stream, text), 40.60 KiB.

[3 Jul 2007 10:55] Alexander Barkov
A simplier test case demonstrating the same problem:

drop table if exists t1, t2;
create table t2 (a char(1));
insert into t2 values ('0'),('1'),('2'),('3'),('4'),('5'),('6'),('7');
insert into t2 values ('8'),('9'),('A'),('B'),('C'),('D'),('E'),('F');
create table t1 (
  a varchar(2) character set cp932
) engine=myisam;
insert into t1
select unhex(concat(t24.a, t23.a, t22.a, t21.a))
from t2 t21, t2 t22, t2 t23, t2 t24;
delete from t1 where a='';
--select hex(a) from t1;
alter table t1 add key(a);
check table t1;
[3 Jul 2007 11:49] Alexander Barkov
The problem appear to be in wrong comparison with trailing space.
This problem was fixed for all other character sets under terms of
bug#7788 "Table is full" occurs during a multitable update").

ctype-cp932.c was not fixed.
[3 Jul 2007 12:04] Alexander Barkov
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

http://lists.mysql.com/commits/30163

ChangeSet@1.2517, 2007-07-03 16:59:54+05:00, bar@mysql.com +3 -0
  Bug#29333 myisam corruption with character set cp932 collate cp932_japanese_ci
  Problem: wrong comparison with trailing space.
  This problem was fixed for all other character sets under terms of
  bug 7788 ""Table is full" occurs during a multitable update".
  ctype-cp932.c was forgotten.
  Fix: applying the same fix for ctype-cp932.c.
[4 Jul 2007 7:49] Alexander Barkov
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

http://lists.mysql.com/commits/30266

ChangeSet@1.2517, 2007-07-04 12:47:48+05:00, bar@mysql.com +3 -0
  Bug#29333 myisam corruption with character set cp932 collate cp932_japanese_ci
  Problem: wrong comparison with trailing space.
  This problem was fixed for all other character sets under terms of
  bug 7788 ""Table is full" occurs during a multitable update".
  ctype-cp932.c was forgotten.
  Fix: applying the same fix for ctype-cp932.c.
  (see ctype-sjis.c as an example of a previously correctly fixed file)
[4 Jul 2007 11:18] Alexander Barkov
New patch version is here:

http://lists.mysql.com/commits/30285
[5 Jul 2007 10:06] Alexander Barkov
Pushed into 5.0.46-rpl
Pushed into 5.1.21-rpl
[20 Jul 2007 23:46] Bugs System
Pushed into 5.1.21-beta
[20 Jul 2007 23:49] Bugs System
Pushed into 5.0.48
[23 Jul 2007 1:36] Paul Dubois
Noted in 5.0.48, 5.1.21 changelogs.

MyISAM corruption could occur with the cp932_japanese_ci collation
for the cp932 character set due to incorrect comparison for trailing
space.