Bug #45012 my_like_range_cp932 generates invalid string
Submitted: 21 May 2009 17:06 Modified: 14 Oct 2010 13:30
Reporter: Sergei Golubchik Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Charsets Severity:S2 (Serious)
Version:4.1, 5.0, 5,1, 6.0 bzr OS:Any
Assigned to: Alexander Barkov CPU Architecture:Any

[21 May 2009 17:06] Sergei Golubchik
Description:
my_like_range_cp932() uses 0xFF as a max_sort_char, but it's not a valid cp932 character. As a result my_like_range_cp932 generates invalid cp932 string, which is then passed down to a storage engine in records_in_range().

In particular, DB2 engine doesn't like invalid strings.
See http://lists.mysql.com/internals/36709

How to repeat:
.
[22 May 2009 8:53] Sveta Smirnova
Thank you for the report.

Verified as described.
[22 May 2009 14:00] Tim Clark
Additionally, the following character sets appear to have a similar problem:
big5
euckr
gb2312
gbk
sjis
tis620
ujis
[22 May 2009 20:19] Tim Clark
I wanted to note that this problem is visible with MyISAM also, via a warning:

>CREATE TABLE t1 (c char(10), v varchar(20), index(c), index(v)) collate big5_chinese_ci engine=myisam;
>insert into t1 values ("abc","def"),("abcd", "def"),("abcde","defg"),("aaaa","bbbb");
>select * from t1 force index(v) where v like "x%";
Empty set, 1 warning (0.02 sec)
>show warnings;
+---------+------+-------------------------------------------------------------------------------+
| Level   | Code | Message                                                                       |
+---------+------+-------------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xFF\xFF\xFF\xFF\xFF\xFF...' for column 'v' at row 1 |
+---------+------+-------------------------------------------------------------------------------+
[1 Jun 2009 21:10] Tim Clark
Not sure whether this should be put in a different bug report, but I believe that the max_sort_char (0xFFFF) for the ucs2_* collations is also invalid according to the Unicode specification (max valid is 0xFFFD).
[21 Jul 2010 13:58] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114042

3497 Alexander Barkov	2010-07-21
      Bug#45012 my_like_range_cp932 generates invalid string
      
      Problem: The functions my_like_range_xxx() returned
      a badly formed maximum strings for Asian character sets,
      which made problems for storage engines.
      
      Fix: 
      - Remove a number my_like_range_xxx() implementations,
        which were in fact dumplicate code.
      - Using generic my_like_range_mb() instead
      - Setting max_sort_char member properly
      
      Notes:
      
      - No additional tests available.
        Old tests covers the affected code well enough.
      
      - Also, it's not possible to demonstrate in tests
        that the problem has really disappeared, because
        my_like_range() results cannot be exposed to SQL level.
        This has been checked manually in gdb:
        my_like_range() now returns well-formed strings for all
        Asian character sets.
[22 Jul 2010 11:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114129

3497 Alexander Barkov	2010-07-22
      Bug#45012 my_like_range_cp932 generates invalid string
      
      Problem: The functions my_like_range_xxx() returned
      badly formed maximum strings for Asian character sets,
      which made problems for storage engines.
      
      Fix: 
      - Removed a number my_like_range_xxx() implementations,
        which were in fact dumplicate code pieces.
      - Using generic my_like_range_mb() instead.
      - Setting max_sort_char member properly for Asian character sets
      - Adding unittest/strings/strings-t.c, 
        to test that my_like_range_xxx() return well-formed 
        min and max strings.
      
      Notes:
      
      - No additional tests in mysql/t/ available.
        Old tests cover the affected code well enough.
[26 Jul 2010 5:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114311

3456 Alexander Barkov	2010-07-26
      Bug#45012 my_like_range_cp932 generates invalid string
      
      Problem: The functions my_like_range_xxx() returned
      badly formed maximum strings for Asian character sets,
      which made problems for storage engines.
      
      Fix: 
      - Removed a number my_like_range_xxx() implementations,
        which were in fact dumplicate code pieces.
      - Using generic my_like_range_mb() instead.
      - Setting max_sort_char member properly for Asian character sets
      - Adding unittest/strings/strings-t.c, 
        to test that my_like_range_xxx() return well-formed 
        min and max strings.
      
      Notes:
      
      - No additional tests in mysql/t/ available.
        Old tests cover the affected code well enough.
[26 Jul 2010 7:02] Alexander Barkov
Pushed into mysql-5.1-bugteam (5.1.50)
Pushed into mysql-trunk-merge (5.5.6)
[29 Jul 2010 6:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/114583

3460 Alexander Barkov	2010-07-29
      Postfix for BUG#45012.
      
      Problem: The original patch didn't compile on debug_werror
      due to wrong format in printf("%d") for size_t variables.
      
      Fix: Adding cast to (int).
[3 Aug 2010 17:45] Paul DuBois
Noted in 5.1.50 changelog.

The my_like_range_xxx() functions returned badly formed maximum
strings for Asian character sets, which caused problems for storage
engines. 

Setting report to Need Merge pending further pushes.
[4 Aug 2010 7:52] Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100731131027-1n61gseejyxsqk5d) (version source revid:alik@sun.com-20100731074942-o840woifuqioxxe4) (merge vers: 5.5.6-m3) (pib:18)
[4 Aug 2010 8:08] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (merge vers: 5.6.99-m4) (pib:18)
[4 Aug 2010 8:24] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (merge vers: 5.6.99-m4) (pib:18)
[4 Aug 2010 9:03] Bugs System
Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (pib:20)
[4 Aug 2010 22:39] Paul DuBois
Noted in 5.5.6 changelog.
For 5.6, bug does not appear in any released 5.6.x version.
[19 Aug 2010 15:41] Bugs System
Pushed into mysql-5.1 5.1.51 (revid:build@mysql.com-20100819151858-muaaor6jojb5ouzj) (version source revid:build@mysql.com-20100819151858-muaaor6jojb5ouzj) (merge vers: 5.1.51) (pib:20)
[28 Sep 2010 11:53] Alexander Barkov
See also Bug#50677
[14 Oct 2010 8:34] Bugs System
Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:martin.skold@mysql.com-20101014082627-jrmy9xbfbtrebw3c) (version source revid:martin.skold@mysql.com-20101014082627-jrmy9xbfbtrebw3c) (merge vers: 5.1.51-ndb-7.0.20) (pib:21)
[14 Oct 2010 8:49] Bugs System
Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.39 (revid:martin.skold@mysql.com-20101014083757-5qo48b86d69zjvzj) (version source revid:martin.skold@mysql.com-20101014083757-5qo48b86d69zjvzj) (merge vers: 5.1.51-ndb-6.3.39) (pib:21)
[14 Oct 2010 9:04] Bugs System
Pushed into mysql-5.1-telco-6.2 5.1.51-ndb-6.2.19 (revid:martin.skold@mysql.com-20101014084420-y54ecj85j5we27oa) (version source revid:martin.skold@mysql.com-20101014084420-y54ecj85j5we27oa) (merge vers: 5.1.51-ndb-6.2.19) (pib:21)
[14 Oct 2010 13:30] Jon Stephens
Already documented in the 5.1.50 changelog; no additional changelog entries required. Set back to Closed state.