| Bug #44811 | Tests with utf8 charset fail with ibmdb2i on 64bit MySQL | ||
|---|---|---|---|
| Submitted: | 12 May 2009 4:59 | Modified: | 14 Jul 2009 15:20 |
| Reporter: | Nidhi Shrotriya | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: DB2SE for IBM i | Severity: | S3 (Non-critical) |
| Version: | 5.1.34 | OS: | IBM i (5.4 & 6.1) |
| Assigned to: | CPU Architecture: | Any | |
[12 May 2009 5:00]
Nidhi Shrotriya
Sql file for ctype_utf8.test
Attachment: ctype_utf8-2.sql (text/x-sql), 654 bytes.
[12 May 2009 5:01]
Nidhi Shrotriya
Sql file for ibmdb2i_rir.test
Attachment: ibmdb2i_rir_1.sql (text/x-sql), 474 bytes.
[29 May 2009 9:34]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/75223 2918 Narayanan V 2009-05-29 Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL wmemset was being used to fill the row buffers. wmemset was intended to fill the buffer with 16-bit UCS2 pad values. However, the 64-bit version of wmemset uses 32-bit wide characters and thus filled the buffer incorrectly. In some cases, the null byte map would be overwritten, causing ctype_utf8.test and ibmdb2i_rir.test to fail, giving the error message CPF5035. This patch eliminates the use of wmemset to fill the row buffer. wmemset has been replaced with memset16, which always fills memory with 16-bit values. @ storage/ibmdb2i/db2i_conversion.cc Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL Eliminate the use of wmemset to fill the row buffer. Replace wmemset with memset16, which always fills memory with 16-bit values. @ storage/ibmdb2i/db2i_misc.h Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL Eliminate the use of wmemset to fill the row buffer. Replace wmemset with memset16, which always fills memory with 16-bit values.
[29 May 2009 10:01]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/75224 2918 Narayanan V 2009-05-29 Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL wmemset was being used to fill the row buffers. wmemset was intended to fill the buffer with 16-bit UCS2 pad values. However, the 64-bit version of wmemset uses 32-bit wide characters and thus filled the buffer incorrectly. In some cases, the null byte map would be overwritten, causing ctype_utf8.test and ibmdb2i_rir.test to fail, giving the error message CPF5035. This patch eliminates the use of wmemset to fill the row buffer. wmemset has been replaced with memset16, which always fills memory with 16-bit values. @ storage/ibmdb2i/db2i_conversion.cc Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL wmemset has been replaced with memset16, which always fills memory with 16-bit values. @ storage/ibmdb2i/db2i_misc.h Bug#44811 Tests with utf8 charset fail with ibmdb2i on 64bit MySQL wmemset has been replaced with memset16, which always fills memory with 16-bit values.
[16 Jun 2009 11:03]
Bugs System
Pushed into 5.1.36 (revid:joro@sun.com-20090616102155-3zhezogudt4uxdyn) (version source revid:azundris@mysql.com-20090529164935-xe3dceff53d7pywb) (merge vers: 5.1.36) (pib:6)
[17 Jun 2009 19:28]
Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090616183122-chjzbaa30qopdra9) (version source revid:azundris@mysql.com-20090529170733-wxq9j0idmss9rllz) (merge vers: 6.0.12-alpha) (pib:11)
[14 Jul 2009 15:20]
MC Brown
A note has been added to the 5.1.36 and 5.4.4 changelog: Running queries on tables with the IBMDB2I storage engine using the utf8 character would fail when using the 64-bit version of MySQL
[12 Aug 2009 22:26]
Paul DuBois
Noted in 5.4.2 changelog because next 5.4 version will be 5.4.2 and not 5.4.4.
[15 Aug 2009 1:41]
Paul DuBois
Ignore previous comment about 5.4.2.
[26 Aug 2009 13:46]
Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[26 Aug 2009 13:46]
Bugs System
Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)
[26 Aug 2009 13:48]
Bugs System
Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)
[27 Aug 2009 16:33]
Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[7 Oct 2009 19:14]
Paul DuBois
The 5.4 fix has been pushed to 5.4.2.
Description: 1) ctype_utf8.test(test modified by IBM) fails with mysqltest: In included file "/home/nidhi/mysql-5.1.33-i5os-power-64bit/mysql-test/t/ctype_utf8-2.sql": At line 3: query 'insert into t1 select repeat('�',87)' failed: 1296: Got error 2021 'See message CPF5035 in joblog for job 177510/QUSER/QSQSRVR.' from IBMDB2I This happens only with 64-bit build of MySQL when using ibmdb2i. Looks like a storage space issue as I tried to do the above repeat() with different nos. and it passes with insert into t1 select repeat('�',86); fails with insert into t1 select repeat('�',87); with the error CPF5035 as below. mysql> source mysql-test/t/ctype_utf8-2.sql; Query OK, 0 rows affected (0.07 sec) Query OK, 1 row affected (0.01 sec) Records: 1 Duplicates: 0 Warnings: 0 Query OK, 1 row affected (0.00 sec) Records: 1 Duplicates: 0 Warnings: 0 +------------+-----------------+ | length(s1) | char_length(s1) | +------------+-----------------+ | 100 | 100 | | 172 | 86 | +------------+-----------------+ 2 rows in set (0.00 sec) Query OK, 0 rows affected (0.03 sec) mysql> source mysql-test/t/ctype_utf8-2.sql; Query OK, 0 rows affected (0.06 sec) Query OK, 1 row affected (0.01 sec) Records: 1 Duplicates: 0 Warnings: 0 ERROR 1296 (HY000): Got error 2021 'See message CPF5035 in joblog for job 175060/QUSER/QSQSRVR.' from IBMDB2I +------------+-----------------+ | length(s1) | char_length(s1) | +------------+-----------------+ | 100 | 100 | +------------+-----------------+ 1 row in set (0.00 sec) Query OK, 0 rows affected (0.03 sec) If character taking 3 bytes may be 86*3=258 > 256+1....something to do with that ?? 2) ibmdb2i_rir.test (test added by IBM) fails as below mysqltest: At line 429: query 'insert into t1 values ('さしすせそかきくけこあいうえお')' failed: 1296: Got error 2021 'See message CPF5035 in joblog for job 177510/QUSER/QSQSRVR.' from IBMDB2I Passes with MyISAM/Innodb. How to repeat: Attaching .sql files to reproduce with ibmdb2i and 64bit build of mysql.