Bug #52121 partition by key on utf32 enum field cause debug assertion: (length % 4) == 0
Submitted: 17 Mar 2010 3:09 Modified: 2 Sep 2010 15:58
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Charsets Severity:S1 (Critical)
Version:5.5.6-debug, 5.6.99-m4-debug OS:Any
Assigned to: Alexander Barkov CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[17 Mar 2010 3:09] Shane Bester
Description:
Version: '5.6.99-m4-debug-log'  socket: ''  port: 3306  Source distribution
Assertion failed: (length % 4) == 0, file .\ctype-ucs2.c, line 2465

mysqld.exe!my_sigabrt_handler()[my_thr_init.c:519]
mysqld.exe!raise()[winsig.c:590]
mysqld.exe!abort()[abort.c:71]
mysqld.exe!_wassert()[assert.c:212]
mysqld.exe!my_lengthsp_utf32()[ctype-ucs2.c:2465]
mysqld.exe!my_hash_sort_uca()[ctype-uca.c:7250]
mysqld.exe!my_hash_sort_any_uca()[ctype-uca.c:8073]
mysqld.exe!Field::hash()[field.cc:1376]
mysqld.exe!calculate_key_value()[sql_partition.cc:2726]
mysqld.exe!get_part_id_key()[sql_partition.cc:2837]
mysqld.exe!get_partition_id_key_nosub()[sql_partition.cc:3546]
mysqld.exe!ha_partition::write_row()[ha_partition.cc:3134]
mysqld.exe!handler::ha_write_row()[handler.cc:4670]
mysqld.exe!write_record()[sql_insert.cc:1658]
mysqld.exe!mysql_insert()[sql_insert.cc:864]
mysqld.exe!mysql_execute_command()[sql_parse.cc:3110]
mysqld.exe!mysql_parse()[sql_parse.cc:5735]
mysqld.exe!dispatch_command()[sql_parse.cc:1024]
mysqld.exe!do_command()[sql_parse.cc:710]
mysqld.exe!do_handle_one_connection()[sql_connect.cc:1174]
mysqld.exe!handle_one_connection()[sql_connect.cc:1113]
mysqld.exe!pthread_start()[my_winthread.c:61]
mysqld.exe!_callthreadstartex()[threadex.c:348]
mysqld.exe!_threadstartex()[threadex.c:331]
kernel32.dll!FlsSetValue()

How to repeat:
#on debug server run:

drop table if exists tt;
create table `tt` (`a` enum('a') charset utf32 collate utf32_spanish2_ci) engine=myisam partition by key(`a`) partitions 2;
insert into tt values ('a');
[17 Mar 2010 3:39] Miguel Solorzano
Thank you for the bug report.

InnoDB: Foreign key constraint system tables created
100317  0:34:16 InnoDB Plugin 1.0.6 started; log sequence number 0
100317  0:34:17 [Note] Event Scheduler: Loaded 0 events
100317  0:34:17 [Note] 5.5/libexec/mysqld: ready for connections.
Version: '5.6.99-m4-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
mysqld: ctype-ucs2.c:2465: my_lengthsp_utf32: Assertion `(length % 4) == 0' failed.
100317  0:34:59 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 338447 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x2d61e08
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7ff73815ee88 thread_stack 0x40000
5.5/libexec/mysqld(my_print_stacktrace+0x35)[0xb5dc8d]
5.5/libexec/mysqld(handle_segfault+0x2a7)[0x696c64]
/lib64/libpthread.so.0[0x3d6ec0f0f0]
/lib64/libc.so.6(gsignal+0x35)[0x3d6e4326c5]
/lib64/libc.so.6(abort+0x175)[0x3d6e433ea5]
/lib64/libc.so.6(__assert_fail+0xf5)[0x3d6e42b7b5]
5.5/libexec/mysqld[0xb8dc6a]
5.5/libexec/mysqld[0xb90317]
5.5/libexec/mysqld[0xb91b26]
5.5/libexec/mysqld(_ZN5Field4hashEPmS0_+0xb6)[0x654eca]
5.5/libexec/mysqld[0x5a7037]
5.5/libexec/mysqld[0x5a7204]
5.5/libexec/mysqld(_Z26get_partition_id_key_nosubP14partition_infoPjPx+0x37)[0x5a83b8]
5.5/libexec/mysqld(_ZN12ha_partition9write_rowEPh+0x1ae)[0x7ffed0]
5.5/libexec/mysqld(_ZN7handler12ha_write_rowEPh+0x5f)[0x7f7245]
5.5/libexec/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0xadd)[0x74d8bb]
5.5/libexec/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xba9)[0x74ba81]
5.5/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x2f58)[0x6aad82]
5.5/libexec/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x280)[0x6b216f]
5.5/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xb08)[0x6a5ff4]
5.5/libexec/mysqld(_Z10do_commandP3THD+0x252)[0x6a52d7]
5.5/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x14d)[0x6a358e]
5.5/libexec/mysqld(handle_one_connection+0x33)[0x6a343a]
/lib64/libpthread.so.0[0x3d6ec06a3a]
/lib64/libc.so.6(clone+0x6d)[0x3d6e4de67d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x7ff714004b98 is an invalid pointer
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
[miguel@hegel dbs]$ 

[miguel@hegel ~]$ dbs/5.5/bin/mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.99-m4-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> drop table if exists tt;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> create table `tt` (`a` enum('a') charset utf32 collate utf32_spanish2_ci) engine=myisam
    -> partition by key(`a`) partitions 2;
Query OK, 0 rows affected (0.07 sec)

mysql> insert into tt values ('a');
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
[23 Mar 2010 11:30] Valeriy Kravchuk
I've got the same assertion with current mysql-next-mr tree:

openxs@ubuntu:/home2/openxs/dbs/next-mr$ bin/mysql --no-defaults -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.99-m4-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> drop table if exists tt;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> create table `tt` (`a` enum('a') charset utf32 collate utf32_spanish2_ci) engine=myisam
    -> partition by key(`a`) partitions 2;
Query OK, 0 rows affected (0.00 sec)

mysql> insert into tt values ('a');
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> 100323 13:27:43 mysqld_safe Number of processes running now: 0
100323 13:27:43 mysqld_safe mysqld restarted

mysql> exit
Bye
openxs@ubuntu:/home2/openxs/dbs/next-mr$ tail -100 var/ubuntu.err
...
100323 13:25:48 mysqld_safe Starting mysqld daemon with databases from /home2/openxs/dbs/next-mr/var
100323 13:25:50 [Note] Buffered information: Performance schema disabled (reason: start parameters).

100323 13:25:50 [Note] Plugin 'FEDERATED' is disabled.
100323 13:25:50 [Note] Plugin 'ndbcluster' is disabled.
InnoDB: The InnoDB memory heap is disabled
InnoDB: Mutexes and rw_locks use GCC atomic builtins
100323 13:25:50  InnoDB: highest supported file format is Barracuda.
100323 13:25:50 InnoDB Plugin 1.0.6 started; log sequence number 35777658
100323 13:25:51 [Note] Event Scheduler: Loaded 0 events
100323 13:25:51 [Note] /home2/openxs/dbs/next-mr/libexec/mysqld: ready for connections.
Version: '5.6.99-m4-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
100323 13:26:38  InnoDB: mbminlen=4, mbmaxlen=4, type->mbminlen=0, type->mbmaxlen=4
mysqld: ctype-ucs2.c:2465: my_lengthsp_utf32: Assertion `(length % 4) == 0' failed.
100323 13:27:43 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337841 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x93ec1a0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xa88483b0 thread_stack 0x30000
/home2/openxs/dbs/next-mr/libexec/mysqld(my_print_stacktrace+0x26)[0x877fd51]
/home2/openxs/dbs/next-mr/libexec/mysqld(handle_segfault+0x2ee)[0x82c7466]
[0xb778d420]
/lib/tls/i686/cmov/libc.so.6(abort+0x101)[0xb75d0a01]
/lib/tls/i686/cmov/libc.so.6(__assert_fail+0xee)[0xb75c810e]
/home2/openxs/dbs/next-mr/libexec/mysqld[0x87ae902]
/home2/openxs/dbs/next-mr/libexec/mysqld[0x87b09e5]
/home2/openxs/dbs/next-mr/libexec/mysqld[0x87b20af]
/home2/openxs/dbs/next-mr/libexec/mysqld(_ZN5Field4hashEPmS0_+0x91)[0x828654f]
/home2/openxs/dbs/next-mr/libexec/mysqld[0x81dfbf2]
/home2/openxs/dbs/next-mr/libexec/mysqld[0x81e0ec0]
/home2/openxs/dbs/next-mr/libexec/mysqld(_Z26get_partition_id_key_nosubP14partition_infoPjPx+0x1d)[0x81e0f53]
/home2/openxs/dbs/next-mr/libexec/mysqld(_ZN12ha_partition9write_rowEPh+0x183)[0x8427adf]
...
[24 Jun 2010 8:32] Shane Bester
repeatable with 5.5.6-debug:
bzr+ssh://XXX@bk-internal.mysql.com/bzrroot/server/mysql-trunk/
Now on revision 3072
[5 Aug 2010 8:19] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/115060

3187 Alexander Barkov	2010-08-05
      Bug#52121 partition by key on utf32 enum field cause debug assertion: (length % 4) == 0
      
      Problem: ENUM columns are sorted and distributed according to their
      numeric value, but Field::hash() incorrectly passed string character set
      (utf32) in combination with numeric value to the hash function,
      which made assertion fail.
      
      Fix: pass "binary" character set in combination with numeric value
      to the hash function.
      
        mysql-test/suite/parts/r/part_ctype_utf32.result
        Adding tests
      
        mysql-test/suite/parts/t/part_ctype_utf32.test
        Adding test
      
        sql/field.cc
        Pass correct character set pointer to the hash function.
[7 Aug 2010 3:38] Shane Bester
i hope mysql_upgrade (check table .. for upgrade) will indicate that table needs repairing so the situation is handled..
[12 Aug 2010 7:27] Alexander Barkov
Upgrade is needed for partitions on ENUM columns with the following collation:

- collations cp1250_czech_cs, latin2_czech_cs

- utf8_unicode_ci, ucs2_unicode_ci, utfmb4_unicode_ci, utf32_unicode_ci,
  utf16_unicode_ci and all other UCA collations.

- collations latin7_general_ci, latin7_general_cs
  with more than 31 elements (0x1F hex)

- any other case insensitive collations having more than 64 elements (0x40 hex).
[23 Aug 2010 10:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/116483

3192 Alexander Barkov	2010-08-23
      Bug#52121 partition by key on utf32 enum field cause debug assertion: (length % 4) == 0
            
      Problem: ENUM columns are sorted and distributed according to their
      numeric value, but Field::hash() incorrectly passed string character set
      (utf32) in combination with numeric value to the hash function,
      which made assertion fail.
      
      Fix: pass "binary" character set in combination with numeric value
      to the hash function.
      
        mysql-test/suite/parts/r/part_ctype_utf32.result
        Adding tests
      
        mysql-test/suite/parts/t/part_ctype_utf32.test
        Adding test
      
        sql/field.cc
        Pass correct character set pointer to the hash function.
[23 Aug 2010 11:11] Alexander Barkov
Pushed into mysql-5.5-bugfixing (5.5.6-m3)
[25 Aug 2010 9:21] Bugs System
Pushed into mysql-5.5 5.5.6-m3 (revid:alik@ibmvm-20100825092002-2yvkb3iwu43ycpnm) (version source revid:alik@ibmvm-20100825092002-2yvkb3iwu43ycpnm) (merge vers: 5.5.6-m3) (pib:20)
[30 Aug 2010 8:29] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@sun.com-20100830082732-n2eyijnv86exc5ci) (version source revid:alik@sun.com-20100830082732-n2eyijnv86exc5ci) (merge vers: 5.6.1-m4) (pib:21)
[30 Aug 2010 8:32] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100830082745-n6sh01wlwh3itasv) (version source revid:alik@sun.com-20100830082745-n6sh01wlwh3itasv) (pib:21)
[2 Sep 2010 0:47] Paul Dubois
Noted in 5.5.6, 5.6.1 changelogs.

PARTITION BY KEY on a utf32 ENUM column raised a debugging assertion.
[30 Jan 2013 17:45] Paul Dubois
Noted in 5.1.69 changelog.