Bug #50542 5.5.x doesn't check length of key prefixes: corruption and crash results
Submitted: 22 Jan 2010 12:40 Modified: 15 Mar 2010 15:48
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: DDL Severity:S1 (Critical)
Version:5.5.2-m2, 5.5.99-m3 OS:Any
Assigned to: Magne Mæhre CPU Architecture:Any
Tags: regression, valgrind
Triage: Triaged: D1 (Critical)

[22 Jan 2010 12:40] Shane Bester
Description:
5.5.x doesn't check that the length of the index prefix is less than or equal to the length of the column.  

innodb crashes:

mysqld-debug.exe!row_sel_convert_mysql_key_to_innobase()[row0sel.c:2331]
mysqld-debug.exe!ha_innobase::index_read()[ha_innodb.cc:5101]
mysqld-debug.exe!ha_innobase::rnd_pos()[ha_innodb.cc:5566]
mysqld-debug.exe!rr_from_tempfile()[records.cc:413]
mysqld-debug.exe!mysql_update()[sql_update.cc:582]
mysqld-debug.exe!mysql_execute_command()[sql_parse.cc:3176]
mysqld-debug.exe!mysql_parse()[sql_parse.cc:6211]
mysqld-debug.exe!dispatch_command()[sql_parse.cc:1240]
mysqld-debug.exe!do_command()[sql_parse.cc:873]
mysqld-debug.exe!handle_one_connection()[sql_connect.cc:1154]
mysqld-debug.exe!pthread_start()[my_winthread.c:61]
mysqld-debug.exe!_callthreadstartex()[threadex.c:348]
mysqld-debug.exe!_threadstartex()[threadex.c:331]
kernel32.dll!FlsSetValue()

innodb might also catch the error and print:
100122 13:43:27  InnoDB: Warning: using a partial-field key prefix in search.
InnoDB: index `PRIMARY` of table test.t468. Last data field length 6 bytes,
InnoDB: key ptr now exceeds key end by 4 bytes.
InnoDB: Key value in the MySQL format:

myisam corrupts:

ERROR 126 (HY000) at line 4: Incorrect key file for table '.\test\t1.MYI'; try to repair it

5.1.42 worked as expected, and didn't allot the table to be created in the first place.

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` char(1), primary key (`a`(255))) engine=innodb default charset=latin1;
insert into `t1` values ();
update `t1` set `a`=1;

drop table if exists `t1`;
create table `t1` (`a` char(1), primary key (`a`(255))) engine=myisam default charset=latin1;
insert into `t1` values ();
update `t1` set `a`=1;
[22 Jan 2010 14:33] Valeriy Kravchuk
Verified just as described with current trunk from bzr.
[8 Feb 2010 17:14] Shane Bester
also seen as valgrind warnings:

Version: '5.5.99-m3-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
Thread 3:
Conditional jump or move depends on uninitialised value(s)
at 0x861D1D5: my_utf8_uni (ctype-utf8.c:1955)
by 0x861E410: my_ismbchar_utf8 (ctype-utf8.c:2613)
by 0x861028A: my_charpos_mb (ctype-mb.c:384)
by 0x8564901: _mi_make_key (mi_key.c:120)
by 0x8574997: mi_write (mi_write.c:125)
by 0x858B7AF: ha_myisam::write_row(unsigned char*) (ha_myisam.cc:785)
by 0x8393A7C: handler::ha_write_row(unsigned char*) (handler.cc:4722)
by 0x82FF221: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1647)
by 0x8304176: mysql_insert (sql_insert.cc:854)
by 0x826444A: mysql_execute_command(THD*) (sql_parse.cc:3086)
by 0x826A956: mysql_parse (sql_parse.cc:5985)
by 0x826BD19: dispatch_command (sql_parse.cc:1143)
by 0x826CF81: do_command(THD*) (sql_parse.cc:813)
by 0x8259E17: do_handle_one_connection(THD*) (sql_connect.cc:1174)
by 0x8259ED2: handle_one_connection (sql_connect.cc:1113)
by 0x4893DA: start_thread (in /lib/libpthread-2.5.so)
by 0x3D606D: clone (in /lib/libc-2.5.so)
Uninitialised value was created by a heap allocation

---
drop table if exists `t3938`;
create table `t3938`(
`col1` nchar (4) not null ,
 index `idx1` (`col1`(255))
) engine=MyISAM pack_keys=1 row_format=compact;
insert ignore into `t3938` values ();
---
[9 Feb 2010 12:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99710

2937 Magne Mahre	2010-02-09
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
      
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
      
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
      
      The fix has been to explicitly test for the 
      condition where the index key is wider during the 
      create phase, and throw an error if so.
      
      error
[11 Feb 2010 10:28] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99907

2940 Magne Mahre	2010-02-11
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
      
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
      
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
      
      The partial key validiation clause has been 
      restructured to (hopefully) better document which
      uses of partial keys are valid.
[11 Feb 2010 17:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99979

2940 Magne Mahre	2010-02-11
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
            
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
            
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
            
      The partial key validiation clause has been 
      restructured to (hopefully) better document which
      uses of partial keys are valid.
[12 Feb 2010 17:40] Bugs System
Pushed into 5.5.2-m2 (revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (version source revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (merge vers: 5.5.2-m2) (pib:16)
[13 Feb 2010 8:37] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100213083436-9pesg4h55w1mekxc) (version source revid:alik@sun.com-20100212100039-eh6atbvijcm71eyb) (merge vers: 6.0.14-alpha) (pib:16)
[13 Feb 2010 8:39] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100213083327-cee4ao3jpg33eggv) (version source revid:alik@sun.com-20100212095912-k3fklaqrxzzls9cd) (pib:16)
[13 Feb 2010 18:09] Paul Dubois
Noted in 5.5.2, 6.0.14 changelogs.

Index prefixes could be specified with a length greater than the
associated column, resulting in a server crash for subsequent table
inserts. 

Setting report to Need Merge pending push of Celosia to release tree.
[15 Mar 2010 15:48] Paul Dubois
Has been merged to mysql-trunk. Closing.