MySQL Bugs: #50542: 5.5.x doesn't check length of key prefixes: corruption and crash results

Bug #50542	5.5.x doesn't check length of key prefixes: corruption and crash results
Submitted:	22 Jan 2010 12:40	Modified:	15 Mar 2010 15:48
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: DDL	Severity:	S1 (Critical)
Version:	5.5.2-m2, 5.5.99-m3	OS:	Any
Assigned to:	Magne Mæhre	CPU Architecture:	Any
Tags:	regression, valgrind

Description:
5.5.x doesn't check that the length of the index prefix is less than or equal to the length of the column.  

innodb crashes:

mysqld-debug.exe!row_sel_convert_mysql_key_to_innobase()[row0sel.c:2331]
mysqld-debug.exe!ha_innobase::index_read()[ha_innodb.cc:5101]
mysqld-debug.exe!ha_innobase::rnd_pos()[ha_innodb.cc:5566]
mysqld-debug.exe!rr_from_tempfile()[records.cc:413]
mysqld-debug.exe!mysql_update()[sql_update.cc:582]
mysqld-debug.exe!mysql_execute_command()[sql_parse.cc:3176]
mysqld-debug.exe!mysql_parse()[sql_parse.cc:6211]
mysqld-debug.exe!dispatch_command()[sql_parse.cc:1240]
mysqld-debug.exe!do_command()[sql_parse.cc:873]
mysqld-debug.exe!handle_one_connection()[sql_connect.cc:1154]
mysqld-debug.exe!pthread_start()[my_winthread.c:61]
mysqld-debug.exe!_callthreadstartex()[threadex.c:348]
mysqld-debug.exe!_threadstartex()[threadex.c:331]
kernel32.dll!FlsSetValue()

innodb might also catch the error and print:
100122 13:43:27  InnoDB: Warning: using a partial-field key prefix in search.
InnoDB: index `PRIMARY` of table test.t468. Last data field length 6 bytes,
InnoDB: key ptr now exceeds key end by 4 bytes.
InnoDB: Key value in the MySQL format:

myisam corrupts:

ERROR 126 (HY000) at line 4: Incorrect key file for table '.\test\t1.MYI'; try to repair it

5.1.42 worked as expected, and didn't allot the table to be created in the first place.

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` char(1), primary key (`a`(255))) engine=innodb default charset=latin1;
insert into `t1` values ();
update `t1` set `a`=1;

drop table if exists `t1`;
create table `t1` (`a` char(1), primary key (`a`(255))) engine=myisam default charset=latin1;
insert into `t1` values ();
update `t1` set `a`=1;

Verified just as described with current trunk from bzr.

also seen as valgrind warnings:

Version: '5.5.99-m3-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
Thread 3:
Conditional jump or move depends on uninitialised value(s)
at 0x861D1D5: my_utf8_uni (ctype-utf8.c:1955)
by 0x861E410: my_ismbchar_utf8 (ctype-utf8.c:2613)
by 0x861028A: my_charpos_mb (ctype-mb.c:384)
by 0x8564901: _mi_make_key (mi_key.c:120)
by 0x8574997: mi_write (mi_write.c:125)
by 0x858B7AF: ha_myisam::write_row(unsigned char*) (ha_myisam.cc:785)
by 0x8393A7C: handler::ha_write_row(unsigned char*) (handler.cc:4722)
by 0x82FF221: write_record(THD*, TABLE*, st_copy_info*) (sql_insert.cc:1647)
by 0x8304176: mysql_insert (sql_insert.cc:854)
by 0x826444A: mysql_execute_command(THD*) (sql_parse.cc:3086)
by 0x826A956: mysql_parse (sql_parse.cc:5985)
by 0x826BD19: dispatch_command (sql_parse.cc:1143)
by 0x826CF81: do_command(THD*) (sql_parse.cc:813)
by 0x8259E17: do_handle_one_connection(THD*) (sql_connect.cc:1174)
by 0x8259ED2: handle_one_connection (sql_connect.cc:1113)
by 0x4893DA: start_thread (in /lib/libpthread-2.5.so)
by 0x3D606D: clone (in /lib/libc-2.5.so)
Uninitialised value was created by a heap allocation

---
drop table if exists `t3938`;
create table `t3938`(
`col1` nchar (4) not null ,
 index `idx1` (`col1`(255))
) engine=MyISAM pack_keys=1 row_format=compact;
insert ignore into `t3938` values ();
---

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99710

2937 Magne Mahre	2010-02-09
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
      
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
      
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
      
      The fix has been to explicitly test for the 
      condition where the index key is wider during the 
      create phase, and throw an error if so.
      
      error

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99907

2940 Magne Mahre	2010-02-11
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
      
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
      
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
      
      The partial key validiation clause has been 
      restructured to (hopefully) better document which
      uses of partial keys are valid.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/99979

2940 Magne Mahre	2010-02-11
      Bug#50542 5.5.x doesn't check length of key prefixes: 
                corruption and crash results
            
      An index creation statement where the index key
      is larger/wider than the column it references 
      should throw an error.
            
      A statement like:
        CREATE TABLE t1 (a CHAR(1), PRIMARY KEY (A(255)))
      did not error, but a segmentation fault followed when
      an insertion was attempted on the table
            
      The partial key validiation clause has been 
      restructured to (hopefully) better document which
      uses of partial keys are valid.

Pushed into 5.5.2-m2 (revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (version source revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (merge vers: 5.5.2-m2) (pib:16)

Pushed into 6.0.14-alpha (revid:alik@sun.com-20100213083436-9pesg4h55w1mekxc) (version source revid:alik@sun.com-20100212100039-eh6atbvijcm71eyb) (merge vers: 6.0.14-alpha) (pib:16)

Pushed into mysql-next-mr (revid:alik@sun.com-20100213083327-cee4ao3jpg33eggv) (version source revid:alik@sun.com-20100212095912-k3fklaqrxzzls9cd) (pib:16)

Noted in 5.5.2, 6.0.14 changelogs.

Index prefixes could be specified with a length greater than the
associated column, resulting in a server crash for subsequent table
inserts. 

Setting report to Need Merge pending push of Celosia to release tree.

Has been merged to mysql-trunk. Closing.