Bug #29087 select hex(col) from .. where col = unhex(...); crashes server
Submitted: 13 Jun 2007 20:13 Modified: 2 Aug 2007 9:34
Reporter: Miguel Solorzano Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:5.0BK/5.1BK OS:Linux (FC 6 32-bit)
Assigned to: Igor Babaev CPU Architecture:Any

[13 Jun 2007 20:13] Miguel Solorzano 
Description:
Testing bug: http://bugs.mysql.com/bug.php?id=29084 I got a server crash:

[miguel@light mysql-5.0]$ bk changes | head
ChangeSet@1.2518, 2007-06-07 16:29:59+02:00, joerg@trift2. +27 -0
  Merge trift2.:/MySQL/M41/bug23504-4.1
  into  trift2.:/MySQL/M50/push-5.0
  MERGE: 1.1616.3048.1

ChangeSet@1.1616.3048.1, 2007-06-07 12:59:31+02:00, joerg@trift2. +27 -0
  netware/*.def  :  Allocate 128K stack for all executables (bug#23504)

ChangeSet@1.2517, 2007-06-06 20:06:59+02:00, joerg@trift2. +4 -0
  Merge trift2.:/MySQL/M50/mysql-5.0

[miguel@light 5.0]$ bin/mysql -uroot test
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.0.44-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create table tt(bin_col binary(20) NOT NULL DEFAULT
    -> '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', unique key(bin_col));
Query OK, 0 rows affected (0.01 sec)

mysql> insert into tt set bin_col = unhex('1F9480179366F2BF567E1C4B964C1EF029087575');
Query OK, 1 row affected (0.01 sec)

mysql> insert into tt set bin_col = unhex('1F9480179366F2BF567E1C4B964C1EF029087520');
Query OK, 1 row affected (0.00 sec)

mysql> select hex(bin_col) from tt where bin_col =
    -> unhex('1F9480179366F2BF567E1C4B964C1EF029087520');
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> 

[New Thread -1303864432 (LWP 15763)]
070613 17:03:58 [Note] /home/miguel/dbs/5.0/libexec/mysqld: ready for connections.
Version: '5.0.44-debug'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
[New Thread -1304065136 (LWP 15873)]
mysqld: mi_search.c:495: _mi_prefix_search: Assertion `vseg < vseg_end' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread -1304065136 (LWP 15873)]
0x009fc402 in __kernel_vsyscall ()
(gdb) bt full
#0  0x009fc402 in __kernel_vsyscall ()
No symbol table info available.
#1  0x00138d40 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0x0013a591 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x0013238b in __assert_fail () from /lib/libc.so.6
No symbol table info available.
#4  0x0850cfcf in _mi_prefix_search (info=0xabcf198, keyinfo=0xabb2ab8, page=0xabcf3e5 "\223\001u", 
    key=0xabcfc08 "\023\037\224\200\027\223fò¿V~\034K\226L\036ð)\bu ", key_len=20, nextflag=1, ret_pos=0xb2456bb8, 
    buff=0xb2456760 "", last_key=0xb2456bbf "") at mi_search.c:495
        vseg_end = (uchar *) 0xabcf3df ""
        left = 0
        k = (uchar *) 0xabcfc1c " "
        packed = 0
        flag = 0
        my_flag = 0
        nod_flag = 0
        length = 26
        len = 20
        matched = 20
        cmplen = 19
        kseg_len = 19
        prefix_len = 0
        suffix_len = 20
        key_len_skip = 20
---Type <return> to continue, or q <return> to quit---
        seg_len_pack = 1
        key_len_left = 0
        end = (uchar *) 0xabcf3ee '¥' <repeats 200 times>...
        kseg = (uchar *) 0xabcfc09 "\037\224\200\027\223fò¿V~\034K\226L\036ð)\bu "
        vseg = (uchar *) 0xabcf3df ""
        sort_order = (uchar *) 0x0
        tt_buff = "º\000\024bE²@bE²\001\000\000\000\035\000\000\000\220\213E²\002\000\000\000H\f\000\0008\v\000\000Ü\037#\000ñâº\000ÉõX\b (\204\b\001>\000\000\000\000\000\000\032\000\000\000Üóº\000\001\000\000\000\002\000\000\000\001\000\000\000Ä\000\000\000\220\213E²\002\000\000\000x »\n\210bE²ô\217$\000 ¡$\000à\037»\nÈbE²ð\226\027\000 ¡$\000à\037»\n\000(`­\200µ\002\000Ã0\fÃØ\037»\n", '\0' <repeats 12 times>, "\001\000\000\000\002\000\000\000x »\n\030cE²\224;W\b:\001\000\000\020cE²\fcE²\bcE²\fcE²\bcE²\004cE²"...
        t_buff = (uchar *) 0xb2456224 "\024bE²@bE²\001"
        saved_from = (uchar *) 0x0
        saved_to = (uchar *) 0x0
        saved_vseg = (uchar *) 0xabcf3cb "\037\224\200\027\223fò¿V~\034K\226L\036ð)\bu "
        saved_length = 0
        saved_prefix_len = 0
        length_pack = 1
        _db_func_ = 0xb2456244 "Ü\037#"
        _db_file_ = 0x1758a1 "\201ÃS7\r"
        _db_level_ = 15873
        _db_framep_ = (char **) 0x3e
        __PRETTY_FUNCTION__ = "_mi_prefix_search"
#5  0x0850bc56 in _mi_search (info=0xabcf198, keyinfo=0xabb2ab8, 
<cut>

How to repeat:
See description.

Suggested fix:
-
[13 Jun 2007 20:19] MySQL Verification Team 
5.1 crashes too:

[miguel@light 5.1]$ bin/mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.20-beta-debug Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create table tt(bin_col binary(20) NOT NULL DEFAULT
    -> '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', unique key(bin_col));
Query OK, 0 rows affected (0.14 sec)

mysql> insert into tt set bin_col = unhex('1F9480179366F2BF567E1C4B964C1EF029087575');
Query OK, 1 row affected (0.00 sec)

mysql> insert into tt set bin_col = unhex('1F9480179366F2BF567E1C4B964C1EF029087520');
Query OK, 1 row affected (0.00 sec)

mysql> select hex(bin_col) from tt where bin_col =
    -> unhex('1F9480179366F2BF567E1C4B964C1EF029087520');
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
[22 Jun 2007 6:29] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29364

ChangeSet@1.2502, 2007-06-21 23:30:59-07:00, igor@olga.mysql.com +2 -0
  Fixed bug #29087. This bug manifested itself for queries that performed
  a lookup into a BINARY index by a key ended with spaces. It caused
  an assertion abort for a debug version and wrong results for non-debug
  versions.
  
  The problem occurred because the function _me_prefix_key assumed that
  the trailing spaces had been stripped off from index entries while
  the function _mi_make_key erroneously did not do it and as a result
  all BINARY keys were inserted into indexes with trailing spaces.
[22 Jun 2007 18:28] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29432

ChangeSet@1.2502, 2007-06-22 11:31:06-07:00, igor@olga.mysql.com +3 -0
  Fixed bug #29087. This bug manifested itself for queries that performed
  a lookup into a BINARY index by a key ended with spaces. It caused
  an assertion abort for a debug version and wrong results for non-debug
  versions.
  
  The problem occurred because the function _me_prefix_key assumed that
  the trailing spaces had been stripped off from index entries while
  the function _mi_make_key erroneously did not do it and as a result
  all BINARY keys were inserted into indexes with trailing spaces.
[26 Jun 2007 5:45] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29571

ChangeSet@1.2502, 2007-06-25 22:44:22-07:00, igor@olga.mysql.com +3 -0
  Fixed bug #29087. This bug manifested itself for queries that performed
  a lookup into a BINARY index by a key ended with spaces. It caused
  an assertion abort for a debug version and wrong results for non-debug
  versions.
  
  The problem occurred because the function _mi_pack_key stripped off 
  the trailing spaces from binary search keys while the function _mi_make_key
  did not do it when keys were inserted into the index.
  
  Now the function _mi_pack_key does not remove the trailing spaces from
  search keys if they are of the binary type.
[1 Jul 2007 19:58] Bugs System 
Pushed into 5.1.21-beta
[1 Jul 2007 20:02] Bugs System 
Pushed into 5.0.46
[4 Jul 2007 1:47] Paul DuBois 
Noted in 5.0.46, 5.1.21 changelogs.

Queries that performed a lookup into a BINARY index containing key
values ending with spaces caused an assertion failure for debug
builds and incorrect results for non-debug builds.
[2 Aug 2007 7:53] Sveta Smirnova 
Bug still exists in 5.0.45, community and Falcon trees.
[2 Aug 2007 7:54] Sveta Smirnova 
Bug #30196 was marked as duplicate of this one.
[2 Aug 2007 9:34] Sveta Smirnova 
Due to internal discussion Bug #30196 has been reclassified as "Packaging" and this one closed again.
[18 Aug 2008 19:00] Sveta Smirnova 
Bug #38819 was marked as duplicate of this one.