Bug #2686 index_merge select on BerkeleyDB table with varchar PK causes mysqld to crash
Submitted: 9 Feb 2004 8:22 Modified: 25 Jun 2004 14:52
Reporter: Sergey Petrunya Email Updates:
Status: Closed Impact on me:
Category:MySQL Server Severity:S2 (Serious)
Version:4.0 OS:Linux (Suse 9.0)
Assigned to: Ingo Strüwing CPU Architecture:Any

[9 Feb 2004 8:22] Sergey Petrunya
index_merge select on BerkeleyDB table with Primary Key key containing parts of long varchar field can cause mysqld to crash. mysqld also may (or may not) crash immediately after being started again.
 The crash happens only with BerkeleyDB tables, tables of other types are OK.

How to repeat:
Run the following:

create table pk_big_blob( 
  pk1 varchar(5000) not null, pk2 varchar(5000) not null, pk3 char(4), 
  key1 int, key2 int, 
  primary key(pk1(4), pk2(4), pk3), key(key1), key(key2) 
) engine=bdb;

insert into pk_big_blob values (concat('aaa-', repeat('A', 4000)),  concat('eee-', repeat('e', 4000)), 'a++a', 1, 1);
insert into pk_big_blob values (concat('bbb-', repeat('B', 4000)),  concat('ggg-', repeat('G', 4000)), 'b++b', 1, 1);
select * from pk_big_blob force index(key1, key2) where key1 < 3 or key2 < 3;

and watch the server to crash.
[9 Feb 2004 8:23] Sergey Petrunya
This seems to be a buffer overrun of some sort, Here is a stack trace:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 8192 (LWP 26261)]
0x4207be77 in memset () from /lib/i686/libc.so.6
(gdb) bt
#0  0x4207be77 in memset () from /lib/i686/libc.so.6
#1  0x086adb96 in ?? ()
#2  0x081f47d1 in ha_berkeley::unpack_key(char*, __db_dbt*, unsigned) (this=0x1,
    record=0x86d0000 <Address 0x86d0000 out of bounds>, key=0x20202020, index=538976288) at ha_berkeley.cc:725
#3  0x081f6499 in ha_berkeley::read_row(int, char*, unsigned, __db_dbt*, __db_dbt*, bool) (this=0x8692600, error=0,
    buf=0x8692738 "?\004b??i\b", keynr=1, row=0xbfffebc0, found_key=0x86926a8, read_next=true) at ha_berkeley.cc:1380
#4  0x081f6d22 in ha_berkeley::index_next(char*) (this=0x8692600, buf=0x20202020 <Address 0x20202020 out of bounds>)
    at ha_berkeley.cc:1525
#5  0x081e4660 in QUICK_RANGE_SELECT::get_next() (this=0x868a3f0) at opt_range.cc:3494
#6  0x081e4364 in QUICK_INDEX_MERGE_SELECT::prepare_unique() (this=0x869d798) at opt_range.cc:3406
#7  0x081de1e9 in QUICK_INDEX_MERGE_SELECT::reset() (this=0x869d798) at opt_range.cc:693
#8  0x081a8fbf in join_init_read_record (tab=0x0) at sql_select.cc:6097
#9  0x081a83a1 in sub_select (join=0x0, join_tab=0x868e910, end_of_records=true) at sql_select.cc:5693
#10 0x081a8087 in do_select (join=0x868d958, fields=0x868e910, table=0x0, procedure=0x20202020) at sql_select.cc:5601
#11 0x0819e060 in JOIN::exec() (this=0x868d958) at sql_select.cc:1491
#12 0x0819e51a in mysql_select(THD*, Item***, st_table_list*, unsigned, List<Item>&, Item*, unsigned, st_order*, st_order*, Item*, st_order*, unsigned long, select_result*, st_select_lex_unit*, st_select_lex*) (thd=0x868c100,
    rref_pointer_array=0x868c2e4, tables=0x868d5a8, wild_num=1, fields=@0x20202020, conds=0x868d8b0, og_num=0, order=0x0,
    group=0x0, having=0x0, proc_param=0x0, select_options=8669696, result=0x868d948, unit=0x868c11c, select_lex=0x868c1f0)
    at sql_select.cc:1616
#13 0x0819ab0d in handle_select(THD*, st_lex*, select_result*) (thd=0x868c100, lex=0x868c110, result=0x868d948)
    at sql_select.cc:183
#14 0x081748e5 in mysql_execute_command(THD*) (thd=0x868c100) at sql_parse.cc:2206
#15 0x08179664 in mysql_parse(THD*, char*, unsigned) (thd=0x868c100, inBuf=0x868c110 "\001", length=76) at sql_parse.cc:4149
#16 0x0817288f in dispatch_command(enum_server_command, THD*, char*, unsigned) (command=COM_QUERY, thd=0x868c100,
    packet=0x8680889 "", packet_length=77) at sql_parse.cc:1396
#17 0x081721fa in do_command(THD*) (thd=0x868c100) at sql_parse.cc:1226
#18 0x08171703 in handle_one_connection (arg=0x20202020) at sql_parse.cc:991
#19 0x08161729 in create_new_thread (thd=0x868c100) at mysqld.cc:3002
#20 0x08161db9 in handle_connections_sockets (arg=0x0) at mysqld.cc:3275
#21 0x08161125 in main (argc=538976288, argv=0xbffff9d4) at mysqld.cc:2672
#22 0x420158d4 in __libc_start_main () from /lib/i686/libc.so.6
[9 Feb 2004 8:26] Sergey Petrunya
In one attempt after the crash I was also unable to start mysqld again, getting the following:

/usr/local/mysql/libexec/mysqld --console
__db_assert: "!DB_REDO(op) || (cmp_p) >= 0 || IS_NOT_LOGGED_LSN(*&(((PAGE *)pagep)->lsn))" failed: file "../../bdb/db/db_rec.c", line 70
mysqld got signal 6;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217596 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xbffff178, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:

Resolved stack trace is:
0x815f4b9 handle_segfault + 455
0x4002e47e _end + 933907790
0x42028cc1 _end + 967439761
0x4202a019 _end + 967444713
0x82b27f7 __db_assert + 85
0x82b9398 __db_addrem_recover + 444
0x82b0583 .L49 + 174
0x825afde __db_apprec + 2626
0x8258bc4 __dbenv_open + 1264
0x81f2e75 _Z13berkeley_initv + 705
0x81eae1a _Z7ha_initv + 20
0x8160cbb _Z22init_server_componentsv + 859
0x8160f84 main + 426
0x420158d4 _end + 967360932
0x80e9751 _start + 33

One can get rid of it by removing the table file. I couldn't repeat this when running mysqld under gdb. I didn't investigate if table data got corrupted or not.
[16 Mar 2004 19:23] MySQL Verification Team
I was able to repeat on Suse 9.0, latest BK 5.0 tree:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 163851 (LWP 25249)]
0x402b0b87 in memset () from /lib/i686/libc.so.6
[28 Mar 2004 5:28] Ingo Strüwing
Keys are packed with the minimum size of the length field for the key part  and were unpacked as if they had a length size of the base column.  For the purpose of optimal key packing we have the method pack_key, while rows are packed with pack (keys may use a small part of a column so that their length can be represented with less bytes).  Now keys are unpacked with unpack_key and no longer with unpack which is used for rows.
[28 Mar 2004 5:35] Ingo Strüwing
The changeset notification is:
bk commit - 5.0 tree (ingo:1.1666)
of 27 Mar 2004 23:59 CET
[7 Jun 2004 12:22] Ingo Strüwing
After the first review the new changeset is:
bk commit - 5.0 tree (ingo:1.1690) BUG#2686
[22 Jun 2004 10:59] Michael Widenius
The patch should be pushed into 4.0 as this may have side effects also in 4.0 (mostly warnings from valgrind)
[24 Jun 2004 16:33] Ingo Strüwing
The changeset for 4.0 is now:
bk commit - 4.0 tree (ingo:1.1888) BUG#2686
[25 Jun 2004 14:52] Ingo Strüwing
Bugfix is pushed into 4.0.