Bug #1087 Crash in in_longlong::set()
Submitted: 18 Aug 2003 17:54 Modified: 19 Aug 2003 12:36
Reporter: Jeremy Cole (Basic Quality Contributor) (OCA) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S1 (Critical)
Version:4.0 OS:Any (All)
Assigned to: CPU Architecture:Any

[18 Aug 2003 17:54] Jeremy Cole
Description:
There seems to be a memory allocation bug in in_vector and specifically in this case being hit in in_longlong::set().

The stack trace is as follows:

0x807474f handle_segfault + 423
0x82a0ad8 pthread_sighandler + 184
0x805776c set__11in_longlongUiP4Item + 36
0x80552dd fix_length_and_dec__12Item_func_in + 497
0x804eb9b fix_fields__9Item_funcP3THDP13st_table_list + 267
0x83244f1 fix_fields__12Item_func_inP3THDP13st_table_list + 57
0x8055830 fix_fields__9Item_condP3THDP13st_table_list + 328
0x8094567 setup_conds__FP3THDP13st_table_listPP4Item + 83
0x8098b44 
mysql_select__FP3THDP13st_table_listRt4List1Z4ItemP4ItemP8st_orderT4T3
T4UlP13select_result + 372
0x80a6fed handle_select__FP3THDP6st_lexP13select_result + 101
0x807f46a mysql_execute_command__Fv + 966
0x8082f6b mysql_parse__FP3THDPcUi + 559
0x807e5b3 dispatch_command__F19enum_server_commandP3THDPcUi + 1471
0x80844ee do_command__FP3THD + 154
0x807d79f handle_one_connection + 635
0x829e28c pthread_start_thread + 220
0x82d199a thread_start + 4

I believe the source of the bug is in item_cmpfunc.h:

   285  /* Functions to handle the optimized IN */
   286  
   287  class in_vector :public Sql_alloc
   288  {
   289   protected:
   290    char *base;
   291    uint size;
   292    qsort_cmp compare;
   293    uint count;
   294  public:
   295    uint used_count;
   296    in_vector(uint elements,uint element_length,qsort_cmp cmp_func)
   297      :base((char*) sql_calloc(elements*element_length)),
   298       size(element_length), compare(cmp_func), count(elements),
   299       used_count(elements) {}
   300    virtual ~in_vector() {}
   301    virtual void set(uint pos,Item *item)=0;
   302    virtual byte *get_value(Item *item)=0;
   303    void sort()
   304    {
   305      qsort(base,used_count,size,compare);
   306    }
   307    int find(Item *item);
   308  };

On line 297 we allocate memory for 'base' without checking the return value of sql_calloc(), and later on, from in_longlong::set() we use that memory without first checking it.

item_cmpfunc.h:

   323  class in_longlong :public in_vector
   324  {
   325    longlong tmp;
   326  public:
   327    in_longlong(uint elements);
   328    void set(uint pos,Item *item);
   329    byte *get_value(Item *item);
   330  };

item_cmpfunc.cc:

   998  void in_longlong::set(uint pos,Item *item)
   999  {
  1000    ((longlong*) base)[pos]=item->val_int();
  1001  }

How to repeat:
Happens on customer system, but is sporadic (originating in OOM condition) and it is not currently possible to make a reproducible test case.

The system that is crashing is often running queries with very large IN() lists, up to several hundred thousands of items.

Suggested fix:
Check the return value of sql_calloc in in_vector.
[18 Aug 2003 21:35] Jeremy Cole
The stack trace above is from the 4.0.14 RPM.
[18 Aug 2003 23:27] Jeremy Cole
Tried to simulate what would happen if sql_calloc() failed in in_vector, by making the following change:

--- item_cmpfunc.h.orig 2003-08-19 02:01:48.000000000 -0400
+++ item_cmpfunc.h      2003-08-19 02:02:12.000000000 -0400
@@ -294,7 +294,7 @@
 public:
   uint used_count;
   in_vector(uint elements,uint element_length,qsort_cmp cmp_func)
-    :base((char*) sql_calloc(elements*element_length)),
+    :base((char*) NULL /*sql_calloc(elements*element_length)*/),
      size(element_length), compare(cmp_func), count(elements),
      used_count(elements) {}
   virtual ~in_vector() {}

And running the simple query:

 select 1 in (1,2,3)

Generates the "exact" same stack trace:

0x8093cfa handle_segfault + 452
0x82653a4 __pthread_sighandler + 116
0x8061c89 _ZN11in_longlong3setEjP4Item + 29
0x8061e7a _ZN12Item_func_in18fix_length_and_decEv + 246
0x8054531 _ZN9Item_func10fix_fieldsEP3THDP13st_table_list + 249
0x8066c74 _ZN12Item_func_in10fix_fieldsEP3THDP13st_table_list + 70
0x80bbb6f _Z12setup_fieldsP3THDP13st_table_listR4ListI4ItemEbPS5_b + 237
0x80c09e1 _Z12mysql_selectP3THDP13st_table_listR4ListI4ItemEPS4_P8st_orderS9_S7_S9_mP13select_result + 379
0x80c083c _Z13handle_selectP3THDP6st_lexP13select_result + 128
0x80a4284 _Z21mysql_execute_commandv + 8250
0x80a6af2 _Z11mysql_parseP3THDPcj + 212
0x80a173a _Z16dispatch_command19enum_server_commandP3THDPcj + 1606
0x80a10e9 _Z10do_commandP3THD + 495
0x80a0667 handle_one_connection + 623
0x8260b89 pthread_start_thread + 193
0x8299d57 clone + 103
[19 Aug 2003 12:36] Michael Widenius
Thank you for your bug report. This issue has been committed to our
source repository of that product and will be incorporated into the
next release.

If necessary, you can access the source repository and build the latest
available version, including the bugfix, yourself. More information 
about accessing the source trees is available at
    http://www.mysql.com/doc/en/Installing_source_tree.html