MySQL Bugs: #41868: crash or memory overrun with concat + upper, date

Bug #41868	crash or memory overrun with concat + upper, date_format functions
Submitted:	5 Jan 2009 14:09	Modified:	9 Mar 2009 14:49
Reporter:	Germán Cáseres	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: DML	Severity:	S1 (Critical)
Version:	5.1.30, 6.0.10, 6.0.8, 5.0.76, 5.0.74,, 5.0.40	OS:	Any
Assigned to:	Alexey Kopytov	CPU Architecture:	Any
Tags:	concat, crash, date_format, upper, valgrind

Description:
Hi,
   I have a problem with a database when executing some queries. Those queries seems to be random, or al least I was unable to reproduce the error.
   I receive exception 0xc0000005 (Access Violation) on mysqld.
   I tried Reinstalling the server, and also tried installing it on another computer.

   The error log shows two occurences of the same error with different queries. I tried executing the queries again, but no error appeared.

I use mysql with Apache 2.2 and PHP5. Last stable version on both.

How to repeat:
Can't find a way to repeat, the error appears almost randomnly

MySQL Config file

Attachment: my.ini (application/octet-stream, text), 8.78 KiB.

MySQL error log

Attachment: mysql.err (application/octet-stream, text), 8.51 KiB.

Thank you for the bug report. Are you able to provide the dump file of the database so we can try on our side?. The dump could be private if you wish.
Thanks in advance.

Thank you for hte problem report. Is it 32-bit or 64-bit Windows? How much RAM do you have? 

Please, try also to identify complete queries that lead to crash (I am not sure second one is complete) and send EXPLAIN results for them.

Miguel,
       i will provide the dump as soon as i have access to the server.

Valeriy,
        its 32-bits with 1Gb of RAM.

        I don't know why the second q

Valeriy,
        its 32-bits with 1Gb of RAM.

        I don't know why the second query is incomplete. Maybe because it's too long. I'll try to identify it.

        Anyway if i execute the first query, it works well (from mysql console or from the web application)

Miguel, i have uploaded the dump. Please take a look at it when you can.

Hi,  Thanks for the dump.  I found a problem with first query:

Version: '5.1.30-enterprise-gpl-advanced-debug'  socket: ''  port: 3306  MySQL Enterprise Server - Advanced Edition Debug (GPL)
Error: Memory allocated at .\sql_string.cc:82 was overrun, discovered at '.\sql_string.cc:51'
Error: Memory allocated at .\sql_string.cc:82 was overrun, discovered at '.\sql_string.cc:74'
Error: Memory allocated at .\sql_string.cc:82 was overrun, discovered at '.\sql_string.cc:74'
Error: Memory allocated at .\sql_string.cc:82 was overrun, discovered at '.\sql_string.cc:74'

Will work to make a smaller testcase, and update the bug report when done.

I updated the synopsis to be more specific.
Here's a minimal testcase to show the problem:

------
drop table if exists `t1`;
create table `t1` (`c` date,aa varchar(30)) engine=myisam;
insert into `t1` values ('2008-12-31','aaaaaa');
select  date_format(`c`,get_format(date,'eur')) `h`,
concat(upper(`aa`),', ', `aa`) `i` from `t1`;
------

valgrind errors and safe-malloc errors, or crashes can be observed on 5.1.30, 6.0.8, 6.0.10, 5.0.74, and 5.0.40. So this is an old bug.

The crash is not repeatable each time, so please use mysqld-debug or valgrind to
check:

1 errors in context 1 of 1:
Invalid write of size 1
 copy_if_not_alloced(String*, String*, unsigned) (sql_string.cc:86)
 Item_str_conv::val_str(String*) (item_strfunc.cc:1010)
 Item_func_concat::val_str(String*) (item_strfunc.cc:286)
 Item::send(Protocol*, String*) (item.cc:5093)
 select_send::send_data(List<Item>&) (sql_class.cc:1040)
 _ZL8end_sendP4JOINP13st_join_tableb (sql_select.cc:11527)
 _ZL9do_selectP4JOINP4ListI4ItemEP8st_tableP9Procedure (sql_select.
 JOIN::exec() (sql_select.cc:2106)
 mysql_select(THD*, Item***, TABLE_LIST*, unsigned, List<Item>&, It
 handle_select(THD*, st_lex*, select_result*, unsigned long) (sql_s

I'm glad that you've reached a testcase, you're really a genius. I'll try to work around that query.
Yesterday the error occured again, but with another query. This time a very simple query:

Insert into usuarios_actividad (idusuario, fecha, idactividad)
values('2795', '2009-01-06 08:22:11', 6)

I'm attaching the entire log for that error, please tell me if i should do this in a new bug notification.

Thank you, you all are really helpful

MYSQL error log updated 06 Jan 2009

Attachment: mysql-06012009.err (application/octet-stream, text), 2.82 KiB.

Germán, it's common in memory corruption/overrun bugs that internal data structures get trashed, and a crash only happens much later - seemingly with an innocent statement.  Especially when free() and malloc() crash, this is usually the case.  So, all these other crashes are probably related to the original one for which I made a testcase.

bug #41927 might be related ..

Two problems here:

1. String::realloc() does not check whether the existing string data fits in the newly allocated buffer for cases when reallocating a String object with external buffer (i.e. alloced == FALSE).  As a result, when shrinking an existing string, the newly allocated buffer gets overrun. This is what was causing crashes/memory overruns in this bug.

2. Even when the above is fixed, we still have a problem with excessive malloc() calls in select_send::send_data(). Before iterating through Items and calling the send() method for each Item, we setup a 'buffer' object pointing to a preallocated memory chunk which is passed to Item::send() and apparently is supposed to be reused for each Item::send(). It turns out it's not always the case, since Item::send() (or rather, internal *::val_str() calls) may reset 'buffer' object to point to another, sometimes statically allocated, memory region. Therefore, next calls to Item::send() will likely not be using the preallocated memory, but instead do their own allocations. This is what was happening during execution of the query from this bug's test case.

#2 is easy to fix by resetting the 'buffer' object to its original state after each Item::send() call.

Bug #41927 is a duplicate of this one.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65445

2741 Alexey Kopytov	2009-02-06
      Temporarily reverted patch for bug #41868 as it was causing problems in PB.

Pushed into 5.0.78 (revid:joro@sun.com-20090206133929-ybpbkkox9wkptopb) (version source revid:alexey.kopytov@sun.com-20090206095111-c7thecg2li0mgbcl) (merge vers: 5.0.78) (pib:6)

Pushed into 5.1.32 (revid:davi.arnaut@sun.com-20090209214102-gj3sb3ujpnvpiy4c) (version source revid:davi.arnaut@sun.com-20090209214102-gj3sb3ujpnvpiy4c) (merge vers: 5.1.32) (pib:6)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/65736

2735 Alexey Kopytov	2009-02-10
      Fix for bug #41868: crash or memory overrun with concat + upper,
                          date_format functions
      
      String::realloc() did not check whether the existing string data fits in
      the newly allocated buffer for cases when reallocating a String object
      with external buffer (i.e.alloced == FALSE).  This could lead to memory
      overruns in some cases.

Pushed into 6.0.10-alpha (revid:matthias.leich@sun.com-20090212211028-y72faag15q3z3szy) (version source revid:alexey.kopytov@sun.com-20090206095654-32mkrlbe7cuvw0v4) (merge vers: 6.0.10-alpha) (pib:6)

Pushed into 5.1.32-ndb-6.3.23 (revid:tomas.ulin@sun.com-20090217131017-6u8qz1edkjfiobef) (version source revid:tomas.ulin@sun.com-20090216083408-rmvyaxjt6mk8sg1y) (merge vers: 5.1.32-ndb-6.3.23) (pib:6)

Pushed into 5.1.32-ndb-6.4.3 (revid:tomas.ulin@sun.com-20090217134419-5ha6xg4dpedrbmau) (version source revid:tomas.ulin@sun.com-20090216083646-m8st11oj1hhfuuh5) (merge vers: 5.1.32-ndb-6.4.3) (pib:6)

Pushed into 5.1.32-ndb-6.2.17 (revid:tomas.ulin@sun.com-20090217134216-5699eq74ws4oxa0j) (version source revid:tomas.ulin@sun.com-20090211111208-wf0acl7c1vl5653e) (merge vers: 5.1.32-ndb-6.2.17) (pib:6)

Pushed into 6.0.10-alpha (revid:sergey.glukhov@sun.com-20090218125737-5y5b2xo3duo1wlvo) (version source revid:azundris@mysql.com-20090213182753-tsqikd41gkwowrqq) (merge vers: 6.0.10-alpha) (pib:6)

Noted in 5.1.32, 6.0.10 changelogs.

String reallocation could cause memory overruns.

Setting report to NDI pending push into 5.0.x.

Pushed into 5.0.79 (revid:joro@sun.com-20090309135922-a0di9ebkxoj4d4wv) (version source revid:alexey.kopytov@sun.com-20090213161707-gax0uem47igu776s) (merge vers: 5.0.79) (pib:6)

Noted in 5.0.79 changelog.

Pushed into 5.1.33 (revid:joro@sun.com-20090313111355-7bsi1hgkvrg8pdds) (version source revid:vvaintroub@mysql.com-20090218093153-sjzxf01i4ezte0xp) (merge vers: 5.1.33) (pib:6)

Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508100057-30ote4xggi4nq14v) (merge vers: 5.1.33-ndb-6.2.18) (pib:6)

Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090508175813-s6yele2z3oh6o99z) (merge vers: 5.1.33-ndb-6.3.25) (pib:6)

Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509073226-09bljakh9eppogec) (merge vers: 5.1.33-ndb-7.0.6) (pib:6)