Bug #44766 valgrind error when using convert() in a subquery
Submitted: 10 May 2009 14:22 Modified: 29 Jan 2011 23:08
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Charsets Severity:S3 (Non-critical)
Version:5.0.70, 5.0.82, 5.1.34, 5.1.35, 6.0.10 OS:Any
Assigned to: Ramil Kalimullin CPU Architecture:Any
Tags: convert, valgrind

[10 May 2009 14:22] Shane Bester
Description:
when converting character sets inside a subquery, memcpy tries to copy memory to the buffer from which it reads.  From 5.1.35:

 Source and destination overlap in memcpy(0x4EBD9AC, 0x4EBD9AC, 3)
: memcpy (mc_replace_strmem.c:116)
: String::copy(char const*, unsigned, charset_info_st*) (sql_string.cc:214)
: String::copy  (sql_string.cc:334)
: Item_func_conv_charset::val_str(String*) (item_strfunc.cc:2717)
: Item::save_in_field(Field*, bool) (item.cc:4892)
: fill_record(THD*, Field**, List<Item>&, bool) (sql_base.cc:8203)
: select_union::send_data(List<Item>&) (sql_union.cc:60)
: end_send     (sql_select.cc:11976)
: do_select    (sql_select.cc:10854)
: JOIN::exec   (sql_select.cc:2199)
: mysql_select (sql_select.cc:2378)
: mysql_derived_filling(THD*, st_lex*, TABLE_LIST*) (sql_derived.cc:293)

How to repeat:
run mysqld under valgrind and then:

drop table if exists `t1`;
create table `t1`(`c1` tinyint)engine=myisam charset=latin1;
insert into `t1` values (127);

select 1 from
(
 select convert(`t2`.`c1` using utf8) 
 from `t1`,`t1` as `t2` limit 1
) as `s` limit 1;
[15 May 2009 9:27] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/74167

2874 Ramil Kalimullin	2009-05-15
      Fix for bug#44743: Join in combination with concat does not always work
              bug#44766: valgrind error when using convert() in a subquery
      
      Problem: converting a string to some charset we didn't take into account
      that input and output buffers may be the same.
      
      Fix: reallocate output buffer in such cases.
     @ mysql-test/r/cast.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/r/func_concat.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/t/cast.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ mysql-test/t/func_concat.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ sql/sql_string.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - String::copy() - if the input string points to the output buffer,
            allocate the buffer anew.
[21 May 2009 8:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/74673

2897 Ramil Kalimullin	2009-05-21
      Fix for bug#44743: Join in combination with concat does not always work
              bug#44766: valgrind error when using convert() in a subquery
      
      Problem: input and output buffers may be the same 
      converting a string to some charset. 
      That may lead to wrong results/valgrind warnings.  
      
      Fix: use different buffers.
     @ mysql-test/r/cast.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/r/func_concat.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/t/cast.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ mysql-test/t/func_concat.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ sql/item.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - comment added.
     @ sql/item_strfunc.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - '&args[0]->str_value' used as a parameter of args[0]->val_str(),
            as 'str' may be equal to 'str_value' which we use as the output buffer
            converting strings.
     @ sql/sql_string.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - input and output buffers must NOT be the same.
[28 May 2009 8:17] Bugs System
Pushed into 5.1.36 (revid:joro@sun.com-20090528073639-yohsb4q1jzg7ycws) (version source revid:ramil@mysql.com-20090521095605-b271vdcz4op3cc0y) (merge vers: 5.1.36) (pib:6)
[1 Jun 2009 19:29] Paul DuBois
Noted in 5.1.36 changelog.

Conversion of a string to a different character set could use the 
same buffer for input and output, leading to incorrect results or
warnings. 

Setting report to NDI pending push into 6.0.x.
[17 Jun 2009 15:34] MySQL Verification Team
was this fixed in 5.0 also ? imho it should have been..
[17 Jun 2009 19:24] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090616183122-chjzbaa30qopdra9) (version source revid:ramil@mysql.com-20090521100743-a64v5anrz4jfhjdv) (merge vers: 6.0.12-alpha) (pib:11)
[26 Jun 2009 2:23] Paul DuBois
Noted in 5.4.4 changelog.
[12 Aug 2009 22:29] Paul DuBois
Noted in 5.4.2 changelog because next 5.4 version will be 5.4.2 and not 5.4.4.
[15 Aug 2009 1:48] Paul DuBois
Ignore previous comment about 5.4.2.
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)
[26 Aug 2009 13:48] Bugs System
Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)
[27 Aug 2009 16:33] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[7 Oct 2009 19:34] Paul DuBois
The 5.4 fix has been pushed to 5.4.2.
[17 Jan 2011 12:15] Bugs System
Pushed into mysql-5.1 5.1.56 (revid:alexander.barkov@oracle.com-20110117121133-64dwqa4aiu382e7y) (version source revid:alexander.barkov@oracle.com-20110117121133-64dwqa4aiu382e7y) (merge vers: 5.1.56) (pib:24)
[17 Jan 2011 12:30] Bugs System
Pushed into mysql-5.5 5.5.10 (revid:alexander.barkov@oracle.com-20110117122613-tlwbdj268iau16rk) (version source revid:alexander.barkov@oracle.com-20110117122613-tlwbdj268iau16rk) (merge vers: 5.5.10) (pib:24)
[17 Jan 2011 12:39] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:alexander.barkov@oracle.com-20110117123558-9z1ce4fbkrt9a7bi) (version source revid:alexander.barkov@oracle.com-20110117123558-9z1ce4fbkrt9a7bi) (merge vers: 5.6.2) (pib:24)