MySQL Bugs: #44766: valgrind error when using convert() in a subquery

Bug #44766	valgrind error when using convert() in a subquery
Submitted:	10 May 16:22	Modified:	26 Jun 4:23
Reporter:	Shane Bester
Status:	Closed
Category:	Server: Charsets	Severity:	S3 (Non-critical)
Version:	5.0.70, 5.0.82, 5.1.34, 5.1.35, 6.0.10	OS:	Any
Assigned to:	Ramil Kalimullin	Target Version:	5.0+
Tags:	valgrind, convert
Triage:	Triaged: D2 (Serious)

Description:
when converting character sets inside a subquery, memcpy tries to copy memory to the
buffer from which it reads.  From 5.1.35:

 Source and destination overlap in memcpy(0x4EBD9AC, 0x4EBD9AC, 3)
: memcpy (mc_replace_strmem.c:116)
: String::copy(char const*, unsigned, charset_info_st*) (sql_string.cc:214)
: String::copy  (sql_string.cc:334)
: Item_func_conv_charset::val_str(String*) (item_strfunc.cc:2717)
: Item::save_in_field(Field*, bool) (item.cc:4892)
: fill_record(THD*, Field**, List<Item>&, bool) (sql_base.cc:8203)
: select_union::send_data(List<Item>&) (sql_union.cc:60)
: end_send     (sql_select.cc:11976)
: do_select    (sql_select.cc:10854)
: JOIN::exec   (sql_select.cc:2199)
: mysql_select (sql_select.cc:2378)
: mysql_derived_filling(THD*, st_lex*, TABLE_LIST*) (sql_derived.cc:293)

How to repeat:
run mysqld under valgrind and then:

drop table if exists `t1`;
create table `t1`(`c1` tinyint)engine=myisam charset=latin1;
insert into `t1` values (127);

select 1 from
(
 select convert(`t2`.`c1` using utf8) 
 from `t1`,`t1` as `t2` limit 1
) as `s` limit 1;

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/74167

2874 Ramil Kalimullin	2009-05-15
      Fix for bug#44743: Join in combination with concat does not always work
              bug#44766: valgrind error when using convert() in a subquery
      
      Problem: converting a string to some charset we didn't take into account
      that input and output buffers may be the same.
      
      Fix: reallocate output buffer in such cases.
     @ mysql-test/r/cast.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/r/func_concat.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/t/cast.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ mysql-test/t/func_concat.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ sql/sql_string.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - String::copy() - if the input string points to the output buffer,
            allocate the buffer anew.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/74673

2897 Ramil Kalimullin	2009-05-21
      Fix for bug#44743: Join in combination with concat does not always work
              bug#44766: valgrind error when using convert() in a subquery
      
      Problem: input and output buffers may be the same 
      converting a string to some charset. 
      That may lead to wrong results/valgrind warnings.  
      
      Fix: use different buffers.
     @ mysql-test/r/cast.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/r/func_concat.result
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test result.
     @ mysql-test/t/cast.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ mysql-test/t/func_concat.test
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - test case.
     @ sql/item.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - comment added.
     @ sql/item_strfunc.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - '&args[0]->str_value' used as a parameter of args[0]->val_str(),
            as 'str' may be equal to 'str_value' which we use as the output buffer
            converting strings.
     @ sql/sql_string.cc
        Fix for bug#44743: Join in combination with concat does not always work
                bug#44766: valgrind error when using convert() in a subquery
          - input and output buffers must NOT be the same.

Pushed into 5.1.36 (revid:joro@sun.com-20090528073639-yohsb4q1jzg7ycws) (version source
revid:ramil@mysql.com-20090521095605-b271vdcz4op3cc0y) (merge vers: 5.1.36) (pib:6)

Noted in 5.1.36 changelog.

Conversion of a string to a different character set could use the 
same buffer for input and output, leading to incorrect results or
warnings. 

Setting report to NDI pending push into 6.0.x.

was this fixed in 5.0 also ? imho it should have been..

Pushed into 5.4.4-alpha (revid:alik@sun.com-20090616183122-chjzbaa30qopdra9) (version
source revid:ramil@mysql.com-20090521100743-a64v5anrz4jfhjdv) (merge vers: 6.0.12-alpha)
(pib:11)

Noted in 5.4.4 changelog.

Noted in 5.4.2 changelog because next 5.4 version will be 5.4.2 and not 5.4.4.

Ignore previous comment about 5.4.2.

Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l)
(version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers:
5.1.37-ndb-7.0.8) (pib:11)

Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc)
(version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers:
5.1.37-ndb-6.3.27) (pib:11)

Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4)
(version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers:
5.1.37-ndb-6.2.19) (pib:11)

Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr)
(version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers:
5.1.37-ndb-7.0.8) (pib:11)

The 5.4 fix has been pushed to 5.4.2.