MySQL Bugs: #52089: overlapping memory locations given in make_join

Bug #52089	overlapping memory locations given in make_join_statistics
Submitted:	16 Mar 2010 12:18	Modified:	13 Nov 2010 13:26
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	MySQL Server: Optimizer	Severity:	S1 (Critical)
Version:	5.1.49, 5.6.1,5.6.99-m4	OS:	Linux
Assigned to:		CPU Architecture:	Any
Tags:	valgrind

Description:
5.6.99-m4 valgrind output:

Source and destination overlap in memcpy(0x15cfc078, 0x15cfc078, 72)
at: memcpy (mc_replace_strmem.c:482)
by: update_ref_and_keys (sql_select.cc:3991)
by: make_join_statistics (sql_select.cc:2737)
by: JOIN::optimize() (sql_select.cc:1015)
by: mysql_select (sql_select.cc:2492)
by: handle_select (sql_select.cc:271)
by: execute_sqlcom_select (sql_parse.cc:4703)
by: mysql_execute_command (sql_parse.cc:2191)
by: mysql_parse (sql_parse.cc:5735)
by: dispatch_command (sql_parse.cc:1024)
by: do_command (sql_parse.cc:710)
by: do_handle_one_connection (sql_connect.cc:1174)
by: handle_one_connection (sql_connect.cc:1113)
by: start_thread (in /lib64/libpthread-2.5.so)
by: clone (in /lib64/libc-2.5.so)

5.1.44 was not affected by this, so there is a possible regression.

How to repeat:
#run mysqld under valgrind then execute this:

drop table if exists `t45`,`t14`;
create  table `t45` (`col0` date default null,key (`col0`)) engine=myisam ;
create table `t14` (`col0` longblob not null) engine=myisam;
set sql_mode='';
insert into `t45` values (),();
insert into `t14` values (),();
select 1 from `t45` inner join `t14` using(`col0`);

Hm, in the code I noticed this comment:

#ifdef HAVE_purify
   /* Valgrind complains about overlapped memcpy when save_pos==use. */
   if (save_pos != use)
#endif
    *save_pos= *use;

How sure are we that overlapping memory is a good idea? Might it be optimized
differently on different compilers/platforms?

got this again on 5.1.49. will get a testcase sometime.

Source and destination overlap in memcpy(0x9f47ee8, 0x9f47ee8, 72)
at : memcpy (mc_replace_strmem.c:482)
by : update_ref_and_keys (sql_select.cc:4016)
by : make_join_statistics (sql_select.cc:2764)
by : JOIN::optimize() (sql_select.cc:1012)
by : mysql_select (sql_select.cc:2495)
by : mysql_explain_union
by : execute_sqlcom_select
by : mysql_execute_command
by : mysql_parse
by : dispatch_command
by : do_command
by : handle_one_connection
by : start_thread
by : clone

hello!  please verify this with a testcase on mysql-trunk

drop table if exists `t1`;
create table `t1` (`a` int,key(`a`)) engine=innodb;
select 1 from `t1` `t2`,`t1` where `t1`.`a`=`t2`.`a`; 

 Source and destination overlap in memcpy(0x13ba0a30, 0x13ba0a30, 72)
at memcpy (mc_replace_strmem.c:482)
by make_join_statistics(sql_select.cc:4027)
by JOIN::optimize (sql_select.cc:1038)
by mysql_select  (sql_select.cc:2516)
by handle_select (sql_select.cc:290)
by execute_sqlcom_select (sql_parse.cc:4797)
by mysql_execute_command (sql_parse.cc:2298)
by mysql_parse (sql_parse.cc:5826)
by dispatch_command (sql_parse.cc:1128)
by do_command (sql_parse.cc:800)
by do_handle_one_connection (sql_connect.cc:1191)
by handle_one_connection
by start_thread
by clone

Valgrind is run like this:
valgrind --tool=memcheck --num-callers=50 --leak-check=full --db-attach=no -v --show-reachable=yes ./bin/mysqld --no-defaults --server-id=2 --skip-gr --skip-name-resolve --console  --log-warnings=2  --skip-name  --datadir=./data --basedir=.  --innodb-buffer-pool-size=100M --innodb-file-per-table --innodb-flush-log-at-trx-commit=0

valgrind-3.5.0

stepping stones to reproduction

Attachment: bug52089_repro.txt (text/plain), 11.91 KiB.

Hello. The code in question (where Valgrind finds the error) is:
#ifdef HAVE_purify
      /* Valgrind complains about overlapped memcpy when save_pos==use. */
      if (save_pos != use)
#endif
        *save_pos= *use;
HAVE_purify should be defined for builds which are used against Valgrind (those are BUILD/*valgrind* scripts). You used a non-Valgrind build, so you hit the Valgrind complaint, which is expected.
We can see that Valgrind's error is stupid:
memcpy(0x13ba0a30, 0x13ba0a30, 72)
the source and destination are the same addresses, this cannot cause a memory problem. I suggest closing as "not a bug".

But this can apparently cause undefined behaviour...that's why I got worried..

http://www.opengroup.org/onlinepubs/009695399/functions/memcpy.html
"If copying takes place between objects that overlap, the behavior is undefined."

So, you're saying that if I catch this error again in valgrind and src!=dest then it's truly a bug ?  If src==dest then you're sure the error is benign and no implementation of memcpy will act weird?  I'm happy with that if you agree.

If you run with Valgrind, and src!=dest then it's truly a bug.
If src==dest: it's not exactly memcpy() here, it's
*save_pos = *use;
i.e. a copy of two KEYUSE structs. In C and C++, if there is overlap, the copy is correct only if the overlap is exact (which is true if src==dest) and the types are identical.

It is probably gcc which incorrectly translates a struct assignment to memcpy():
Davi found http://gcc.gnu.org/bugzilla/show_bug.cgi?id=19410 about this.

see also BUG#56138

addressed in patch of BUG#56138