Bug #38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar

Description:
calling a stored procedure intermittently crashes with stack trace similar to:

mysqld-nt.exe!my_utf8_uni
mysqld-nt.exe!my_well_formed_len_mb
mysqld-nt.exe!well_formed_copy_nchars
mysqld-nt.exe!Field_blob::store
mysqld-nt.exe!Item::save_in_field
mysqld-nt.exe!sp_eval_expr
mysqld-nt.exe!sp_rcontext::set_variable
mysqld-nt.exe!select_dumpvar::send_data
mysqld-nt.exe!JOIN::exec
mysqld-nt.exe!mysql_select
mysqld-nt.exe!handle_select
mysqld-nt.exe!mysql_execute_command
mysqld-nt.exe!sp_instr_stmt::exec_core
mysqld-nt.exe!sp_lex_keeper::reset_lex_and_exec_core
mysqld-nt.exe!sp_instr_stmt::execute
mysqld-nt.exe!sp_head::execute
mysqld-nt.exe!sp_head::execute_procedure
mysqld-nt.exe!mysql_execute_command
mysqld-nt.exe!mysql_parse
mysqld-nt.exe!dispatch_command
mysqld-nt.exe!do_command
mysqld-nt.exe!handle_one_connection
mysqld-nt.exe!pthread_start
mysqld-nt.exe!_threadstart

How to repeat:
testcase is on it's way

full stack1

Attachment: bug38469_full_stack_5.0.66a.txt (text/plain), 3.56 KiB.

memory overruns are sensitive to compilers, and platform differences.  at least on windows 5.0.66a, the following testcase shows a problem:

---------
delimiter $
drop procedure if exists `p2` $
create procedure `p2`(in `a` text)
begin
        declare `pos` int default 1;
        declare `str` text;
        set `str` := `a`;
        select substr(`str`, `pos`+ 1 )into `str`;
end $
delimiter ;
call `p2`('s s s s   , ,');
show warnings;
---------

notice the output of 'show warnings' is pointing to uninitialized or freed memory:

mysql> call `p2`('s s s s   , ,');
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+-------------------------------------------------------
| Level   | Code | Message
+---------+------+-------------------------------------------------------
| Warning | 1366 | Incorrect string value: '\xA5\xA5\xA5\xA5\xA5\xA5...' '
+---------+------+-------------------------------------------------------
1 row in set (0.00 sec)

under valgrind on 5.0.66a on linux there are errors. they are interrmittent though

Attachment: bug38469_valgrind_errors_and_testcase.txt (text/plain), 8.08 KiB.

setting to verified on 5.0.66a with the following testcase.  crash on linux is very hard to archieve with a simple testcase, but valgrind errors show the problem.

start mysqld-debug with clean datadir under valgrind with these options:

valgrind --tool=memcheck   --leak-check=yes -v --show-reachable=yes  ./bin/mysqld-debug  --skip-grant-tables --skip-name-resolve  --basedir=. --datadir=./data --default-character-set=utf8

now run:

mysql -uroot
use test
delimiter $
drop procedure if exists `p2` $
create procedure `p2`(in `a` text)
begin
        declare `pos` int default 1;
        declare `str` text;
        set `str` := `a`;
        select substr(`str`, `pos`+ 1 )into `str`;
end $
delimiter ;
call `p2`('s s s s   , ,');
show warnings;

See the above attached file for the output of valgrind.

bug #38674 is likely the same as this.

valgrind output from 5.1.28

Attachment: bug38469.txt (text/plain), 7.50 KiB.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54163

2677 Kristofer Pettersson	2008-09-15
      Bug#38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar
      
      A stored procedure involving substrings could crash the server on certain
      platforms because of invalid memory reads.
      
      This patch fixes this issue.

Observations made while debugging this:
* Where does the String object originate from? ie. where is the '*from' pointer pointing to?
 - `a` Item_splocal item:  

* Which Item is used? The 'field' will be Field_blob; what is the corresponding Item ?
    const Item *
    Item_splocal::this_item() const
    {
      DBUG_ASSERT(m_sp == m_thd->spcont->sp);

      return m_thd->spcont->get_item(m_var_idx); /* get value from value store */
    }
  - Item_string is used to represent the current value of `Str` and is wrapped by Item_splocal.   

* The variable value is either copied or referenced by `Str`-Item_splocal variable. rcontext::set_variable():
    int
    sp_rcontext::set_variable(THD *thd, Field *field, Item **value)
    {
      if (!value)
      {
        field->set_null();
        return 0;
      }

      return sp_eval_expr(thd, field, value);
    }
    Setting will be rcontext::set_variable -> ... -> Item_string::save_in_field -> Item_string::save_str_value_in_field -> Field_blob::store

* The Item_func_substr(..) is referencing the Item_splocal `Str`; a new temporary string is created which references Item_splocal `Str` using address 
  to cached value in associated Field member, Field_blob.

Is the String conversion happening at the right place? I think it should happen inside the String class or inside a String-help class attached to the String object, but we won't make this change today.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54404

2677 Kristofer Pettersson	2008-09-20
      Bug#38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar
                  
      A stored procedure involving substrings could crash the server on certain
      platforms because of invalid memory reads.
                
      During storing the new blob-field value, the cached value's address range
      overlapped that of the new field value. This caused problems when the 
      cached value storage was reallocated to provide access for a new 
      characater set representation. The patch checks the address ranges, and if
      they overlap, the new field value is copied to a new storage before it is
      converted to the new character set.

The patch looks fine.

Thanks for the comments added!

Noted in 5.1.29 changelog.

Stored procedures involving substrings could crash the server on
certain platforms due to invalid memory reads.  

Leaving report status unchanged; this is early documentation of an upcoming push into 5.1.29.

Pushed into 5.1.30  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:kpettersson@mysql.com-20080920145234-bxwpkhohzmar9njy) (pib:4)

Setting report to NDI pending push into 5.0.x, 6.0.x.

Pushed into 6.0.8-alpha  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:kpettersson@mysql.com-20080925114617-i25jmvw6nwwwcl0z) (pib:5)

Noted in 6.0.8 changelog.

Setting report to NDI pending push into 5.0.x.

Noted in 5.0.72 changelog.

Pushed into 5.1.29-ndb-6.2.17  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:tomas.ulin@sun.com-20081028140209-u4emkk1xphi5tkfb) (pib:5)

Pushed into 5.1.29-ndb-6.3.19  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:tomas.ulin@sun.com-20081028194045-0353yg8cvd2c7dd1) (pib:5)

Pushed into 5.1.29-ndb-6.4.0  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:jonas@mysql.com-20081101082305-qx5a1bj0z7i8ueys) (pib:5)

Bug #31107 is set as duplicate of this bug here.