Bug #38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar
Submitted: 30 Jul 2008 18:21 Modified: 20 Oct 2008 15:31
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Stored Routines Severity:S1 (Critical)
Version:5.0.66a, mysql-5.1.26-rc-linux-i686-glibc23, 5.1.28 OS:Any
Assigned to: Kristofer Pettersson CPU Architecture:Any
Triage: Triaged: D1 (Critical)

[30 Jul 2008 18:21] Shane Bester
Description:
calling a stored procedure intermittently crashes with stack trace similar to:

mysqld-nt.exe!my_utf8_uni
mysqld-nt.exe!my_well_formed_len_mb
mysqld-nt.exe!well_formed_copy_nchars
mysqld-nt.exe!Field_blob::store
mysqld-nt.exe!Item::save_in_field
mysqld-nt.exe!sp_eval_expr
mysqld-nt.exe!sp_rcontext::set_variable
mysqld-nt.exe!select_dumpvar::send_data
mysqld-nt.exe!JOIN::exec
mysqld-nt.exe!mysql_select
mysqld-nt.exe!handle_select
mysqld-nt.exe!mysql_execute_command
mysqld-nt.exe!sp_instr_stmt::exec_core
mysqld-nt.exe!sp_lex_keeper::reset_lex_and_exec_core
mysqld-nt.exe!sp_instr_stmt::execute
mysqld-nt.exe!sp_head::execute
mysqld-nt.exe!sp_head::execute_procedure
mysqld-nt.exe!mysql_execute_command
mysqld-nt.exe!mysql_parse
mysqld-nt.exe!dispatch_command
mysqld-nt.exe!do_command
mysqld-nt.exe!handle_one_connection
mysqld-nt.exe!pthread_start
mysqld-nt.exe!_threadstart

How to repeat:
testcase is on it's way
[30 Jul 2008 18:23] Shane Bester
full stack1

Attachment: bug38469_full_stack_5.0.66a.txt (text/plain), 3.56 KiB.

[30 Jul 2008 20:52] Shane Bester
memory overruns are sensitive to compilers, and platform differences.  at least on windows 5.0.66a, the following testcase shows a problem:

---------
delimiter $
drop procedure if exists `p2` $
create procedure `p2`(in `a` text)
begin
        declare `pos` int default 1;
        declare `str` text;
        set `str` := `a`;
        select substr(`str`, `pos`+ 1 )into `str`;
end $
delimiter ;
call `p2`('s s s s   , ,');
show warnings;
---------

notice the output of 'show warnings' is pointing to uninitialized or freed memory:

mysql> call `p2`('s s s s   , ,');
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show warnings;
+---------+------+-------------------------------------------------------
| Level   | Code | Message
+---------+------+-------------------------------------------------------
| Warning | 1366 | Incorrect string value: '\xA5\xA5\xA5\xA5\xA5\xA5...' '
+---------+------+-------------------------------------------------------
1 row in set (0.00 sec)
[31 Jul 2008 8:00] Shane Bester
under valgrind on 5.0.66a on linux there are errors. they are interrmittent though

Attachment: bug38469_valgrind_errors_and_testcase.txt (text/plain), 8.08 KiB.

[31 Jul 2008 8:10] Shane Bester
setting to verified on 5.0.66a with the following testcase.  crash on linux is very hard to archieve with a simple testcase, but valgrind errors show the problem.

start mysqld-debug with clean datadir under valgrind with these options:

valgrind --tool=memcheck   --leak-check=yes -v --show-reachable=yes  ./bin/mysqld-debug  --skip-grant-tables --skip-name-resolve  --basedir=. --datadir=./data --default-character-set=utf8

now run:

mysql -uroot
use test
delimiter $
drop procedure if exists `p2` $
create procedure `p2`(in `a` text)
begin
        declare `pos` int default 1;
        declare `str` text;
        set `str` := `a`;
        select substr(`str`, `pos`+ 1 )into `str`;
end $
delimiter ;
call `p2`('s s s s   , ,');
show warnings;

See the above attached file for the output of valgrind.
[8 Aug 2008 20:50] Shane Bester
bug #38674 is likely the same as this.
[15 Sep 2008 8:07] Shane Bester
valgrind output from 5.1.28

Attachment: bug38469.txt (text/plain), 7.50 KiB.

[15 Sep 2008 21:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54163

2677 Kristofer Pettersson	2008-09-15
      Bug#38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar
      
      A stored procedure involving substrings could crash the server on certain
      platforms because of invalid memory reads.
      
      This patch fixes this issue.
[19 Sep 2008 13:25] Kristofer Pettersson
Observations made while debugging this:
* Where does the String object originate from? ie. where is the '*from' pointer pointing to?
 - `a` Item_splocal item:  

* Which Item is used? The 'field' will be Field_blob; what is the corresponding Item ?
    const Item *
    Item_splocal::this_item() const
    {
      DBUG_ASSERT(m_sp == m_thd->spcont->sp);

      return m_thd->spcont->get_item(m_var_idx); /* get value from value store */
    }
  - Item_string is used to represent the current value of `Str` and is wrapped by Item_splocal.   

* The variable value is either copied or referenced by `Str`-Item_splocal variable. rcontext::set_variable():
    int
    sp_rcontext::set_variable(THD *thd, Field *field, Item **value)
    {
      if (!value)
      {
        field->set_null();
        return 0;
      }

      return sp_eval_expr(thd, field, value);
    }
    Setting will be rcontext::set_variable -> ... -> Item_string::save_in_field -> Item_string::save_str_value_in_field -> Field_blob::store

* The Item_func_substr(..) is referencing the Item_splocal `Str`; a new temporary string is created which references Item_splocal `Str` using address 
  to cached value in associated Field member, Field_blob.

Is the String conversion happening at the right place? I think it should happen inside the String class or inside a String-help class attached to the String object, but we won't make this change today.
[20 Sep 2008 8:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/54404

2677 Kristofer Pettersson	2008-09-20
      Bug#38469 invalid memory read and/or crash with utf8 text field, stored procedure, uservar
                  
      A stored procedure involving substrings could crash the server on certain
      platforms because of invalid memory reads.
                
      During storing the new blob-field value, the cached value's address range
      overlapped that of the new field value. This caused problems when the 
      cached value storage was reallocated to provide access for a new 
      characater set representation. The patch checks the address ranges, and if
      they overlap, the new field value is copied to a new storage before it is
      converted to the new character set.
[20 Sep 2008 12:47] Alexander Barkov
The patch looks fine.

Thanks for the comments added!
[7 Oct 2008 19:21] Paul Dubois
Noted in 5.1.29 changelog.

Stored procedures involving substrings could crash the server on
certain platforms due to invalid memory reads.  

Leaving report status unchanged; this is early documentation of an upcoming push into 5.1.29.
[9 Oct 2008 18:09] Bugs System
Pushed into 5.1.30  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:kpettersson@mysql.com-20080920145234-bxwpkhohzmar9njy) (pib:4)
[9 Oct 2008 18:39] Paul Dubois
Setting report to NDI pending push into 5.0.x, 6.0.x.
[17 Oct 2008 16:45] Bugs System
Pushed into 6.0.8-alpha  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:kpettersson@mysql.com-20080925114617-i25jmvw6nwwwcl0z) (pib:5)
[17 Oct 2008 17:50] Paul Dubois
Noted in 6.0.8 changelog.

Setting report to NDI pending push into 5.0.x.
[20 Oct 2008 15:31] Paul Dubois
Noted in 5.0.72 changelog.
[28 Oct 2008 21:05] Bugs System
Pushed into 5.1.29-ndb-6.2.17  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:tomas.ulin@sun.com-20081028140209-u4emkk1xphi5tkfb) (pib:5)
[28 Oct 2008 22:23] Bugs System
Pushed into 5.1.29-ndb-6.3.19  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:tomas.ulin@sun.com-20081028194045-0353yg8cvd2c7dd1) (pib:5)
[1 Nov 2008 9:49] Bugs System
Pushed into 5.1.29-ndb-6.4.0  (revid:kpettersson@mysql.com-20080920085103-z6pem6ae29r0fuir) (version source revid:jonas@mysql.com-20081101082305-qx5a1bj0z7i8ueys) (pib:5)
[7 Nov 2008 16:18] Susanne Ebrecht
Bug #31107 is set as duplicate of this bug here.