Bug #58165 "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes
Submitted: 12 Nov 2010 14:25 Modified: 2 Mar 2011 3:27
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: DML Severity:S1 (Critical)
Version:5.1.54, 5.5.7 OS:Any
Assigned to: Martin Hansson CPU Architecture:Any
Tags: LOAD DATA, my_empty_string, select into outfile
Triage: Triaged: D2 (Serious)

[12 Nov 2010 14:25] Shane Bester
Description:
The symptoms of the problem are that LOAD DATA starts failing, and SELECT .. INTO OUTFILE fails with nonsensical errors, like this:

ERROR 1083 (42000): Field separator argument is not what is expected; check the manual

or warnings like:

Warning | 1638 | Non-ASCII separator arguments are not fully supported

LOAD DATA might simply refuse to load any rows and return no error.

The reason for failure is "my_empty_string" variable in item_strfunc.cc is used as the default value for some of the sql_exchange items.

But some string functions are [incorrectly?] modifying my_empty_string so that it is not empty anymore.  Here is evidence of this modification:

(gdb) watch my_empty_string.Ptr
Hardware watchpoint 10: my_empty_string.Ptr
(gdb) continue
Continuing.
[New Thread 0x7ffff40a8710 (LWP 22428)]
[Switching to Thread 0x7ffff40a8710 (LWP 22428)]
Hardware watchpoint 10: my_empty_string.Ptr

Old value = 0xab09c6 ""
New value = 0x7fffe00081a0 ""
String::realloc (this=0xfd8a40, alloc_length=1) at sql_string.cc:89
89          Alloced_length= len;
(gdb) bt
#0  String::realloc (this=0xfd8a40, alloc_length=1) at sql_string.cc:89
#1  0x000000000065fadb in String::replace (this=0xfd8a40, offset=0, arg_length=0, to=0x7fffe00080d0 "1", to_length=1) at sql_string.cc:646
#2  0x000000000065f9f2 in String::replace (this=0xfd8a40, offset=0, arg_length=0, to=...) at sql_string.cc:626
#3  0x00000000005ea76b in Item_func_insert::val_str (this=0x7fffe0007850, str=0x7ffff40a5850) at item_strfunc.cc:1056
#

Further down we see it is not empty:

(gdb) print my_empty_string
$32 = {
  Ptr = 0x7fffe00081a0 "1", 
  str_length = 1, 
  Alloced_length = 8, 
  alloced = true, 
  str_charset = 0xfd1460
}
(gdb) 

How to repeat:
you can start mysqld in gdb, hit ctrl-c to break into it.

watch my_empty_string.Ptr
continue

Now in a mysql window, do this:

drop table if exists t1;
create table t1(a text)engine=myisam;
select 'aaaaaaaaaaaaaa' into outfile 'in.txt';
load data infile 'in.txt' into table t1;
show warnings;
select load_file(substring((0x26),('98/12/31'),(1)));
show warnings;
select linefromwkb(insert((substring_index((0xfa),('xm4x'),('E\\Rm'))),((('') <= (''))),(degrees(';PvD')),(@@global.auto_increment_increment)));
show warnings;
load data infile 'in.txt' into table t1;
show warnings;
-----------

The second load data will not load anything...

Suggested fix:
there are too many places in the code where my_empty_string is returned, and might be subject to in adverse modification.

It should be const really.  Let me know if you need a more complete list of 
functions that might modify this value.
[12 Nov 2010 14:48] Shane Bester
Here is the initialization of line_start and enclosed from the global variable
my_empty_string:

sql_exchange::sql_exchange(char *name,bool flag)
  :file_name(name), opt_enclosed(0), dumpfile(flag), skip_lines(0)
{
  field_term= &default_field_term;
  enclosed=   line_start= &my_empty_string;
[12 Nov 2010 15:08] Andrii Nikitin
Verified on 5.1.52  Mac 10.5 64bit after several executions of testcase:

mysql [localhost] {msandbox} (test) > select 1 into outfile '/tmp/sdfsadfsd';
ERROR 1083 (42000): Field separator argument is not what is expected; check the manual
[12 Nov 2010 15:09] Andrii Nikitin
full log

Attachment: corruption.log (application/octet-stream, text), 7.69 KiB.

[12 Nov 2010 15:26] Shane Bester
workaround is to always specify the 'fields enclosed by' and  'lines starting by' values explicitly
[15 Nov 2010 7:47] Shane Bester
my_empty_string destructor also crashes debug server at shutdown. it was a trigger to cause of bug 52353
[15 Dec 2010 11:00] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/126915

3525 Martin Hansson	2010-12-15
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      The string manipulating function insert() used a shared string buffer intended
      to always contain an empty string. Fixed by copying the buffer before
      modifying it.
      
      Relevant code has also been documented.
[16 Dec 2010 12:36] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127062

3525 Martin Hansson	2010-12-16
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      The string manipulating SQL function insert() used a shared string buffer
      intended to contain an immutable empty string causing errors. With our code
      base this design faced two obstacles. The implementations of string valued SQL
      functions return pointers to String objects rather than the objects themselves
      and even if this were rectified we still have no protection against
      manipulation of shared string buffers.
      
      Fixed by always allocating a new String object whenever a function returns an
      empty string.
      
      Relevant code has also been documented.
[22 Dec 2010 9:50] Shane Bester
How sure are we that this problem is limited to the INSERT() string function?  Is there a way to catch this problem with a debug assertion or something so we can test all functions in an automated way?
[22 Dec 2010 10:57] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127496

3525 Martin Hansson	2010-12-22
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      The string manipulating SQL function insert() used a shared string buffer
      intended to contain an immutable empty string causing errors. With our code
      base this design faced two obstacles. The implementations of string valued SQL
      functions return pointers to String objects rather than the objects themselves
      and even if this were rectified we still have no protection against
      manipulation of shared string buffers.
      
      Fixed by always allocating a new String object whenever a function returns an
      empty string.
      
      Relevant code has also been documented.
[22 Dec 2010 12:30] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127501

3534 Martin Hansson	2010-12-22
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      Some string manipulating SQL functions use a shared string buffer intended to
      contain an immutable empty string. This buffer was used by the SQL function
      SUBSTRING_INDEX() to return an empty string when one argument was of the wrong
      datatype. If the string was then modified by the sql function INSERT(),
      undefined behavior ensued.
      
      Fixed by allocating a new String buffer when SUBSTRING_INDEX() returns an
      empty string due to errors, and returning NULL should it fail.
      
      Relevant code has also been documented.
[28 Dec 2010 8:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127616

3534 Martin Hansson	2010-12-28
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      Some string manipulating SQL functions use a shared string object intended to
      contain an immutable empty string. This objectt was used by the SQL function
      SUBSTRING_INDEX() to return an empty string when one argument was of the wrong
      datatype. If the string object was then modified by the sql function INSERT(),
      undefined behavior ensued.
      
      Fixed by instead modifying the string object representing the function's
      result value when SUBSTRING_INDEX() returns an empty string due to errors.
      
      Relevant code has also been documented.
[30 Dec 2010 9:19] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127704

3537 Martin Hansson	2010-12-30
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      Some string manipulating SQL functions use a shared string object intended to
      contain an immutable empty string. This object was used by the SQL function
      SUBSTRING_INDEX() to return an empty string when one argument was of the wrong
      datatype. If the string object was then modified by the sql function INSERT(),
      undefined behavior ensued.
      
      Fixed by instead modifying the string object representing the function's
      result value whenever string manipulating SQL functions return an empty
      string.
      
      Relevant code has also been documented.
[30 Dec 2010 9:41] Alexander Barkov
http://lists.mysql.com/commits/127704 is Ok to push.

Thanks for adding code comments!
[30 Dec 2010 10:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127707

3229 Martin Hansson	2010-12-30 [merge]
      Merge of fix for Bug#58165.
[30 Dec 2010 13:00] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/127717

3475 Martin Hansson	2010-12-30 [merge]
      Merge of fix for Bug#58165.
[13 Jan 2011 7:57] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128585

3547 Martin Hansson	2011-01-13
      Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and
      other crashes
      
      Some string manipulating SQL functions use a shared string object intended to
      contain an immutable empty string. This object was used by the SQL function
      SUBSTRING_INDEX() to return an empty string when one argument was of the wrong
      datatype. If the string object was then modified by the sql function INSERT(),
      undefined behavior ensued.
      
      Fixed by instead modifying the string object representing the function's
      result value whenever string manipulating SQL functions return an empty
      string.
      
      Relevant code has also been documented.
[13 Jan 2011 8:08] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128586

3233 Martin Hansson	2011-01-13 [merge]
      Merge of fix for Bug#58165.
[13 Jan 2011 8:20] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128587

3494 Martin Hansson	2011-01-13 [merge]
      Merge of fix for Bug#58165.
[13 Jan 2011 8:21] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:martin.hansson@oracle.com-20110113081952-n7mg2swpu1xb5fjp) (version source revid:martin.hansson@oracle.com-20110113081952-n7mg2swpu1xb5fjp) (merge vers: 5.6.2) (pib:24)
[13 Jan 2011 8:22] Bugs System
Pushed into mysql-5.1 5.1.55 (revid:martin.hansson@oracle.com-20110113075715-2q5k9c2xk19lq2bg) (version source revid:martin.hansson@oracle.com-20110113075715-2q5k9c2xk19lq2bg) (merge vers: 5.1.55) (pib:24)
[13 Jan 2011 8:22] Bugs System
Pushed into mysql-5.5 5.5.9 (revid:martin.hansson@oracle.com-20110113080721-bu92czxpaw9nht9k) (version source revid:martin.hansson@oracle.com-20110113080721-bu92czxpaw9nht9k) (merge vers: 5.5.9) (pib:24)
[2 Mar 2011 3:27] Paul Dubois
Noted in 5.1.56, 5.5.10, 5.6.2 changelogs.

Some string manipulating SQL functions use a shared string object
intended to contain an immutable empty string. This object was used
by the SQL function SUBSTRING_INDEX() to return an empty string when
one argument was of the wrong datatype. If the string object was then
modified by the SQL function INSERT(), undefined behavior ensued.

CHANGESET - http://lists.mysql.com/commits/128587