| Bug #58165 | "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes | ||
|---|---|---|---|
| Submitted: | 12 Nov 2010 14:25 | Modified: | 2 Mar 2011 3:27 |
| Reporter: | Shane Bester (Platinum Quality Contributor) | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: DML | Severity: | S1 (Critical) |
| Version: | 5.1.54, 5.5.7 | OS: | Any |
| Assigned to: | Martin Hansson | CPU Architecture: | Any |
| Tags: | LOAD DATA, my_empty_string, select into outfile | ||
[12 Nov 2010 14:48]
MySQL Verification Team
Here is the initialization of line_start and enclosed from the global variable
my_empty_string:
sql_exchange::sql_exchange(char *name,bool flag)
:file_name(name), opt_enclosed(0), dumpfile(flag), skip_lines(0)
{
field_term= &default_field_term;
enclosed= line_start= &my_empty_string;
[12 Nov 2010 15:08]
Andrii Nikitin
Verified on 5.1.52 Mac 10.5 64bit after several executions of testcase:
mysql [localhost] {msandbox} (test) > select 1 into outfile '/tmp/sdfsadfsd';
ERROR 1083 (42000): Field separator argument is not what is expected; check the manual
[12 Nov 2010 15:09]
Andrii Nikitin
full log
Attachment: corruption.log (application/octet-stream, text), 7.69 KiB.
[12 Nov 2010 15:26]
MySQL Verification Team
workaround is to always specify the 'fields enclosed by' and 'lines starting by' values explicitly
[15 Nov 2010 7:47]
MySQL Verification Team
my_empty_string destructor also crashes debug server at shutdown. it was a trigger to cause of bug 52353
[15 Dec 2010 11:00]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/126915 3525 Martin Hansson 2010-12-15 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes The string manipulating function insert() used a shared string buffer intended to always contain an empty string. Fixed by copying the buffer before modifying it. Relevant code has also been documented.
[16 Dec 2010 12:36]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127062 3525 Martin Hansson 2010-12-16 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes The string manipulating SQL function insert() used a shared string buffer intended to contain an immutable empty string causing errors. With our code base this design faced two obstacles. The implementations of string valued SQL functions return pointers to String objects rather than the objects themselves and even if this were rectified we still have no protection against manipulation of shared string buffers. Fixed by always allocating a new String object whenever a function returns an empty string. Relevant code has also been documented.
[22 Dec 2010 9:50]
MySQL Verification Team
How sure are we that this problem is limited to the INSERT() string function? Is there a way to catch this problem with a debug assertion or something so we can test all functions in an automated way?
[22 Dec 2010 10:57]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127496 3525 Martin Hansson 2010-12-22 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes The string manipulating SQL function insert() used a shared string buffer intended to contain an immutable empty string causing errors. With our code base this design faced two obstacles. The implementations of string valued SQL functions return pointers to String objects rather than the objects themselves and even if this were rectified we still have no protection against manipulation of shared string buffers. Fixed by always allocating a new String object whenever a function returns an empty string. Relevant code has also been documented.
[22 Dec 2010 12:30]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127501 3534 Martin Hansson 2010-12-22 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes Some string manipulating SQL functions use a shared string buffer intended to contain an immutable empty string. This buffer was used by the SQL function SUBSTRING_INDEX() to return an empty string when one argument was of the wrong datatype. If the string was then modified by the sql function INSERT(), undefined behavior ensued. Fixed by allocating a new String buffer when SUBSTRING_INDEX() returns an empty string due to errors, and returning NULL should it fail. Relevant code has also been documented.
[28 Dec 2010 8:26]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127616 3534 Martin Hansson 2010-12-28 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes Some string manipulating SQL functions use a shared string object intended to contain an immutable empty string. This objectt was used by the SQL function SUBSTRING_INDEX() to return an empty string when one argument was of the wrong datatype. If the string object was then modified by the sql function INSERT(), undefined behavior ensued. Fixed by instead modifying the string object representing the function's result value when SUBSTRING_INDEX() returns an empty string due to errors. Relevant code has also been documented.
[30 Dec 2010 9:19]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127704 3537 Martin Hansson 2010-12-30 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes Some string manipulating SQL functions use a shared string object intended to contain an immutable empty string. This object was used by the SQL function SUBSTRING_INDEX() to return an empty string when one argument was of the wrong datatype. If the string object was then modified by the sql function INSERT(), undefined behavior ensued. Fixed by instead modifying the string object representing the function's result value whenever string manipulating SQL functions return an empty string. Relevant code has also been documented.
[30 Dec 2010 9:41]
Alexander Barkov
http://lists.mysql.com/commits/127704 is Ok to push. Thanks for adding code comments!
[30 Dec 2010 10:47]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127707 3229 Martin Hansson 2010-12-30 [merge] Merge of fix for Bug#58165.
[30 Dec 2010 13:00]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/127717 3475 Martin Hansson 2010-12-30 [merge] Merge of fix for Bug#58165.
[13 Jan 2011 7:57]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/128585 3547 Martin Hansson 2011-01-13 Bug#58165: "my_empty_string" gets modified and causes LOAD DATA to fail and other crashes Some string manipulating SQL functions use a shared string object intended to contain an immutable empty string. This object was used by the SQL function SUBSTRING_INDEX() to return an empty string when one argument was of the wrong datatype. If the string object was then modified by the sql function INSERT(), undefined behavior ensued. Fixed by instead modifying the string object representing the function's result value whenever string manipulating SQL functions return an empty string. Relevant code has also been documented.
[13 Jan 2011 8:08]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/128586 3233 Martin Hansson 2011-01-13 [merge] Merge of fix for Bug#58165.
[13 Jan 2011 8:20]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/128587 3494 Martin Hansson 2011-01-13 [merge] Merge of fix for Bug#58165.
[13 Jan 2011 8:21]
Bugs System
Pushed into mysql-trunk 5.6.2 (revid:martin.hansson@oracle.com-20110113081952-n7mg2swpu1xb5fjp) (version source revid:martin.hansson@oracle.com-20110113081952-n7mg2swpu1xb5fjp) (merge vers: 5.6.2) (pib:24)
[13 Jan 2011 8:22]
Bugs System
Pushed into mysql-5.1 5.1.55 (revid:martin.hansson@oracle.com-20110113075715-2q5k9c2xk19lq2bg) (version source revid:martin.hansson@oracle.com-20110113075715-2q5k9c2xk19lq2bg) (merge vers: 5.1.55) (pib:24)
[13 Jan 2011 8:22]
Bugs System
Pushed into mysql-5.5 5.5.9 (revid:martin.hansson@oracle.com-20110113080721-bu92czxpaw9nht9k) (version source revid:martin.hansson@oracle.com-20110113080721-bu92czxpaw9nht9k) (merge vers: 5.5.9) (pib:24)
[2 Mar 2011 3:27]
Paul DuBois
Noted in 5.1.56, 5.5.10, 5.6.2 changelogs. Some string manipulating SQL functions use a shared string object intended to contain an immutable empty string. This object was used by the SQL function SUBSTRING_INDEX() to return an empty string when one argument was of the wrong datatype. If the string object was then modified by the SQL function INSERT(), undefined behavior ensued. CHANGESET - http://lists.mysql.com/commits/128587
Description: The symptoms of the problem are that LOAD DATA starts failing, and SELECT .. INTO OUTFILE fails with nonsensical errors, like this: ERROR 1083 (42000): Field separator argument is not what is expected; check the manual or warnings like: Warning | 1638 | Non-ASCII separator arguments are not fully supported LOAD DATA might simply refuse to load any rows and return no error. The reason for failure is "my_empty_string" variable in item_strfunc.cc is used as the default value for some of the sql_exchange items. But some string functions are [incorrectly?] modifying my_empty_string so that it is not empty anymore. Here is evidence of this modification: (gdb) watch my_empty_string.Ptr Hardware watchpoint 10: my_empty_string.Ptr (gdb) continue Continuing. [New Thread 0x7ffff40a8710 (LWP 22428)] [Switching to Thread 0x7ffff40a8710 (LWP 22428)] Hardware watchpoint 10: my_empty_string.Ptr Old value = 0xab09c6 "" New value = 0x7fffe00081a0 "" String::realloc (this=0xfd8a40, alloc_length=1) at sql_string.cc:89 89 Alloced_length= len; (gdb) bt #0 String::realloc (this=0xfd8a40, alloc_length=1) at sql_string.cc:89 #1 0x000000000065fadb in String::replace (this=0xfd8a40, offset=0, arg_length=0, to=0x7fffe00080d0 "1", to_length=1) at sql_string.cc:646 #2 0x000000000065f9f2 in String::replace (this=0xfd8a40, offset=0, arg_length=0, to=...) at sql_string.cc:626 #3 0x00000000005ea76b in Item_func_insert::val_str (this=0x7fffe0007850, str=0x7ffff40a5850) at item_strfunc.cc:1056 # Further down we see it is not empty: (gdb) print my_empty_string $32 = { Ptr = 0x7fffe00081a0 "1", str_length = 1, Alloced_length = 8, alloced = true, str_charset = 0xfd1460 } (gdb) How to repeat: you can start mysqld in gdb, hit ctrl-c to break into it. watch my_empty_string.Ptr continue Now in a mysql window, do this: drop table if exists t1; create table t1(a text)engine=myisam; select 'aaaaaaaaaaaaaa' into outfile 'in.txt'; load data infile 'in.txt' into table t1; show warnings; select load_file(substring((0x26),('98/12/31'),(1))); show warnings; select linefromwkb(insert((substring_index((0xfa),('xm4x'),('E\\Rm'))),((('') <= (''))),(degrees(';PvD')),(@@global.auto_increment_increment))); show warnings; load data infile 'in.txt' into table t1; show warnings; ----------- The second load data will not load anything... Suggested fix: there are too many places in the code where my_empty_string is returned, and might be subject to in adverse modification. It should be const really. Let me know if you need a more complete list of functions that might modify this value.