Bug #51876 crash/memory underrun when loading data with ucs2 and reverse() function

• Submitted: 9 Mar 2010 16:45
• Modified: 14 Oct 2010 14:18
• Reporter: Shane Bester (Platinum Quality Contributor)
• Status: Closed
• Category: MySQL Server: Charsets
• Severity: S1 (Critical)
• Version: 5.0.90,5.1.44, 5.5.2-m2, 5.6.99
• OS: Any
• Assigned to: Georgi Kodinov
• CPU Architecture: Any
• Tags: reverse, ucs2, valgrind

[9 Mar 2010 16:45] Shane Bester

Description:
5.1.44 valgrind output:

Invalid write of size 1                                        
at: memcpy (mc_replace_strmem.c:482)                           
by: Item_func_reverse::val_str(String*) (item_strfunc.cc:827)  
by: Item::save_in_field(Field*, bool) (item.cc:5106)           
by: fill_record_n_invoke_before_triggers (sql_base.cc:8156)    
by: mysql_load (sql_load.cc:995)                               
by: mysql_execute_command(THD*) (sql_parse.cc:3459)            
by: mysql_parse (sql_parse.cc:5971)                            
by: dispatch_command (sql_parse.cc:1233)                       
by: do_command(THD*) (sql_parse.cc:874)                        
by: handle_one_connection (sql_connect.cc:1127)                
by: start_thread (in /lib64/libpthread-2.5.so)                 
by: clone (in /lib64/libc-2.5.so)                              
 Address 0x5eedbdf is 1 bytes before a block of size 8 alloc'd 
at: malloc (vg_replace_malloc.c:195)                           
by: my_malloc (my_malloc.c:34)                                 
by: String::realloc(unsigned int) (sql_string.cc:77)           
by: Item_func_reverse::val_str(String*) (item_strfunc.cc:807)  
by: Item::save_in_field(Field*, bool) (item.cc:5106)           
by: fill_record_n_invoke_before_triggers (sql_base.cc:8156)    
by: mysql_load (sql_load.cc:995)                               
by: mysql_execute_command(THD*) (sql_parse.cc:3459)            
by: mysql_parse (sql_parse.cc:5971)                            
by: dispatch_command (sql_parse.cc:1233)                       
by: do_command(THD*) (sql_parse.cc:874)                        
by: handle_one_connection (sql_connect.cc:1127)                
by: start_thread (in /lib64/libpthread-2.5.so)                 
by: clone (in /lib64/libc-2.5.so)                              

Windows debug binaries print safemalloc errors to the error log.

How to repeat:
#run server under valgrind, or use debug build

select '00' union select '10' into outfile 'tmpp.txt';
drop table if exists `t1`;
create table `t1`(`a` int)engine=myisam;
load data infile 'tmpp.txt'into table `t1`
character set ucs2 
(@`b`) set `a`=reverse(@`b`);

[9 Mar 2010 21:37] Sveta Smirnova

Thank you report.

Verified as described. Valgrind started as described in bug #51001.

[16 Mar 2010 0:10] Omer Barnir

triage: setting tag to SR51MRU. SR55RC (valgrind)

[24 Mar 2010 15:58] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/104229

3413 Georgi Kodinov	2010-03-24
      Bug #51876: crash/memory underrun when loading data with ucs2
      and reverse() function
      
      3 problems fixed : 
      1. The reported problem : caused by incorrect parsing of 
      the file as ucs data resulting in wrong length of the parsed
      string. Fixed by truncating the invalid trailing bytes 
      (non-complete multibyte characters) when reading from the file
      2. LOAD DATA when reading from a proper UCS2 file wasn't 
      recognizing the new line characters. Fixed by first looking 
      if a byte is a new line (or any other special) character before
      reading it as a part of a multibyte character.
      3. When using user variables to hold the column data in LOAD
      DATA the character set of the user variable was set incorrectly
      to the database charset. Fixed by setting it to the charset
      specified by LOAD DATA (if any).

[30 Jun 2010 6:20] Alexander Barkov

Ok to push, as quick solution to fix crash in 5.1.

We'll need to do a real fix into some development version soon,
to make separators use 2 bytes for both "SELECT INTO OUTFILE"
and "LOAD DATA INFILE" in case of ucs2.

[14 Jul 2010 11:25] Georgi Kodinov

The remark for the full ucs2 compliance is spawn off as bug #55254

[14 Jul 2010 11:55] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/113555

3481 Georgi Kodinov	2010-07-14
      Bug #51876: crash/memory underrun when loading data with ucs2
      and reverse() function
            
      3 problems fixed : 
      1. The reported problem : caused by incorrect parsing of 
      the file as ucs data resulting in wrong length of the parsed
      string. Fixed by truncating the invalid trailing bytes 
      (non-complete multibyte characters) when reading from the file
      2. LOAD DATA when reading from a proper UCS2 file wasn't 
      recognizing the new line characters. Fixed by first looking 
      if a byte is a new line (or any other special) character before
      reading it as a part of a multibyte character.
      3. When using user variables to hold the column data in LOAD
      DATA the character set of the user variable was set incorrectly
      to the database charset. Fixed by setting it to the charset
      specified by LOAD DATA (if any).

[23 Jul 2010 12:23] Bugs System

Pushed into mysql-trunk 5.5.6-m3 (revid:[e-mail]-20100723121820-jryu2fuw3pc53q9w) (version source revid:[e-mail]-20100531152341-x2d4hma644icamh1) (merge vers: 5.5.5-m3) (pib:18)

[23 Jul 2010 12:30] Bugs System

Pushed into mysql-next-mr (revid:[e-mail]-20100723121929-90e9zemk3jkr2ocy) (version source revid:[e-mail]-20100531152341-x2d4hma644icamh1) (pib:18)

[30 Jul 2010 2:43] Paul DuBois

Noted in 5.1.50, 5.5.6 changelogs.

Reading a ucs2 data file with LOAD DATA INFILE was subject to three
problems. 1) Incorrect parsing of the file as ucs2 data, resulting in
incorrect length of the parsed string. This is fixed by truncating
the invalid trailing bytes (incomplete multibyte characters) when
reading from the file. 2) Reads from a proper ucs2 file did not
recognize newline characters. This is fixed by first checking whether 
a byte is a newline (or any other special character) before reading 
it as a part of a multibyte character. 3) When using user variables
to hold column data, the character set of the user variable was set
incorrectly to the database charset. This is fixed by setting it to
the character set specified in the LOAD DATA INFILE statement, if
any.

[19 Aug 2010 15:42] Bugs System

Pushed into mysql-5.1 5.1.51 (revid:[e-mail]-20100819151858-muaaor6jojb5ouzj) (version source revid:[e-mail]-20100819151858-muaaor6jojb5ouzj) (merge vers: 5.1.51) (pib:20)

[14 Oct 2010 8:27] Bugs System

Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:[e-mail]-20101014082627-jrmy9xbfbtrebw3c) (version source revid:[e-mail]-20101014082627-jrmy9xbfbtrebw3c) (merge vers: 5.1.51-ndb-7.0.20) (pib:21)

[14 Oct 2010 8:42] Bugs System

Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.39 (revid:[e-mail]-20101014083757-5qo48b86d69zjvzj) (version source revid:[e-mail]-20101014083757-5qo48b86d69zjvzj) (merge vers: 5.1.51-ndb-6.3.39) (pib:21)

[14 Oct 2010 8:57] Bugs System

Pushed into mysql-5.1-telco-6.2 5.1.51-ndb-6.2.19 (revid:[e-mail]-20101014084420-y54ecj85j5we27oa) (version source revid:[e-mail]-20101014084420-y54ecj85j5we27oa) (merge vers: 5.1.51-ndb-6.2.19) (pib:21)

[14 Oct 2010 14:18] Jon Stephens

Already documented in the 5.1.50 changelog; no additional changelog entries required. Set back to Closed state.