Bug #42253 StringIndexOutOfBoundsException in EscapeProcessor.processTimeToken
Submitted: 21 Jan 2009 21:16 Modified: 23 Jun 2009 15:20
Reporter: Steve Revilak Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / J Severity:S2 (Serious)
Version:5.1.7 OS:Any
Assigned to: Jess Balint CPU Architecture:Any

[21 Jan 2009 21:16] Steve Revilak
Description:
MySQL Developers,

I'd like to report an unhandled StringIndexOutOfBoundsException in
Version 5.1.7 of MySQL Connector/J:

   java.lang.StringIndexOutOfBoundsException: String index out of range: -1
	at java.lang.String.substring(String.java:1938)
	at com.mysql.jdbc.EscapeProcessor.processTimeToken(EscapeProcessor.java:353)
	at com.mysql.jdbc.EscapeProcessor.escapeSQL(EscapeProcessor.java:257)
	at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1546)
	at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1524)

I will provide a small test case which reproduces the error.  I have
experienced this error with the following configurations:

* Connector/J version 5.1.7 on Mac OS X 10.5.6, using MySQL 5.0.51b,
  and java 1.6.0_07

* Connector/J version 5.1.5 on CentOS 4.4 (kernel: Linux
  2.6.9-42.0.10.ELsmp #1 SMP Tue Feb 27 10:11:19 EST 2007 i686 athlon
  i386 GNU/Linux), using MySQL 4.1.16 and java 1.6.0_06.

(The stack trace comes from the former Configuration, using
Connector/J 5.1.7 and java 1.6.0_07 -- "How to repeat" section uses
this configuration).

How to repeat:
Steps to reproduce:

(1) In MySQL's `test' database, create the following table:

------------------------------------------------------------------
CREATE TABLE `my_table` (
  `c1` date NOT NULL,
  `c2` varchar(32) NOT NULL default '',
  `c3` varchar(5) NOT NULL default '',
  `c4` tinyint(4) NOT NULL default '0',
  `c5` datetime NOT NULL,
  `c6` varchar(32) NOT NULL default '',
  `c7` varchar(32) NOT NULL default '',
  `c8` varchar(12) NOT NULL default '',
  `c9` varchar(32) NOT NULL default '',
  `c10` varchar(16) NOT NULL default '',
  `c11` varchar(96) character set latin1 collate latin1_bin NOT NULL default 'N/A',
  `c12` int(11) NOT NULL default '0',
  `c13` varchar(32) NOT NULL default '',
  `c14` int(10) unsigned default NULL,
  `c15` int(10) unsigned default NULL,
  `c16` varchar(28) character set latin1 collate latin1_bin NOT NULL default 'N/A',
  `c17` tinyint(4) NOT NULL default '0',
  `c18` decimal(9,6) default NULL,
  KEY `datestamp_idx` (`c1`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
------------------------------------------------------------------

(2) Save the following java code to the file 'Demo.java':

------------------------------------------------------------------
import java.io.IOException;
import java.io.BufferedReader;
import java.io.FileReader;
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.DriverManager;
import java.sql.Statement;

public class Demo {

    private String mFile;
    private String mHost;
    private String mUser;
    private String mPasswd;
    private String mDB;
    private String mQuery;

    private Demo(String file,
                 String host,
                 String user,
                 String passwd,
                 String db) {
        this.mFile = file;  /* read query from file */
        this.mHost = host;
        this.mUser = user;
        this.mPasswd = passwd;
        this.mDB = db;
    }

    public void doIt() throws IOException, SQLException, ClassNotFoundException {
        mQuery = readQuery();
        execQuery();
    }

    private String readQuery() throws IOException {
        BufferedReader r = new BufferedReader(new FileReader(mFile));
        StringBuilder sb = new StringBuilder();
        String line = null;
        while ((line = r.readLine()) != null) {
            sb.append(line);
        }
        r.close();
        return sb.toString();
    }

    private void execQuery() throws SQLException, ClassNotFoundException {
        System.out.println("loading driver");
        Class.forName("com.mysql.jdbc.Driver");

        System.out.println("making connection");
        Connection conn = DriverManager.getConnection(makeURL());
        
        System.out.println("executing update");
        Statement st = conn.createStatement();
        st.executeUpdate(mQuery);
        
        System.out.println("closing connection");
        st.close();
        conn.close();
    }

    private String makeURL() {
        return "jdbc:mysql://" + mHost + "/" + mDB
            + "?user=" + mUser + "&password=" + mPasswd;
    }

    public static void main(String argv[]) {
        if (argv.length != 5) {
            System.err.println("usage: java Demo FILENAME HOST USER PASSWD DBNAME");
            System.exit(1);
        }

        Demo demo = new Demo(argv[0], argv[1], argv[2], argv[3], argv[4]);

        try {
            demo.doIt();
        }
        catch (Throwable t) {
            t.printStackTrace();
            System.exit(1);
        }
        System.exit(0);
    }
}

------------------------------------------------------------------

(3) Compile Demo.java, with the command line "javac Demo.java"

(4) Save the following SQL statement to a file named "test.sql"

------------------------------------------------------------------
insert into my_table 
  (c1, c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12, c13, c16, c17, c18)  
values  
('2009-01-20','c2','c3','4','2009-01-20 17:36:25.0','c6','c7','c8','c9','c10','\'c_11_+_c11\'','12','c13','c16','17','18.1'),
('2009-01-20','c2','c3','4','2009-01-20 19:37:05.0','c6','c7','c8','c9','c10','{tgv_c11\'s}/c11/c11','12','c13','c16','17','18.2')
------------------------------------------------------------------

(5) Run Demo.java with the command line below (substituting "HOST",
    "USER", and "PASSWD" with your own hostname, username, and
    password).

    $ java -cp .:mysql-connector-java-5.1.7-bin.jar Demo test.sql HOST USER PASSWD test
    loading driver
    making connection
    executing update
    java.lang.StringIndexOutOfBoundsException: String index out of range: -1
    	at java.lang.String.substring(String.java:1938)
    	at com.mysql.jdbc.EscapeProcessor.processTimeToken(EscapeProcessor.java:353)
    	at com.mysql.jdbc.EscapeProcessor.escapeSQL(EscapeProcessor.java:257)
    	at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1546)
    	at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1524)
    	at Demo.execQuery(Demo.java:62)
    	at Demo.doIt(Demo.java:37)
    	at Demo.main(Demo.java:83)

It appears that the JDBC driver is trying to treat the quoted string
"{tgv_c11\'s}/c11/c11" as a timestamp escape.

Contrast this behavior with the following.  Below, I'll source
"test.sql" with the mysql command line client, and the insert
statement will succeed.

   $ mysql -u USER -pPASSWD -h HOST test
   Reading table information for completion of table and column names
   You can turn off this feature to get a quicker startup with -A
   
   Welcome to the MySQL monitor.  Commands end with ; or \g.
   Your MySQL connection id is 310
   Server version: 5.0.51b-log MySQL Community Server (GPL)
   
   Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
   
   test> source test.sql
   Query OK, 2 rows affected (0.00 sec)
   Records: 2  Duplicates: 0  Warnings: 0
   
   test> select * from my_table;
   oatmeal:test> select * from my_table;
   +------------+----+----+----+---------------------+----+----+----+----+-----+---------------------+-----+-----+------+------+-----+-----+-----------+
   | c1         | c2 | c3 | c4 | c5                  | c6 | c7 | c8 | c9 | c10 | c11                 | c12 | c13 | c14  | c15  | c16 | c17 | c18       |
   +------------+----+----+----+---------------------+----+----+----+----+-----+---------------------+-----+-----+------+------+-----+-----+-----------+
   | 2009-01-20 | c2 | c3 |  4 | 2009-01-20 17:36:25 | c6 | c7 | c8 | c9 | c10 | 'c_11_+_c11'        |  12 | c13 | NULL | NULL | c16 |  17 | 18.100000 | 
   | 2009-01-20 | c2 | c3 |  4 | 2009-01-20 19:37:05 | c6 | c7 | c8 | c9 | c10 | {tgv_c11's}/c11/c11 |  12 | c13 | NULL | NULL | c16 |  17 | 18.200000 | 
   +------------+----+----+----+---------------------+----+----+----+----+-----+---------------------+-----+-----+------+------+-----+-----+-----------+

The behavior of the mysql command-line client appears correct.  I
would expect the same behavior from Connector/J, but Connector/J
throws a StringIndexOutOfBoundsException instead.

Suggested fix:
Unfortunately, I have not found a workaround, but I would welcome any suggestions.
[21 Jan 2009 21:21] Steve Revilak
SQL for create table statement

Attachment: create.sql (application/octet-stream, text), 822 bytes.

[21 Jan 2009 21:22] Steve Revilak
Java program to reproduce StringIndexOutOfBoundsException

Attachment: Demo.java (text/plain), 2.27 KiB.

[21 Jan 2009 21:22] Mark Matthews
A work around would be to set escape processing to "false" on the statement (Statement.setEscapeProcessing(false)).

It appears the tokenizer is getting confused on \' entries in your SQL.
[21 Jan 2009 21:24] Steve Revilak
Extended Insert statement that causes the StringIndexOutOfBoundsException

Attachment: test.sql (application/octet-stream, text), 364 bytes.

[21 Jan 2009 21:34] Steve Revilak
Mark,

Thanks for you quick response.  I will try your suggestion.
[21 Jan 2009 22:01] Steve Revilak
With Statement.setEscapeProcessing(false) set, the StringIndexOutOfBoundsException does not occur.
[22 Jan 2009 10:03] Tonci Grgin
Steve, verified as described using latest snapshot (5.1-nightly-20090122):
loading driver
making connection
executing update
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
        at java.lang.String.substring(String.java:1768)
        at com.mysql.jdbc.EscapeProcessor.processTimeToken(EscapeProcessor.java:
353)
        at com.mysql.jdbc.EscapeProcessor.escapeSQL(EscapeProcessor.java:257)
        at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1549)
        at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1527)
        at Demo.execQuery(Demo.java:55)
        at Demo.doIt(Demo.java:32)
        at Demo.main(Demo.java:76)
[9 Feb 2009 21:08] Jess Balint
fix + test

Attachment: bug42253.diff (text/x-diff), 1.45 KiB.

[2 Mar 2009 18:40] Jess Balint
Bug#31457 marked as a duplicate of this.
[2 Jun 2009 5:59] Jess Balint
Pushed for release in 5.1.8
[23 Jun 2009 15:20] Tony Bedford
An entry has been added to the 5.1.8 changelog:

Connector/J generated an unhandled StringIndexOutOfBoundsException:

java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1938)
at com.mysql.jdbc.EscapeProcessor.processTimeToken(EscapeProcessor.java:353)
at com.mysql.jdbc.EscapeProcessor.escapeSQL(EscapeProcessor.java:257)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1546)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1524)