Bug #57641 Substring out of range exception in ConsumeQuotedToken
Submitted: 21 Oct 2010 21:27 Modified: 12 Nov 2010 16:35
Reporter: Andreas Johansson Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / NET Severity:S3 (Non-critical)
Version:6.3.5 OS:Any
Assigned to: Tony Bedford CPU Architecture:Any
Tags: consumequotedtoken, out of range exception, substring, tracingdriver

[21 Oct 2010 21:27] Andreas Johansson
Description:
When the TracingDriver is used and the sql statement is longer than 300 characters and contains quoted token and the 300 character is in the middle of a quoted token it will throw ArgumentOutOfRangeException.

How to repeat:
For the TracingDriver to be used one of the following must be true.

MySqlTrace.QueryAnalysisEnabled
Logging or UseUsageAdvisor (set in querystring)

The SQL statement must be longer than 300 characters and 300th character in the middle of a quoted token

SELECT 1 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1`,  2 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2`, 3 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA3`, 4 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4`, 5 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA5`, 6 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA6`;

Open a connection with logging=true and create a command with the sql above and execute it with ExecuteNonQuery to reproduce.

Suggested fix:
Since the cut is arbitrary at 300 characters a possible solution would be just to ignore that there is no end quote for a quoted token and include it or just drop the incomplete token.

Since it only happens in the TracingDriver a workaround would be to make sure the following are all set to false:

MySqlTrace.QueryAnalysisEnabled (call MySqlTrace.DisableQueryAnalyzer())
Logging=false and UseUsageAdvisor=false (set in querystring)

Patch to use incomplete token
*** QueryNormalizer.cs	2010-10-21 23:12:14.000000000 +0200
--- QueryNormalizer.cs	2010-10-21 23:20:53.364904001 +0200
***************
*** 324,331 ****
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else

!                 tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }

  

          private void ConsumeUnquotedToken()

--- 324,331 ----
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else if (pos > fullSql.Length)

!                 tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start)));

          }

  

          private void ConsumeUnquotedToken()

Patch to drop incomplete token
*** QueryNormalizer.cs	2010-10-21 23:12:14.000000000 +0200
--- QueryNormalizer.cs	2010-10-21 23:17:42.924904001 +0200
***************
*** 324,330 ****
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else

                  tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }

  

--- 324,330 ----
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else if (pos <= fullSql.Length)

                  tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }
[21 Oct 2010 21:28] Andreas Johansson
Patch to allow incomplete quoted token

Attachment: IncompleteToken.patch (text/x-patch), 773 bytes.

[21 Oct 2010 21:28] Andreas Johansson
Drop the last token if it is quoted and incomplete

Attachment: DropToken.patch (text/x-patch), 695 bytes.

[21 Oct 2010 21:30] Andreas Johansson
Small app to reproduce

Attachment: Main.cs (text/x-csharp), 972 bytes.

[22 Oct 2010 19:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/121721

939 Reggie Burnett	2010-10-22
      - Fixed TracingDriver so that it normalizes long queries before truncation so we don't get exceptions when quoted
        tokens land on the 300th character (bug #57641)
[22 Oct 2010 19:03] Reggie Burnett
Fixed in 6.3.6+
[12 Nov 2010 16:35] Tony Bedford
An entry has been added to the 6.3.6 changelog:

When the tracing driver was used and a SQL statement was longer than 300 characters, an ArgumentOutOfRangeExcpetion occurred if the statement also contained a quoted character, and the 300th character was in the middle of a quoted token.