MySQL Bugs: #57641: Substring out of range exception in ConsumeQuotedToken

Bug #57641	Substring out of range exception in ConsumeQuotedToken
Submitted:	21 Oct 2010 21:27	Modified:	12 Nov 2010 16:35
Reporter:	Andreas Johansson	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Connector / NET	Severity:	S3 (Non-critical)
Version:	6.3.5	OS:	Any
Assigned to:	Tony Bedford	CPU Architecture:	Any
Tags:	consumequotedtoken, out of range exception, substring, tracingdriver

Description:
When the TracingDriver is used and the sql statement is longer than 300 characters and contains quoted token and the 300 character is in the middle of a quoted token it will throw ArgumentOutOfRangeException.

How to repeat:
For the TracingDriver to be used one of the following must be true.

MySqlTrace.QueryAnalysisEnabled
Logging or UseUsageAdvisor (set in querystring)

The SQL statement must be longer than 300 characters and 300th character in the middle of a quoted token

SELECT 1 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA1`,  2 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2`, 3 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA3`, 4 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA4`, 5 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA5`, 6 AS `AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA6`;

Open a connection with logging=true and create a command with the sql above and execute it with ExecuteNonQuery to reproduce.

Suggested fix:
Since the cut is arbitrary at 300 characters a possible solution would be just to ignore that there is no end quote for a quoted token and include it or just drop the incomplete token.

Since it only happens in the TracingDriver a workaround would be to make sure the following are all set to false:

MySqlTrace.QueryAnalysisEnabled (call MySqlTrace.DisableQueryAnalyzer())
Logging=false and UseUsageAdvisor=false (set in querystring)

Patch to use incomplete token
*** QueryNormalizer.cs	2010-10-21 23:12:14.000000000 +0200
--- QueryNormalizer.cs	2010-10-21 23:20:53.364904001 +0200
***************
*** 324,331 ****
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else

!                 tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }

  

          private void ConsumeUnquotedToken()

--- 324,331 ----
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else if (pos > fullSql.Length)

!                 tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start)));

          }

  

          private void ConsumeUnquotedToken()

Patch to drop incomplete token
*** QueryNormalizer.cs	2010-10-21 23:12:14.000000000 +0200
--- QueryNormalizer.cs	2010-10-21 23:17:42.924904001 +0200
***************
*** 324,330 ****
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else

                  tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }

  

--- 324,330 ----
              pos++;

              if (c == '\'')

                  tokens.Add(new Token(TokenType.String, "?"));

!             else if (pos <= fullSql.Length)

                  tokens.Add(new Token(TokenType.Identifier, fullSql.Substring(start, pos - start)));

          }

Patch to allow incomplete quoted token

Attachment: IncompleteToken.patch (text/x-patch), 773 bytes.

Drop the last token if it is quoted and incomplete

Attachment: DropToken.patch (text/x-patch), 695 bytes.

Small app to reproduce

Attachment: Main.cs (text/x-csharp), 972 bytes.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/121721

939 Reggie Burnett	2010-10-22
      - Fixed TracingDriver so that it normalizes long queries before truncation so we don't get exceptions when quoted
        tokens land on the 300th character (bug #57641)

Fixed in 6.3.6+

An entry has been added to the 6.3.6 changelog:

When the tracing driver was used and a SQL statement was longer than 300 characters, an ArgumentOutOfRangeExcpetion occurred if the statement also contained a quoted character, and the 300th character was in the middle of a quoted token.