Bug #54614 MySqlTokenizer.FindToken() cannot use operators - and / with quoted identifiers

• Submitted: 18 Jun 2010 11:21
• Modified: 20 Jun 2010 11:04
• Reporter: göktuğ akgün
• Status: Closed
• Category: Connector / NET
• Severity: S2 (Serious)
• Version: 6.3.1
• OS: Windows (vista)
• CPU Architecture: Any
• Tags: MySqlTokenizer FindToken() Comment Operator ReadSpecialToken()

[18 Jun 2010 11:21] göktuğ akgün

Description:
This is not special bug for me but I couldn't find it in this bug system.

MySqlTokenizer.FindToken() cannot get tokens correctly when use ansi quotes after operators - and /. Then it fails by
"Index and length must refer to a location within the string." at
line 136: string token = sql.Substring(startIndex, stopIndex - startIndex).Trim();
in NextToken(), MySqlTokenizer.cs.

When I use breakpoint at line 137: return token; see that some tokens are incorrect like in this command:

..
`a`-`b` AS `c`,
..

in this command, FindToken() gets these tokens:
`Per. TL`
-`
Alis
TL`
..

How to repeat:
CREATE TABLE deneme (a int,b int);
insert deneme values (3,1),(4,5);

select `a`-`b` from deneme;
SELECT `a`/`b` from deneme;

Suggested fix:
I changed the code in FindToken() like this:
        public bool FindToken()
        {
            isComment = quoted = false;  // reset our flags
            startIndex = stopIndex = -1;

            while (pos < sql.Length)
            {
                char c = sql[pos++];
                if (Char.IsWhiteSpace(c)) continue;
                
                if (c == '`' || c == '\'' || c == '"' || (c == '[' && SqlServerMode))
                    ReadQuotedToken(c);
                else if (c == '#' || c == '-' || c == '/')
                {
					if( !ReadComment( c ) )
						//ReadSpecialToken(); !!!CHANGED!
						ReadUnquotedToken();
                }
                else
                    ReadUnquotedToken();
                if (startIndex != -1) return true;
            }
            return false;
        }

and it works now. And it works unquoted identifiers as well (`a`-`b` and a-b).

[18 Jun 2010 11:31] göktuğ akgün

I changed my previous message
"`Per. TL`-`Alis TL` AS `Birim Kar`,"
to "`a`-`b` AS `c`," and didn't change the tokens produced, they should be:

`a`
-`
b`
AS
`c`

[20 Jun 2010 11:04] göktuğ akgün

It is solved in 6.3.2.