Bug #10452 Apostrophe (0x91 and 0x92) in Script not read by Query Browser
Submitted: 8 May 2005 18:52 Modified: 30 Sep 2005 9:11
Reporter: James Pearson Email Updates:
Status: Not a Bug Impact on me:
Category:MySQL Query Browser Severity:S3 (Non-critical)
Version:1.1.15 OS:Microsoft Windows (Windows XP Home)
Assigned to: Mike Lischke CPU Architecture:Any

[8 May 2005 18:52] James Pearson
When *opening* a script that contains apostrophes within a string, the apostrophes are removed.  This does not happen when running the script from the command line or if you paste the script into the Query toolbar.

A string in SQL is contained within the characters 0x27 so the popular bulletin board software phpbb replaces apostrophes with characters 0x91 and 0x92, but these are removed by Query Browser when opening a script.

This is a problem because in phpbb, strings are used as a Primary Key ('_search_wordlist' table) and by removing these characters "duplicate entry" errors are given when trying to run the script (e.g. if 'becoming' exists as a primary key and '‘becoming’' has its apostrophes removed).

How to repeat:
try *opening* a script with an insert command containing data with 0x91 and 0x92 characters, e.g.

INSERT INTO `table` VALUES ('‘becoming’');

Suggested fix:
Allow Query Browser to read files with 0x91 and 0x92 characters.
[26 Aug 2005 14:30] Norbert Heyermeyer
You have the same problem, when the script contains german umlaute (äöüß). These will also be removed from the script, when you open it.

When you copy the script into the edit window, these chars are processed correctly.
[26 Aug 2005 15:31] Michael G. Zinner
Did you open the file as UTF8, UTF16 or ANSI?
[26 Sep 2005 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[27 Sep 2005 10:05] James Pearson
I was opening the script using UTF-8.  Having now tried UTF-16 the characters 0x91 and 0x92 are read correctly.
[27 Sep 2005 12:14] Mike Lischke
James, can you please try again with utf-8? Does it work now? If not can you please attach the file you tried to open to this bug entry?

[27 Sep 2005 12:27] James Pearson
Example script with Apostrophes (0x91 and 0x92)

Attachment: insert.sql (text/plain), 44 bytes.

[27 Sep 2005 12:30] James Pearson
I have now installed version 1.1.15.  Using the attached file "Insert.sql" I get the following results:

ANSI       Correct
UTF-8     Missing Apostrophes
UTF-16   File does not load at all
[30 Sep 2005 9:11] Mike Lischke
The attached file is encoded using ANSI and hence it loads fine using the ANSI option in the open dialog. Since it is ANSI encoded it does not load well with any of the Unicode settings. So what do you expect? If you want to load this as UTF-8 file then convert it first to UTF-8 (e.g. by using Notepad). Btw. The editor is smart enough to detect a UTF-8 encoded file even if you specify ANSI in the open dialog (provided the file contains the recommended BOM header, which is usually automatically inserted by the text program, e.g. Notepad does this).