Bug #7563 Query Browser won't execute script output from mysqldump with quotes in fields
Submitted: 28 Dec 2004 22:03 Modified: 29 Dec 2004 22:27
Reporter: John McClenahan Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Query Browser Severity:S2 (Serious)
Version:1.1.4 OS:Linux (Fedora Core 3, fully patched)
Assigned to: CPU Architecture:Any

[28 Dec 2004 22:03] John McClenahan
Description:
I have a mysql dump backup of our website MySQL database tables (see www.abbeytheatre.org/programme/prog0102.htm for the formatted output including the entries below which give me trouble!). It contains a mixture of numeric, date/time and text fields. Some of the text fields contain apostrophes (single quotes) in the text. The dump file 'escapes' the quote marks (precedes them with a \ character), as expected.

When I use File/Load Script, and execute the dumped file in MySQL Query Browser, I get an error when it attempts to INSERT the first record which has such an escaped quote mark. It sees the quote in the field as an unmatched quote opening a string, in spite of the escaping \ , and then can't read the rest of the INSERT statement properly. Syntax highlighting in the Query Browser script window shows the problem with the selection shown below.

If I edit the field so as to put a second quote mark after the escaped one, the rest of the INSERT line then parses correctly.

This didn't happen on the Windows machine where I did the equivalent, with the same SQL dumped file, but I think I may then have been using MySQL Control Centre to re-run the dump file.

The error complaining about invalid SQL Syntax appears to refer to the second SQL clause of the selection below when executed although it is the first statement which causes the error - and the error message suggests that it is reading the following line as starting a new SQL statement with the word NSERT, not INSERT.

The error is first triggered in the field that contains the words ... our Island\'s story ... (line 1848 in the whole SQL dump file of 31,000-odd lines) - the first INSERT statement below).

A similar error is caused a couple of statements  later by the phrase ...Hans Christian Andersen\'s The Ugly Duckling ...

Our web hosting company is using version 4.0.17 of MySQL server, and I am using here v4.0.23, with v1.1.4 of the Query Browser, which was the latest RPM version a few days ago.

I don't know where the mismatch between writing and reading the quotes is occuring. The file I downloaded seems to be in the latin1 character set, while Query Browser will only read UTF8 - I converted when prompted.

Any other suggestions for a workaround, please?

John McClenahan

How to repeat:
-- Table structure for table `events`
--

CREATE TABLE events (
  ID int(11) NOT NULL default '0',
  ID_old int(11) default NULL,
  DateFrom datetime default NULL,
  DateTo datetime default NULL,
  Start_sale datetime default NULL,
  End_sale datetime default NULL,
  name mediumtext,
  author mediumtext,
  Status smallint(6) default NULL,
  venue_ID int(11) default NULL,
  event_type_ID int(11) default NULL,
  description mediumtext,
  ImgSource mediumtext,
  link_desc_1 mediumtext,
  link_1 mediumtext,
  link_desc_2 mediumtext,
  link_2 mediumtext,
  link_desc_3 mediumtext,
  link_3 mediumtext,
  link_desc_4 mediumtext,
  link_4 mediumtext,
  link_desc_5 mediumtext,
  link_5 mediumtext,
  PRIMARY KEY  (ID)
) TYPE=MyISAM;

--
-- Dumping data for table `events` (critical few lines)
INSERT INTO events VALUES (700,NULL,'2002-09-09 00:00:00','2002-09-14 00:00:00',NULL,'2002-09-14 00:00:00','May It Please Your Majesty',NULL,0,5,NULL,'The Tudor Sisters, Stuart Sisters, Victoria and our present Queen have all played an important part in our Island\'s story. Five are found towards the end of their lives recalling significant moments during their reigns, as death and the inevitable funeral beckons. In the case of our present Queen it is her courtiers and subjects who can recall events with pride and wit.\r\n<p>As she herself once said, commenting on her predecessors: \"This is history and therefore so thrilling.\"</p>\r\n<p><b>Humour, Sarcasm, Wit, Frustration, Anger, Sadness &#150; History!</b></p>','2002-03/pleaseyourmajesty/jubileelogo.gif','Photos','2002-03/pleaseyourmajesty/index.htm','Preview photos','2002-03/pleaseyourmajesty/layout.htm','Poster (pdf)','2002-03/pleaseyourmajesty/poster.pdf',NULL,NULL,NULL,NULL);
INSERT INTO events VALUES (698,NULL,'2002-05-12 00:00:00','2002-05-12 00:00:00',NULL,'2002-05-12 00:00:00','An Evening of Dance',NULL,0,4,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
INSERT INTO events VALUES (705,NULL,'2002-12-05 00:00:00','2002-12-05 00:00:00',NULL,'2002-12-05 00:00:00','The Man With Size Twelve Feet',NULL,0,5,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
INSERT INTO events VALUES (706,NULL,'2002-12-20 00:00:00','2003-01-04 00:00:00',NULL,'2003-01-04 00:00:00','Honk',NULL,0,4,NULL,'Winner of Best Musical in the Laurence Olivier Awards 2000, Honk! is  the hilarious musical based on Hans Christian Andersen\'s The Ugly Duckling. Honk! contains an exhilarating score that is brimming with wit, melody and unexpected emotion. This inspirational fairy-tale is now a surprisingly sophisticated comedy that will charm theatregoers of all ages. The London Times called it \"delightful,\" while the Guardian said it \"hits all the right notes...(with) an elegance that is rare.\" A real treat for all the family!','2001-02/honk/poster.jpg',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);

Suggested fix:
Several possibilities occur to me, but I'm not a professional programmer. As an experienced user, I would find any of the following helpful.

Warn the user if it is the character set conversion that is causing the problem.

Warn if a mismatch in version numbers is causing the problem.

Manually edit the SQL dump file (to replace \' with \'' throughout) - which is what I'll have to do in the short term 

Deal with the quotes consistently in dump and script execution - this seems to me the proper solution.

I couldn't find anything relevant even after spending an hour searching the whole web, the bug report site for Query Browser, and forums for anything I could find that seemed related.
[29 Dec 2004 0:12] MySQL Verification Team
I tested your script on Slackware and QB 1.1.4 without any problems.
[29 Dec 2004 11:52] John McClenahan
Thank you for trying this. I still have the problem, consistently repeatable on Fedora Core 3, but have found a different way round most of it - which I should probably have used in the first place, if I had remembered about it first.

Using the command line tool 'mysql' reads almost all of the input without problems, and gets way past the error I reported previously. I found in QB that even after I edited the SQL text to replace \' with ~ (tilde),  QB fell over at other input characters - those characters with numeric values (in UTF8) showing in little boxes as 0085, 0091 and 0092 when viewed in the QB script tab. I think these correspond to the display characters 'long dash', and curly open or close quotes but I'm not sure about the latter - they may be ordinary double quote characters. There seemed also to be some un-escaped single quotes left in the dump file too, in the middle of text fields, which converted oddly from latin1 to UTF8 and also caused QB to fail.

However, even the command line tool mysql falls over with an error on a much smaller table, reproduced below - this time on the table definition, not the data. 

When I tried re-creating this table in the Query Browser it too wouldn't read it, giving the same error message about a SQL error in the table definition (without saying what the problem was). When I had edited the text in a couple of places to see if I could fix it, then re-executed, QB exited abruptly without an error message. I tried first removing the default '' entry for the Text field, then the NOT NULL entry for the DUMMY Timestamp - the latter edit on its own caused QB to crash.

So there still seem to be some inconsistencies between the output of mysqldump and various input tools including mysql as well as QB.

I have now got all the data in, except for this non-critical table, by cutting out the lines in the SQL file for this table alone, and re-importing the rest using the command line tool.

Thank you again for responding so promptly. 

John McC
--
-- Table structure for table `testautoinc`
--

CREATE TABLE testautoinc (
  ID int(20) unsigned NOT NULL auto_increment,
  Text field varchar(100) NOT NULL default '',
  Dummy timestamp(14) NOT NULL,
  PRIMARY KEY  (ID)
) TYPE=MyISAM;

--
-- Dumping data for table `testautoinc`
--

INSERT INTO testautoinc VALUES (1,'Test record 1',20040106230300);
INSERT INTO testautoinc VALUES (2,'Test record 2',20040106230454);
INSERT INTO testautoinc VALUES (3,'Test record 3',20040106230614);
INSERT INTO testautoinc VALUES (4,'Test record 4',20040106231141);
[29 Dec 2004 22:27] MySQL Verification Team
Well I tested again your small script without problem. However you have
a syntax error:

Text field varchar(100) NOT NULL default '',

and I changed to:

Text_field varchar(100) NOT NULL default '',
[29 Dec 2004 23:24] John McClenahan
Again, thanks for looking at this so promptly and helping me understand where the error came from - I certainly would NOT have spotted it.

All I did was cut and paste the entries (on the Linux system) from the mysqldump output file when sending extracts to you, and also before trying to edit the extract in QB to get it to work. The syntax error (which first appeared when mysql command line method was reading in from the SQL dump output) appears due, you think, to the missing underscore between Text and Field. It was missing in the dump file! 

I see now that it is (confusingly) the name of the field (Column name = 'Text field'), and was generated when I originally designed the table in MS Access - which allows spaces in field names. When exported to a MySQL table, the space persisted, and is still there in the original table on our web host. (Maybe that explains something else I can't get to work properly - converting autoincrementing fields in an existing table from MS Access to MySQL - which is what that table was trying to test out in a simple case.)

At least I now understand where and how the error was generated, and perhaps it isn't a bug -- if MySQL isn't supposed to allow spaces in column (field) names then neither the command line tool mysql nor mysqldump would waste time checking for them. 

Since I didn't remember reading this anywhere, I've gone back and looked at the manual, and still can't find it! I looked at the Tutorial - Defining a Table - and also searched for Column Name and Column Naming. I'm sure it must be there somewhere, but I still can't see it! 

This Linux and MySQL learning process reminds me of an adventure game - every time you get to one level of understanding, there are another two to come, full of traps! I've been using computers since my school days in the late 1950s as a hobby, but I still have a long way to go, clearly.

J McC
[29 Dec 2004 23:33] John McClenahan
Tried to be too clever when searching - asking for Column Name instead of Field Name. The latter does find it in section 9.2.

I still had to read it a few times to clarify that the space character is NOT ok in unquoted field names, but might be (I think) in quoted ones surround by back-ticks.

JMcC