Bug #18877 Documenting syntax changes
Submitted: 7 Apr 2006 7:38 Modified: 16 Oct 2009 11:05
Reporter: Keith Roberts Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Documentation Severity:S4 (Feature request)
Version:5.0.18 OS:Linux (SuSE Linux 9.2 pro)
Assigned to: MC Brown CPU Architecture:Any

[7 Apr 2006 7:38] Keith Roberts
Description:
I would like to suggest some ways of keeping the documentation of mysql
syntax changes up to date, as those changes are made by the developers
of mysql.

Any changes made to the internals of mysql may not be immediately
apparent, or of any particular concern, to joe bloggs average user of
mysql.

As the syntax is joe bloggs way of communicating with mysql, he needs
to know about these changes right away.

IMHO, There needs to be some sort of methodology in place that will
ensure changes made to mysql syntax are properly documented by the
developers as those changes are made.

How can this be accomplished?

Well, I did start Jack Crenshaw's  "Let's build a compiler" course,
around 98-98. http://compilers.iecc.com/crenshaw/

(I realise that Jack only covers top-down resursive-descent one-pass
compilation techniques, and hand written parsers, but I'm sure that the
method mentioned below is applicable to other types of parsers as well.)

One of the tips Jack mentions is how to go about designing your parser.

He suggests that parser design is first done using BNF syntax diagrams.
(IIRC he actually suggests the use of XBNF diagrams.)

Then use the BNF (or XBNF) as a blueprint to convert that particular syntax
construct into parser code.

By using this method, all mysql syntax documentation is guaranteed to be
up to date, as the documentation is actually designed first, before
changes to the code have been implemeted. Bearing this idea of Jack's in
mind, may I suggest the following methodology for documenting the changes
made to mysql syntax.

1. Changes to the mysql syntax are limited to core parser developers of
   MySQL AB only.

   (Maybe there needs to be a small team of MySQL AB persons that are
   directly responsible for implementing changes to the mysql syntax,
   using the method mentioned above.)

2. Anyone can make suggestions to improve mysql syntax, by submitting a bug
   report.

3. Only the MySQL AB parser team can implement changes to mysql syntax,
   whether it be suggestions submitted via a bug report, or ideas from any
   other sources.

4. The parser team designs the new mysql syntax with (X)BNF diagrams.

5. The syntax changes are made to the mysql program code by the parser
   team, or a member of the team.

6. The changes are tested and approved for inclusion in the next release of
   mysql.

7. When the changes have been approved for the next release, the (X)BNF is
   then passed along to Stefan Hinz for inclusion in the reference manual.

Hopefully, this will keep all changes to mysql syntax current and up to
date.

Kind Regards

Keith Roberts

How to repeat:
Just test the documented syntax using the mysql monitor program.

Suggested fix:
Limit mysql syntax changes to a small number of core developers only, and use some sort of methodology for implementing those syntax changes that will ensure those changes are properly documented as soon as they have been implemented and accepted into the next release of mysql.
[10 Apr 2006 11:13] Valeriy Kravchuk
Thank you for a problem report. Sounds like a reasonable (documentation) feature request (using exact BNFs, not their more "informal" representation) or even a build/QA procedure request.
[10 Apr 2006 15:31] Keith Roberts
YW Valeriy - thankyou for your reply.
[15 Oct 2009 14:14] MC Brown
We  have added a new index that outlines the syntax for MySQL with references: 

http://dev.mysql.com/doc/refman/5.1/en/dynindex-statement.html

A documentation project to handle the rest of this process and improve the output of the SQL Syntax, BNF and changes will be tackled in the future.
[15 Oct 2009 20:28] Keith Roberts
The manual has improved greatly since my original report.

I'm still concerned about user's having to wade through BNF to learn the mysql syntax. Would it not be better to produce proper syntax railroad diagrams for users?

I find real syntax railroad diagrams a breeze to read. Just follow the arrows.

SQLite uses a Tcl/Tk script to generate gif syntax diagrams.

I think the script is under the same license as SQLite.

Do you not think this would be worth doing for mysql?
[15 Oct 2009 20:31] Keith Roberts
Richard Hipp's TCL bubble-generator.tcl wish script

Attachment: bubble-generator.tcl (text/x-tcl), 30.65 KiB.

[15 Oct 2009 20:38] Keith Roberts
An example railroad syntax diagram for SQLite as created by Dr. R. Hipp's bubble-generator.tcl is at:

http://www.sqlite.org/lang_createtable.html

This is for the CREATE TABLE syntax. A lot easier to read than the equivalent BNF.

Something like this would make understanding mysql CREATE TABLE (and all of the other mysql syntax) alot easier and quicker to comprehend IMHO :)
[16 Oct 2009 8:41] MC Brown
Yes, railroad diagrams and improvements to the display and layout of the BNF are part of the project I mentioned.
[16 Oct 2009 11:05] Keith Roberts
What I was going to suggest would be some type of meta-data for the mysql language syntax.

That could then be used as the canonical reference for generating the mysql parser code, and also generating the mysql syntax railroad diagrams.

So any changes the developers make to the mysql language could be made in one place - ie the language meta-data.

This could them be used by a script to automatically generate the mysql syntax parser code, and the appropriate syntax railroad diagrams, all from that one meta-data source.

This would then keep the railroad syntax diagrams in sync with the actual mysql program code that generates the mysql parser.

( Got this idea from Pragmatic Programmers by Andy Hunt & Dave Thomas. They call this the DRY principle - Don't Repeat Yourself! )