Bug #21029 Dependencies between sql_yacc.cc and dependent headers not detected
Submitted: 13 Jul 2006 9:35 Modified: 26 Jan 2007 16:22
Reporter: Konstantin Osipov (OCA) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Compiling Severity:S3 (Non-critical)
Version:5.1 OS:
Assigned to: Marc ALFF CPU Architecture:Any

[13 Jul 2006 9:35] Konstantin Osipov
Description:
When I change errmsg.txt and type 'make' in sql/ directory, sql_yacc.yy is not rebuilt. This can lead to subtle errors if the change affects the errors used in sql_yacc.yy.

How to repeat:
cd mysql-5.1/sql/
touch share/errmsg.txt
make

Observe the list of rebuilt files and last modification time of sql_yacc.cc

Suggested fix:
Don't know.
[13 Jul 2006 16:22] Hartmut Holzgraefe
verified on 5.0, too
[19 Dec 2006 16:08] Marc ALFF
sql_yacc.cc only depends on sql_yacc.yy, and has no dependency on the
file sql/share/errmsg.txt or any header file derived from it.

So, even after adding an error message, the bison generated code sql_yacc.cc
would be identical, and does not need to be compiled again.

Closing as "not a bug"

What *should* happen is recompiling sql_yacc.o from sql_yacc.cc, since the
generated code contain dependencies to header files, like:
#include "mysql_priv.h"

Note however, that changing sql_yacc.yy and sql/share/errmsg.txt 
are "maintainer" operations, and require much more than relying on make alone,
since configure options, etc, can change.

For a maintainer, the safe way to build the code is to use the build scripts:
./BUILD/compile-xxx
which are safe since they execute a full configure / make clean / make cycle.
[11 Jan 2007 17:38] Konstantin Osipov
Suggested patch: remove the custom dependencies for sql_yacc.*. With this patch, everything works okay:

===== Makefile.am 1.128 vs edited =====
--- 1.128/sql/Makefile.am	2007-01-11 20:34:41 +03:00
+++ edited/Makefile.am	2007-01-11 20:34:37 +03:00
@@ -142,21 +142,6 @@ link_sources: mysql_tzinfo_to_sql.cc
 mysql_tzinfo_to_sql.o:	$(mysql_tzinfo_to_sql_SOURCES)
 			$(CXXCOMPILE) -c $(INCLUDES) -DTZINFO2SQL $<
 
-# Try to get better dependencies for the grammar. Othervise really bad
-# things like different grammars for different pars of MySQL can
-# happen if you are unlucky.
-sql_yacc.cc:	sql_yacc.yy
-sql_yacc.h:	sql_yacc.yy
-
-# Be careful here, note that we use VPATH and might or might not have
-# a pregenerated "sql_yacc.cc" in $(srcdir) or one we just built in
-# $(builddir). And it has to work if $(srcdir) == $(builddir).
-sql_yacc.o:	sql_yacc.cc sql_yacc.h $(HEADERS)
-		@SED@ -e 's/__attribute__ ((__unused__))//' $< > sql_yacc.cc-new
-		@MV@ sql_yacc.cc-new sql_yacc.cc
-		@echo "Note: The following compile may take a long time."
-		@echo "If it fails, re-run configure with --with-low-memory"
-		$(CXXCOMPILE) $(LM_CFLAGS) -c sql_yacc.cc
 
 # This generates lex_hash.h
 # NOTE Built sources should depend on their sources not the tool
[11 Jan 2007 17:38] Marc ALFF
Re-opening the bug, since there *is* a dependency problem with sql_yacc.cc
[12 Jan 2007 22:12] Marc ALFF
See related Bug#24619, Bug#24557.
[12 Jan 2007 22:18] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/18082

ChangeSet@1.2380, 2007-01-12 15:18:12-07:00, malff@weblab.(none) +4 -0
  Bug#21029 (Dependencies between sql_yacc.cc and dependent headers not detected)
  
  The build scripts in general, using automake, autoconf, etc, contain several
  special commands and work around all related to the way the bison code in the
  parser is built, for sql/sql_yacc.yy. These work arounds, accumulated over
  time during development, ultimately cause the build scripts to be unstable
  and cause build defects by not enforcing dependencies.
  
  This fix simplifies the build process and aligns it with the automake tooling,
  which provides native support for bison and *.yy files.
  
  In particular, the following problem have been fixed:
  - dependencies with sql_yacc.cc were not honored (Bug 21029), leading to
    corrupted builds,
  - the work around introduced by Bug 24557, to cleanup the generated files
    sql_yacc.h and sql_yacc.cc, has been removed,
  - the generated makefile, in a source distribution, used to destroy the files
    sql_yacc.h and sql_yacc.cc on a 'make clean' target. This has been fixed.
  - The root cause of the problem found with gcc 4.1 (see Bug 24619) has been
    clearly documented, and the "sed" hack has been replaced by a cleaner
    work around, when building the code with bison 1.875 and gcc 4.1.
  - Removed the file sql/sql_yacc.yy.bak, added by WL 3031 by accident.
  
  The build platform used to test this fix was:
  Linux 2.6.15-gentoo-r7, AMD x86_64
  GNU M4 1.4.7
  GNU Autoconf 2.60
  GNU automake 1.9.6
  GNU Make 3.81
  GNU Bison 2.2
  GCC 4.1.1 (Gentoo 4.1.1-r1)
  
  Build performed:
  
  ./BUILD/compile-dist
  --> builds properly
  make distcheck
  --> builds properly, which proves that sql_yacc.cc and sql_yacc.o are
  built correctly in a VPATH build.
  
  ./sql/.deps/sql_yacc.Po
  --> properly generated
  
  touch include/hash.h
  make
  --> does not rebuild sql_yacc.h, sql_yacc.output, sql_yacc.cc
  --> rebuilds sql_yacc.o, then dependent binaries
  
  in the sql sub directory:
  make clean
  --> does not remove sql_yacc.h and sql_yacc.cc
  --> does remove sql_yacc.output
  
  make distclean
  --> does remove sql_yacc.h and sql_yacc.cc, then the Makefile
  
  After uninstalling bison 2.2 and installing bison 1.875 instead,
  
  ./BUILD/compile-dist
  --> builds properly
  
  Verified that the bison 1.875 generated code does contain a broken
  construct with __attribute__ ((unused)),
  verified that GCC 4.1.1 fails without the work around implemented
  in sql_yacc.yy, and verified that the build is performed properly
  with the work around.
  
  make distcheck
  --> builds properly.
[24 Jan 2007 3:27] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/18679

ChangeSet@1.2380, 2007-01-23 20:26:47-07:00, malff@weblab.(none) +4 -0
  Bug#21029 (Dependencies between sql_yacc.cc and dependent headers not detected)
  
  The build scripts in general, using automake, autoconf, etc, contain several
  special commands and work around all related to the way the bison code in the
  parser is built, for sql/sql_yacc.yy. These work arounds, accumulated over
  time during development, ultimately cause the build scripts to be unstable
  and cause build defects by not enforcing dependencies.
  
  This fix simplifies the build process and aligns it with the automake tooling,
  which provides native support for bison and *.yy files.
  
  In particular, the following problem have been fixed:
  - dependencies with sql_yacc.cc were not honored (Bug 21029), leading to
    corrupted builds,
  - the work around introduced by Bug 24557, to cleanup the generated files
    sql_yacc.h and sql_yacc.cc, has been removed,
  - the generated makefile, in a source distribution, used to destroy the files
    sql_yacc.h and sql_yacc.cc on a 'make clean' target. This has been fixed:
    these files are now removed by make maintainer-clean.
  - The root cause of the problem found with gcc 4.1 (see Bug 24619) has been
    clearly documented, and the "sed" hack has been replaced by a cleaner
    work around, when building the code with bison 1.875.
  - Removed the file sql/sql_yacc.yy.bak, added by WL 3031 by accident.
  - Removed the unnecessary AM_YFLAG= --debug introduced by WL 3432, since
    the compiling option DBUG_OFF takes precedence when setting YYDEBUG.
  
  =============================================================================
  Note: everything below this line is for review only, it will be cut
  from the final CSET comments when the patch is pushed.
  
  The build platform used to test this fix was:
  Linux 2.6.15-gentoo-r7, AMD x86_64
  GNU M4 1.4.7
  GNU Autoconf 2.60
  GNU automake 1.9.6
  GNU Make 3.81
  GNU Bison 2.2
  GCC 4.1.1 (Gentoo 4.1.1-r1)
  
  Build performed:
  
  ./BUILD/compile-dist
  --> builds properly
  make distcheck
  --> builds properly (it involves a VPATH build)
  
  ./sql/.deps/sql_yacc.Po
  --> properly generated, including the content
  
  ./sql/Makefile.in, ./sql/Makefile
  --> contain dependencies for sql_yacc.h and sql_yacc.cc
  
  touch include/hash.h
  make
  --> does not rebuild sql_yacc.h, sql_yacc.output, sql_yacc.cc
  --> rebuilds sql_yacc.o, then dependent binaries
  
  touch sql/sql_yacc.yy
  make
  --> rebuilds sql_yacc.{h|cc}, then sql_yacc.o, then dependent files
  
  in the sql sub directory:
  make clean
  --> does not remove sql_yacc.h and sql_yacc.cc
  --> does remove sql_yacc.output
  
  make distclean
  --> keeps sql_yacc.{h|cc}
  
  ./BUILD/compile-dist
  make
  make maintainer-clean:
  --> removes sql_yacc.{h|cc}
  
  After un installing bison 2.2 and installing bison 1.875 instead,
  
  ./BUILD/compile-dist
  --> builds properly
  
  Verified that the bison 1.875 generated code does contain a broken
  construct with __attribute__ ((unused)),
  verified that GCC 4.1.1 fails without the work around implemented
  in sql_yacc.yy, and verified that the build is performed properly
  with the work around.
  
  make distcheck
  --> builds properly.
[24 Jan 2007 21:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/18752

ChangeSet@1.2380, 2007-01-24 14:40:39-07:00, malff@weblab.(none) +4 -0
  Bug#21029 (Dependencies between sql_yacc.cc and dependent headers not detected)
  
  The build scripts in general, using automake, autoconf, etc, contain several
  special commands and work around all related to the way the bison code in the
  parser is built, for sql/sql_yacc.yy. These work arounds, accumulated over
  time during development, ultimately cause the build scripts to be unstable
  and cause build defects by not enforcing dependencies.
  
  This fix simplifies the build process and aligns it with the automake tooling,
  which provides native support for bison and *.yy files.
  
  In particular, the following problem have been fixed:
  - dependencies with sql_yacc.cc were not honored (Bug 21029), leading to
    corrupted builds,
  - the work around introduced by Bug 24557, to cleanup the generated files
    sql_yacc.h and sql_yacc.cc, has been removed,
  - the generated makefile, in a source distribution, used to destroy the files
    sql_yacc.h and sql_yacc.cc on a 'make clean' target. This has been fixed:
    these files are now removed by make maintainer-clean.
  - The root cause of the problem found with gcc 4.1 (see Bug 24619) has been
    clearly documented, and the "sed" hack has been replaced by a cleaner
    work around, when building the code with bison 1.875.
  - Removed the file sql/sql_yacc.yy.bak, added by WL 3031 by accident.
  - Removed the unnecessary AM_YFLAG= --debug introduced by WL 3432, since
    the compiling option DBUG_OFF takes precedence when setting YYDEBUG.
[26 Jan 2007 16:22] Paul DuBois
No changelog entry needed.