Bug #43614 make distcheck failure (ndb/Makefile is made, but not subdirs of ndb)
Submitted: 12 Mar 2009 22:03 Modified: 13 Apr 2009 23:42
Reporter: Timothy Smith Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Compiling Severity:S2 (Serious)
Version:5.1-bugteam OS:Any
Assigned to: Timothy Smith CPU Architecture:Any
Tags: distcheck autoconf plugin

[12 Mar 2009 22:03] Timothy Smith
Description:
This is in the 5.1-bugteam branch only.

The new plugin.m4 code for handling DIST_SUBDIRS separate from SUBDIRS causes a problem during the VPATH build that is done in 'make distcheck'.  The problem is that during the VPATH build, the storage/ndb/Makefile is generated by autoconf, but the storage/ndb/*/Makefile are not.

During the build, there are some recursive targets which use DIST_SUBDIRS; those targets will go into storage/ndb, and will try to go into the subdirs, and fail.

6.0 will also be affected, but 5.0 will not.

How to repeat:
cd mysql-5.1-bugteam
./BUILD/compile-dist && make distcheck

The dist target will succeed (and mysql-5.1.??.tar.gz will be created), but the distcheck target will fail.  If you look in the resulting _build/storage/ndb directory, you'll see a Makefile only.  In that Makefile, DIST_SUBDIRS will show several directories which weren't created by autoconf.

Suggested fix:
The fix is to ensure that autoconf creates all the Makefiles even during a vpath build.  This can be done in plugin.m4 with improved AC_CONFIG_FILES() call.
[12 Mar 2009 23:12] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69084

2839 Timothy Smith	2009-03-12
      Bug #43614: make distcheck failure (ndb/Makefile is made, but not subdirs of ndb)
      
      There are some recursive targets that automake generates which reference
      DIST_SUBDIRS.  It's critical, then, for such subdirs to exist even if they
      won't be built as part of SUBDIRS.
      
      During a VPATH build, it is the configure script which creates the subdirs
      (when it processes the AC_CONFIG_FILES() for each subdir's Makefile).  If
      autoconf doesn't create a subdir's Makefile, then the recursive make will fail
      when it is unable to cd into that subdir.
      
      This isn't a problem in non-VPATH builds, because the subdirs are all present
      in the source tarball.  So the problem only shows up during 'make distcheck',
      which does a VPATH build.
      
      The fix is to look, when configure is being created by autoconf, for any
      plugin subdirectories.  These are the dynamic subdirectories which need to be
      handled specially.  It's enough to tell autoconf to generate a Makefile for
      any Makefile.am found in the plugin directory - all plugin subdirectories
      using automake (i.e., listed in the plugin's DIST_SUBDIRS) will have a
      Makefile.am.
      
      This is done by calling 'find'.  This means that 'find' must be in the PATH on
      the host that is running autoconf.  'find' is NOT needed when calling
      configure, so it is not an additional dependency for the user.
      
      Finally, ha_ndbcluster.m4 had called AC_CONFIG_FILES() on all those subdir
      Makefiles, but only when the plugin was actually being built.  So it didn't
      work in the case that NDB was not being built.  All of those Makefiles have to
      be removed from this static list, since the plugin machinery is now adding
      them automatically.  autoconf fails if a file is duplicated in
      AC_CONFIG_FILES().
[12 Mar 2009 23:44] Timothy Smith
A few more comments.

This problem is not NDB-specific.  NDB happens to be the only plugin that defines SUBDIRS (i.e., that uses automake recursively).  So it happens to be the only one that has the problem, but in the future certainly this would affect other plugins.

The closest I found to a solution to this when searching the web was

http://www.mail-archive.com/autoconf@gnu.org/msg16753.html

NDB is also the only plugin that did anything with AC_CONFIG_FILES() at all.
[13 Mar 2009 9:53] Magnus BlÄudd
Applied patch and tested to create dist, unpack dist and finally _VPATH build. Nice to avoid the need for adding each Makefile to "plug.in"(in this case ha_ndbcluster.m4)

Only comment I would have would be about the comments that first use # and then continue with dnl, is that intentional?

+dnl to if they need to pass a specific path.  This is *NOT* used when calling
+dnl running configure!
      ^^
There is also a smalling double verb mistake here.
[13 Mar 2009 11:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69132

2837 Georgi Kodinov	2009-03-13
      Bug #43614: make distcheck failure (ndb/Makefile is made, but not subdirs of ndb)
            
      There are some recursive targets that automake generates which reference
      DIST_SUBDIRS.  It's critical, then, for such subdirs to exist even if they
      won't be built as part of SUBDIRS.
      
      During a VPATH build, it is the configure script which creates the subdirs
      (when it processes the AC_CONFIG_FILES() for each subdir's Makefile).  If
      autoconf doesn't create a subdir's Makefile, then the recursive make will fail
      when it is unable to cd into that subdir.
      
      This isn't a problem in non-VPATH builds, because the subdirs are all present
      in the source tarball.  So the problem only shows up during 'make distcheck',
      which does a VPATH build.
      
      The fix is to look, when configure is being created by autoconf, for any
      plugin subdirectories.  These are the dynamic subdirectories which need to be
      handled specially.  It's enough to tell autoconf to generate a Makefile for
      any Makefile.am found in the plugin directory - all plugin subdirectories
      using automake (i.e., listed in the plugin's DIST_SUBDIRS) will have a
      Makefile.am.
      
      This is done by calling 'find'.  This means that 'find' must be in the PATH on
      the host that is running autoconf.  'find' is NOT needed when calling
      configure, so it is not an additional dependency for the user.
      
      Finally, ha_ndbcluster.m4 had called AC_CONFIG_FILES() on all those subdir
      Makefiles, but only when the plugin was actually being built.  So it didn't
      work in the case that NDB was not being built.  All of those Makefiles have to
      be removed from this static list, since the plugin machinery is now adding
      them automatically.  autoconf fails if a file is duplicated in
      AC_CONFIG_FILES().
[13 Mar 2009 11:15] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69134

2838 Georgi Kodinov	2009-03-13
      Bug #43614: make distcheck failure (ndb/Makefile is made, but not subdirs of ndb)
            
      There are some recursive targets that automake generates which reference
      DIST_SUBDIRS.  It's critical, then, for such subdirs to exist even if they
      won't be built as part of SUBDIRS.
      
      During a VPATH build, it is the configure script which creates the subdirs
      (when it processes the AC_CONFIG_FILES() for each subdir's Makefile).  If
      autoconf doesn't create a subdir's Makefile, then the recursive make will fail
      when it is unable to cd into that subdir.
      
      This isn't a problem in non-VPATH builds, because the subdirs are all present
      in the source tarball.  So the problem only shows up during 'make distcheck',
      which does a VPATH build.
      
      The fix is to look, when configure is being created by autoconf, for any
      plugin subdirectories.  These are the dynamic subdirectories which need to be
      handled specially.  It's enough to tell autoconf to generate a Makefile for
      any Makefile.am found in the plugin directory - all plugin subdirectories
      using automake (i.e., listed in the plugin's DIST_SUBDIRS) will have a
      Makefile.am.
      
      This is done by calling 'find'.  This means that 'find' must be in the PATH on
      the host that is running autoconf.  'find' is NOT needed when calling
      configure, so it is not an additional dependency for the user.
      
      Finally, ha_ndbcluster.m4 had called AC_CONFIG_FILES() on all those subdir
      Makefiles, but only when the plugin was actually being built.  So it didn't
      work in the case that NDB was not being built.  All of those Makefiles have to
      be removed from this static list, since the plugin machinery is now adding
      them automatically.  autoconf fails if a file is duplicated in
      AC_CONFIG_FILES().
[13 Mar 2009 13:20] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/69142

3119 Georgi Kodinov	2009-03-13 [merge]
      merged bug #43614 to 6.0
[13 Mar 2009 19:04] Bugs System
Pushed into 5.1.33 (revid:joro@sun.com-20090313111355-7bsi1hgkvrg8pdds) (version source revid:joro@sun.com-20090313111355-7bsi1hgkvrg8pdds) (merge vers: 5.1.33) (pib:6)
[18 Mar 2009 13:18] Bugs System
Pushed into 6.0.11-alpha (revid:joro@sun.com-20090318122208-1b5kvg6zeb4hxwp9) (version source revid:joro@sun.com-20090317133112-41qn6aly7arljtlq) (merge vers: 6.0.11-alpha) (pib:6)
[13 Apr 2009 23:42] Paul DuBois
Noted in 5.1.33, 6.0.11 changelogs.

make distcheck failed to properly handle subdirectories of
storage/ndb.
[9 May 2009 16:42] Bugs System
Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508100057-30ote4xggi4nq14v) (merge vers: 5.1.33-ndb-6.2.18) (pib:6)
[9 May 2009 17:39] Bugs System
Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090508175813-s6yele2z3oh6o99z) (merge vers: 5.1.33-ndb-6.3.25) (pib:6)
[9 May 2009 18:36] Bugs System
Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509073226-09bljakh9eppogec) (merge vers: 5.1.33-ndb-7.0.6) (pib:6)