MySQL Bugs: #24809: Libedit compile fails in "make distcheck" with automake 1.10

Bug #24809	Libedit compile fails in "make distcheck" with automake 1.10
Submitted:	5 Dec 2006 1:31	Modified:	31 Oct 2007 9:45
Reporter:	Kent Boortz	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S3 (Non-critical)
Version:	5.0.32-pre	OS:	Any (All)
Assigned to:	Kent Boortz	CPU Architecture:	Any

Description:
Libedit compile fails in "make distcheck" with automake 1.10.
Comparing with earlier automake the include line differs for
compiling "chared.c". The working line is

 gcc -DUNDEF_THREADS_HACK -DHAVE_CONFIG_H -DNO_KILL_INTR \ 
 -I. -I../../../cmd-line-utils/libedit -I../.. -I../../include \ 
 -I../../../include -I../../../cmd-line-utils/libedit/../.. -I.. \ 
 -O3 -DDBUG_OFF    -MT chared.o -MD -MP -MF ".deps/chared.Tpo" -c \ 
 -o chared.o ../../../cmd-line-utils/libedit/chared.c

The line not working is

 gcc -DUNDEF_THREADS_HACK -DHAVE_CONFIG_H -DNO_KILL_INTR \ 
 -I. -I../.. -I../../../cmd-line-utils/libedit -I../../include \
 -I../../../include -I../../../cmd-line-utils/libedit/../.. -I.. \ 
 -O3 -DDBUG_OFF    -MT chared.o -MD -MP -MF .deps/chared.Tpo -c \ 
 -o chared.o ../../../cmd-line-utils/libedit/chared.c

The "-I../.." is causing the failure, the main "config.h" is picked
up instead of the one in the "linedit" directory. Because of this
the libedit "sys.h" will not be included, that defines "protected"
to the empty string. And then gcc will complain about syntax error
because of this keyword not being removed from the function
declarations.

 In file included from ../../../cmd-line-utils/libedit/el.h:96,
                  from ../../../cmd-line-utils/libedit/chared.c:41:
 ../../../cmd-line-utils/libedit/tty.h:459: error: expected '=', ',', ';', 'asmâ'or '__attribute__' before 'int'
 ../../../cmd-line-utils/libedit/tty.h:460: error: expected '=', ',', ';', 'asmâ'or '__attribute__' before 'void'
 .
 .

How to repeat:
On a system with automake-1.10 do

 BUILD/compile-dist
 make distcheck

Automake will put the directory path to config.h specified
in the configure script into a variable DEFAULT_INCLUDES in
the generated make files. So that all make files can find
the "central" header files.

Now, in automake 1.10 an incompatible change was made.
Earlier the line was in our case

  DEFAULT_INCLUDES = -I. -I$(srcdir) -I$(top_builddir)/include

in effect always honor header files in the source directory
first (even if source/object directory are separate). In
1.10 the line is

  DEFAULT_INCLUDES = -I. -I$(top_builddir)/include -I$(srcdir)

in effect honoring the central include directory before the
one in the static source directory.

In the case of "libedit" we try force the build to use the
"config.h" in libedit before the one in the central sources.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/36695

ChangeSet@1.2551, 2007-10-30 20:54:31+01:00, kent@mysql.com +1 -0
  Makefile.am:
    Ensure use of libedit "config.h" by adding "-I. -I$(srcdir)" to DEFS,
    work around for problem with automake 1.10 (bug#24809)

Pushed into 6.0.4-alpha

Pushed into 5.1.23-rc

Pushed into 5.0.52

The fix for this bug is no longer in the trees(checked 5.1.41)  so it still occurs when doing "make distcheck" on OSX