MySQL Bugs: #38263: Error compiling mysql-5.1.24-ndb-6.3.16 on HP-UX 11.11

Bug #38263	Error compiling mysql-5.1.24-ndb-6.3.16 on HP-UX 11.11
Submitted:	21 Jul 2008 17:40	Modified:	25 Jul 2008 20:55
Reporter:	Anatoly Pidruchny (Candidate Quality Contributor)
Status:	Verified
Category:	Server: Cluster	Severity:	S3 (Non-critical)
Version:	mysql-5.1-telco-6.3	OS:	HP/UX (11.11)
Assigned to:		Target Version:
Tags:	qc, 5.1.24-ndb-6.3.16
Triage:	Triaged: D3 (Medium) / R2 (Low) / E3 (Medium)

Description:
I tried to compile the sources from the file mysql-5.1.24-ndb-6.3.16-telco.tar.gz 
downloaded from ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/ on HP-UX 11.11 and
I got the following error:

aCC +O2 +O2 +DA2.0N +Z +ESnolit -Aa -DHPUX11 -DSNPRINTF_RETURN_TRUNC -DHAVE_BROKEN_PREAD
-D_INCLUDE_LONGLONG -DNO_FCNTL_NONBLOCK -DDO_NOT_REMOVE_THREAD_WRAPPERS
-DHAVE_BROKEN_PTHREAD_COND_TIMEDWAIT -o mysql mysql.o readline.o sql_string.o
completion_hash.o  ../cmd-line-utils/readline/libreadline.a -lcurses -lpthread
../libmysql/.libs/libmysqlclient.a -lpthread -lpthread -lrt -lpthread -lpthread
../zlib/.libs/libzlt.a -lnsl -lm
/usr/ccs/bin/ld: Unsatisfied symbols:
  _my_thread_var (first referenced in ../libmysql/.libs/libmysqlclient.a(strtoll.o))
(code)

This is the configure line that I used (it is the same one that I used for compiling the
5.1.x sources and had no problems):

./configure '--prefix=/usr/local/mysql' '--localstatedir=/usr/local/mysql/data'
'--libexecdir=/usr/local/mysql/bin' '--with-comment=MySQL Community Server (GPL)'
'--with-server-suffix=' '--enable-thread-safe-client' '--enable-local-infile'
'--with-pic' '--disable-shared' '--with-client-ldflags=-static'
'--with-mysqld-ldflags=-static' '--with-zlib-dir=bundled' '--with-big-tables'
'--with-readline' '--with-embedded-server' '--with-archive-storage-engine'
'--with-blackhole-storage-engine' '--with-ndbcluster' '--with-csv-storage-engine'
'--with-example-storage-engine' '--with-federated-storage-engine' '--with-partition'
'--with-innodb' '--with-extra-charsets=all' 'CC=cc' 'CFLAGS=+O2 +DA2.0N +Z +ESnolit'
'CPPFLAGS=-DDEFAULT_THREAD_STACK=262144' 'CXXFLAGS=+O2 +DA2.0N +Z +ESnolit -Aa'
'CXX=aCC'

/Anatoly

How to repeat:
Download the file
ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.24-ndb-6.3.16-telco.tar.gz,
unpack on HP-UX 11.11 system;
Run configure with the parameters as in the description;
Run gmake.

Suggested fix:
From the quick look at the problem, the strtoll function is not available on HP-UX and so
it is defined in the mystrings library. But, as you can see in the description, mystrings
library is not present on the command line when mysql binary is linked. I think the fix
should be to add mystrings library for the command used to link mysql and possibly other
binaries.

Sorry, my suggested fix is not accurate, please disregard. It is _my_thread_var symbol is
not found during linking and strtoll.o object file is actually found. _my_thread_var
function is defined in libmysql/my_thr_init.c (and libmysql_r/my_thr_init.c). Looks like
the problem is that the THREAD macro is not defined when my_thr_init.c is compiled,
because:

$ nm my_thr_init.o | grep my_
Symbols from my_thr_init.o:

In other words, no functions with prefix my_ were compiled and that can only happen when
THREAD macro was not defined. When I grepped the sources, I see:

./include/config.h:1197:#define THREAD 1
./include/my_config.h:1197:#define THREAD 1

I can try to dig more. Please let me know how I can help.

Thanks,

Anatoly.

I fixed the compilation problems for myself with the following changes in the
client/Makefile.in file:

*** mysql-5.1.24-ndb-6.3.16-telco.orig/client/Makefile.in       Fri Jun 27 08:23:43 2008
--- mysql-5.1.24-ndb-6.3.16-telco/client/Makefile.in    Thu Jul 24 17:33:01 2008
***************
*** 218,224 ****
  DARWIN_MWCC_TRUE = @DARWIN_MWCC_TRUE@
  
  # Fix for mit-threads
! DEFS = -DUNDEF_THREADS_HACK \
                        -DDEFAULT_MYSQL_HOME="\"$(prefix)\"" \
                        -DDATADIR="\"$(localstatedir)\""
  
--- 218,224 ----
  DARWIN_MWCC_TRUE = @DARWIN_MWCC_TRUE@
  
  # Fix for mit-threads
! DEFS = \
                        -DDEFAULT_MYSQL_HOME="\"$(prefix)\"" \
                        -DDATADIR="\"$(localstatedir)\""
  
***************
*** 474,480 ****
                                  $(openssl_includes)
  
  LDADD = @CLIENT_EXTRA_LDFLAGS@ $(CLIENT_THREAD_LIBS) \
!                               $(top_builddir)/libmysql/libmysqlclient.la
  
  noinst_HEADERS = sql_string.h completion_hash.h my_readline.h \
                                client_priv.h
--- 474,480 ----
                                  $(openssl_includes)
  
  LDADD = @CLIENT_EXTRA_LDFLAGS@ $(CLIENT_THREAD_LIBS) \
!                               $(LIBMYSQLCLIENT_LA)
  
  noinst_HEADERS = sql_string.h completion_hash.h my_readline.h \
                                client_priv.h

The first change I tried was to use LIBMYSQLCLIENT_LA variable in LDADD variable
definition instead of the hard-coded string libmysqlclient.la. I want to get threaded
code for both server and client and I think it is clearly a bug that libmysqlclient.la
was always used during compilation. After this change the mysql binary was built
successfully, but I got another error linking mysqltest:

cc +O2 +DA2.0N +Z +ESnolit -DHPUX11 -DSNPRINTF_RETURN_TRUNC -DHAVE_BROKEN_PREAD
-DHAVE_BROKEN_GETPASS -DNO_FCNTL_NONBLOCK -DDO_NOT_REMOVE_THREAD_WRAPPERS
-DHAVE_BROKEN_PTHREAD_COND_TIMEDWAIT -DHAVE_BROKEN_INLINE -o mysqltest mysqltest.o
my_getsystime.o my_copy.o my_mkdir.o  ../regex/libregex.a -lpthread
../libmysql_r/.libs/libmysqlclient_r.a -lpthread -lpthread -lpthread -lpthread -lpthread
-lpthread -lrt -lpthread -lpthread ../zlib/.libs/libzlt.a -lnsl -lm
/usr/ccs/bin/ld: Unsatisfied symbols:
   my_errno (first referenced in my_copy.o) (data)

Then I made the second change, remove -DUNDEF_THREADS_HACK from the DEFS variable. The
thing is, some source files (including my_copy.c) from mysys directory are compiled again
in the client directory. But since UNDEF_THREADS_HACK macro is defined then they are built
with no threads support, and this is clearly wrong. The comment just above the changed
line says "Fix for mit-threads", so, I am not sure if removing UNDEF_THREADS_HACK can
cause problems on other platforms. But UNDEF_THREADS_HACK definitely causes problems on
HP-UX.

Many thanks for writing a bug report.

I could reproduce your problem but I got different error messages by using HP-UX 11.31.

Fact is, that CGE won't compile on HP-UX.

After hours of investigation and trying, I managed to compile the software on HP-UX 11.11
(did not try to install and run it yet). Below is the list of changes that I had to
make:

1. The configure line that worked is attached as file configure_line. Essential there is
that it is necessary to add _HPUX_SOURCE and _XOPEN_SOURCE_EXTENDED macros in the
CPPFLAGS. It was not necessary to do this with non-CGE 5.1.x software. I want to suggest
that these macros should always be added automatically by configure when HP-UX platform
is detected. By the way, the --with-ssl option still causes compilation problems and so I
had to remove it.

2. As I described in my previous comment, the changes are necessary in client/Makefile.in
file. Patch file is attached as Makefile.in.patch.

3. With the above changes, the following compilation errors occur during compilation of
ndb storage engine:

Error 173: "ndbfs/PosixAsyncFile.cpp", line 458 # Redefined symbol 'seek_val'; previously
defined at ["ndbfs/PosixAsyncFile.cpp", line 447].
      off_t seek_val;
            ^^^^^^^^ 
Error 182: "ndbfs/PosixAsyncFile.cpp", line 530 # "void PosixAsyncFile::readvReq(Request
*)" cannot access private member "void AsyncFile::readReq(Request *)".
      readReq(request);
      ^^^^^^^^^^^^^^^^ 

To fix them, I made some small changes in two files, see the patch files
AsyncFile.hpp.patch and PosixAsyncFile.cpp.patch.

Configure line

Attachment: configure_line (application/octet-stream, text), 1.02 KiB.

Makefile changes

Attachment: Makefile.in.patch (application/octet-stream, text), 1.25 KiB.

Patch for AsyncFile.hpp

Attachment: AsyncFile.hpp.patch (application/octet-stream, text), 487 bytes.

Patch for PosixAsyncFile.cpp

Attachment: PosixAsyncFile.cpp.patch (application/octet-stream, text), 886 bytes.

the reporter has been very nice, helping us finding many critical bugs,
so I think we should fix it therefor.

I assigned it to mtaylor as he changed the makefiles when fixing the linking problem for
java.

given that patches exists for all the problems, it doesnt sound to hard to fix...
but configure/make is always a bit messy