MySQL Bugs: #38263: Error compiling mysql-5.1.24-ndb-6.3.16 on HP-UX 11.11

Bug #38263	Error compiling mysql-5.1.24-ndb-6.3.16 on HP-UX 11.11
Submitted:	21 Jul 2008 15:40	Modified:	2 Dec 2010 11:33
Reporter:	Anatoly Pidruchny (Candidate Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	mysql-5.1-telco-6.3	OS:	HP/UX (11.11)
Assigned to:		CPU Architecture:	Any
Tags:	5.1.24-ndb-6.3.16, qc

Description:
I tried to compile the sources from the file mysql-5.1.24-ndb-6.3.16-telco.tar.gz  downloaded from ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/ on HP-UX 11.11 and I got the following error:

aCC +O2 +O2 +DA2.0N +Z +ESnolit -Aa -DHPUX11 -DSNPRINTF_RETURN_TRUNC -DHAVE_BROKEN_PREAD -D_INCLUDE_LONGLONG -DNO_FCNTL_NONBLOCK -DDO_NOT_REMOVE_THREAD_WRAPPERS -DHAVE_BROKEN_PTHREAD_COND_TIMEDWAIT -o mysql mysql.o readline.o sql_string.o completion_hash.o  ../cmd-line-utils/readline/libreadline.a -lcurses -lpthread ../libmysql/.libs/libmysqlclient.a -lpthread -lpthread -lrt -lpthread -lpthread ../zlib/.libs/libzlt.a -lnsl -lm
/usr/ccs/bin/ld: Unsatisfied symbols:
  _my_thread_var (first referenced in ../libmysql/.libs/libmysqlclient.a(strtoll.o)) (code)

This is the configure line that I used (it is the same one that I used for compiling the 5.1.x sources and had no problems):

./configure '--prefix=/usr/local/mysql' '--localstatedir=/usr/local/mysql/data' '--libexecdir=/usr/local/mysql/bin' '--with-comment=MySQL Community Server (GPL)' '--with-server-suffix=' '--enable-thread-safe-client' '--enable-local-infile' '--with-pic' '--disable-shared' '--with-client-ldflags=-static' '--with-mysqld-ldflags=-static' '--with-zlib-dir=bundled' '--with-big-tables' '--with-readline' '--with-embedded-server' '--with-archive-storage-engine' '--with-blackhole-storage-engine' '--with-ndbcluster' '--with-csv-storage-engine' '--with-example-storage-engine' '--with-federated-storage-engine' '--with-partition' '--with-innodb' '--with-extra-charsets=all' 'CC=cc' 'CFLAGS=+O2 +DA2.0N +Z +ESnolit' 'CPPFLAGS=-DDEFAULT_THREAD_STACK=262144' 'CXXFLAGS=+O2 +DA2.0N +Z +ESnolit -Aa' 'CXX=aCC'

/Anatoly

How to repeat:
Download the file ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.24-ndb-6.3.16-telco.tar.gz, unpack on HP-UX 11.11 system;
Run configure with the parameters as in the description;
Run gmake.

Suggested fix:
From the quick look at the problem, the strtoll function is not available on HP-UX and so it is defined in the mystrings library. But, as you can see in the description, mystrings library is not present on the command line when mysql binary is linked. I think the fix should be to add mystrings library for the command used to link mysql and possibly other binaries.

Sorry, my suggested fix is not accurate, please disregard. It is _my_thread_var symbol is not found during linking and strtoll.o object file is actually found. _my_thread_var function is defined in libmysql/my_thr_init.c (and libmysql_r/my_thr_init.c). Looks like the problem is that the THREAD macro is not defined when my_thr_init.c is compiled, because:

$ nm my_thr_init.o | grep my_
Symbols from my_thr_init.o:

In other words, no functions with prefix my_ were compiled and that can only happen when THREAD macro was not defined. When I grepped the sources, I see:

./include/config.h:1197:#define THREAD 1
./include/my_config.h:1197:#define THREAD 1

I can try to dig more. Please let me know how I can help.

Thanks,

Anatoly.

I fixed the compilation problems for myself with the following changes in the client/Makefile.in file:

*** mysql-5.1.24-ndb-6.3.16-telco.orig/client/Makefile.in       Fri Jun 27 08:23:43 2008
--- mysql-5.1.24-ndb-6.3.16-telco/client/Makefile.in    Thu Jul 24 17:33:01 2008
***************
*** 218,224 ****
  DARWIN_MWCC_TRUE = @DARWIN_MWCC_TRUE@
  
  # Fix for mit-threads
! DEFS = -DUNDEF_THREADS_HACK \
                        -DDEFAULT_MYSQL_HOME="\"$(prefix)\"" \
                        -DDATADIR="\"$(localstatedir)\""
  
--- 218,224 ----
  DARWIN_MWCC_TRUE = @DARWIN_MWCC_TRUE@
  
  # Fix for mit-threads
! DEFS = \
                        -DDEFAULT_MYSQL_HOME="\"$(prefix)\"" \
                        -DDATADIR="\"$(localstatedir)\""
  
***************
*** 474,480 ****
                                  $(openssl_includes)
  
  LDADD = @CLIENT_EXTRA_LDFLAGS@ $(CLIENT_THREAD_LIBS) \
!                               $(top_builddir)/libmysql/libmysqlclient.la
  
  noinst_HEADERS = sql_string.h completion_hash.h my_readline.h \
                                client_priv.h
--- 474,480 ----
                                  $(openssl_includes)
  
  LDADD = @CLIENT_EXTRA_LDFLAGS@ $(CLIENT_THREAD_LIBS) \
!                               $(LIBMYSQLCLIENT_LA)
  
  noinst_HEADERS = sql_string.h completion_hash.h my_readline.h \
                                client_priv.h

The first change I tried was to use LIBMYSQLCLIENT_LA variable in LDADD variable definition instead of the hard-coded string libmysqlclient.la. I want to get threaded code for both server and client and I think it is clearly a bug that libmysqlclient.la was always used during compilation. After this change the mysql binary was built successfully, but I got another error linking mysqltest:

cc +O2 +DA2.0N +Z +ESnolit -DHPUX11 -DSNPRINTF_RETURN_TRUNC -DHAVE_BROKEN_PREAD -DHAVE_BROKEN_GETPASS -DNO_FCNTL_NONBLOCK -DDO_NOT_REMOVE_THREAD_WRAPPERS -DHAVE_BROKEN_PTHREAD_COND_TIMEDWAIT -DHAVE_BROKEN_INLINE -o mysqltest mysqltest.o my_getsystime.o my_copy.o my_mkdir.o  ../regex/libregex.a -lpthread ../libmysql_r/.libs/libmysqlclient_r.a -lpthread -lpthread -lpthread -lpthread -lpthread -lpthread -lrt -lpthread -lpthread ../zlib/.libs/libzlt.a -lnsl -lm
/usr/ccs/bin/ld: Unsatisfied symbols:
   my_errno (first referenced in my_copy.o) (data)

Then I made the second change, remove -DUNDEF_THREADS_HACK from the DEFS variable. The thing is, some source files (including my_copy.c) from mysys directory are compiled again in the client directory. But since UNDEF_THREADS_HACK macro is defined then they are built with no threads support, and this is clearly wrong. The comment just above the changed line says "Fix for mit-threads", so, I am not sure if removing UNDEF_THREADS_HACK can cause problems on other platforms. But UNDEF_THREADS_HACK definitely causes problems on HP-UX.

Many thanks for writing a bug report.

I could reproduce your problem but I got different error messages by using HP-UX 11.31.

Fact is, that CGE won't compile on HP-UX.

After hours of investigation and trying, I managed to compile the software on HP-UX 11.11 (did not try to install and run it yet). Below is the list of changes that I had to make:

1. The configure line that worked is attached as file configure_line. Essential there is that it is necessary to add _HPUX_SOURCE and _XOPEN_SOURCE_EXTENDED macros in the CPPFLAGS. It was not necessary to do this with non-CGE 5.1.x software. I want to suggest that these macros should always be added automatically by configure when HP-UX platform is detected. By the way, the --with-ssl option still causes compilation problems and so I had to remove it.

2. As I described in my previous comment, the changes are necessary in client/Makefile.in file. Patch file is attached as Makefile.in.patch.

3. With the above changes, the following compilation errors occur during compilation of ndb storage engine:

Error 173: "ndbfs/PosixAsyncFile.cpp", line 458 # Redefined symbol 'seek_val'; previously defined at ["ndbfs/PosixAsyncFile.cpp", line 447].
      off_t seek_val;
            ^^^^^^^^ 
Error 182: "ndbfs/PosixAsyncFile.cpp", line 530 # "void PosixAsyncFile::readvReq(Request *)" cannot access private member "void AsyncFile::readReq(Request *)".
      readReq(request);
      ^^^^^^^^^^^^^^^^ 

To fix them, I made some small changes in two files, see the patch files AsyncFile.hpp.patch and PosixAsyncFile.cpp.patch.

Configure line

Attachment: configure_line (application/octet-stream, text), 1.02 KiB.

Makefile changes

Attachment: Makefile.in.patch (application/octet-stream, text), 1.25 KiB.

Patch for AsyncFile.hpp

Attachment: AsyncFile.hpp.patch (application/octet-stream, text), 487 bytes.

Patch for PosixAsyncFile.cpp

Attachment: PosixAsyncFile.cpp.patch (application/octet-stream, text), 886 bytes.

the reporter has been very nice, helping us finding many critical bugs,
so I think we should fix it therefor.

I assigned it to mtaylor as he changed the makefiles when fixing the linking problem for java.

given that patches exists for all the problems, it doesnt sound to hard to fix...
but configure/make is always a bit messy

HP-UX not supported platform.