Bug #6521 mysql 4.1.x (the client binary) links bad to libmysqlclient.so.14 on OpenBSD
Submitted: 9 Nov 2004 13:44 Modified: 5 May 2006 17:35
Reporter: pineau benjamin Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Command-line Clients Severity:S1 (Critical)
Version:4.1.7, 4.1.15 OS:Other (OpenBSD 3.7)
Assigned to: Kent Boortz CPU Architecture:Any

[9 Nov 2004 13:44] pineau benjamin
Description:
Seems like there is a bug on the use of ld (through libtool ?) when linking the client binary (named 'mysql') of 4.1.7 on OpenBSD.
He gets linked to '../libmysql/.libs/libmysqlclient.so.14.0' instead of '/usr/local/mysql/lib/libmysqlclient.so.14.0'.
Other binaries are linked ok.

bash-2.05b$ ldd /usr/local/mysql/bin/mysql
/usr/local/mysql/bin/mysql:
/usr/local/mysql/bin/mysql: can't load library '../libmysql/.libs/libmysqlclient.so.14.0'
/usr/local/mysql/bin/mysql: exit status 4

bash-2.05b$ libtool --version
ltmain.sh (GNU libtool) 1.5.6 (1.1220.2.95 2004/04/11 05:50:42)

bash-2.05b$ ld --version
GNU ld version 2.14 20030612

bash-2.05b$ gcc --version
2.95.3

bash-2.05b$ uname -a
OpenBSD oshima 3.6 GENERIC.MP#103 i386

How to repeat:
I did the following on several OpenBSD systems (from 3.4 to 3.6), with mysql-4.1.7, and obtained the same bug in every case:

env CFLAGS=" -D__BSD__ " \
    CXXFLAGS="-D__BSD__ -felide-constructors -fno-exceptions -fno-rtti" \
     ./configure --prefix=/usr/local/mysql \
                --enable-thread-safe-client \
                --with-berkeley-db \
                --with-innodb \
                --with-csv-storage-engine \
                --with-libwrap \
                --with-mysqld-user="_mysql" \
                --with-openssl \
                --enable-assembler \
                --enable-local-infile \
                --with-vio \
                --with-extra-charsets=complex \
                --with-ndbcluster \
                --with-ndb-shm
                --with-innodb \
                --with-csv-storage-engine \
                --with-libwrap \
                --with-mysqld-user="_mysql" \
                --with-openssl \
                --enable-assembler \
                --enable-local-infile \
                --with-vio \
                --with-extra-charsets=complex \
                --with-ndbcluster \
                --with-ndb-shm

gmake
sudo gmake install

bash-2.05b$ ldd /usr/local/mysql/bin/mysql
/usr/local/mysql/bin/mysql:
/usr/local/mysql/bin/mysql: can't load library '../libmysql/.libs/libmysqlclient.so.14.0'
/usr/local/mysql/bin/mysql: exit status 4

This happens only on OpenBSD (I did try on FreeBSD 4.9, the bug didn't appear).
[12 Nov 2004 21:07] Edward Frye
g++ -O3 -DDBUG_OFF -fno-implicit-templates -fno-exceptions \
	-fno-rtti -o .libs/mysql mysql.o readline.o \
	sql_string.o completion_hash.o  \
	../cmd-line-utils/libedit/libedit.a -lncurses \
	../libmysql/.libs/libmysqlclient.so.14.0 -L/usr/lib\
	-lm -lz -lssl -lcrypto \
	-Wl,--rpath -Wl,/usr/local/lib/mysql

I changed the library definition from a litteral to a -L and -l by
issuing the following command in the client directory:

    g++ -O3 -DDBUG_OFF -fno-implicit-templates -fno-exceptions \
	-fno-rtti -o .libs/mysql mysql.o readline.o \
	sql_string.o completion_hash.o  \
	../cmd-line-utils/libedit/libedit.a -lncurses \
	-L ../libmysql/.libs/ -lmysqlclient -L/usr/lib\
	-lm -lz -lssl -lcrypto \
	-Wl,--rpath -Wl,/usr/local/lib/mysql

This solved the problem and I was able to then re 'make install' and
now everything works.

My plaftorm is a default install of OpenBSD 3.5 on i386
I would write a patch for OpenBSD to solve this problem but don't have the time and am not programmer
[22 Nov 2004 5:59] Matthew Lord
I was able to verify this on the ds9.orbus.fr build machine.

uname -a
OpenBSD ds9.orbus.fr 3.6 J#1 i386

I was able to repeat the problem by just doing ./configure --prefix=/home/mysqldev/mlord/4.1.8 
&& make && make install

You can see the binaries on the machine there.
[27 Dec 2004 16:06] Lenz Grimmer
This could be a bug in our libtool script. Could you please retest this with the 4.1.8a source tarball I am about to publish today? This one is built using updated libtool scripts (ltmain.sh was updated).
[2 Jan 2005 4:37] Gavin MacFarlane
I have also recently reproduced the same problem on an OpenBSD 3.6 x86 config.  The g++ fix that the previous gentleman posted seemed to solve my issue as well.  

mysql-4.1.8a
[12 Jan 2005 10:41] pineau benjamin
To respond to Lenz Grimmer:
The problem subsists on mysql-4.1.8 (release):

/usr/local/mysql/bin/mysql:
/usr/local/mysql/bin/mysql: can't load library '../libmysql/.libs/libmysqlclient.so.14.0'
/usr/local/mysql/bin/mysql: exit status 4
[12 Jan 2005 10:52] Lenz Grimmer
Benjamin: What about the 4.1.8_a_ sources? The "a" is important here.
[4 Feb 2005 11:34] Bent Vangli
Hi

I can confirm that the same linking error occurs on 4.1.9 when compiled and installed on a Fedora Core3 - 64 bit (dual Opteron system). Suggest that the error get a higher severity because you are not able to enter a freshly installed system to make your modifications

However, after exercising Edward Frye's genious fix it all works. :-)

With very best regards 

Bent Vangli, Oslo, Norway
[4 Feb 2005 21:34] pineau benjamin
Still occurs with 4.1.8a (yes, "a" this time ;) and 4.1.9.
[10 Feb 2005 5:47] Aaron Huntsman
I tried Mr. Frye's solution, and it works for mysql and most of the other command-line tools, but mysqladmin still gives me the "can't load library '../libmysql/.libs/libmysqlclient.so.14.0'" error.  Running mysql 4.1.9 on OpenBSD 3.6/i386.  Any suggestions?
[21 Feb 2005 13:55] pineau benjamin
As a workaround, we can compile the three broken binaries (mysql, mysqladmin, mysqlbinlog) against the static libmysqlclient. 

To keep the coherency with flags from autoconf, I proceed as this:

1) configure and compile mysql
2) cd client/ && make clean
3) eval $(make | egrep '^g.*\.libs/mysql( |admin|binlog)'|sed -e s/.so....0/.a/g -e s/$/\;/)
4) cd .. && make install

Can be adapted to link against the dynamic libmysqlclient (change the sed replaces).

P.S.: The problem subsist in mysql 4.1.10. The question is: why libtool fails to provide "-L../libmysql/.libs -lmysqlclient" on those plateforms for those 3 clients binaries while he don't for the other binaries in clients/* (mysqldump, mysqlimport, ...). Should be noticed that those 3 are the ones that use g++ (while others use gcc, $(CXXLINK) vs $(LINK) in Makefile).
[31 Mar 2005 6:55] Bent Vangli
This error i still persistent in 4.1.10a on Fedora Core 3 - 64 bit dual Opteron. However workarounds described above seems mostly to works for my installation, but it is a mess making updating and patching cumbersome. I hope its possible to fix as the reason seems to be pinpointet in pineau benjamin's [21 Feb 2:55pm] message above.

Despite this, MySQL rocks - Keep on the good works :-)

Bent Vangli, Oslo, Norway
[4 May 2005 20:45] D.J. Harbaugh
I'm also having the same problem with OpenBSD 3.6 on a Pentium III 550.  Using mysql version 4.1.11, so this bug hasn't been squashed yet :-(
[5 May 2005 6:57] Lenz Grimmer
I am a bit stuck with this bug - sorry for the long delay. I will try to assign it to another developer with a better clue.
[17 May 2005 16:04] D.J. Harbaugh
I've found a much easier workaround for those of us who don't regularly work with compilers :-)  Unfortunately, this requires you to have static binaries, but for those of you like me who don't care, just make sure you pop a --enable-static --disable-shared onto the configure script and there's no problem after that.  I realize those are redundant commands, but I wasn't taking any chances, and it did work ;-)
[28 Jun 2005 22:16] Han Boetes
I think I just got it compiled properly:

./configure --foo --bar
perl -pi -e 's|^LIBTOOL.*|LIBTOOL = libtool --preserve-dup-deps|' client/Makefile
make

Though I use my own buildsystem so I don't know how reproducable this is. And this is still just a workaround.
[15 Jul 2005 13:38] Renaud Allard
The bug is still present with OpenBSD3.7 and MySQL 5.0.7-beta.
[22 Jul 2005 5:42] [ name withheld ]
I compiled from source on OpenBSD 3.7 using mysql-4.1.13.tar.gz. Edward Frye's work around didn't work for me, sadly. MySQL asks for libmysqlclient.so.14 when I run command line php scripts even when I use the OpenBSD distribution package.
[28 Aug 2005 7:09] Brad Smith
The libtool that comes with the latest releases of MySQL 4.1.x and 5.0.x are really old. It would be best if MySQL syncs up to libtool 1.5.18.
[30 Aug 2005 1:18] Brad Smith
This should be fixed with libtool 1.5.8 or newer.

http://www.openbsd.org/cgi-bin/cvsweb/ports/devel/libtool/patches/patch-ltmain_in

rev 1.2
[16 Sep 2005 19:51] D.J. Harbaugh
Unfortunately, my workaround doesn't work on version 4.1.14 on OpenBSD 3.7.  How is this a "non-critical" bug?  It would seem to me if you can't install the program and there's no workaround that appears to work for everyone that it's a pretty "critical" bug.
[16 Sep 2005 19:54] D.J. Harbaugh
Also, we upgraded our libtool package to version 1.5.10p2, and this problem still rears it's head.  Does the installer care what my libtool is, or does it always use something that came with the source?  If the latter, is there an easy way to  either get it to use mine, or upgrade the source?
[26 Sep 2005 7:08] Sergei Golubchik
Yes, you can make MySQL to use your autotools, you'll need to regenerate files that come with the source. This is how our build script does it (simplified):

aclocal
autoheader
libtoolize --automake --force
automake --add-missing --force
autoconf
(cd bdb/dist && sh s_all)
(cd innobase && aclocal && autoheader && aclocal && automake && autoconf)
[1 Oct 2005 12:43] Valeriy Kravchuk
So, does anybody tried to build 4.1.14 on OpenBSD using a newer version of tools as suggested in the last comment by Sergei Golubchik? This is the sequence of actions used when building from recent development (BK) sources.

I think, this bug report is so long and contains so much different ideas, workarounds and platforms mentioned, that it should be closed by a reasonable summary. We need your feedback to be able to create it.
[2 Oct 2005 23:06] pineau benjamin
In reply to Valeriy Kravchuk questions, I tested the latests solutions today
with MySQL 4.1.14 on OpenBSD 3.7 and (the upcoming in November version) 3.8, 
with each one latests ports systems provided auto*/libtool tools.

Without modifications, the bug still affect this latest MySQL version on
both OpenBSD installations.

Sergei's propositions works well with OpenBSD 3.8 (that has automake 1.9,
among others).

But it don't work with OpenBSD 3.7, port's automake being too old there 
(1.4 only). Here is the transcript on 3.7:

$ uname -rs
OpenBSD 3.7
$ libtool --version  # latest from OpenBSD 3.7 ports
ltmain.sh (GNU libtool) 1.5.10 (1.1220.2.131 2004/09/19 12:46:56)
$ export AUTOMAKE_VERSION=1.4 AUTOCONF_VERSION=2.59 # latests from ports
$ aclocal
aclocal: configure.in: 567: macro `AM_PROG_AS' not found in library
$ autoheader
$ libtoolize --automake --force
$ automake --add-missing --force
automake: unrecognized option -- `--force'
$ automake --add-missing       # let's try without the '--force' option, then
aclocal.m4: 6378: `automake requires `AM_CONFIG_HEADER', not `AC_CONFIG_HEADER'
ndb/tools/Makefile.am:10: variable `LDADD_LOC' not defined
[... many others complaints about LDADD_LOC ...]
$ autoconf
$ cd bdb/dist && sh s_all && cd -
$ cd innobase && aclocal && autoheader && aclocal && automake && autoconf
$ ./configure && make
make  all-recursive
Making all in os
Bad modifier: )
Unclosed variable specification
"Makefile", line 262: Need an operator
Fatal errors encountered

Someone reported to have a similar problem on Fedora Core/amd64 on above
comments ; would be useful to know if the problem is fixed there also.

ps: INSTALL-SOURCE file is inexact: there's no BUILD/autorun.sh on 4.1.14
sources tarball.
[4 Oct 2005 11:41] Valeriy Kravchuk
I was able to repeat the behaviour originally described on our openbsd machine, with the following sequence of actions being applied to  mysql-4.1.15-nightly-20050930.tar.gz (latest nightly build available, I do not found a way to put a -BK copy on the machine yet):

./configure --prefix=/home/mysqldev/valeriy/dbs/4.1
make
make install
cd ../dbs/4.1
bin/mysql_install_db
bin/mysqld_safe &

All these work OK. But then:

mysqldev@ds9:~/valeriy/dbs/4.1> bin/mysql -uroot
bin/mysql: can't load library '../libmysql/.libs/libmysqlclient.14.0'
mysqldev@ds9:~/valeriy/dbs/4.1> ldd /home/mysqldev/valeriy/dbs/4.1/bin/mysql
/home/mysqldev/valeriy/dbs/4.1/bin/mysql:
/home/mysqldev/valeriy/dbs/4.1/bin/mysql: can't load library '../libmysql/.libs/
libmysqlclient.14.0'
/home/mysqldev/valeriy/dbs/4.1/bin/mysql: exit status 4
mysqldev@ds9:~/valeriy/dbs/4.1> bin/mysqladmin -uroot shutdown
bin/mysqladmin: can't load library '../libmysql/.libs/libmysqlclient.14.0'
mysqldev@ds9:~/valeriy/dbs/4.1> uname -a
OpenBSD ds9.orbus.fr 3.7 GENERIC.MP#0 i386

So, the problem still exists (and sources are still there), as described by previous reporters.

Yes, and a reconfiguration process proposed by Sergei does not work on 3.7, sorry. Moreover, it is really needed for -BK sources only. Standard sources .tar.gz do not need these command to compile and work successfully (on other platfroms).

Yes, there is no BUILD/autorun.sh in 4.1.x (it is in 5.0.x only), but its content is a set of commands provided by Sergei.
[12 Jan 2006 19:49] Ian Van Hoven
I followed Edward Frye's advice to fix mysql 4.1.16 on obsd 3.5. after configure/make/make install, i was seeing the same error.

to resolve...

% cd $INSTALLDIR/client

% g++ -O3 -DDBUG_OFF -fno-implicit-templates -fno-exceptions -fno-rtti -o .libs/mysql mysql.o readline.o sql_string.o completion_hash.o ../cmd-line-utils/libedit/libedit.a -lncurses -L ../libmysql/.libs/ -lmysqlclient -L/usr/lib -lm -lz -lssl -lcrypto -Wl,--rpath -Wl,/usr/local/lib/mysql

% cd ..

% make install

worked like a charm
[5 May 2006 13:58] pineau benjamin
The problem was still present on MySQL 4.1.18 and disapeared in 4.1.19 (at least, on my OpenBSD 3.8 i386 system).

Can someone confirm that the bug is fixed on OpenBSD 3.9 with MySQL > 4.1.18 ?

Btw, as a related note: the 4.1.19 OpenBSD 3.9 binaries provided by MySQL A.B. aren't  so usefull: they only contains statics libs (.a but no .so) (and subsenquently: all executables are staticly linked). Could this be corrected ?
[5 May 2006 17:32] Kent Boortz
We have upgraded automake and libtool, no other change
for OpenBSD. So this was likely corrected by the upgrade.

The binaries should not be statically linked, but executables
are deliberately linked static with libmysqlclient or other
libraries built during the server build. This is to avoid
problems with mismatch between MySQL libraries and
MySQL executables.