MySQL Bugs: #48960: Compile from src.rpm with Federated or max config fails on Fedora 12 x86

Bug #48960	Compile from src.rpm with Federated or max config fails on Fedora 12 x86
Submitted:	21 Nov 2009 13:25	Modified:	23 Nov 2009 12:05
Reporter:	Michael Mussulis	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S1 (Critical)
Version:	5.1.39	OS:	Linux (Fedora 12 x86)
Assigned to:		CPU Architecture:	Any
Tags:	.spec, build, compile, config, federated, MAX, rpm, src

Description:
Hi,

On a newly built Fedora 12 x86 box, I needed to enable mysql with Federated engine. Enabling Federated either in my.cnf or manually from mysql using INSTALL plugin results with:

091120 14:28:35 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
091120 14:28:36 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
091120 14:28:36 [Note] Plugin 'ndbcluster' is disabled.
091120 14:28:36 [ERROR] Can't open shared library '/usr/lib/mysql/plugin/ha_federated.so' (errno: 0 undefined symbol: dynstr_append_mem)
091120 14:28:36  InnoDB: Started; log sequence number 0 367948
091120 14:28:36 [Note] Event Scheduler: Loaded 0 events
091120 14:28:36 [Note] /usr/libexec/mysqld: ready for connections.
Version: '5.1.39'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  Source distribution

So I decided to re-compile mysql from src.rpm, using a modified .spec file as follows:

removed:
--with-plugin-innobase \
--with-plugin-ndbcluster \

added

--with-plugins=max

no other changes made to the spec file. 

How to repeat:
1. Download mysql-5.1.39-4.fc12.src.rpm
2. rpm -ivh mysql-5.1.39-4.fc12.src.rpm
3. modify spec file as follows:

remove:
--with-plugin-innobase \
--with-plugin-ndbcluster \

add

--with-plugins=max \

4. Build with rpmbuild -bb mysql.spec

Build results

Attachment: build_results.txt (text/plain), 17.95 KiB.

Thank you for the bug report. Please see it is the same issue that the bug http://bugs.mysql.com/bug.php?id=48864. Thanks in advance.

Hi Migeul,

It's not the same bug from what I can see. The platforms differ, mine is on x86 (32 bit not 64), and the failure differs, mine fails due to the 'Federated' plugin with:

libfederated_a-ha_federated.o: In function `ha_federated::records_in_range(unsigned int, st_key_range*, st_key_range*)':
/root/rpmbuild/BUILD/mysql-5.1.39/storage/federated/../../sql/handler.h:1296: multiple definition of `ha_federated::records_in_range(unsigned int, st_key_range*, st_key_range*)'
ha_federated.o:/root/rpmbuild/BUILD/mysql-5.1.39/libmysqld/ha_federated.cc:1575: first defined here

See attached build results file.

fyi, this failure occurs also with mysql-5.1.37-1.fc11.src.rpm.

Also fails with mysql-5.1.40-1.fc13.src.rpm.

Hi Michael,

As I understand mysql-5.1.39-4.fc12.src.rpm is patched (by Redhat) mysql sources. 

Could you please check mysql sources from mysql.com and confirm that bug still presents?

Hi Alexey,

As requested, I tried building from mysql source tarball mysql-5.1.41.tar.gz. Built using:

[root@eurofeeds mysql-5.1.41]# cat mk.sh
#!/bin/sh

./configure \
        --with-readline \
        --with-ssl=/usr \
        --without-debug \
        --enable-shared \
        --with-embedded-server \
        --localstatedir=/var/lib/mysql \
        --with-unix-socket-path=/var/lib/mysql/mysql.sock \
        --with-mysqld-user="mysql" \
        --with-extra-charsets=all \
        --with-big-tables \
        --with-pic \
        --with-plugin-innobase \
        --with-plugin-ndbcluster \
        --with-plugin-archive \
        --with-plugin-blackhole \
        --with-plugin-csv \
        --with-plugin-federated \
        --with-plugin-partition \
        --without-example-storage-engine \
        --without-plugin-daemon_example \
        --without-plugin-ftexample \
        --enable-local-infile \
        --enable-largefile \
        --enable-thread-safe-client \
        --disable-dependency-tracking

make
make test

Result - builds but fails test:

rpl.rpl_row_stop_middle_update           [ skipped ]  Test needs debug binaries
rpl.rpl_slave_load_remove_tmpfile        [ skipped ]  Test needs debug binaries
rpl_ndb.rpl_ndb_2ndb                     [ disabled ]  Bug#45974: rpl_ndb_2ndb fails sporadically
ndb.ndb_binlog_ignore_db                 [ fail ]
        Test ended at 2009-11-22 13:54:08

CURRENT_TEST: ndb.ndb_binlog_ignore_db
mysqltest: In included file "./include/ndb_not_readonly.inc": At line 25: Failed while waiting for mysqld to come out of readonly mode
Failed to read from '/root/mysql-5.1.41/mysql-test/var/log/ndb_binlog_ignore_db.log', errno: 22

 - saving '/root/mysql-5.1.41/mysql-test/var/log/ndb.ndb_binlog_ignore_db/' to '/root/mysql-5.1.41/mysql-test/var/log/ndb.ndb_binlog_ignore_db/'

Only  301  of 1076 completed.
mysql-test-run: *** ERROR: Not all tests completed
make: *** [test-ns] Error 1

Also, the configure script does not recognize all plugin options, see attached config.log.

Regards,
Michael.
PS: I will now try your mysql source rpm and let you know.

config log from tarball source

Attachment: config.log-tarball-source.gz (application/x-gzip, text), 42.10 KiB.

Alexey,

Build from MySQL-5.1.41-0.glibc23.src.rpm, using default spec file, unaltered. Builds, however many tests fail, and then finally rpm packaging fails with:

Obsoletes: MySQL mysql mysql-server
Processing files: MySQL-client-5.1.41-0.glibc23.i686
Provides: mysql-client
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /bin/sh /usr/bin/perl libc.so.6 libc.so.6(GLIBC_2.0) libc.so.6(GLIBC_2.1) libc.so.6(GLIBC_2.1.2) libc.so.6(GLIBC_2.1.3) libc.so.6(GLIBC_2.11) libc.so.6(GLIBC_2.2) libc.so.6(GLIBC_2.3) libc.so.6(GLIBC_2.3.4) libc.so.6(GLIBC_2.4) libcrypt.so.1 libm.so.6 libm.so.6(GLIBC_2.0) libncursesw.so.5 libnsl.so.1 libpthread.so.0 libpthread.so.0(GLIBC_2.0) libpthread.so.0(GLIBC_2.1) libpthread.so.0(GLIBC_2.2) libpthread.so.0(GLIBC_2.3.2) libtinfo.so.5 perl(Exporter) perl(Fcntl) perl(File::Temp) perl(Getopt::Long) perl(IPC::Open3) perl(Sys::Hostname) rtld(GNU_HASH)
Obsoletes: mysql-client
Processing files: MySQL-test-5.1.41-0.glibc23.i686
Processing files: MySQL-devel-5.1.41-0.glibc23.i686
error: File not found: /root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/lib/mysql/libmygcc.a
Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.MGT3fi
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd mysql-5.1.41
+ DOCDIR=/root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/share/doc/MySQL-devel-5.1.41
+ export DOCDIR
+ rm -rf /root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/share/doc/MySQL-devel-5.1.41
+ /bin/mkdir -p /root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/share/doc/MySQL-devel-5.1.41
+ cp -pr mysql-release-5.1.41/EXCEPTIONS-CLIENT /root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/share/doc/MySQL-devel-5.1.41
+ exit 0

RPM build errors:
    File not found: /root/rpmbuild/BUILDROOT/MySQL-5.1.41-0.glibc23.i386/usr/lib/mysql/libmygcc.a

Alexey,

The build breaks due to a conflict between "ha_federated.o" and "libfederated_a-ha_federated.o" - both linked when compiling libmysqld.

Modifying the .spec file to remove ha_federated.o just before the compile command solves the problem, as in:

--- mysql.spec  2009-10-19 17:36:40.000000000 +0300
+++ mysql.5.1.39.spec   2009-11-23 02:02:03.182375378 +0200
@@ -217,6 +217,14 @@
        --with-pic \
        --with-plugin-innobase \
        --with-plugin-ndbcluster \
+       --with-plugin-archive \
+       --with-plugin-blackhole \
+       --with-plugin-csv \
+       --with-plugin-federated \
+       --with-plugin-partition \
+       --without-example-storage-engine \
+       --without-plugin-daemon_example \
+       --without-plugin-ftexample \
        --enable-local-infile \
        --enable-largefile \
        --enable-thread-safe-client \
@@ -232,6 +240,10 @@
 mkdir libmysqld/work
 cd libmysqld/work
 ar -x ../libmysqld.a
+
+# multiple definitions error - breaks build???
+rm ha_federated.o
+
 gcc $CFLAGS $LDFLAGS -shared -Wl,-soname,libmysqld.so.0 -o libmysqld.so.0.0.1 \
        *.o \
        -lpthread -lcrypt -lnsl -lssl -lcrypto -lz -lrt -lstdc++ -lm -lc

There is similar bug #47759

I opened bug #48977 about NDB tests failures.

Can you run server compiled from our sources? Can you use federated storage engine with it?

Sveta,

Indeed bug 47759 is similar, but not identical. That bug talks about 'loading' the plugin (most likely the default ha_federated.so shared library that is intalled with the default mysql rpm package).

My bug talks about both loading and compiling the federated plugin, compiling being a totally different kettle of fish.

I have not yet managed to build mysql from your source (tarball); the build fails during testing stage - see earlier comment plus build results.

I have now managed to get past the ha_federated multiple symbol error (in the Fedora 12 source rpm), thanks to the build made by Remi, by removing ha_federated.so during the build of libmysqld.so. However, the build fails during the test stage, which I reckon I can resolve. I will post my results shortly.

All in all, the most obvious pattern emerging from all these builds I've performed (and I've built all versions from 5.0.67 to 5.1.40 (src.rpm)) is that building fails - either during core build or testing. So far, the only build that succeeded was v5.0.84 (src.rpm).

Thanks,
Michael.

Thank you for the feedback.

> the build fails during testing stage - see earlier comment plus build results.

Correct. But this is different problem from one reported originally.

Please cd to mysql-test directory inside MySQL source directory and send us output of

./mysql-test-run.pl --do-test=federated

Sveta,

I have finally completed the build of mysql-5.1.39-4.fc12.src.rpm, again with the help of Remi spec file. Apparently the 'workaround' is to perform a minimal set of tests as in:

  #make test-ns <= remmed

  # RC minimal test
  cd mysql-test
  %{__perl} ./mysql-test-run.pl \
  --ssl --mysqld=--binlog-format=mixed --suite=main || :

Replacing mysql with the new build rpms and modifying my.cnf with

plugin-load=federated=ha_federated.so

results with:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.1.39-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show engines;
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine     | Support | Comment                                                        | Transactions | XA   | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| ndbcluster | NO      | Clustered, fault-tolerant tables                               | NULL         | NULL | NULL       |
| MRG_MYISAM | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| BLACKHOLE  | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| CSV        | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| FEDERATED  | NO      | Federated MySQL storage engine                                 | NULL         | NULL | NULL       |
| ARCHIVE    | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
| MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great performance         | NO           | NO   | NO         |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

replacing the above with:

#plugin-load=federated=ha_federated.so
federated

results with:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.39-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show engines;
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| Engine     | Support | Comment                                                        | Transactions | XA   | Savepoints |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
| ndbcluster | NO      | Clustered, fault-tolerant tables                               | NULL         | NULL | NULL       |
| MRG_MYISAM | YES     | Collection of identical MyISAM tables                          | NO           | NO   | NO         |
| BLACKHOLE  | YES     | /dev/null storage engine (anything you write to it disappears) | NO           | NO   | NO         |
| CSV        | YES     | CSV storage engine                                             | NO           | NO   | NO         |
| MEMORY     | YES     | Hash based, stored in memory, useful for temporary tables      | NO           | NO   | NO         |
| FEDERATED  | YES     | Federated MySQL storage engine                                 | NO           | NO   | NO         |
| ARCHIVE    | YES     | Archive storage engine                                         | NO           | NO   | NO         |
| InnoDB     | YES     | Supports transactions, row-level locking, and foreign keys     | YES          | YES  | YES        |
| MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great performance         | NO           | NO   | NO         |
+------------+---------+----------------------------------------------------------------+--------------+------+------------+
9 rows in set (0.00 sec)

I've also tested it with an actual federated table and it works.

Sveta,

The result of the test you requested. Note, it's from the successful build. Let me know if you require a different test case.

Thanks,
Michael.

[root@eurofeeds mysql-test]# ./mysql-test-run.pl --do-test=federated
Logging: ./mysql-test-run.pl  --do-test=federated
091123 13:36:19 [ERROR] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!

091123 13:36:19 [ERROR] Aborting

MySQL Version 5.1.39
Checking supported features...
 - using ndbcluster when necessary, mysqld supports it
 - SSL connections supported
Collecting tests...
vardir: /root/rpmbuild/BUILD/mysql-5.1.39/mysql-test/var
Checking leftover processes...
 - found old pid 29650 in 'mysqld.2.pid', killing it...
   process did not exist!
 - found old pid 29647 in 'mysqld.1.pid', killing it...
   process did not exist!
Removing old var directory...
Creating var directory '/root/rpmbuild/BUILD/mysql-5.1.39/mysql-test/var'...
Installing system database...
Using server port 42686

==============================================================================

TEST                                      RESULT   TIME (ms)
------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
worker[1] mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped
federated.federated_innodb               [ skipped ]  No innodb support
federated.federated_transactions         [ disabled ]  Bug#29523 Transactions do not work
federated.federated                      [ pass ]   5698
federated.federated_archive              [ pass ]    170
federated.federated_bug_13118            [ pass ]    108
federated.federated_bug_25714            [ pass ]    197
federated.federated_server               [ pass ]   8152
------------------------------------------------------------
The servers were restarted 0 times
Spent 14.325 of 18 seconds executing testcases

All 5 tests were successful.

Thank you for the feedback.

So Federated table works with our sources? Please file bug report to Fedora so they fix their RPMs.

Regarding your compiling/make test errors I opened new bug #48977. Please subscribe to this bug and follow discussion.

If NDB are not only tests which fail please open new report.

Thank you for the interest in MySQL!

Sveta,

Sorry, I think there's been a misunderstanding. I managed to get Federated  to work with the source from Fedora, i.e. mysql-5.1.39-4.fc12.src.rpm (once the spec file is modified as described earlier).

I have not managed to get it to work with your sources yet. I have already logged a bug with Redhat regarding this, but so far nothing has happened.

Thanks,
Michael.

Michael,

thank you for the feedback. Please try if federated storage engine works with our sources for you and reopen the report if it doesn't. Thanks in advance.

Yes, this is broken with YOUR/MySQL sources... as of 5.1.47 and before.  When building with '--with-federated-storage-engine':

..snip...
libfederated_a-ha_federated.o: In function `ha_federated::append_stmt_insert(String*)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:1709: multiple definition of `ha_federated::append_stmt_insert(String*)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:1709: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::repair(THD*, st_ha_check_opt*)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2049: multiple definition of `ha_federated::repair(THD*, st_ha_check_opt*)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2049: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::read_range_first(st_key_range const*, st_key_range const*, bool, bool)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2470: multiple definition of `ha_federated::read_range_first(st_key_range const*, st_key_range const*, bool, bool)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2470: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::update_row(unsigned char const*, unsigned char*)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2094: multiple definition of `ha_federated::update_row(unsigned char const*, unsigned char*)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2094: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::delete_row(unsigned char const*)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2247: multiple definition of `ha_federated::delete_row(unsigned char const*)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2247: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::write_row(unsigned char*)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:1788: multiple definition of `ha_federated::write_row(unsigned char*)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:1788: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::index_read_idx_with_result_set(unsigned char*, unsigned int, unsigned char const*, unsigned int, ha_rkey_function, st_mysql_res**)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2375: multiple definition of `ha_federated::index_read_idx_with_result_set(unsigned char*, unsigned int, unsigned char const*, unsigned int, ha_rkey_function, st_mysql_res**)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2375: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::index_read_idx(unsigned char*, unsigned int, unsigned char const*, unsigned int, ha_rkey_function)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2345: multiple definition of `ha_federated::index_read_idx(unsigned char*, unsigned int, unsigned char const*, unsigned int, ha_rkey_function)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2345: first defined here
libfederated_a-ha_federated.o: In function `ha_federated::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function)':
/builddir/build/BUILD/mysql-5.1.47/storage/federated/ha_federated.cc:2320: multiple definition of `ha_federated::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function)'
ha_federated.o:/builddir/build/BUILD/mysql-5.1.47/libmysqld/ha_federated.cc:2320: first defined here
libfederated_a-ha_federated.o:(.data+0x0): multiple definition of `federated_storage_engine'
ha_federated.o:(.data+0x0): first defined here
libfederated_a-ha_federated.o:(.data+0x4): multiple definition of `builtin_federated_plugin_interface_version'
ha_federated.o:(.data+0x4): first defined here
libfederated_a-ha_federated.o:(.data+0x8): multiple definition of `builtin_federated_sizeof_struct_st_plugin'
ha_federated.o:(.data+0x8): first defined here
libfederated_a-ha_federated.o:(.data.rel+0x0): multiple definition of `builtin_federated_plugin'
ha_federated.o:(.data.rel+0x0): first defined here

The reason I am not building federated as a shared object is because this bug has not been resolved for 5.1 since I reported it in 2008 (the patch committed appears to only effect 5.5):

http://bugs.mysql.com/bug.php?id=40942