Bug #67790 MySQL 5.5 and 5.6 fails to build if GCC Link Time Optimization (-flto) is used
Submitted: 2 Dec 2012 18:24 Modified: 12 Mar 2013 15:49
Reporter: Laurynas Biveinis (OCA) Email Updates:
Status: Won't fix Impact on me:
None 
Category:MySQL Server: Compiling Severity:S3 (Non-critical)
Version:5.5 revno 3953, 5.6 revno 4458, 5.5.28, 5.6.8 OS:Linux (Ubuntu 12.10 x86_64)
Assigned to: Tor Didriksen CPU Architecture:Any

[2 Dec 2012 18:24] Laurynas Biveinis
Description:
If -flto is added to GCC flags, build fails with: 5.5:

Linking CXX shared library libmysqlclient.so
libmysql.c.o (symbol from plugin): warning: memset used with constant zero length parameter; this could be due to transposed parameters
In file included from :0:0:
/home/laurynas/percona/src/5.5/obj-lto/libmysql/libmysql_exports_file.cc:101:15: error: function ‘mysql_embedded’ redeclared as variable
In file included from /usr/include/unistd.h:1103:0,
                 from /home/laurynas/percona/src/5.5/obj-lto/libmysql/libmysql_exports_file.cc:111,
                 from :0:
/home/laurynas/percona/src/5.5/libmysql/libmysql.c:1096:9: note: previously declared here
In file included from :0:0:
/home/laurynas/percona/src/5.5/obj-lto/libmysql/libmysql_exports_file.cc:60:15: error: function ‘mysql_read_query_result’ redeclared as variable
In file included from /usr/include/unistd.h:1139:0,
                 from /home/laurynas/percona/src/5.5/obj-lto/libmysql/libmysql_exports_file.cc:111,
                 from :0:
/home/laurynas/percona/src/5.5/libmysql/libmysql.c:4861:9: note: previously declared here

... and so on.

How to repeat:
$ export CFLAGS=-flto
$ export CXXFLAGS=-flto
$ cmake .. -DBUILD_CONFIG=mysql_release -DCMAKE_BUILD_TYPE=RelWithDebInfo -DWITH_EMBEDDED_SERVER=OFF
[2 Dec 2012 18:51] Laurynas Biveinis
$ gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc-4.7.real
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.7.2-2ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.7 --enable-shared --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.7.2 (Ubuntu/Linaro 4.7.2-2ubuntu1)
[2 Dec 2012 18:57] Laurynas Biveinis
5.6:

[ 48%] Built target mytap
Linking CXX executable pfs-t
pfs_global.cc.o (symbol from plugin): In function `pfs_malloc(unsigned long, int)':
(.text+0x0): multiple definition of `pfs_malloc(unsigned long, int)'
CMakeFiles/pfs-t.dir/pfs-t.cc.o (symbol from plugin):(.text+0x0): first defined here
pfs_global.cc.o (symbol from plugin): In function `pfs_malloc(unsigned long, int)':
(.text+0x0): multiple definition of `pfs_free(void*)'
CMakeFiles/pfs-t.dir/pfs-t.cc.o (symbol from plugin):(.text+0x0): first defined here
pfs_global.cc.o (symbol from plugin): In function `pfs_malloc(unsigned long, int)':
(.text+0x0): multiple definition of `pfs_print_error(char const*, ...)'
CMakeFiles/pfs-t.dir/pfs-t.cc.o (symbol from plugin):(.text+0x0): first defined here
pfs_global.cc.o (symbol from plugin): In function `pfs_malloc(unsigned long, int)':
(.text+0x0): multiple definition of `pfs_initialized'
CMakeFiles/pfs-t.dir/pfs-t.cc.o (symbol from plugin):(.text+0x0): first defined here

... and a bunch of other errors
[4 Dec 2012 22:31] Sveta Smirnova
Thank you for the report.

I can not repeat described behavior.

I also noticed you use Percona sources. Please try with Oracle sources and inform us if problem still exists.
[5 Dec 2012 5:24] Laurynas Biveinis
Sveta -

Thanks for getting back to me. Is it because of the paths that you say these are Percona sources? These are actually the Oracle bzr sources. The exact versions used I specified in the Version field. I will try with source tarballs too.
[5 Dec 2012 5:45] Laurynas Biveinis
Exact same error with mysql-5.5.28.tar.gz.
[5 Dec 2012 6:09] Laurynas Biveinis
Exact same error with mysql-5.6.8-rc.tar.gz.
[5 Dec 2012 18:04] Sveta Smirnova
Thank you for the feedback.

Compile of version 5.5.28 fails with different error for me:

[ 30%] Generating ../include/mysqld_error.h, ../sql/share/english/errmsg.sys
/bin/sh: line 1: 10120 Segmentation fault      (core dumped) ./comp_err --charset=/home/sveta/Downloads/mysql-5.5.28/sql/share/charsets --out-dir=/home/sveta/Downloads/mysql-5.5.28/sql/share/ --header_file=/home/sveta/Downloads/mysql-5.5.28/include/mysqld_error.h --name_file=/home/sveta/Downloads/mysql-5.5.28/include/mysqld_ername.h --state_file=/home/sveta/Downloads/mysql-5.5.28/include/sql_state.h --in_file=/home/sveta/Downloads/mysql-5.5.28/sql/share/errmsg-utf8.txt
make[2]: *** [include/mysqld_error.h] Error 139
make[1]: *** [extra/CMakeFiles/GenError.dir/all] Error 2

$ /usr/local/bin/gcc --version
gcc (GCC) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Current development sources compiles without error. But I set this bug as "Verified", because our released packages still suffer from this issue.
[5 Dec 2012 19:06] Laurynas Biveinis
Sveta -

I suspect that LTO might work only partially in your environment and that would explain why you see no issues or different issues for this bug and bug 67789.

"Basic" LTO would work for each static library and executable, but would not cross their boundaries. I.e. if mysqld is linked with libinnobase.a, LTO would not apply across their boundary.

For that to happen, i.e. for full LTO, a linker plugin must be used. Its requirements are either GNU ld 2.21+ either Gold linker and then GCC option -fuse-linker-plugin in addition to -flto. Ubuntu's GCC is configured to add it silently when -flto is used it seems. But I am not aware of any general way to confirm that the linker plugin is used. Perhaps you could try adding -fuse-linker-plugin too and see if anything fails then.

I tried building with -flto -fno-use-linker-plugin (i.e. what I suspect your configuration is) and the build does not fail for me then neither.
[5 Dec 2012 19:15] Laurynas Biveinis
Another indication of linker plugin in use might be that the build produces many warnings in the form

ctype-czech.c.o (symbol from plugin): warning: memset used with constant zero length parameter; this could be due to transposed parameters
[21 Feb 2013 17:47] Raghavendra Prabhu
I am able to build with this patch:

 === modified file 'Percona-Server/cmake/libutils.cmake'
 --- Percona-Server/cmake/libutils.cmake 2011-06-30 15:46:53 +0000
 +++ Percona-Server/cmake/libutils.cmake 2013-02-21 17:29:25 +0000
 @@ -249,7 +249,7 @@
          ENDIF()
        ENDFOREACH()
      ENDIF()
 -    CREATE_EXPORT_FILE(SRC ${TARGET} "${ARG_EXPORTS}")
 +    #CREATE_EXPORT_FILE(SRC ${TARGET} "${ARG_EXPORTS}")
      IF(NOT ARG_NOINSTALL)
        ADD_VERSION_INFO(${TARGET} SHARED SRC)
      ENDIF()

I used following CFLAGS:

export CFLAGS="-Wall -Ofast -g -static-libgcc -fno-omit-frame-pointer  -flto "
export CXXFLAGS="-Ofast -fno-omit-frame-pointer -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fno-exceptions  -flto"

What are the implications of that patch? I didn't understand the requirement of the exports file created -- libmysql_exports_file.cc
[12 Mar 2013 15:05] Tor Didriksen
See comments for CREATE_EXPORT_FILE in cmake/libutils.cmake

If you remove libmysql_exports_file.cc you will end up with
a mostly empty libmysqlclient.so
That can be fixed by the --whole-archive flag, see 'man ld'
[12 Mar 2013 15:48] Tor Didriksen
Note, from 'man gcc':
Combining -flto with -g is currently experimental and expected
to produce wrong results.
[11 Aug 2019 17:08] Björn Voigt
The bug still exists. openSUSE Tumbleweed now uses LTO by default. See https://lists.opensuse.org/opensuse-factory/2019-07/msg00240.html

MySQL server was removed from the official openSUSE Tumbleweed repositories in favor of MariaDB. I removed the LTO flags from the default compilation flags in my MySQL home repository: https://build.opensuse.org/package/show/home:bjoernv/mysql-community-server

Did anyone tested if MySQL 5.7 and 5.8 work with LTO enabled?
[12 Aug 2019 14:34] Tor Didriksen
Posted by developer:
 
should be fixed in 8.0.13 (although I admit we haven't started using it actively until quite recently)

commit 2bbea0037d7c4d00e6d21bbc04701c5400edc670
Author: Tor Didriksen <tor.didriksen@oracle.com>
Date:   Thu Jun 14 12:42:06 2018 +0200

    Bug#28184537 ADD SUPPORT FOR -FLTO LINK-TIME OPTIMIZATION
    
    Add an option -DWITH_LTO which essentially does:
    cmake -DCMAKE_C_FLAGS=-flto -DCMAKE_CXX_FLAGS=-flto
          -DCMAKE_CXX_LINK_FLAGS="-flto=8" -DWITH_SYSTEM_LIBS=1
    
    Fix linker/compiler warnings:
      [-Wmaybe-uninitialized]
      [-Wuninitialized]
      [-Wlto-type-mismatch]
      [-Wodr]
      type 'struct xxx' should match type 'struct xxx'

WITH_SYSTEM_LIBS doesn't work on all platforms though.