MySQL Bugs: #96698: Use gold as default on x86

Bug #96698	Use gold as default on x86_64 only
Submitted:	29 Aug 2019 13:19	Modified:	25 Nov 2019 19:37
Reporter:	Sam Avril	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S2 (Serious)
Version:	8.0.17 from github	OS:	Linux (Armbian on OrangePiPC)
Assigned to:		CPU Architecture:	ARM (armv7l)
Tags:	build, Memory

Description:
I have 17Go of swap space but only 1Go of real RAM. And every time i try to compile, i run into this error

[ 97%] Linking CXX executable ../../runtime_output_directory/merge_large_tests-t
/usr/bin/ld.gold: out of memory
collect2: error: ld returned 1 exit status
unittest/gunit/CMakeFiles/merge_large_tests-t.dir/build.make:2136: recipe for target 'runtime_output_directory/merge_large_tests-t' failed
make[2]: *** [runtime_output_directory/merge_large_tests-t] Error 1
CMakeFiles/Makefile2:12005: recipe for target 'unittest/gunit/CMakeFiles/merge_large_tests-t.dir/all' failed
make[1]: *** [unittest/gunit/CMakeFiles/merge_large_tests-t.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

How to repeat:
I cloned the repository, then create the build directory, get into it. I ran cmake :
cmake .. -DCMAKE_INSTALL_PREFIX=/opt/mysql -DCOMPILATION_COMMENT="Version pipi" -DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_unicode_ci -DDOWNLOAD_BOOST=1 -DWITH_BOOST=./boost -DENABLE_DOWNLOADS=1 -DENABLE_GCOV=1 -DENABLE_GPROF=1 -DENABLED_PROFILING=1 -DOPTIMIZER_TRACE=1 -DWITH_UNIXODBC=1

Then I ran make -j4, it crashed. I also tested with make

This issue got into the released tarball of 8.0.18 (mysql-boost-8.0.18.tar.gz)

The issue is then triggered on Fedora on ARMv7hl.

--

I am a Fedora maintainer who takes care of the community-mysql package.
Without the successful build for the armv7hl arch, I can't build and release the update to 8.0.18 into the Fedora.
Without the fix or workaround it will block any further affected version from entering Fedora.

--

The issue looks like this:

[100%] Linking CXX executable ../../../runtime_output_directory/pfs_connect_attr-t
cd /builddir/build/BUILD/mysql-8.0.18/build/storage/perfschema/unittest && /usr/bin/cmake -E cmake_link_script CMakeFiles/pfs_connect_attr-t.dir/link.txt --verbose=1
/usr/bin/c++  -std=c++14 -fno-omit-frame-pointer -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -march=armv7-a -mfpu=vfpv3-d16 -mtune=generic-armv7-a -mabi=aapcs-linux -mfloat-abi=hard -Wall -Wextra -Wformat-security -Wvla -Wundef -Woverloaded-virtual -Wcast-qual -Wimplicit-fallthrough=2 -Wlogical-op -DDBUG_OFF -ffunction-sections -fdata-sections -O2 -g -DNDEBUG  -fuse-ld=gold -Wl,--gc-sections -pie -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld CMakeFiles/pfs_connect_attr-t.dir/pfs_connect_attr-t.cc.o CMakeFiles/pfs_connect_attr-t.dir/__/__/__/sql/sql_builtin.cc.o  -o ../../../runtime_output_directory/pfs_connect_attr-t -lpthread ../../../unittest/mytap/libmytap.a ../libperfschema.a ../../../sql/libsql_main.a ../../archive/libarchive.a /usr/lib/libz.so ../../blackhole/libblackhole.a ../../csv/libcsv.a ../../federated/libfederated.a ../../heap/libheap.a ../../heap/libheap_library.a ../../innobase/libinnobase.a ../../../sql/libsql_dd.a ../../../sql/libsql_gis.a /usr/lib/libz.so /usr/lib/liblz4.so ../../myisam/libmyisam.a ../../myisam/libmyisam_library.a ../../myisammrg/libmyisammrg.a ../libperfschema.a /usr/lib/libicuuc.so /usr/lib/libicuio.so /usr/lib/libicudata.so /usr/lib/libicui18n.so ../../../sql/librpl.a ../../../sql/libmaster.a ../../../sql/libslave.a ../../../sql/libbinlog.a ../../../sql/libsql_main.a ../../innobase/libinnobase.a ../../../sql/libsql_dd.a ../../../sql/libsql_gis.a ../../../sql/librpl.a ../../../sql/libmaster.a ../../../sql/libslave.a ../../../sql/libbinlog.a ../../../sql/libsql_main.a ../../innobase/libinnobase.a ../../../sql/libsql_dd.a ../../../sql/libsql_gis.a ../../../sql/librpl.a ../../../sql/libmaster.a ../../../sql/libslave.a ../../../sql/libbinlog.a ../libperfschema.a ../../archive/libarchive.a ../../blackhole/libblackhole.a ../../csv/libcsv.a ../../federated/libfederated.a ../../heap/libheap.a ../../heap/libheap_library.a ../../myisam/libmyisam.a ../../myisam/libmyisam_library.a ../../myisammrg/libmyisammrg.a ../../temptable/libtemptable.a ../../../plugin/fulltext/libngram_parser.a ../../../plugin/x/libmysqlx.a /usr/lib/libevent_core.so /usr/lib/libevent_extra.so /usr/lib/libicuuc.so /usr/lib/libicuio.so /usr/lib/libicudata.so /usr/lib/libicui18n.so /usr/lib/libprotobuf-lite.so ../../../plugin/x/protocol/protobuf/libmysqlxmessages_lite.a ../../../components/mysql_server/component_mysql_server.a ../../../archive_output_directory/libvio.a -lcrypt -laio /usr/lib/liblz4.so ../../../libbinlogevents/lib/libbinlogevents.a ../../../archive_output_directory/libmysys.a /usr/lib/libz.so ../../../archive_output_directory/libstrings.a ../../../archive_output_directory/libmytime.a -lpthread /usr/lib/libzstd.so -lm -lrt /usr/lib/libssl.so /usr/lib/libcrypto.so -ldl 

/usr/bin/ld.gold: out of memory
collect2: error: ld returned 1 exit status

--

I have a interesting observation though: the issue is not triggered on the Fedora 29. So there may be chance to trace the issue to a specific package which changed (as all of them did) between F29 & F30.

I can try some, if you give me a tip. However the armv7hl builds are considerably slow, so finding it by random (check each buildroot package) is a no-go.

Build log on the Fedora Rawhide Armv7hl

Attachment: build.log.tar.gz (application/gzip, text), 334.72 KiB.

Buildroot log on the Fedora Rawhide Armv7hl

Attachment: root.log (text/x-log), 99.25 KiB.

I disabled liking with GOLD in Fedora on Armv7hl, until fix is found.

  -DUSE_LD_GOLD=OFF

Without GOLD, if build fine.

hi!

gold seems to be problematic on several archs, would make sense to disable
gold completely or use is on x86_64 only.

gold as default linker was added for faster build times I think, 
these days lld (the LLVM linker) would any way be more naturally choice
for a performant linker.

Posted by developer:
 
Fixed in 8.0.19.

The GNU gold loader could cause memory exhaustion on some platforms.
Now it is used by default only on Intel 64-bit platforms.