Bug #90211 Various warnings and errors when compiling MySQL 8 with Clang
Submitted: 26 Mar 2018 3:18 Modified: 30 Nov 2020 13:28
Reporter: Roel Van de Paar Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: Compiling Severity:S1 (Critical)
Version:8.0.4rc OS:Any
Assigned to: CPU Architecture:Any

[26 Mar 2018 3:18] Roel Van de Paar
As per attached file

How to repeat:
1) Install the latest clang from Chromium devs (and this automatically updates previous version installed with this method too);
sudo yum remove clang    # Or sudo apt-get remove clang    # Only required if this procedure has never been followed yet
cd ~
git clone --depth=1 https://chromium.googlesource.com/chromium/src/tools/clang
cd ..

2) Download mysql-server from github, checkout 8.0
3) Make a copy of the download dir and cd to that dir (or build out of source I suppose)
3) cmake . -DCMAKE_C_COMPILER=/home/$(whoami)/third_party/llvm-build/Release+Asserts/bin/clang -DCMAKE_CXX_COMPILER=/home/$(whoami)/third_party/llvm-build/Release+Asserts/bin/clang++ -DCMAKE_BUILD_TYPE=Debug -DWITH_SSL=system -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DDEBUG_EXTNAME=OFF -DENABLE_DOWNLOADS=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/tmp -DENABLED_LOCAL_INFILE=1 -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DWITH_ZLIB=system -DWITH_KEYRING_TEST=ON -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON
4) make -j10
[26 Mar 2018 3:19] Roel Van de Paar
Issues seen

Attachment: bug_90211.txt (text/plain), 25.11 KiB.

[26 Mar 2018 5:40] Roel Van de Paar
1) This was using Clang 7, but I suspect that Clang 6 will give similar results. 
2) When not using make -j10 but just make (i.e. single threaded), there are no such issues, though a few warnings are seen. I note the "Warning Free" at https://mysqlserverteam.com/mysql-8-0-source-code-improvements/
[26 Mar 2018 8:32] Roel Van de Paar
Hrm. I am seeing the SEGV on /git/mysql-server_dbg/rapid/plugin/x/protocol/mysqlx.proto in single threaded builds as well, but much later in the build

[ 72%] Running C++ protocol buffer compiler (lite) on /git/mysql-server_dbg/rapid/plugin/x/protocol/mysqlx.proto
==7285==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x0000009fb9e7 bp 0x7fff0a4fc030 sp 0x7fff0a4fbff0 T0)
==7285==The signal is caused by a READ memory access.
==7285==Hint: address points to the zero page.
    #0 0x9fb9e6 in google::protobuf::FileOptions::Clear() (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9fb9e6)
    #1 0xcfab04 in google::protobuf::(anonymous namespace)::InlineParseFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #2 0xcfab04 in google::protobuf::(anonymous namespace)::InlineParseFromArray(void const*, int, google::protobuf::MessageLite*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #3 0xcfab04 in google::protobuf::MessageLite::ParseFromString(std::string const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #4 0x985851 in void google::protobuf::DescriptorBuilder::AllocateOptionsImpl<google::protobuf::FileDescriptor>(std::string const&, std::string const&, google::protobuf::FileDescriptor::OptionsType const&, google::protobuf::FileDescriptor*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x985851)
    #5 0x918f4c in google::protobuf::DescriptorBuilder::AllocateOptions(google::protobuf::FileOptions const&, google::protobuf::FileDescriptor*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x918f4c)
    #6 0x911550 in google::protobuf::DescriptorBuilder::BuildFile(google::protobuf::FileDescriptorProto const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x911550)
    #7 0x8f4e93 in google::protobuf::DescriptorPool::BuildFileFromDatabase(google::protobuf::FileDescriptorProto const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8f4e93)
    #8 0x8ec1da in google::protobuf::DescriptorPool::TryFindFileInFallbackDatabase(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ec1da)
    #9 0x8ebb11 in google::protobuf::DescriptorPool::FindFileByName(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ebb11)
    #10 0x9e8dce in google::protobuf::protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9e8dce)
    #11 0xc9c8d7 in google::protobuf::internal::FunctionClosure0::Run() (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc9c8d7)
    #12 0xc9f3f6 in google::protobuf::GoogleOnceInitImpl(long*, google::protobuf::Closure*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc9f3f6)
    #13 0x6ef92e in google::protobuf::GoogleOnceInit(long*, void (*)()) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x6ef92e)
    #14 0xa68565 in google::protobuf::FileOptions::GetMetadata() const (/git/mysql-server_dbg/runtime_output_directory/protoc+0xa68565)
    #15 0xc759fc in google::protobuf::compiler::Parser::ParseOption(google::protobuf::Message*, google::protobuf::compiler::Parser::LocationRecorder const&, google::protobuf::FileDescriptorProto const*, google::protobuf::compiler::Parser::OptionStyle) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc759fc)
    #16 0xc72bc4 in google::protobuf::compiler::Parser::ParseTopLevelStatement(google::protobuf::FileDescriptorProto*, google::protobuf::compiler::Parser::LocationRecorder const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc72bc4)
    #17 0xc70c12 in google::protobuf::compiler::Parser::Parse(google::protobuf::io::Tokenizer*, google::protobuf::FileDescriptorProto*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc70c12)
    #18 0xc5c9bb in google::protobuf::compiler::SourceTreeDescriptorDatabase::FindFileByName(std::string const&, google::protobuf::FileDescriptorProto*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc5c9bb)
    #19 0x8ec1be in google::protobuf::DescriptorPool::TryFindFileInFallbackDatabase(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ec1be)
    #20 0x8ebb11 in google::protobuf::DescriptorPool::FindFileByName(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ebb11)
    #21 0x6759b1 in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x6759b1)
    #22 0x66c5f9 in main (/git/mysql-server_dbg/runtime_output_directory/protoc+0x66c5f9)
    #23 0x7f148d8dcc04 in __libc_start_main /usr/src/debug/glibc-2.17-c758a686/csu/../csu/libc-start.c:274
    #24 0x59636b in _start (/git/mysql-server_dbg/runtime_output_directory/protoc+0x59636b)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9fb9e6) in google::protobuf::FileOptions::Clear()
make[2]: *** [rapid/plugin/x/generated/protobuf_lite/mysqlx.pb.cc] Error 1
make[1]: *** [rapid/plugin/x/protocol/CMakeFiles/mysqlxmessages_lite.dir/all] Error 2
make: *** [all] Error 2
[26 Mar 2018 8:38] Roel Van de Paar
Now testing with Clang 6. Warnings are seen all the same, so "Warning Free" in https://mysqlserverteam.com/mysql-8-0-source-code-improvements/ is not accurate.
[26 Mar 2018 9:00] Roel Van de Paar
Yes, so the 10 threads seems unrelated to the issue. Updated title to match description more accurately; Various warnings and errors when compiling MySQL 8 with Clang (though it may be that GCC has them too).
[26 Mar 2018 14:41] Steinar Gunderson

First of all, these issues are in protobuf, not MySQL per se. There are some warnings with Clang 6, but since they're in third-party code, we have simply disabled them when compiling protobuf (will be part of 8.0.11). Optionally, -DUSE_SYSTEM_LIBS=1 will use system protobuf and thus not compile the files in question.

As for the crash, we don't support prerelease compilers (more generally, we support platforms, not compilers). Given the stack trace, it is highly likely that the issue either is in the prerelease Clang, or in protobuf.
[27 Mar 2018 6:14] Roel Van de Paar
Thank you for the input. Two notes;
1) This happens with Clang 6 AND Clang 7
2) Using -DWITH_RAPID=OFF makes the solution compile fine
Any updates based on these two items?
[27 Mar 2018 7:34] Tor Didriksen
clang and sanitizers and the bundled version of protobuf that comes with our source do not play well together at all. I do not know whether this is a clang bug or a protoc bug. Stripping away -fsanitize=xxx when building protoc will make the build work, but then the mysql executables which are generated crash at runtime. Use -DWITH_PROTOBUF=system.
[27 Mar 2018 7:36] Tor Didriksen
We have recently added:
extra/protobuf/CMakeLists.txt:  MY_CHECK_CXX_COMPILER_FLAG("-Wenum-compare-switch" HAVE_ENUM_COMPARE_SWITCH)
extra/protobuf/CMakeLists.txt:    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-enum-compare-switch")
[30 Mar 2018 15:14] MySQL Verification Team

I guess that these explanations are sufficient and that we can close this bug ...
[1 May 2018 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[25 Nov 2020 3:38] Roel Van de Paar
> I guess that these explanations are sufficient and that we can close this bug

Not sure I am following this.
[30 Nov 2020 13:28] MySQL Verification Team
Changing status as agreed .....