MySQL Bugs: #90211: Various warnings and errors when compiling MySQL 8 with Clang

Bug #90211	Various warnings and errors when compiling MySQL 8 with Clang
Submitted:	26 Mar 2018 3:18	Modified:	30 Nov 2020 13:28
Reporter:	Roel Van de Paar	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S1 (Critical)
Version:	8.0.4rc	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
As per attached file

How to repeat:
1) Install the latest clang from Chromium devs (and this automatically updates previous version installed with this method too);
sudo yum remove clang    # Or sudo apt-get remove clang    # Only required if this procedure has never been followed yet
cd ~
mkdir TMP_CLANG
cd TMP_CLANG
git clone --depth=1 https://chromium.googlesource.com/chromium/src/tools/clang
cd ..
TMP_CLANG/clang/scripts/update.py

2) Download mysql-server from github, checkout 8.0
3) Make a copy of the download dir and cd to that dir (or build out of source I suppose)
3) cmake . -DCMAKE_C_COMPILER=/home/$(whoami)/third_party/llvm-build/Release+Asserts/bin/clang -DCMAKE_CXX_COMPILER=/home/$(whoami)/third_party/llvm-build/Release+Asserts/bin/clang++ -DCMAKE_BUILD_TYPE=Debug -DWITH_SSL=system -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DDEBUG_EXTNAME=OFF -DENABLE_DOWNLOADS=1 -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/tmp -DENABLED_LOCAL_INFILE=1 -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 -DWITH_ZLIB=system -DWITH_KEYRING_TEST=ON -DWITH_ASAN=ON -DWITH_ASAN_SCOPE=ON -DWITH_UBSAN=ON
4) make -j10

Issues seen

Attachment: bug_90211.txt (text/plain), 25.11 KiB.

1) This was using Clang 7, but I suspect that Clang 6 will give similar results. 
2) When not using make -j10 but just make (i.e. single threaded), there are no such issues, though a few warnings are seen. I note the "Warning Free" at https://mysqlserverteam.com/mysql-8-0-source-code-improvements/

Hrm. I am seeing the SEGV on /git/mysql-server_dbg/rapid/plugin/x/protocol/mysqlx.proto in single threaded builds as well, but much later in the build

[ 72%] Running C++ protocol buffer compiler (lite) on /git/mysql-server_dbg/rapid/plugin/x/protocol/mysqlx.proto
AddressSanitizer:DEADLYSIGNAL
=================================================================
==7285==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x0000009fb9e7 bp 0x7fff0a4fc030 sp 0x7fff0a4fbff0 T0)
==7285==The signal is caused by a READ memory access.
==7285==Hint: address points to the zero page.
    #0 0x9fb9e6 in google::protobuf::FileOptions::Clear() (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9fb9e6)
    #1 0xcfab04 in google::protobuf::(anonymous namespace)::InlineParseFromCodedStream(google::protobuf::io::CodedInputStream*, google::protobuf::MessageLite*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #2 0xcfab04 in google::protobuf::(anonymous namespace)::InlineParseFromArray(void const*, int, google::protobuf::MessageLite*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #3 0xcfab04 in google::protobuf::MessageLite::ParseFromString(std::string const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xcfab04)
    #4 0x985851 in void google::protobuf::DescriptorBuilder::AllocateOptionsImpl<google::protobuf::FileDescriptor>(std::string const&, std::string const&, google::protobuf::FileDescriptor::OptionsType const&, google::protobuf::FileDescriptor*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x985851)
    #5 0x918f4c in google::protobuf::DescriptorBuilder::AllocateOptions(google::protobuf::FileOptions const&, google::protobuf::FileDescriptor*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x918f4c)
    #6 0x911550 in google::protobuf::DescriptorBuilder::BuildFile(google::protobuf::FileDescriptorProto const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x911550)
    #7 0x8f4e93 in google::protobuf::DescriptorPool::BuildFileFromDatabase(google::protobuf::FileDescriptorProto const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8f4e93)
    #8 0x8ec1da in google::protobuf::DescriptorPool::TryFindFileInFallbackDatabase(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ec1da)
    #9 0x8ebb11 in google::protobuf::DescriptorPool::FindFileByName(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ebb11)
    #10 0x9e8dce in google::protobuf::protobuf_AssignDesc_google_2fprotobuf_2fdescriptor_2eproto() (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9e8dce)
    #11 0xc9c8d7 in google::protobuf::internal::FunctionClosure0::Run() (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc9c8d7)
    #12 0xc9f3f6 in google::protobuf::GoogleOnceInitImpl(long*, google::protobuf::Closure*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc9f3f6)
    #13 0x6ef92e in google::protobuf::GoogleOnceInit(long*, void (*)()) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x6ef92e)
    #14 0xa68565 in google::protobuf::FileOptions::GetMetadata() const (/git/mysql-server_dbg/runtime_output_directory/protoc+0xa68565)
    #15 0xc759fc in google::protobuf::compiler::Parser::ParseOption(google::protobuf::Message*, google::protobuf::compiler::Parser::LocationRecorder const&, google::protobuf::FileDescriptorProto const*, google::protobuf::compiler::Parser::OptionStyle) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc759fc)
    #16 0xc72bc4 in google::protobuf::compiler::Parser::ParseTopLevelStatement(google::protobuf::FileDescriptorProto*, google::protobuf::compiler::Parser::LocationRecorder const&) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc72bc4)
    #17 0xc70c12 in google::protobuf::compiler::Parser::Parse(google::protobuf::io::Tokenizer*, google::protobuf::FileDescriptorProto*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc70c12)
    #18 0xc5c9bb in google::protobuf::compiler::SourceTreeDescriptorDatabase::FindFileByName(std::string const&, google::protobuf::FileDescriptorProto*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0xc5c9bb)
    #19 0x8ec1be in google::protobuf::DescriptorPool::TryFindFileInFallbackDatabase(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ec1be)
    #20 0x8ebb11 in google::protobuf::DescriptorPool::FindFileByName(std::string const&) const (/git/mysql-server_dbg/runtime_output_directory/protoc+0x8ebb11)
    #21 0x6759b1 in google::protobuf::compiler::CommandLineInterface::Run(int, char const* const*) (/git/mysql-server_dbg/runtime_output_directory/protoc+0x6759b1)
    #22 0x66c5f9 in main (/git/mysql-server_dbg/runtime_output_directory/protoc+0x66c5f9)
    #23 0x7f148d8dcc04 in __libc_start_main /usr/src/debug/glibc-2.17-c758a686/csu/../csu/libc-start.c:274
    #24 0x59636b in _start (/git/mysql-server_dbg/runtime_output_directory/protoc+0x59636b)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (/git/mysql-server_dbg/runtime_output_directory/protoc+0x9fb9e6) in google::protobuf::FileOptions::Clear()
==7285==ABORTING
make[2]: *** [rapid/plugin/x/generated/protobuf_lite/mysqlx.pb.cc] Error 1
make[1]: *** [rapid/plugin/x/protocol/CMakeFiles/mysqlxmessages_lite.dir/all] Error 2
make: *** [all] Error 2

Now testing with Clang 6. Warnings are seen all the same, so "Warning Free" in https://mysqlserverteam.com/mysql-8-0-source-code-improvements/ is not accurate.

Yes, so the 10 threads seems unrelated to the issue. Updated title to match description more accurately; Various warnings and errors when compiling MySQL 8 with Clang (though it may be that GCC has them too).

Hi,

First of all, these issues are in protobuf, not MySQL per se. There are some warnings with Clang 6, but since they're in third-party code, we have simply disabled them when compiling protobuf (will be part of 8.0.11). Optionally, -DUSE_SYSTEM_LIBS=1 will use system protobuf and thus not compile the files in question.

As for the crash, we don't support prerelease compilers (more generally, we support platforms, not compilers). Given the stack trace, it is highly likely that the issue either is in the prerelease Clang, or in protobuf.

Thank you for the input. Two notes;
1) This happens with Clang 6 AND Clang 7
2) Using -DWITH_RAPID=OFF makes the solution compile fine
Any updates based on these two items?

clang and sanitizers and the bundled version of protobuf that comes with our source do not play well together at all. I do not know whether this is a clang bug or a protoc bug. Stripping away -fsanitize=xxx when building protoc will make the build work, but then the mysql executables which are generated crash at runtime. Use -DWITH_PROTOBUF=system.

We have recently added:
extra/protobuf/CMakeLists.txt:  MY_CHECK_CXX_COMPILER_FLAG("-Wenum-compare-switch" HAVE_ENUM_COMPARE_SWITCH)
extra/protobuf/CMakeLists.txt:    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-enum-compare-switch")

Roel,

I guess that these explanations are sufficient and that we can close this bug ...

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

> I guess that these explanations are sufficient and that we can close this bug

Not sure I am following this.

Changing status as agreed .....