MySQL Bugs: #89615: Building Shell 1.0.10 and above from source fails for undefined vio* functions

Bug #89615	Building Shell 1.0.10 and above from source fails for undefined vio* functions
Submitted:	10 Feb 2018 18:35	Modified:	23 Aug 2018 6:10
Reporter:	Georgi Sotirov	Email Updates:
Status:	Unsupported	Impact on me:	None
Category:	Shell General / Core Client	Severity:	S2 (Serious)
Version:	1.0.10, 1.0.11	OS:	Linux (Slackware 14.2)
Assigned to:		CPU Architecture:	Any

Description:
It's impossible for me to build MySQL Shell 1.0.10 or 1.0.11 on Slackware 14.2 from source, because of undefined references. The related part from the build output is:

[ 97%] Linking CXX executable mysqlsh
/usr/bin/cmake -E cmake_link_script CMakeFiles/mysqlsh.dir/link.txt --verbose=1
/usr/bin/c++   -O3 -march=i586 -mtune=i686 -DHAVE_OPENSSL -std=c++11 -fPIC -O3 -DNDEBUG   CMakeFiles/mysqlsh.dir/src/main.cc.o CMakeFiles/mysqlsh.dir/src/get_password.cc.o CMakeFiles/mysqlsh.dir/src/shell_cmdline_options.cc.o CMakeFiles/mysqlsh.dir/src/cmdline_shell.cc.o CMakeFiles/mysqlsh.dir/src/boost_code.cc.o  -o mysqlsh -rdynamic -lpthread src/libedit/libedit.a shellcore/libmysqlshcore.a mysqlxtest/libmysqlxtest.a -lpython2.7 -lmysqlclient -lpthread -lz -lm -lrt -lssl -lcrypto -ldl -lprotobuf /usr/lib/libssl.a /usr/lib/libcrypto.a -ldl -lcurses -lpthread -lprotobuf
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::ssl_init_error::message(int) const':
mysqlx_sync_connection.cc:(.text+0x62): undefined reference to `sslGetErrString'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::Mysqlx_sync_connection(char const*, char const*, char const*, char const*, char const*, char const*, char const*, char const*, unsigned int)':
mysqlx_sync_connection.cc:(.text+0x1d4): undefined reference to `process_tls_version'
mysqlx_sync_connection.cc:(.text+0x21b): undefined reference to `new_VioSSLConnectorFd'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::connect(sockaddr_in*, unsigned int)':
mysqlx_sync_connection.cc:(.text+0x2e3): undefined reference to `vio_new'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::activate_tls()':
mysqlx_sync_connection.cc:(.text+0x495): undefined reference to `sslconnect'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::shutdown(mysqlx::Shutdown_type)':
mysqlx_sync_connection.cc:(.text+0x529): undefined reference to `vio_fd'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::close()':
mysqlx_sync_connection.cc:(.text+0x805): undefined reference to `vio_fd'
mysqlxtest/libmysqlxtest.a(mysqlx_sync_connection.cc.o): In function `mysqlx::Mysqlx_sync_connection::~Mysqlx_sync_connection()':
mysqlx_sync_connection.cc:(.text+0x85d): undefined reference to `free_vio_ssl_acceptor_fd'
collect2: error: ld returned 1 exit status
CMakeFiles/mysqlsh.dir/build.make:210: recipe for target 'mysqlsh' failed
make[2]: *** [mysqlsh] Error 1
make[2]: Leaving directory '/usr/src/tmp/mysql-shell-1.0.11-src/build'
CMakeFiles/Makefile2:72: recipe for target 'CMakeFiles/mysqlsh.dir/all' failed
make[1]: *** [CMakeFiles/mysqlsh.dir/all] Error 2
make[1]: Leaving directory '/usr/src/tmp/mysql-shell-1.0.11-src/build'
Makefile:152: recipe for target 'all' failed
make: *** [all] Error 2

I'm trying to build dynamically to OpenSSL 1.0.2n by adding -DWITH_SSL=/usr to CMake configuration in order to have HAVE_SSL defined on compile, as otherwise, there is error for undefined function. What am I missing? Verion 1.0.9 built without problem even without -DWITH_SSL=/usr, but with OpenSSL 1.0.2k at the time.

How to repeat:
1. Download MySQL Shell 1.0.11:

wget 'https://dev.mysql.com/get/Downloads/MySQL-Shell/mysql-shell-1.0.11-src.tar.gz'

2. Unpack:

tar -xvf mysql-shell-1.0.11-src.tar.gz ; cd mysql-shell-1.0.11-src

3. Configure:

mkdir build; cd build
cmake -LAH \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CXX_FLAGS="-O3 -march=i586 -mtune=i686" \
      -DCMAKE_C_FLAGS="-O3 -march=i586 -mtune=i686" \
      -DCMAKE_INSTALL_PREFIX="/usr" \
      -DCMAKE_VERBOSE_MAKEFILE=TRUE \
      -DMYSQL_LIB=/usr/lib/libmysqlclient.so \
      -DMYSQLCLIENT_STATIC_LINKING=FALSE \
      -DHAVE_PYTHON=1 \
      -DWITH_EDITLINE=bundled \
      -DWITH_NUMA=1 \
      -DWITH_PIC=ON \
      -DWITH_SSL=/usr \
      ..

4. Try to build:

make VERBOSE=1

That's HAVE_OPENSSL (not HAVE_SSL) of course.

Yep, if I add libmysqlclient.a (static library) to the command line for linking mysqlsh, everything's fine, which reminds me of bug 82202. Apparently, some symbols are still not exposed in libmysqlclient.so dynamic library.

Spent some time digging into the CMake scripts in MySQL Shell distribution to finally find out that adding -DMYSQL_EXTRA_LIBRARIES=/usr/lib${LIBDIRSUFFIX}/libmysqlclient.a to configure solves my problem. Still it would be very good if I'm able to link directly to the dynamic library. With MySQL versions released every 1-2 months I do not want to have to relink MySQL Shell with the static library after every new release.

Unfortunately, my previous "solution" (i.e. with adding -DMYSQL_EXTRA_LIBRARIES=/usr/lib/libmysqlclient.a) wasn't meant to be. The resulting binary has no problem connecting to MySQL using the X protocol, but classic connections fail with segmentation fault. I was able to produce the following with the debugger:

Program received signal SIGSEGV, Segmentation fault.
0x00007f8da43c87f0 in mysql_options () from /usr/lib64/libmysqlclient.so.20
(gdb) bt
#0  0x00007f8da43c87f0 in mysql_options () from /usr/lib64/libmysqlclient.so.20
#1  0x00000000006320c7 in mysqlsh::mysql::Connection::setup_ssl (this=this@entry=0x28035c0, ssl_info=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/modules/mysql_connection.cc:337
#2  0x0000000000633892 in mysqlsh::mysql::Connection::Connection (this=0x28035c0, host=..., port=3306, socket=..., user=..., password=..., schema=...,
    ssl_info=...) at /usr/src/tmp/mysql-shell-1.0.11-src/modules/mysql_connection.cc:281
#3  0x000000000061cb85 in mysqlsh::mysql::ClassicSession::connect (this=0x2805250, args=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/modules/mod_mysql_session.cc:112
#4  0x000000000060303f in mysqlsh::connect_session (args=..., session_type=<optimized out>)
    at /usr/src/tmp/mysql-shell-1.0.11-src/modules/base_session.cc:109
#5  0x00000000005a4ac5 in shcore::Shell_core::connect_dev_session (this=0x26f8460, args=..., session_type=<optimized out>)
    at /usr/src/tmp/mysql-shell-1.0.11-src/shellcore/shell_core.cc:439
#6  0x0000000000581a3c in mysqlsh::Base_shell::connect_session (this=this@entry=0x7ffdabd18cc0, args=..., session_type=mysqlsh::SessionType::Auto,
    recreate_schema=<optimized out>) at /usr/src/tmp/mysql-shell-1.0.11-src/shell/base_shell.cc:309
#7  0x00000000005830bf in mysqlsh::Base_shell::connect (this=this@entry=0x7ffdabd18cc0, primary_session=primary_session@entry=false)
    at /usr/src/tmp/mysql-shell-1.0.11-src/shell/base_shell.cc:270
#8  0x00000000005839e0 in mysqlsh::Base_shell::cmd_connect (this=0x7ffdabd18cc0, args=...) at /usr/src/tmp/mysql-shell-1.0.11-src/shell/base_shell.cc:646
#9  0x00000000005a788a in std::function<bool (std::vector<std::string, std::allocator<std::string> > const&)>::operator()(std::vector<std::string, std::allocator<std::string> > const&) const (__args#0=..., this=<optimized out>) at /usr/include/c++/5.5.0/functional:2267
#10 shcore::Shell_command_handler::process (this=this@entry=0x7ffdabd18e48, command_line=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/shellcore/shell_core.cc:832
#11 0x000000000058075d in mysqlsh::Base_shell::do_shell_command (this=this@entry=0x7ffdabd18cc0, line=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/shell/base_shell.cc:826
#12 0x0000000000584043 in mysqlsh::Base_shell::process_line (this=this@entry=0x7ffdabd18cc0, line=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/shell/base_shell.cc:838
#13 0x000000000053a2f2 in mysqlsh::Command_line_shell::command_loop (this=this@entry=0x7ffdabd18cc0)
    at /usr/src/tmp/mysql-shell-1.0.11-src/src/cmdline_shell.cc:164
#14 0x00000000004e0ab6 in main (argc=<optimized out>, argv=0x7ffdabd193f8) at /usr/src/tmp/mysql-shell-1.0.11-src/src/main.cc:265
(gdb) up
#1  0x00000000006320c7 in mysqlsh::mysql::Connection::setup_ssl (this=this@entry=0x28035c0, ssl_info=...)
    at /usr/src/tmp/mysql-shell-1.0.11-src/modules/mysql_connection.cc:337
337       mysql_options(_mysql, MYSQL_OPT_SSL_MODE, &value);
(gdb) l
332       if (ssl_info.mode)
333         value = ssl_info.mode;
334       else
335         value = static_cast<int>(shcore::SslMode::Preferred);
336
337       mysql_options(_mysql, MYSQL_OPT_SSL_MODE, &value);
338
339       return true;
340     }
341
(gdb) p _mysql
$1 = (MYSQL *) 0x0

So it appears to be null pointer dereference. Therefore, I tried linking directly to the static mysqlclient library (i.e. with -DMYSQL_LIB=/usr/lib/libmysqlclient.a) and this works, but the resulting binary is almost 10 times bigger.

How can we make MySQL Shell working again with dynamically linked mysqlclient library like it was the case with 1.0.9? I know Oracle is providing only statically linked binaries, but my aim is to link dynamically.

Hello Georgi,

Thank you for the report and feedback.

Thanks,
Umesh

Build details

Attachment: 89615_1.0.11.build (application/octet-stream, text), 257.23 KiB.

Yes, exactly. I could have provided you the full build log if you have asked :-)

Posted by developer:
 
MySQL Shell 1.0 is no longer supported and version 8.0 should be used instead.
Build instructions are described in the INSTALL file.

"MySQL Shell 1.0 is no longer supported and version 8.0 should be used instead."

Are you really serious with this statement!? MySQL Shell 1.0.11 is an _official_ GA release (available from https://dev.mysql.com/downloads/shell/), which I'm still building for older versions of Slackware. I consider building 8.0, but only for the next Slackware version.

"Build instructions are described in the INSTALL file."

How are general build instructions going to help me when the source itself doesn't  compile!? Have you even looked at the patch I provided?