MySQL Bugs: #94725: zlib symbols of libmysqlclient.so conflict with ones of zlib.so

Bug #94725	zlib symbols of libmysqlclient.so conflict with ones of zlib.so
Submitted:	20 Mar 2019 19:52	Modified:	22 Mar 2019 16:49
Reporter:	Garri Djavadyan	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: C API (client library)	Severity:	S4 (Feature request)
Version:	5.6.43	OS:	CentOS (7.6.1810)
Assigned to:		CPU Architecture:	x86 (x86_64)
Tags:	zlib

Description:
I experience problems with Perl application using both MySQL and Zlib modules. In particular, /usr/lib64/perl5/vendor_perl/auto/Compress/Raw/Zlib/Zlib.so calls inflateReset function from /usr/lib64/mysql/libmysqlclient.so.18.1.0 library instead of /usr/lib64/libz.so.1.2.7 one and crashes as a result.

I posted more details on Stackoverflow page:
https://stackoverflow.com/questions/55234441/stream-error-while-trying-to-initialize-defla...

How to repeat:
1. Install mysql-community-server from official MySQL repository for CentOS 7.
2. Run Perl application using MySQL and Zlib modules.

Below is a relevant output from gdb:

# gdb
...
(gdb) file /usr/bin/perl
Reading symbols from /usr/bin/perl...Reading symbols from /usr/lib/debug/usr/bin/perl5.16.3.debug...done.
done.

(gdb) set args libexec/billd test

(gdb) b deflateReset
Function "deflateReset" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (deflateReset) pending.

(gdb) r
Starting program: /usr/bin/perl libexec/billd test
...
Breakpoint 1, deflateReset (strm=strm@entry=0x2808138)
    at /export/home/pb2/build/sb_0-32010456-1545380068.25/rpm/BUILD/mysql-5.6.43/mysql-5.6.43/zlib/deflate.c:507
507	/export/home/pb2/build/sb_0-32010456-1545380068.25/rpm/BUILD/mysql-5.6.43/mysql-5.6.43/zlib/deflate.c: No such file or directory.
...
(gdb) bt
#0  deflateReset (strm=strm@entry=0x2808138)
    at /export/home/pb2/build/sb_0-32010456-1545380068.25/rpm/BUILD/mysql-5.6.43/mysql-5.6.43/zlib/deflate.c:507
#1  0x00007fffedfdcb58 in deflateInit2_ (strm=strm@entry=0x2808138, level=level@entry=9, method=method@entry=8, windowBits=<optimized out>, 
    windowBits@entry=15, memLevel=<optimized out>, strategy=strategy@entry=0, version=version@entry=0x7fffeb71e47a "1.2.7", 
    stream_size=stream_size@entry=112) at /export/home/pb2/build/sb_0-32010456-1545380068.25/rpm/BUILD/mysql-5.6.43/mysql-5.6.43/zlib/deflate.c:347
#2  0x00007fffeb71c8cb in XS_Compress__Raw__Zlib__deflateInit (my_perl=0x603010, cv=<optimized out>) at Zlib.xs:796
#3  0x00007ffff7b0941f in Perl_pp_entersub (my_perl=0x603010) at pp_hot.c:2778
#4  0x00007ffff7b01b96 in Perl_runops_standard (my_perl=0x603010) at run.c:41
#5  0x00007ffff7a9e985 in S_run_body (oldscope=<optimized out>, my_perl=<optimized out>) at perl.c:2402
#6  perl_run (my_perl=0x603010) at perl.c:2320
#7  0x0000000000400ce9 in main (argc=3, argv=0x7fffffffe458, env=0x7fffffffe478) at perlmain.c:120

(gdb) info b
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   <MULTIPLE>         
	breakpoint already hit 1 time
1.1                         y     0x00007fffedd44180 in deflateReset at deflate.c:429
1.2                         y     0x00007fffedfda5c0 in deflateReset 
                                                   at /export/home/pb2/build/sb_0-32010456-1545380068.25/rpm/BUILD/mysql-5.6.43/mysql-5.6.43/zlib/deflate.c:507

Hi,

Thank you for your bug report.

Let me inform you that all binaries in MySQL binary packages are built with zlib that is a part of the MySQL source code tree. If you build MySQL yourself, you have to build all executables and binaries with the static version of the zlib that is supplied with MySQL source code. 

If you build MySQL binaries with system installed zlib library, you are on your own.

Also, we are not maintainers of the  Perl programming language and its modules.

Hence, I do not see how is this our bug ????

Hi Sinisa,

I think I failed to explain the issue. The core problem is that any binary loading both system's zlib.so and official MySQL's libmysqlclient.so shared libraries at the same time has a chance to call wrong implementation of a zlib function. I believe, it is because the zlib symbols of libmysqlclient.so are globally accessible for any binary which use libmysqlclient.so shared library. As a result, libmysqlclient.so poses itself as an alternative zlib implementation, effectively overriding system's zlib functions.

For example, our Perl application uses MySQL and PDF functions, as a result, perl process loads both libmysqlclient.so and zlib.so. But because zlib functions (deflateInit, deflateReset ...) are available in both libraries, some functions are called from one library and some of them from another.

As I understand correctly, MySQL uses static zlib for internal functions only and therefore the zlib symbols should not be exported globally.

I'm sorry, I'm not a developer. Therefore, my explanation may sound a bit wrong. I just tried to explain the situation I see at the moment from a sysadmin's point of view. Thanks.

Hi,

All our programs and libraries have zlib library statically linked. Hence, there are no symbols exported. If you look at the dynamic loader dependencies, you will find that our packaged binaries do not have zlib.so dependencies.

If you have built MySQL yourself, you should have  used zlib that is supplied therein.

Hence, this is not a bug. 

You can download and use our binary packages if your build is not done as described above.

> All our programs and libraries have zlib library statically linked. Hence, there are no symbols exported.

That is not true. The library libmysqlclient.so exports zlib functions, so they are available for any third party binary that loads libmysqlclient.so.

$ nm -D /usr/lib64/mysql/libmysqlclient.so.18.1.0 | fgrep inflate
000000000008a8a0 T inflate
000000000008d850 T inflateCodesUsed
000000000008d490 T inflateCopy
00000000000e6de0 R inflate_copyright
000000000008cdc0 T inflateEnd
000000000008e400 T inflate_fast
000000000008ce50 T inflateGetDictionary
000000000008d0f0 T inflateGetHeader
000000000008a7f0 T inflateInit_
000000000008a6e0 T inflateInit2_
000000000008d7d0 T inflateMark
000000000008a810 T inflatePrime
000000000008a5b0 T inflateReset
000000000008a610 T inflateReset2
000000000008a4e0 T inflateResetKeep
000000000008cef0 T inflateSetDictionary
000000000008d150 T inflateSync
000000000008d430 T inflateSyncPoint
000000000008d8b0 T inflate_table
000000000008d720 T inflateUndermine
000000000008d770 T inflateValidate

> You can download and use our binary packages if your build is not done as described above.

I use official binaries from repo.mysql.com.

$ rpm -qa | grep mysql
mysql-community-libs-5.6.43-2.el7.x86_64
mysql-community-server-5.6.43-2.el7.x86_64
mysql80-community-release-el7-2.noarch
mysql-community-common-5.6.43-2.el7.x86_64
mysql-community-client-5.6.43-2.el7.x86_64

Hi,

You are correct.

They are exported also on my macOS and Linux.

Verified as a feature request.

Thank you very much for the confirmation! I am really concerned about severity level set though. For example, we are in the process of migration of our business critical application (ISP billing system) from FreeBSD to CentOS 7 OS. So, at the moment the described behaviour present serious problem for us (documentation subsystem does not work properly). And we cannot find a workaround at the moment.

If you really think the problem deserves a "feature request" severity level, would you mind to propose us a workaround, please? The only solution that I can think about at the moment is switching to another MySQL implementation. There were no problems with system's provided MariaDB, but we had to upgrade to 5.6 branch because of INET6 functions.

Thanks.

Hi,

First of all, you are welcome.

The only possible workaround is to make your system zlib library version / release identical to the one in MySQL.  zlib/ directory contains the details. The other workaround is to link your Perl with static version of zlib library.

What would be the problem of adapting the linker script (libmysql/libmysql.ver.in) to hide the symbols:

```
libmysqlclient_18 { global: *; local: deflate*; };
```

Hi,

That would be a significant problem. However, even if it was incredibly simple, we would not implement it as we do not intend to change our CMake scripts at all, unless we have to accommodate new OS versions and new compilers.

Hi Sinisa,

could you elaborate on the nature of the problem? Maybe there is a solution that fulfills all requirements.
Also technically it's not a cmake script but a linker script.

It is not possible to adapt the system zlib library because this would require a complete recompilation of all other software.

This is a clear bug in libmysqlclient shipped by mysql.com. Distribution libraries do not export these symbols.

This is a forum for bug reports with repeatable test cases.

This is not a forum where we expound our development and build methodology.

Hi,

You can not report a new bug within an already verified bug. 

Hence, as far as I can see, you have to report two new bugs. One for Connector/J and the other for our C API.

Also , you can only involve our products in the test cases.