Bug #112949 missing mysql-community-libs-compat rpms for 8.2.0
Submitted: 3 Nov 2023 7:25 Modified: 15 Dec 2023 14:40
Reporter: Simon Mudd (OCA) Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Packaging Severity:S3 (Non-critical)
Version:8.2.0 OS:Any
Assigned to: CPU Architecture:Any (RHEL rpms)
Tags: .so, dependency, library, rpm

[3 Nov 2023 7:25] Simon Mudd
Description:
I noticed that 8.2.0 comes with rpms for RHEL 8 which is what I use.
However, there is no mysql-community-libs-compat package providing libraries for applications built against the older .so.21 libraries in 8.0.

How to repeat:
Check rpm list if you choose from here:  https://dev.mysql.com/downloads/mysql/
with 8.2 innovation, RHEL, RHEL 8 x86_64.

You'll see all the expected rpms but not the lib-compat.

An 8.0 setup (on RHEL8) will show these libraries:

[user@host ~]$ rpm -qa | grep mysql-community-lib
mysql-community-libs-8.0.35-1.el8.x86_64
[user@host ~]$ rpm -ql mysql-community-libs | grep '\.so\.'
/etc/ld.so.conf.d/mysql-x86_64.conf
/usr/lib64/mysql/libmysqlclient.so.21
/usr/lib64/mysql/libmysqlclient.so.21.2.35
[user@host ~]$

The 8.2 rpms include these libraries:

[user@host820 ~]$ rpm -qlp mysql-community-libs-8.2.0-1.el8.x86_64.rpm | grep '\.so\.'
/etc/ld.so.conf.d/mysql-x86_64.conf
/usr/lib64/mysql/libmysqlclient.so.22
/usr/lib64/mysql/libmysqlclient.so.22.1.0
[user@host820 ~]$

So to run 8.2.0 now with other apps built against 8.0 libraries I have to have a mix of rpms installed which looks wrong:

[user@host820 ~]$ rpm -qa |grep mysql-community
mysql-community-libs-8.0.34-1.el8.x86_64
mysql-community-icu-data-files-8.2.0-1.el8.x86_64
mysql-community-server-8.2.0-1.el8.x86_64
mysql-community-client-8.0.34-1.el8.x86_64
mysql-community-client-plugins-8.0.34-1.el8.x86_64
mysql-community-common-8.2.0-1.el8.x86_64

If I try to upgrade the client rpm then this pulls mysql-community-client-plugins rpm which requires the 8.2.0 mysql-community-libs rpm and  will fail as there are other dependencies against 8.0 libs:

[user@host820 ~]$ sudo rpm -Uvh mysql-community-{client,client-plugins,libs}-8.2.0-1.el8.x86_64.rpm
error: Failed dependencies:
        libmysqlclient.so.21()(64bit) is needed by (installed) python-mysql-python-1.2.5-14.el8.x86_64
        libmysqlclient.so.21(libmysqlclient_21.0)(64bit) is needed by (installed) python-mysql-python-1.2.5-14.el8.x86_64

Others may have other dependency issues but the problem remains the same.

Suggested fix:
Please ensure that 8.2.0 libs-compat rpms are built to build libraries for 8.0 apps.
[3 Nov 2023 7:46] Terje Røsten
Hi,

thanks for your report!
[3 Nov 2023 21:49] Simon Mudd
The interesting thing about this bug is that if it's not fixed it makes it harder and more messy to try 8.2 out on rpm based systems as the basic problem is true for RHEL 7/8/9 setups and I'd guess it's probably similar for SuSE.

If you don't fix this until the next release, eg. 8.3 in January then people won't be able to easily use and test 8.2, which may mean they just skip it something I guess that you don't want.

So please consider.

Updating the packaging setup so that the appropriate lib-compat rpms are built to cover "compatibility" with 8.0 libraries and release new rpms, e.g. versioned 8.2.1 or 8.2.0-2 or whatever.  However, to do this given the rpms are built from a spec file from the git source code you'd need to release an update to the git tree on ensure the code which I guess you use from packages/rpm-oel/mysql.spec.in is updated and the rpm build process based on that can be run.

That would be good and allow us to upgrade to 8.2 while being able to use existing apps built/compiled against 8.0 libraries without change as change takes time.

The other option would be to patch the src 8.2.0 src rpms appropriately so that the 8.2.0-2 or 8.2.1 rpms could be built. This might sound weird as you may not be aware of tooling to rebuild src.rpms and patch the content but I have done precisely that and you can see tooling to do that here: https://github.com/sjmudd/mysql-rpm-builder

Pay attention to my last commit which to provide an example allows for 8.2.0 to be rebuilt from the original 8.0.2 src.rpms provided by Oracle but with the hypergraph optimiser enabled, if configured, a change compared to the default build which has it disabled. (The code seems to work from initial testing I've seen on this.)

So following the same process you could provide a patch to my repo which would allow me to rebuild the 8.2.0 rpms and include the appropriate compat-lib rpms as part of the build and this would work on CentOS 8/9 and probably if I fixed my tooling on CentOS 7 too.  [ Here as you see this already works on OEL 8/9 , CentOS 8/9 and I've tested it works. ]

So an update to fix this would be really nice and would allow me to test more easily.

Thanks for considering this.
[8 Dec 2023 13:51] Terje Røsten
I have trouble to reproduce the issue:

$ rpm -qa |grep mysql|sort
mysql80-community-release-el8-8.noarch
mysql-community-client-8.0.35-1.el8.x86_64
mysql-community-client-plugins-8.0.35-1.el8.x86_64
mysql-community-common-8.0.35-1.el8.x86_64
mysql-community-icu-data-files-8.0.35-1.el8.x86_64
mysql-community-libs-8.0.35-1.el8.x86_64
mysql-community-server-8.0.35-1.el8.x86_64
mysql-connector-python3-8.2.0-1.el8.x86_64

MySQL server and Python module from 
 http://repo.mysql.com/yum/mysql-connectors-community/el/8/x86_64/

Upgrade to MySQL 8.2.0 seems to work fine:

$ yum update mysql-community\*

Dependencies resolved.
========================================================================================================================================
 Package                                     Architecture        Version                  Repository                               Size
========================================================================================================================================
Upgrading:
 mysql-community-client                      x86_64              8.2.0-1.el8              mysql-innovation-community               16 M
 mysql-community-client-plugins              x86_64              8.2.0-1.el8              mysql-innovation-community              5.7 M
 mysql-community-common                      x86_64              8.2.0-1.el8              mysql-innovation-community              674 k
 mysql-community-icu-data-files              x86_64              8.2.0-1.el8              mysql-innovation-community              2.2 M
 mysql-community-libs                        x86_64              8.2.0-1.el8              mysql-innovation-community              1.5 M
 mysql-community-server                      x86_64              8.2.0-1.el8              mysql-innovation-community               65 M

Transaction Summary
========================================================================================================================================
Upgrade  6 Packages

I can't find package python-mysql-python at all.
[8 Dec 2023 14:39] Daniël van Eeden
I think I was able to reproduce this with a OEL8 container:

To setup the container:

    docker run -it oraclelinux:8

Then in the container:

    # Setup the MySQL YUM Repo
    yum module -y disable mysql
    rpm -ivh https://dev.mysql.com/get/mysql80-community-release-el8-9.noarch.rpm
    yum-config-manager --disable mysql80-community
    yum-config-manager --enable mysql-innovation-community
    dnf makecache

    # This fails, there is no mysql-community-libs-compat
    dnf install -y mysql-community-libs-compat 

    # This works as expected
    dnf install -y mysql-community-libs

    # This shows mysql-community-libs only provides libmysqlclient.so.22 and not libmysqlclient.so.21
    rpm -ql mysql-community-libs | grep 'libmysqlclient.so'

    # See also this:
    dnf whatprovides /usr/lib64/mysql/libmysqlclient.so.22
    dnf whatprovides /usr/lib64/mysql/libmysqlclient.so.21

    # This shows that `mysql-libs` from `ol8_appstream` provides a `libmyqlclient.so.21`, but that's from 8.0.13 somehow, which isn't good.

Some of the output:

=============================================================
[root@5f696fc304fc /]# rpm -ql mysql-community-libs | grep 'libmysqlclient.so'
/usr/lib64/mysql/libmysqlclient.so.22
/usr/lib64/mysql/libmysqlclient.so.22.1.0
[root@5f696fc304fc /]#
=============================================================

=============================================================
[root@5f696fc304fc /]# dnf whatprovides /usr/lib64/mysql/libmysqlclient.so.22
Last metadata expiration check: 0:03:45 ago on Fri Dec  8 14:24:07 2023.
mysql-community-libs-8.1.0-1.el8.x86_64 : Shared libraries for MySQL database client applications
Repo        : mysql-innovation-community
Matched from:
Filename    : /usr/lib64/mysql/libmysqlclient.so.22

mysql-community-libs-8.2.0-1.el8.x86_64 : Shared libraries for MySQL database client applications
Repo        : @System
Matched from:
Filename    : /usr/lib64/mysql/libmysqlclient.so.22

mysql-community-libs-8.2.0-1.el8.x86_64 : Shared libraries for MySQL database client applications
Repo        : mysql-innovation-community
Matched from:
Filename    : /usr/lib64/mysql/libmysqlclient.so.22

[root@5f696fc304fc /]# dnf whatprovides /usr/lib64/mysql/libmysqlclient.so.21
Last metadata expiration check: 0:03:51 ago on Fri Dec  8 14:24:07 2023.
mysql-libs-8.0.13-1.module+el8+5199+1dce7bb2.x86_64 : The shared libraries required for MySQL clients
Repo        : ol8_appstream
Matched from:
Filename    : /usr/lib64/mysql/libmysqlclient.so.21
=============================================================

So this means that with 8.2.0 installed it might be difficult to use software that was compiled for libmysqlclient.so.21

iirc the aim of the `mysql-community-libs-compat` was to ship the libraries for software compiled against the version of MySQL that the OS shipped with. This unfortunately doesn't help with cases where software is compiled for a newer ABI version than the OS shipped with but not yet the latest available version. However that isn't in play here as OEL8 has a mysql-libs that is from 8.0.32 as you can see below.

=============================================================
# rpm -ql mysql-libs | grep mysqlclient
/usr/lib64/mysql/libmysqlclient.so.21
/usr/lib64/mysql/libmysqlclient.so.21.2.32
=============================================================

I think it would be good if the innovation releases would ship libmysqlclient for ABI 21 (8.0), 22 and whatever newer ABI versions might be between 21 and the next LTS.
[15 Dec 2023 14:38] Simon Mudd
Terje Røsten,

Daniël has provided you with some useful data but I think what is also possibly missing while 8.X+ is not a final version is that people may have other code on their systems which expects to run against the MySQL 8.0 libraries and these libraries are not present and can not be easily installed at the same time as the 8.2+ libraries on the same system.

I haven't checked but in the past the mysql-community packages obsoleted the mysql-libs from the OS (to avoid rpm conflicts) and I believe that the current 8.2 packaging does the same.

At least I see from the mysql.spec.in file:

line 570: Obsoletes:      mysql-libs < %{version}-%{release}

which looks to me to prevent the OS 8.0 package libraries being installed.
That's precisely why the libs-compat package was created to provide support for older libraries still being used by applications.

You may argue that all code should be upgraded but code depending on MySQL dynamic system libraries may require the upstream code to be rebuilt and this takes time.

So developers of applications will upgrade but only once the upgrade process for MySQL is seen as stable. In the meantime making it easy for people to try 8.1+ without having to touch existing applications using MySQL to make makes most sense and with 8.0 you provided 5.7 libraries so it would seem sensible with 8.2+ to do the same thing and also provide 8.0 library support.
[15 Dec 2023 14:40] Simon Mudd
fix typo in tags
[15 Dec 2023 14:46] Terje Røsten
Hi,

thanks for feedback.

Indeed, mysql-community-libs package from different major versions are
not parallel installable at the moment which is a defect.