MySQL Bugs: #113045: Build failure with Protobuf 24.2 in Linux

Bug #113045	Build failure with Protobuf 24.2 in Linux
Submitted:	10 Nov 2023 14:54	Modified:	24 May 2024 6:32
Reporter:	gordon wang (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S3 (Non-critical)
Version:	8.0	OS:	Linux
Assigned to:		CPU Architecture:	Any (x86_64，arm64)

Description:
Question:

In Linux, if you compile mysql-8.0.35 with protobuf 24.2, the error "/usr/bin/ld: /usr/lib64/libprotoc.so: undefined reference to symbol  '_ZN4absl12lts_2023080213base_internal12SpinLockWaitEPSt6atomicIjEiPKNS1_22SpinLockWaitTransitionENS1_14SchedulingModeE'

/usr/bin/ld: /usr/lib64/libabsl_spinlock_wait.so.2308.0.0: error adding symbols: DSO missing from command line.

According to the release notes of mysql-8.0.35, this issue should have been fixed in the current release, but this is not the case.

“Building with WITH_PROTOBUF=system failed with Protobuf 22 or newer due to Protobuf 22 adding the Abseil dependency.  (Bug #111469, Bug #111623, Bug #35546459, Bug #35550389)”

Reading the mysql source code, it seems that protobuf 22+ is only fixed on MacOS, so the problem still exists on Linux.

How to repeat:
How to reproduce:

Environment: Linux x86-64/arm64 platform

Dependency: protobuf 24.2

Compile: Open -DWITH_PROTOBUF="system", then compile the mysql-8.0.35 code

Suggested fix:
My fix:

1. In cmake/protobuf.cmake, add fixes related to IF(LINUX AND WITH_PROTOBUF STREQUAL "system" AND PB_MINOR_VERSION VERSION_GREATER 21)

2. In cmake/fileutils.cmake, improve the FIND_OBJECT_DEPENDENCIES function in Linux.

The git link to my fix code is as follows:

Question 2:

Following the previous question, I found a new one, described as follows:

Run the FIND_OBJECT_DEPENDENCIES function to find dependent libraries. If the output of objdump is as follows:

NEEDED               libstdc++.so.6
NEEDED               libc.so.6
NEEDED ld-Linux-x86-64.2 ".

At this point, libc.so.6 can be processed normally and converted to "-lc";
However, "libstdc++.so.6" cannot be captured by the regular expression because it contains "++", so it cannot install the transposition "-lstdc++";
Although "ld-Linux-x86-64.2" can be captured by the regular expression, it is incorrectly converted to "-ld-Linux-x86-64", and the correct result should be "-ldl".

Function output error results are as follows:

-labsl_spinlock_wait -llibstdc -lgcc_s -lc -lld-linux-x86-64

My fix code is here: https://github.com/mysql/mysql-server/pull/504

Hi Mr. wang,

Thank you for your bug report.

However, for the building of MySQL you have to use protobuf version that is BUNDLED with our source code.

Hence, if you do not follow that, you will hit the problems like the ones that you describe.

Let us know how did it go with the bundled protobuf.

Hi,

One more comment.

This is the only version and release of the protobuf that we support in our builds.

Hence, we can not accept patches for some other release.

hello
Your answer involves two questions, please let me reply one by one.

1. Are there any problems with bundled protobuf?
bundled protobuf is version 19.x, this version of absl can be linked normally, so there is no problem.

2. You can not accept patches for some other release.
To correct my description, this compilation problem exists in all versions of protobuf 22+ (not just protobuf 24.2).

As far as I know, a user in #111469 reported a compilation problem with protobuf 22+ on MacOS to mysql. Bug link below: https://bugs.mysql.com/bug.php?id=111469

Then mysql fixed protobuf compilation issues for MacOS in 8.0.35, 8.2.0. Does this mean that mysql needs to consider protobuf 22+ fixes?

Then I discovered that protobuf 22+ compilation issues also exist on Linux (yes, you only fixed MacOS).
Now that you fixed the issue on MacOS, do you need to fix it on Linux?

Looking forward to your reply~

Hi Mr. wang,

Thank you for your discussion.

We shall definitely consider fixing this build problem on Linux as well.

This bug is now verified.

We now have an agreement on whether to fix protobuf 22+ on Linux.
Next, we can discuss what can be fixed. It would be even better if I could talk to the mysql developers about the fix.

In fact, mysql has two areas where the code needs to be changed:
1. In cmake/protobuf.cmake, support protobuf 22+ on Linux;
2. In cmake/fileutils.cmake, the FIND_OBJECT_DEPENDENCIES function in Linux needs to be improved (this function does not handle some so libraries printed by objdump correctly)

In my PR, the recurrence and repair of these problems are explained in more detail. I recommend that mysql developers look directly at my PR for a clearer description.
My PR links is as follows: https://github.com/mysql/mysql-server/pull/504

By the way, does the mysql community not accept code from regular developers like me?
I am willing to adjust my code based on the input of the mysql community if needed.
If my PR is accepted, I will be very happy.

HI Mr. Wang,

First of all, thanks again for your contribution.

We are very happy to accept patches from anyone in the community, who wishes to contribute. You can not discuss your case with developers, but our Team is quite capable of analysing your contribution and your idea.  That does not mean that the final patch will be exactly the same as the one that you proposed.

We think that your suggestions are on the right track and we shall forward them to our Development Team.

Hi，
Thank you for your team's recognition.

I can adjust my patch code if necessary.
Looking forward to contributing to the mysql code.

mysql-8.0.37 has now been released, but the issue is still not fixed.
As a Linux distribution worker, we have to patch and maintain ourselves in the distribution, but this is not a long-term solution.

I hope the mysql community will consider this fix again (https://github.com/mysql/mysql-server/pull/544). This fix has been verified in the release.

Hi Mr. Wang,

Unfortunately, our Development is under considerable load. They change their schedules weekly, so that nobody knows when will some verified bug be fixed .......

Sorry for the inconvenience .....