MySQL Bugs: #89688: server crash when group replication starts with mysqld-debug

Bug #89688	server crash when group replication starts with mysqld-debug
Submitted:	16 Feb 2018 8:49	Modified:	9 Jun 2018 6:49
Reporter:	Giuseppe Maxia (OCA)	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S4 (Feature request)
Version:	5.7.21, 8.0.3, 8.0.4	OS:	Any (linux and macos)
Assigned to:		CPU Architecture:	Any
Tags:	crash, debug, GROUP, replication

Description:
When starting a group replication with mysqdl-debug instead of mysqld, the server crashes.

Here's a sample from MySQL 5.7.21 (see the complete error log attached)
mysqld-debug: /export/home/pb2/build/sb_0-26514852-1514433850.9/mysql-5.7.21/sql/rpl_gtid.h:527: void Checkable_rwlock::assert_some_lock() const: Assertion `get_state() != 0' failed.
08:33:46 UTC - mysqld got signal 6 ;

MySQL 8.0.4 fails with signal 11, but the end result is the same, i.e. group replication does not start.

select * from performance_schema.replication_group_members
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+
| CHANNEL_NAME              | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION |
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+
| group_replication_applier |           |             |        NULL | OFFLINE      |             |                |
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+

How to repeat:
I start three nodes on the same host. Here's the my.cnf of the first node

port               = 19822
socket             = /tmp/mysql_sandbox19822.sock
basedir            = /home/gmax/opt/mysql/5.7.21
datadir            = /home/gmax/sandboxes/group_msb_5_7_21/node1/data
tmpdir             = /home/gmax/sandboxes/group_msb_5_7_21/node1/tmp
pid-file           = /home/gmax/sandboxes/group_msb_5_7_21/node1/data/mysql_sandbox19822.pid
bind-address       = 127.0.0.1
log-error=msandbox.err
server-id=100
relay-log-index=mysql-relay
relay-log=mysql-relay
log-bin=mysql-bin
log-error=msandbox.err
binlog_checksum=NONE
log_slave_updates=ON
plugin-load=group_replication.so
group_replication=FORCE_PLUS_PERMANENT
group_replication_start_on_boot=OFF
group_replication_bootstrap_group=OFF
transaction_write_set_extraction=XXHASH64
report-host=127.0.0.1
loose-group_replication_group_name="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
loose-group-replication-single-primary-mode=off
master-info-repository=table
relay-log-info-repository=table
gtid_mode=ON
log-slave-updates
enforce-gtid-consistency
loose-group-replication-local-address=127.0.0.1:19947
loose-group-replication-group-seeds=127.0.0.1:19947,127.0.0.1:19948,127.0.0.1:19949

Start the server using mysqld-debug

mysql 5.,7.21 error log

Attachment: stack-trace.txt (text/plain), 7.41 KiB.

An important clarification: using the regular mysqld, the server starts and replication works as expected.

Hello Giuseppe,

Thank you for the report.
Observed that issue is seen on debug build(5.7.21 in my tests).

Thanks,
Umesh

Posted by developer:
 
When using mysqld-debug, you need to load plugins from <root>/lib/plugin/debug rather than <root>/lib/plugin.
We should have had a runtime check for this, but alas we do not.
So a crash like this is actually not unexpected.

Posted by developer:
 
As comment above suggests, this is an artifact of the plugin architecture, and I suggest to close this as "not a bug".

Thanks for your suggestion.
Is there a method to force loading debug plugins in 8.0.11, where the Xplugin is enabled by default?

never mind, I found it.
We need to add plugin_dir=/home/username/usr/mysql/lib/plugin/debug in the options file

Posted by developer:
 
The issue was caused by loading a non-debug build plugin on a debug build server, thence closing as not a bug.

It would be great to have an error when non debug plugins are loaded with a debug built and the reverse too.