Bug #115087 SSL connection server_capabilities has overflow
Submitted: 23 May 3:10 Modified: 23 May 15:42
Reporter: peng gao Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: Group Replication Severity:S3 (Non-critical)
Version:8.0.36 OS:Any
Assigned to: CPU Architecture:Any

[23 May 3:10] peng gao
Description:
When MGR use mysql protocol,when ssl connect error, The error is as follows

2024-05-22T11:53:07.919600+08:00 0 [ERROR] [MY-013780] [Repl] Plugin group_replication reported: 'Failed to establish MySQL client connection in Group Replication. Error establishing connection. Please refer to the manual to make sure that you configured Group Replication properly to work with MySQL Protocol connections.'
2024-05-22T11:53:07.919780+08:00 0 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] The group communication engine failed to test connectivity to the local group communication engine on 192.168.1.86:3320. This may be due to one or more invalid configuration settings. Double-check your group replication 
local address, firewall, SE Linux and TLS configurations and try restarting Group Replication on this server.'

I think this error message is quite vague.We should confirm the error in establishing an SSL connection.

And we found that there is a possibility of overflow in client mysql->server_capabilities, which may be uncertain.

Breakpoint 7, cli_establish_ssl (mysql=0x7fff4011baf0) at /pxc/mysql-8.0.36/sql-common/client.cc:4387
4387      NET *net = &mysql->net;
(gdb) n
4390      if (mysql->options.extension &&
(gdb) n
4391          mysql->options.extension->ssl_mode >= SSL_MODE_REQUIRED &&
(gdb) n
4390      if (mysql->options.extension &&
(gdb) n
4392          !(mysql->server_capabilities & CLIENT_SSL)) {
(gdb) n
4391          mysql->options.extension->ssl_mode >= SSL_MODE_REQUIRED &&
(gdb) n
4393        set_mysql_extended_error(mysql, CR_SSL_CONNECTION_ERROR, unknown_sqlstate,
(gdb) p mysql->options.extension->ssl_mode
$17 = 3
(gdb) p mysql->options.extension
$18 = (st_mysql_options_extention *) 0x7fff4007ec80
(gdb) p mysql->server_capabilities
$19 = 18446744072098936831

Continuing.
Hardware watchpoint 9: *((unsigned long *) 0x7fff2000c3e0)

Old value = 63487
New value = 18446744072098936831
csm_parse_handshake (ctx=0x7fff027f2ff0) at /pxc/mysql-8.0.36/sql-common/client.cc:6858

How to repeat:
see code

Suggested fix:
see code
[23 May 9:31] MySQL Verification Team
HI Mr. gao,

Thank you for your bug report.

Please, confirm that you get this error only in Group Replication and not in the client connection.

Also, please provide us with a full test case. It should consist of your MySQL Replication setup and any SQL statements that provoked this error.

Also, please try to repeat this behaviour with our own binary available from https://dev.mysql.com.

We aer waiting on your feedback.
[23 May 13:48] peng gao
These are two issues,

One、MySQL protocol error is very vague

We often encounter similar errors in Online environment, but the configuration is correct. I can only simulate similar errors by turning off SSL, but in my simulation, the error under the MySQL protocol is very vague

first,skip ssl ,add tls_version='' in my.cnf to disable ssl .

| have_openssl                                      | DISABLED        |
| have_ssl                                          | DISABLED        |

1、use mysql

| group_replication_recovery_use_ssl                  | ON       |
| group_replication_ssl_mode                          | REQUIRED |
| group_replication_communication_stack               | MYSQL                                |
| group_replication_group_seeds                       | 192.168.1.84:3320,192.168.1.85:3320  |
| group_replication_local_address                     | 192.168.1.86:3320                    |

boot first node
set global group_replication_bootstrap_group=on;
start group_replication;

the error:
2024-05-23T21:26:08.294897+08:00 0 [Note] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Using MySQL as Communication Stack for XCom'
2024-05-23T21:26:08.295174+08:00 0 [Note] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Successfully connected to the local XCom via anonymous pipe'
2024-05-23T21:26:08.318713+08:00 20 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
2024-05-23T21:26:08.318999+08:00 0 [ERROR] [MY-013780] [Repl] Plugin group_replication reported: 'Failed to establish MySQL client connection in Group Replication. Error establishing connection. Please refer to the manual to make sure that you configured Group Replication properly to work with MySQL Protocol connections.'
2024-05-23T21:26:08.319018+08:00 20 [Note] [MY-010914] [Server] Got an error reading communication packets
2024-05-23T21:26:08.319077+08:00 0 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] The group communication engine failed to test connectivity to the local group communication engine on 192.168.1.86:3320. This may be due to one or more invalid configuration settings. Double-check your group replication local address, firewall, SE Linux and TLS configurations and try restarting Group Replication on this server.'
2024-05-23T21:26:08.406893+08:00 0 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] The member was unable to join the group. Local port: 3320'
2024-05-23T21:26:08.406939+08:00 0 [Note] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Sleeping for 5 seconds before retrying to join the group. There are 9 more attempt(s) before giving up.'

here can't connection local address 192.168.1.86:3320,But error very vague,

2、use xcom

| group_replication_recovery_use_ssl                  | ON       |
| group_replication_ssl_mode                          | REQUIRED |
| group_replication_communication_stack               | XCOM                                  |
| group_replication_group_seeds                       | 192.168.1.85:33201,192.168.1.84:33201 |
| group_replication_local_address                     | 192.168.1.86:33201                    |

boot first node
set global group_replication_bootstrap_group=on;
start group_replication;

the error:
2024-05-23T21:30:06.935157+08:00 0 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] TLS version is invalid: '
2024-05-23T21:30:06.935239+08:00 0 [Note] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error initializing SSL'
2024-05-23T21:30:06.935355+08:00 9 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error starting SSL in the group communication engine.'
2024-05-23T21:30:06.956616+08:00 9 [ERROR] [MY-011735] [Repl] Plugin group_replication reported: '[GCS] Error initializing the group communication engine.'
2024-05-23T21:30:06.977953+08:00 9 [ERROR] [MY-011674] [Repl] Plugin group_replication reported: 'Unable to initialize the group communication engine'
2024-05-23T21:30:06.977998+08:00 9 [ERROR] [MY-011637] [Repl] Plugin group_replication reported: 'Error on group communication engine initialization'
2024-05-23T21:30:06.978025+08:00 9 [Note] [MY-011649] [Repl] Plugin group_replication reported: 'Requesting to leave the group despite of not being a member'

here can't connection local address 192.168.1.86:33201,The error is very clear,I know it's an SSL configuration issue.

Two、SSL connection server_capabilities has overflow

I think this is an obvious overflow, not just MGR 

Old value = 63487
New value = 18446744072098936831
csm_parse_handshake (ctx=0x7fff027f2ff0) at /pxc/mysql-8.0.36/sql-common/client.cc:6858

code,
 mysql->server_capabilities |= uint2korr((uchar *)end + 5) << 16;
[23 May 15:42] MySQL Verification Team
Hi,

I am having issues reproducing this but I'll verify this so that we can work on it more. Expect more questions from our team as we inspect this more.