| Bug #87500 | Cluster.addInstance: ERROR: Error joining instance to cluster: - Access denied f | ||
|---|---|---|---|
| Submitted: | 22 Aug 2017 8:13 | Modified: | 6 Sep 2017 6:26 |
| Reporter: | kkumar Santosh | Email Updates: | |
| Status: | Not a Bug | Impact on me: | |
| Category: | MySQL Server: Group Replication | Severity: | S1 (Critical) |
| Version: | 5.7 | OS: | Red Hat |
| Assigned to: | MySQL Verification Team | CPU Architecture: | Any |
| Tags: | Access denied for user, Cluster.addInstance:Error | ||
[22 Aug 2017 9:55]
kkumar Santosh
As I am checking there is some Access issue to the user. But on the another note i am able to log into the host Ip with same username and password. I have also grant the permissions to the user as required and suggested in below link. https://dev.mysql.com/doc/refman/5.7/en/mysql-innodb-cluster-working-with-production-deplo...
[22 Aug 2017 10:29]
MySQL Verification Team
Hi, Not sure what you are doing wrong as I can't reproduce this on Oracle Linux. It's obviously the auth issue, so check if you can actually connect to that mysql from the "main node" with that user/pass, check selinux on the new node.. anyhow, this does not appear to be MySQL bug. best regards Bogdan
[24 Aug 2017 7:41]
kkumar Santosh
Thanks, But yes i am able to connect to MySQL from the main node. From my one host machine. Using the details mysql -u #### -h ###.##.##.### -p It is successfully connected. But the same credentials are not working while adding the cluster. I have removed and install the MySQL many times.
[24 Aug 2017 8:01]
kkumar Santosh
I am following steps provided in the below-given URL. http://mysqlserverteam.com/mysql-innodb-cluster-real-world-cluster-tutorial-for-oel-fedora...
[28 Aug 2017 2:35]
kkumar Santosh
Sorry my OS is Linux version 3.8.13-118.19.3.el7uek.x86_64 (mockbuild@x86-ol7-builder-01.us.oracle.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #2 SMP Thu Jul 13 09:41:05 PDT 2017 Please suggest and help.
[31 Aug 2017 4:25]
kkumar Santosh
Please Help...
After Adding an instance i am facing this error in the remote node.
2017-08-31T04:22:00.573186Z 190 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='hk02mysqlpoc2', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='hk02mysqlpoc2', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.
2017-08-31T04:22:00.579298Z 190 [Note] Plugin group_replication reported: 'Establishing connection to a group replication recovery donor b33baa22-8d2d-11e7-b400-005056be7212 at hk02mysqlpoc2 port: 3306.'
2017-08-31T04:22:00.582905Z 196 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
2017-08-31T04:22:00.591479Z 196 [ERROR] Slave I/O for channel 'group_replication_recovery': error connecting to master 'mysql_innodb_cluster_rp616941570@hk02mysqlpoc2:3306' - retry-time: 60 retries: 1, Error_code: 2005
2017-08-31T04:22:00.591507Z 196 [Note] Slave I/O thread for channel 'group_replication_recovery' killed while connecting to master
2017-08-31T04:22:00.591512Z 196 [Note] Slave I/O thread exiting for channel 'group_replication_recovery', read up to log 'FIRST', position 4
2017-08-31T04:22:00.591579Z 190 [ERROR] Plugin group_replication reported: 'There was an error when connecting to the donor server. Check group replication recovery's connection credentials.'
This is my Cluster Status:
mysql-js> cluster.status();
{
"clusterName": "WiseCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "10.32.181.32:3306",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures. 2 members are not active",
"topology": {
"10.32.181.32:3306": {
"address": "10.32.181.32:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"10.32.181.33:3306": {
"address": "10.32.181.33:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "RECOVERING"
},
"10.32.181.34:3306": {
"address": "10.32.181.34:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "(MISSING)"
}
}
}
}
Can you please help.....
[1 Sep 2017 21:29]
MySQL Verification Team
Hi, If I understand you correctly now you solved the login problem but you encounter further errors. Nothing I see here is a bug related, you have issues configuring and setting up group replication. To get help with that you can contact Support team or try to get community help on the https://www.mysql.com/ but the bugs system is certainly not a place to get support. kind regards Bogdan
[6 Sep 2017 6:26]
kkumar Santosh
Hi Thanks for your reply,
i don't know exactly as i am new to this cluster.
Previously i have done setup on Ubuntu and its work fine.
Not an issue. I have moved forward as well with the setup. Now i am able to create a Cluster and at the time of adding i am facing this Error on my node.
2017-09-06T06:14:00.638489Z 12 [Warning] IP address '10.32.181.32' could not be resolved: Name or service not known
2017-09-06T06:14:00.998156Z 13 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.
2017-09-06T06:14:01.006015Z 13 [Note] Plugin group_replication reported: 'Group communication SSL configuration: group_replication_ssl_mode: "REQUIRED"; server_key_file: "server-key.pem"; server_cert_file: "server-cert.pem"; client_key_file: "server-key.pem"; client_cert_file: "server-cert.pem"; ca_file: "ca.pem"; ca_path: ""; cipher: ""; tls_version: "TLSv1,TLSv1.1"; crl_file: ""; crl_path: ""'
2017-09-06T06:14:01.006107Z 13 [Note] Plugin group_replication reported: '[GCS] Added automatically IP ranges 10.32.181.33/24,127.0.0.1/8 to the whitelist'
2017-09-06T06:14:01.006292Z 0 [Note] Plugin group_replication reported: 'Configuring SSL for the server'
2017-09-06T06:14:01.006329Z 0 [Note] Plugin group_replication reported: 'Initializing SSL with key_file: 'server-key.pem' cert_file: 'server-cert.pem' ca_file: 'ca.pem' ca_path: 'NULL''
2017-09-06T06:14:01.006346Z 0 [Note] Plugin group_replication reported: 'Additional SSL configuration is cipher: '' crl_file: 'NULL' crl_path: 'NULL''
2017-09-06T06:14:01.006485Z 0 [Note] Plugin group_replication reported: 'Success initializing SSL'
2017-09-06T06:14:01.006499Z 0 [Note] Plugin group_replication reported: 'Configuring SSL for the client'
2017-09-06T06:14:01.006506Z 0 [Note] Plugin group_replication reported: 'Initializing SSL with key_file: 'server-key.pem' cert_file: 'server-cert.pem' ca_file: 'ca.pem' ca_path: 'NULL''
2017-09-06T06:14:01.006514Z 0 [Note] Plugin group_replication reported: 'Additional SSL configuration is cipher: '' crl_file: 'NULL' crl_path: 'NULL''
2017-09-06T06:14:01.006607Z 0 [Note] Plugin group_replication reported: 'Success initializing SSL'
2017-09-06T06:14:01.006643Z 13 [Note] Plugin group_replication reported: 'Initialized group communication with configuration: group_replication_group_name: "794b0f28-92ca-11e7-becd-005056be7212"; group_replication_local_address: "10.32.181.33:13306"; group_replication_group_seeds: "10.32.181.32:13306"; group_replication_bootstrap_group: false; group_replication_poll_spin_loops: 0; group_replication_compression_threshold: 1000000; group_replication_ip_whitelist: "AUTOMATIC"'
2017-09-06T06:14:01.006689Z 13 [ERROR] Plugin group_replication reported: 'Can't start group replication on secondary member with single primary-mode while asynchronous replication channels are running.'
2017-09-06T06:14:02.006816Z 13 [Note] Plugin group_replication reported: 'Requesting to leave the group despite of not being a member'
2017-09-06T06:14:02.006879Z 13 [ERROR] Plugin group_replication reported: '[GCS] The member is leaving a group without being on one.'
2017-09-06T06:14:02.006945Z 0 [Note] Plugin group_replication reported: 'Destroying SSL'
2017-09-06T06:14:02.006974Z 0 [Note] Plugin group_replication reported: 'Success destroying SSL'
These are my Group Replication Options :
# group replication specific options
plugin-load = group_replication.so
group_replication = FORCE_PLUS_PERMANENT
transaction-write-set-extraction = XXHASH64
group_replication_start_on_boot = ON
group_replication_bootstrap_group = OFF
group_replication_group_name = 550fa9ee-a1f8-4b6d-9bfe-c03c12cd1c72
group_replication_local_address = '10.32.181.33:13306'
group_replication_group_seeds = '10.32.181.32:13306,10.32.181.34:13306'
And on Node 1 (Slave):
I have follow these step;
mysql> CHANGE MASTER TO MASTER_HOST='10.32.181.32',
-> MASTER_USER='rpl_user',
-> MASTER_PASSWORD='rpl2user*Santosh',
-> MASTER_LOG_FILE='hk02mysqlpoc2-bin.000002',
-> MASTER_LOG_POS=4561;
Query OK, 0 rows affected, 2 warnings (0.02 sec)
mysql>
mysql>
mysql> START SLAVE;
Query OK, 0 rows affected (0.00 sec)
And Every thing looks fine and when i am adding a node to my Main cluster node;
mysql-js> cluster.addInstance({user:"wise", password:"Wise*Santosh1", host:"10.32.181.33", port:3306});
A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.
Adding instance to the cluster ...
Cluster.addInstance: WARNING: Not running locally on the server and can not access its error log.
ERROR:
Group Replication join failed.
ERROR: Error joining instance to cluster: '10.32.181.33:3306' - Query failed. 3092 (HY000): The server is not configured properly to be an active member of the group. Please see more details on error log.. Query: START group_replication (RuntimeError)
This is the error.
[28 Feb 2018 8:03]
saravana kumar
@ kkumar Santosh Hi Kkumar i have a same problem.. did you resolve it.. if yes kindly let me know how to resolve this problem.. Thanks Saravanakumar. M
[30 Mar 2018 8:41]
Shubhra Prakash Nandi
Hi, the issue I am facing is when I configure the cluster in a subnet/LAN it's not giving this error, but while trying to configure the cluster for two separated server over the internet I am facing the below issue. The steps followed to create the cluster in both cases is the same. [Note] Access denied for user 'user'@'host' (using password: YES) [Note] Aborted connection 27 to db: 'unconnected' user: 'user' host: 'host' (Got an error reading communication packets)
[30 Mar 2018 8:48]
Shubhra Prakash Nandi
This is from mysqlsh log. I can successfully connect to this server from mysqlsh using the same credentials from the seed node. Is network latency an issue to drop the connection mid way? Can we configure something to increase connection time for the cluster? Cluster.addInstance: ERROR: Error joining instance to cluster: 'host:3306' - mysql.get_classic_session: Access denied for user 'user'@'host' (using password: YES)
[1 Apr 2018 21:18]
Shubhra Prakash Nandi
Well, found out the issue in my case. I had created the user 'ca' with different password on other instances. This was causing the access denied issue. Once I made the password same on all nodes for user 'ca' the issue got resolved. One more important point which I found was when adding an instance if the ip whitelist has hostnames then the hostnames should match hostname of the instance and should resolve to an IP address which can be reached by other instances.
Description: After Creating Cluster on the main Master Node while adding the slave node using the command var cluster = dba.getCluster('WiseCluster'); cluster.addInstance({user:"wise", password:"#Cluster2", host:"10.32.181.33", port:3306}); getting this error in the command line. A new instance will be added to the InnoDB cluster. Depending on the amount of data on the cluster this might take from a few seconds to several hours. Adding instance to the cluster ... Cluster.addInstance: ERROR: Error joining instance to cluster: '10.32.181.32:3306' - Access denied for user 'wise'@'10.32.181.32' (using password: YES) (RuntimeError) Please Suggest what i am missing. The same steps i have followed on Ubuntu will work fine but on Oracle Linux they are showing error. How to repeat: [root@hk02mysqlpoc2 testuser]# mysqlsh mysql-js> shell.connect('wise@10.32.181.32:3306'); mysql-js> var cluster = dba.createCluster('WiseCluster'); mysql-js> var cluster = dba.getCluster('WiseCluster'); mysql-js> cluster.addInstance({user:"wise", password:"#Cluster2", host:"10.32.181.33", port:3306});