Bug #89734 MySQL Enterprise Monitor Agent : Bugs with instance deletion
Submitted: 20 Feb 2018 16:56 Modified: 2 Mar 2018 17:48
Reporter: Maxence LEBORGNE Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Enterprise Monitor: Agent Severity:S3 (Non-critical)
Version:4.0.3 OS:Red Hat
Assigned to: Mark Leith CPU Architecture:x86

[20 Feb 2018 16:56] Maxence LEBORGNE
Description:
Two bugs discovered :
> When removing an instance from the agent (using agent.sh --delete-connection), other remaining instances lose their group tags
> When trying to register back a deleted instance, it does not work.

How to repeat:
[ root@myhostname:~]$ echo "mypassword" | sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --create-connection --socket=$MYSOCK1 --admin-user=monitoragent --connection-group="FIRST GROUP" --connection-group="SECOND GROUP" -f >/dev/null
Admin user password:
Operation completed successfully
[ root@myhostname:~]$ echo "mypassword" | sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --create-connection --socket=$MYSOCK2 --admin-user=monitoragent --connection-group="FIRST GROUP" --connection-group="SECOND GROUP" -f >/dev/null
Admin user password:
Operation completed successfully
[ root@myhostname:~]$ sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --show
Connection ID: 4b8cf414-e36e-324f-aa0c-405bbd570770
Socket: /mysql/instance1_data/mysql_instance1.sock
Groups Tags: [FIRST GROUP, SECOND GROUP]

Connection ID: 58982d66-a87f-3d8d-b302-dab1acf5f879
Socket: /mysql/instance2_data/mysql_instance2.sock
Groups Tags: [FIRST GROUP, SECOND GROUP]

[ root@myhostname:~]$ service mysql-monitor-agent restart
Shutting down MySQL Enterprise Agent service... SUCCESS!
Starting MySQL Enterprise Agent service... SUCCESS!

[ root@myhostname:~]$ sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --show
Connection ID: 4b8cf414-e36e-324f-aa0c-405bbd570770
Socket: /mysql/instance1_data/mysql_instance1.sock
Groups Tags: [FIRST GROUP, SECOND GROUP]

Connection ID: 58982d66-a87f-3d8d-b302-dab1acf5f879
Socket: /mysql/instance2_data/mysql_instance2.sock
Groups Tags: [FIRST GROUP, SECOND GROUP]

[ root@myhostname:~]$ sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --delete-connection --connection-id=58982d66-a87f-3d8d-b302-dab1acf5f879
Operation completed successfully
[ root@myhostname:~]$ service mysql-monitor-agent restart
Shutting down MySQL Enterprise Agent service... SUCCESS!
Starting MySQL Enterprise Agent service... SUCCESS!
[ root@myhostname:~]$  sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --show
Connection ID: 4b8cf414-e36e-324f-aa0c-405bbd570770
Socket: /mysql/instance1_data/mysql_instance1.sock
                                                               <<<<<<<<< instance2 indeed deleted... but groups have disappeared from instance1 !

[ root@myhostname:~]$ echo "mypassword" | sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --create-connection --socket=$MYSOCK2 --admin-user=monitoragent --connection-group="FIRST GROUP" --connection-group="THIRD GROUP" -f >/dev/null
Admin user password:
Operation completed successfully

[ root@myhostname:~]$ sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --show
Connection ID: 4b8cf414-e36e-324f-aa0c-405bbd570770
Socket: /mysql/instance1_data/mysql_instance1.sock

Connection ID: 58982d66-a87f-3d8d-b302-dab1acf5f879            <<<<<<<<< Instance2 registered back
Socket: /mysql/instance2_data/mysql_instance2.sock
Groups Tags: [FIRST GROUP, THIRD GROUP]
                                                               
                                                               
[ root@myhostname:~]$ service mysql-monitor-agent restart
Shutting down MySQL Enterprise Agent service... SUCCESS!
Starting MySQL Enterprise Agent service... SUCCESS!
[ root@myhostname:~]$ sudo -u mysql /opt/mysql/enterprise/agent/bin/agent.sh --show
Connection ID: 4b8cf414-e36e-324f-aa0c-405bbd570770
Socket: /mysql/instance1_data/mysql_instance1.sock

                                                               <<<<<<<<< ... but does not survive after an agent restart
[21 Feb 2018 20:24] MySQL Verification Team
2nd part, this is related/duplicate of:

Bug 25238558 : AGENT.SH --SHOW DOES NOT SHOW CONNECTION IF CREATED BACK AFTER REMOVAL	

Work around in that case (which right is not ideal if this isn't a one off procedure) is you can remove the instance from the service manager , restart the agent and re-create.

1st part on the groups, i'm not sure yet, let me test and check if we have something similar
[2 Mar 2018 17:33] MySQL Verification Team
1st part is easily reproducible as customer described. Testing is confusing due to hitting the bug/behavior of 2nd part. Confirmed adding and removing connection with the agent down (restarting in between) behaves similarly

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent start
Starting MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: 7448753e-060e-38a9-b0eb-9a0bfb897e81
Socket: /tmp/mysql1.soc
Groups Tags: [SECOND GROUP, FIRST GROUP]

Connection ID: d2247d78-c9db-3128-a1fa-be75931925e8
Socket: /tmp/mysql.sock
Groups Tags: [SECOND GROUP, FIRST GROUP]

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent stop
Shutting down MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh -d -id2247d78-c9db-3128-a1fa-be75931925e8
Operation completed successfully
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: 7448753e-060e-38a9-b0eb-9a0bfb897e81
Socket: /tmp/mysql1.soc
Groups Tags: [SECOND GROUP, FIRST GROUP]

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent start
Starting MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: 7448753e-060e-38a9-b0eb-9a0bfb897e81
Socket: /tmp/mysql1.soc 

^ Note there the removal of group from agent.sh --show is after the agent is started.

Just to be sure i started with new server_uuids

shannons-MBP:agent shannon$ ./bin/agent.sh --show

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent stop
Shutting down MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh --create-connection --socket=/tmp/mysql.sock --admin-user=root --connection-group="FIRST GROUP" --connection-group="SECOND GROUP" -f
Admin user password: wade
Operation completed successfully
shannons-MBP:agent shannon$ ./bin/agent.sh --create-connection --socket=/tmp/mysql1.soc --admin-user=root --connection-group="FIRST GROUP" --connection-group="SECOND GROUP" -f
Admin user password: wade
No such file or directory
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: d2247d78-c9db-3128-a1fa-be75931925e8
Socket: /tmp/mysql.sock
Groups Tags: [SECOND GROUP, FIRST GROUP]

Connection ID: 7448753e-060e-38a9-b0eb-9a0bfb897e81
Socket: /tmp/mysql1.soc
Groups Tags: [SECOND GROUP, FIRST GROUP]

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent start
Starting MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: d2247d78-c9db-3128-a1fa-be75931925e8
Socket: /tmp/mysql.sock
Groups Tags: [SECOND GROUP, FIRST GROUP]

Connection ID: 7448753e-060e-38a9-b0eb-9a0bfb897e81
Socket: /tmp/mysql1.soc
Groups Tags: [SECOND GROUP, FIRST GROUP]

shannons-MBP:agent shannon$ ./bin/agent.sh -d -i7448753e-060e-38a9-b0eb-9a0bfb897e81
Operation completed successfully
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: d2247d78-c9db-3128-a1fa-be75931925e8
Socket: /tmp/mysql.sock
Groups Tags: [SECOND GROUP, FIRST GROUP]

shannons-MBP:agent shannon$ ./etc/init.d/mysql-monitor-agent restart
Shutting down MySQL Enterprise Agent service...
 SUCCESS! 
Starting MySQL Enterprise Agent service...
 SUCCESS! 
shannons-MBP:agent shannon$ ./bin/agent.sh --show
Connection ID: d2247d78-c9db-3128-a1fa-be75931925e8
Socket: /tmp/mysql.sock

same. Confirmed in the ui it does remove the connection and from group (though you will hit confusing other bug if you're trying to add back again)
[2 Mar 2018 17:48] MySQL Verification Team
Verified as described. Part is duplicate of known bug, other part is new bug (this one).