Description:
When agent has some obsolete instances in configuration, which are now being monitored by another agent:
1. Instance is shown as crossed out in dashboard, but graphs are updated properly.
2. "Invalid UUID" exceptions are flooded in MEM Server log.
$ grep "Invalid UUID" mysql-monitor.log.* |wc -l
5429
java.lang.IllegalArgumentException: Invalid UUID string:
java.util.UUID.fromString(Unknown Source)
com.mysql.etools.monitor.bo.Agent.processServers(Agent.java:1131)
com.mysql.etools.monitor.bo.Agent.processInventoryType(Agent.java:962)
com.mysql.etools.monitor.bo.Agent.processInventoryTasks(Agent.java:953)
com.mysql.etools.monitor.bo.Agent.processTasks(Agent.java:578)
com.mysql.etools.monitor.bo.Agent.lockedAcceptHeartbeat(Agent.java:417)
com.mysql.etools.monitor.bo.Agent.lockedAcceptHeartbeat(Agent.java:394)
com.mysql.etools.monitor.bo.Agent.access$100(Agent.java:77)
com.mysql.etools.monitor.bo.Agent$3.call(Agent.java:383)
com.mysql.etools.monitor.bo.Agent$3.call(Agent.java:382)
com.mysql.etools.concurrent.LockingExecutor.execute(LockingExecutor.java:39)
com.mysql.etools.monitor.bo.Agent.acceptHeartbeat(Agent.java:381)
com.mysql.etools.monitor.bo.HeartbeatAcceptor.acceptHeartbeat(HeartbeatAcceptor.java:60)
com.mysql.etools.monitor.bo.HeartBeatCommandProcessor.processRequest(HeartBeatCommandProcessor.java:102)
com.mysql.merlin.server.MerlinServlet$1.processRequest(MerlinServlet.java:328)
com.mysql.merlin.server.agent.HeartbeatServlet$1.processRequest(HeartbeatServlet.java:62)
com.mysql.merlin.server.MerlinServlet.doCommon(MerlinServlet.java:297)
com.mysql.merlin.server.MerlinServlet.doPost(MerlinServlet.java:419)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
com.mysql.merlin.server.RequestCounterFilter.doFilter(RequestCounterFilter.java:117)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
com.mysql.etools.monitor.rest.AuthFilter.doFilter(AuthFilter.java:121)
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:263)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:584)
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
How to repeat:
This situation may be observed when agent was monitoring several MySQL instances and later at least one instance is moved to other box (and is monitored by another agent).
Here are steps how to reproduce the problem:
1. Configure agent to monitor two MySQL instances, let's say on ports 13306 and 3306 and make sure that dashboard shows them properly.
2. Shutdown server on 3306 and start it in new location, let's say on port 3307.
3. Configure second agent instance to monitor MySQL instance on port 3307.
4. Wait couple minutes and observe that instance in MySQL dashboard is crossed out but its graphs are being updated properly
5. Restart tomcat
6. After a while mysql-monitor.log is full of messages "Invalid UUID String" with stacktrace
Suggested fix:
1. Clear error message about obsolete instance in agent configuration instead of "Invalid UUID String"
2. Do not flood log with identical stacktraces