MySQL Bugs: #45688: Dashboard reports spike on graph after mem restart

Bug #45688	Dashboard reports spike on graph after mem restart
Submitted:	23 Jun 2009 17:43	Modified:	6 Nov 2009 16:53
Reporter:	Diego Medina	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Enterprise Monitor: Server	Severity:	S3 (Non-critical)
Version:	2.1.0.1070	OS:	Any
Assigned to:	Darren Oldag	CPU Architecture:	Any

Description:
When you restart the Service manager, or if you have the service manager turned off for a few minutes, and then you turn it back on, some graphs, specially "Database Activity", will show a spike on traffic.

To the end user, this could look as if the database server went from handling 3000 queries/sec, to 6000 queries/sec for a short period of time.

(I'll attach a sample graph)

How to repeat:
1- Install and start the agent and service manager
2- Run this on one terminal

$ for i in {1..1000000}; 
  do mysql -h127.0.0.1 -P24040 -umsandbox -pmsandbox -e "select $i";
  sleep 5;
done;

(change ports and user/pass as needed)
3- Go to the UI->Graphs tab
4- Expand the Database Activity graph
5- refresh the page until you see s straight line, showing that you are getting  a constant (or more or less) constant number of queries per second 
6- Stop tomcat
7- Wait 5 minutes
5- Start tomcat again
6- Look at the Database Activity graph, once the mysqld is online again on the UI, you will see a spike on the graph.
But you know that you did not send that many queries all at once.

(I'll attach the values for COM_SELECT taken from the xml file)

Suggested fix:
Do not show a spike

COM_SELECT values from the xml file

Attachment: file.csv (text/plain), 1.14 KiB.

Database Activity graph showing spike

Attachment: spike.png (image/png, text), 30.58 KiB.

please attach the smallest possible mem repository data dump which reproduces the problem.

Diego Medina writes: 
Change status because I attached a file.

Per Leith, let's reassess after current resync work is done

Mark Matthews writes: 
We're thinking this might not have anything to do with resync. Might just be a graph corner case, or agent counter bug?

Diego Medina writes: 
It is not an agent counter bug, please see this file

http://bugs.mysql.com/file.php?id=12328

(I attached it to the bug on the bugs' db) the values increase normally)

Darren Oldag writes: 
since the graphs presume a continuous/consecutive data stream at a known frequency,
we have to ignore the first delta data point after a time gap (even though the
delta is accurate for the given time range).  in this case, the graph render
engine thusly requires two consecutive (in terms of frequency) data points
in order to display a known-good graph data point.

Andy Bang writes: 
In build 2.1.0.1078

Diego Medina writes: 
Verified fixed on 2.1.0.1078

An entry was added to the 2.1.0 changelog:

When the Service Manager was restarted, or if the Service Manager was turned off for a brief period, and then turned back on, some graphs, such as Database Activity, showed an erroneous spike in traffic.