MySQL Bugs: #28529: Heat Chart status link not valid if event has de-escalated and still open

Bug #28529	Heat Chart status link not valid if event has de-escalated and still open
Submitted:	18 May 2007 18:09	Modified:	24 May 2010 11:41
Reporter:	Bill Weber	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Enterprise Monitor: Web	Severity:	S3 (Non-critical)
Version:	1.1.1.5370, 2.0.1.7125	OS:	Any
Assigned to:	Josh Sled	CPU Architecture:	Any
Tags:	mer 1.2.1 ?

Description:
If a Heat Chart rule has de-escalated and the corresponding higher event has not been closed, the Heat Chart dot link will not show the event on the Events tab. This is because we show the de-escalated state in the Heat Chart but don't generate an Event for it until the more serious Event is closed.

How to repeat:
- set max_connections to 20
- connect to server with 20 clients
| Critical Event is fired and Heat Chart do is red
- reduce connections to 18
| Heat Chart dot (eventually) turns to yellow
- click on yellow dot
| taken to Events tab with filter on Warning - no Warning Event listed

This should be resolved in conjunction with displaying current state on the events page. A few options are as follows:

1. return where worst state matches (current behavior) and where current state matches (solves reported problem) - pragmatic solution for 1.2
2. return where worst state matches or is less than found state. This solves the reported problem but would begin to return too many results for the info state. This is especially true when the rule only has a critical threshold. - not especially fond of this option, too much noise
3. revisit the concept of a nodal event structure where each state change represents a sub event. This way we could match on all sub events and show the encapsulation. - I like this but there's likely non-trivial design work and implementation work. Would suggest for post 1.2

Bill Weber writes: 
correction on previous comment...i meant to say we should change it from the specific alert ("Critical", "Warning", "Info") to the generic "Alert"

Sloan Childers writes: 
pushed to 2.2/trunk

Keith Russell writes: 
Patch installed in versions => 2.2.0.1523.

Bill Weber writes: 
verified fixed in build 2.2.0.1523

An entry has been added to the 2.2.0 changelog:

       If a Heat Chart rule had de-escalated and the corresponding
        higher event had not been closed, the Heat Chart dot link did
        not show the event on the Events tab. This was because the
        de-escalated state was shown in the Heat Chart, but an Event was
        not generated for it until the more serious Event was closed.

Josh Sled writes: 
See EM-3854 as well.

Josh Sled writes: 
<jsled> They superficially seem in conflict, but I think the solution is the 3 right-most "level"/"number" columns should be linked by worst severity...
<jsled> ... but the dots should always be linked by current severity.
<jsled> Dot shows Warning?  Link to Warning-level events.  (Even if the event used to be Crit.)
<jsled> Dot shows Crit?  Link to the Crit-level events.  (Even if there are other Warning-level events open at the time;  Presumably the user wants to deal with the Crit thing first, then circle-around for the others)
<Leith> jsled: hmm yea, it's funky though how we actually do the filtering though iirc
<Leith> i.e if you have a warning shown as current on the heatchart - but the top open level for that event is critical, doesn't the link you click on warnings come up as empty (because the event is actually currently classed at the higher critical level)?
<Leith> jsled: I think, if the icon filter link was ">= <level>" instead of "<level>" it would work (because it would show any event that is higher than the current status
<jsled> Leith: confirmed, we group by the most sever level.
<jsled> severe, even.
<Leith> I also think that if we had an "event history" this would be so much nicer
<jsled> le sigh
<jsled> yup.
<Leith> le sigh indeed
<jsled> Well, it shouldn't be too hard to add ">=level" filters.
<Leith> I reckon that's the best way to do it personally - mark you alright with that approach?
<mark> Yeah, it seems to make more sense.
<jsled> So: heat chart icons would be {current-status >= $level}.
<jsled> And the crit/warn/info would be unchanged as {worst-severity = $level}
<Leith> right

Josh Sled writes: 
branches/post-2.2.0/, ready for merge:

revno: 8061
revision-id: jsled@asynchronous.org-20100505191545-67uwn0b0i0prw1gc
parent: jsled@asynchronous.org-20100505191301-aqposancetxo5m8c
committer: Josh Sled <jsled@asynchronous.org>
branch nick: post-2.2.0
timestamp: Wed 2010-05-05 15:15:45 -0400
message:
  EM-2950: Add a [Ui]AlarmLevel option for ">=WARN"; have heatchart dots link to current-severity>=Info (All Alerts), >=Warn or >=Crit instead of just the specific level
, to make sure all Alarms are shown.

Josh Sled writes: 
revno: 8066
revision-id: jsled@asynchronous.org-20100505191545-xd86lbjr8day60l6
parent: jsled@asynchronous.org-20100505191301-njllasbrdzeit13d
committer: Josh Sled <jsled@asynchronous.org>
branch nick: post-2.2.0
timestamp: Wed 2010-05-05 15:15:45 -0400
message:
  EM-2950: Add a [Ui]AlarmLevel option for ">=WARN"; have heatchart dots link to current-severity>=Info (All Alerts), >=Warn or >=Crit instead of just the specific level, to make sure all Alarms are shown.

Keith Russell writes: 
Patch installed in versions 2.2.1.1711.

Bill Weber writes: 
verified fixed in build 2.2.1.1711

Already documented; version number updated.