Description:
Enabling "Attempted Connections To The Server Have Failed" advisor on an installation with 200+ instances and "auto-close" can cause a flood of notifications. A notification is sent if the condition recovers. This generates a problem of too many notifications if the condition happens regularly and recovers regularly.
How to repeat:
Enable "Attempted Connections To The Server Have Failed" advisor on a busy installation with 200+ instances and "auto-close". Then, generate some failed connections.
Suggested fix:
An appropriate feature to add would be ways to disable or combine alerts if the state changes too frequently.
If possible provide some sort of indication of how many mails may be sent IF this information can be provided when enabling the advisor.
If this can not be done then perhaps provide some sort of limit/throttle, after which merlin will stop sending more mails of that type and will advise the throttle has been reached.
Enabling 4 rules on large installation MEM server (with 200+ instances) caused 2000 mails. That was certainly more than expected and a throttle at perhaps 100 mails per "period" (say 60 minutes) would be the limit.
To make this easier provide a page similar to the events page which shows the events sent, and if necessary a flag of "throttled/non-sent events", from where the same details can be viewed as on the main merlin page.
This sort of scenario:
1. inform of a burst in one type of message, and stop sending more for a period
2. when that period is reached to inform that the period is over and after that go back to a normal state again where alerts may be sent.
3. indicate in merlin if this "burst/throttling condition" is active and for which alerts.