| Bug #52767 | Logs fill with "alarm already closed" when agent out of sync with dashboard | ||
|---|---|---|---|
| Submitted: | 12 Apr 2010 15:09 | Modified: | 13 Sep 2010 20:07 |
| Reporter: | Bill Weber | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Enterprise Monitor: Server | Severity: | S3 (Non-critical) |
| Version: | 2.2.0.1692 | OS: | Any |
| Assigned to: | Darren Oldag | CPU Architecture: | Any |
[12 Apr 2010 15:09]
Bill Weber
[12 Apr 2010 21:47]
Enterprise Tools JIRA Robot
Sloan Childers writes: I changed the exception that is thrown from a runtime to one of our server coded exceptions. That should keep the tomcat logs from being flooded as we handle repeating log entries within our own logs automatically. This does *not* fix the problem so I will leave this bug in the verified state. It doesn't look like we can automatically recover from this scenario without knowing the underlying cause which rolled out of the logs. Most likely a Hibernate exception (deadlock, duplicate key, etc.) was thrown that we need to handle specifically like the windows timer issue that caused this before.
[15 Apr 2010 15:44]
Enterprise Tools JIRA Robot
Diego Medina writes:
This could be related to a foreign key error :
{noformat}
------------------------
LATEST FOREIGN KEY ERROR
------------------------
100415 6:12:06 Transaction:
TRANSACTION 43A1AAF0, ACTIVE 0 sec, OS thread id 177224 inserting
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1216, 1 row lock(s), undo log entries 1
MySQL thread id 438682, query id 788193100 localhost 127.0.0.1 service_manager update
insert into rule_eval_results (alarm_id, time, level, result_id) values (256078, 1271304727048, 'failure', 647646)
Foreign key constraint fails for table `mem`.`rule_eval_results`:
,
CONSTRAINT `FK202E43B67E0A1519` FOREIGN KEY (`alarm_id`) REFERENCES `rule_alarms` (`alarm_id`)
Trying to add in child table, in index `alarm_id_time` tuple:
DATA TUPLE: 3 fields;
0: len 4; hex 8003e84e; asc N;;
1: len 8; hex 80000127ffaaaa08; asc ' ;;
2: len 4; hex 8009e1de; asc ;;
But in parent table `mem`.`rule_alarms`, in index `PRIMARY`,
the closest match we can find is record:
PHYSICAL RECORD: n_fields 9; compact format; info bits 0
0: len 4; hex 8003e84f; asc O;;
1: len 6; hex 000043904c75; asc C Lu;;
2: len 7; hex 00000000390084; asc 9 ;;
3: len 4; hex 80000306; asc ;;
4: len 4; hex 8009c759; asc Y;;
5: len 8; hex 80000127ff357ea5; asc ' 5~ ;;
6: SQL NULL;
7: len 30; hex 4175746f6d61746963616c6c7920636c6f736564206261736564206f6e20; asc Automatically closed based on ; (total 49 bytes);
8: len 4; hex 80000001; asc ;;
{noformat}
more logs:
{noformat}
2010-04-15 14:11:07,391 ERROR [em-worker-pool-thread-3:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 14:12:07,430 ERROR [em-worker-pool-thread-4:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 14:13:07,484 ERROR [em-worker-pool-thread-5:com.mysql.dataCollection] Last message repeated 360 times
2010-04-15 14:59:08,755 ERROR [em-worker-pool-thread-3:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 30 times
2010-04-15 15:01:08,789 ERROR [em-worker-pool-thread-5:com.mysql.dataCollection] Last message repeated 390 times
2010-04-15 15:11:06,196 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 41 times
2010-04-15 15:11:06,196 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:12:06,248 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:24:06,811 ERROR [em-worker-pool-thread-2:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 11 times
2010-04-15 15:24:06,811 ERROR [em-worker-pool-thread-2:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:26:06,803 ERROR [em-worker-pool-thread-3:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:37:07,071 ERROR [em-worker-pool-thread-0:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 3 times
2010-04-15 15:37:07,071 ERROR [em-worker-pool-thread-0:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:38:07,121 ERROR [em-worker-pool-thread-3:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 15:47:07,233 ERROR [em-worker-pool-thread-6:com.mysql.dataCollection] Last message repeated 420 times
2010-04-15 16:01:07,905 ERROR [em-worker-pool-thread-5:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 17 times
2010-04-15 16:01:07,905 ERROR [em-worker-pool-thread-5:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:02:07,934 ERROR [em-worker-pool-thread-7:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:09:08,219 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 7 times
2010-04-15 16:09:08,219 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:10:08,914 ERROR [em-worker-pool-thread-3:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:12:09,919 ERROR [em-worker-pool-thread-5:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 2 times
2010-04-15 16:12:09,919 ERROR [em-worker-pool-thread-5:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:13:08,391 ERROR [em-worker-pool-thread-4:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:14:08,447 ERROR [em-worker-pool-thread-1:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(2 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:15:08,800 ERROR [em-worker-pool-thread-5:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:17:08,528 ERROR [em-worker-pool-thread-7:com.mysql.dataCollection] Last message repeated 450 times
2010-04-15 16:27:08,821 ERROR [em-worker-pool-thread-4:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 11 times
2010-04-15 16:27:08,821 ERROR [em-worker-pool-thread-4:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:28:08,807 ERROR [em-worker-pool-thread-0:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:30:09,954 ERROR [em-worker-pool-thread-2:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:31:08,915 ERROR [em-worker-pool-thread-2:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:46:06,131 ERROR [em-worker-pool-thread-0:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] Last message repeated 6 times
2010-04-15 16:46:06,131 ERROR [em-worker-pool-thread-0:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(1 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:47:06,167 ERROR [em-worker-pool-thread-6:com.mysql.advisor.com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator] excessive_number_of_locked_processes.name: %num_locked% > THRESHOLD(0 > 2)
com.mysql.etools.exception.AlreadyClosedException: E1310: AlreadyClosedException: [excessive_number_of_locked_processes.name, 6a1838fc-4943-4855-86f8-a2a5afcd6700]
at com.mysql.etools.monitor.pom.hib.HibAlarm.close(HibAlarm.java:99)
at com.mysql.etools.monitor.bo.rule.Schedule.evaluated(Schedule.java:274)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluationComplete(ListeningRuleEvaluator.java:354)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.scanThresholds(ListeningRuleEvaluator.java:409)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.evaluate(ListeningRuleEvaluator.java:214)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.processDatum(ListeningRuleEvaluator.java:345)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator.access$000(ListeningRuleEvaluator.java:49)
at com.mysql.etools.monitor.bo.rule.ListeningRuleEvaluator$1.saved(ListeningRuleEvaluator.java:71)
at com.mysql.etools.monitor.pom.InstanceAttributeListenerManager.notifyListeners(InstanceAttributeListenerManager.java:122)
at com.mysql.etools.monitor.pom.hib.HibDcReaderWriter3$1.run(HibDcReaderWriter3.java:145)
at com.mysql.etools.concurrent.LogServiceTaskDecorator$LoggingRunnable.run(LogServiceTaskDecorator.java:72)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2010-04-15 16:58:06,563 ERROR [em-worker-pool-thread-0:com.mysql.dataCollection] Last message repeated 480 times
2010-04-15 17:00:11,403 INFO [http-28080-4:com.mysql.timing.accessLog.action] /Monitor.action 1178 ms
2010-04-15 17:00:13,537 INFO [http-28080-5:com.mysql.timing.accessLog.action] /Graph.action 1086 ms
2010-04-15 17:00:13,580 INFO [http-28080-7:com.mysql.timing.accessLog.action] /Graph.action 1156 ms
{noformat}
[16 Apr 2010 18:40]
Enterprise Tools JIRA Robot
Gary Whizin writes: important request: if you see this recurring, first check app to make sure it behaves as expected/desired - 1. close notes persisted 2. event history looks right 3. new alarms fire as expected Maybe contact Sloan and walk through this list together since this is hard to reproduce
[3 May 2010 18:35]
Enterprise Tools JIRA Robot
Gary Whizin writes: cannot reproduce, hoping someone sees this and can help us track it down
[30 Aug 2010 19:04]
Enterprise Tools JIRA Robot
Attachment: 10438_mysql-monitor.log.zip (application/x-zip-compressed, text), 803.11 KiB.
[13 Sep 2010 20:07]
Enterprise Tools JIRA Robot
Bill Weber writes: verified no AlreadClosedExceptions filling logs using build 2.3.0.2023
