Bug #41377 SSL memory leaks cause the dashboard to become unresponsive under heavy load

• Submitted: 10 Dec 2008 22:13
• Modified: 21 Feb 2014 19:33
• Reporter: Gary Whizin
• Status: Closed
• Category: MySQL Enterprise Monitor: Server
• Severity: S2 (Serious)
• Version: 2.0.0.7105
• OS: Any
• Assigned to: Mark Leith
• CPU Architecture: Any
• Tags: mem200_readme

[10 Dec 2008 22:13] Gary Whizin

Description:
Using SSL (agent <--> service manager <--> dashboard), and with very heavy load (monitoring 136 servers), the dashboard becomes unacceptably slow and unresponsive every few days.

A heap dump shows hundreds of thousands of soft references to SSL objects which are not being cleaned up.

A quick search turned up the following memory leak bugs open against JCE.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6578538
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6750401
https://glassfish.dev.java.net/issues/show_bug.cgi?id=5250

How to repeat:
Configure your agents to use SSL and with each agent heartbeat connection there will be a small leak. The seriousness is directly dependent on the number of agents you have using SSL.

Suggested fix:
One possibility is to configure Tomcat to use native SSL.  Early tests show a slight increase in CPU utilization with OpenSSL vs JCE.

A second possibility is to get a patched JRE to ship.

temporary workaround: restart tomcat or reduce use of SSL if you are monitoring a substantial number of agents.

[21 Feb 2014 19:33] Mark Leith

many improvements have been made here, anf SSL is now enabled by default in 3.0 using APR and tomcat native.

Closing.