Description:
Universal proxy were generated for all JDBC return with a Failover Connection.
And their "equals" method are not implemented.
DBCP use ArrayList, which use "equals" method. Because of Proxy object always return false as real object != proxy. (Ref: www.javaspecialists.eu/archive/Issue126.html)
As a result, following stack could happen with use of DBCP, due the growing amount of elements in AbandonedTrace's ArrayList.
"main" prio=10 tid=0x000000004fad9800 nid=0x338c runnable [0x0000000041e6c000]
java.lang.Thread.State: RUNNABLE
at com.mysql.jdbc.LoadBalancingConnectionProxy$ConnectionErrorFiringInvocationHandler.invoke(LoadBalancingConnectionProxy.java:106)
at com.mysql.jdbc.FailoverConnectionProxy$FailoverInvocationHandler.invoke(FailoverConnectionProxy.java:51)
at $Proxy9.equals(Unknown Source)
at org.apache.commons.dbcp.DelegatingPreparedStatement.equals(DelegatingPreparedStatement.java:77)
at java.util.ArrayList.remove(ArrayList.java:423)
at org.apache.commons.dbcp.AbandonedTrace.removeTrace(AbandonedTrace.java:225)
- locked <0x00000000f0537eb0> (a org.apache.commons.dbcp.PoolableConnection)
at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:151)
at org.springframework.jdbc.support.JdbcUtils.closeStatement(JdbcUtils.java:87)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:608)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:636)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:665)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:673)
at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:733)
....
The memory leak also causing high CPU usage.
How to repeat:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="=jdbc:mysql://a:3306,b:3306,c:3306" />
<property name="minEvictableIdleTimeMillis" value="50000" />
<property name="timeBetweenEvictionRunsMillis" value="50000" />
<property name="numTestsPerEvictionRun" value="3" />
<property name="defaultAutoCommit" value="true" />
<property name="maxActive" value="1" />
<property name="maxIdle" value="1" />
<property name="defaultReadOnly" value="true" />
</bean>
while(true){
PreparedStatement pstmt=con.prepareStatement("SELECT 1");
JdbcUtils.closeStatement(stmt);
}
Suggested fix:
Replace proxy with a subclass. Or, implement the equals correctly.
Description: Universal proxy were generated for all JDBC return with a Failover Connection. And their "equals" method are not implemented. DBCP use ArrayList, which use "equals" method. Because of Proxy object always return false as real object != proxy. (Ref: www.javaspecialists.eu/archive/Issue126.html) As a result, following stack could happen with use of DBCP, due the growing amount of elements in AbandonedTrace's ArrayList. "main" prio=10 tid=0x000000004fad9800 nid=0x338c runnable [0x0000000041e6c000] java.lang.Thread.State: RUNNABLE at com.mysql.jdbc.LoadBalancingConnectionProxy$ConnectionErrorFiringInvocationHandler.invoke(LoadBalancingConnectionProxy.java:106) at com.mysql.jdbc.FailoverConnectionProxy$FailoverInvocationHandler.invoke(FailoverConnectionProxy.java:51) at $Proxy9.equals(Unknown Source) at org.apache.commons.dbcp.DelegatingPreparedStatement.equals(DelegatingPreparedStatement.java:77) at java.util.ArrayList.remove(ArrayList.java:423) at org.apache.commons.dbcp.AbandonedTrace.removeTrace(AbandonedTrace.java:225) - locked <0x00000000f0537eb0> (a org.apache.commons.dbcp.PoolableConnection) at org.apache.commons.dbcp.DelegatingStatement.close(DelegatingStatement.java:151) at org.springframework.jdbc.support.JdbcUtils.closeStatement(JdbcUtils.java:87) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:608) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:636) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:665) at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:673) at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:733) .... The memory leak also causing high CPU usage. How to repeat: <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="com.mysql.jdbc.Driver" /> <property name="url" value="=jdbc:mysql://a:3306,b:3306,c:3306" /> <property name="minEvictableIdleTimeMillis" value="50000" /> <property name="timeBetweenEvictionRunsMillis" value="50000" /> <property name="numTestsPerEvictionRun" value="3" /> <property name="defaultAutoCommit" value="true" /> <property name="maxActive" value="1" /> <property name="maxIdle" value="1" /> <property name="defaultReadOnly" value="true" /> </bean> while(true){ PreparedStatement pstmt=con.prepareStatement("SELECT 1"); JdbcUtils.closeStatement(stmt); } Suggested fix: Replace proxy with a subclass. Or, implement the equals correctly.