Bug #22038 http://lists.mysql.com/java/8849
Submitted: 6 Sep 2006 1:05 Modified: 31 Mar 2014 11:52
Reporter: P Eger Email Updates:
Status: Can't repeat Impact on me:
None 
Category:Connector / J Severity:S2 (Serious)
Version:3.1.13 OS:Any
Assigned to: Alexander Soklakov CPU Architecture:Any

[6 Sep 2006 1:05] P Eger
Description:
We recently recieved an OutOfMemoryError on one of our servers, while examining the heap histogram as provided by jmap, we ran into the following, what appears to be a memory leak of some sort in connector/j:

It seems similar to this issue, which i am not sure if it was resolved or not:
http://lists.mysql.com/java/8849

We are not using connection pooling of any sort. Connections are all acquired from a single instance of MysqlDataSource. I have gone over our application code  for possible unreleased Connections, but all acquire / release pairs are using proper try {} finally {} blocks. The server has a max connections setting of 500 if this matters.

Top entries from "jmap -histo" output:

Size    Count   Class description
-------------------------------------------------------
247797088       3864237 char[]
150094640       3752366 java.lang.String
53277472        159824  byte[]
36046176        279576  java.lang.String[]
25714200        189075  com.mysql.jdbc.ConnectionProperties$BooleanConnectionProperty
24280176        505837  java.util.HashMap$Entry
22790016        474792  java.util.Hashtable$Entry
16519000        412975  java.util.LinkedList$Entry
12238040        5919    java.util.HashMap$Entry[]
9787856 141060  java.lang.Object[]
7635408 318142  java.lang.Boolean
7516320 5048    java.util.Hashtable$Entry[]
5600776 156753  short[]
5272272 36613   com.mysql.jdbc.ConnectionProperties$IntegerConnectionProperty
3676760 27035   com.mysql.jdbc.ConnectionProperties$StringConnectionProperty
3448384 53881   java.util.LinkedHashMap$Entry
3278832 2469    com.mysql.jdbc.Connection
3124728 130197  java.lang.Integer
2650760 66269   java.util.LinkedList
2299264 12496   java.net.SocksSocketImpl
1651480 41287   java.util.ArrayList
912184  16289   java.nio.HeapCharBuffer
856896  13389   java.lang.ref.Finalizer
827088  17231   java.util.concurrent.locks.ReentrantLock$NonfairSync
789744  16453   java.util.concurrent.ConcurrentHashMap$Segment
704384  11006   java.net.SocketInputStream
659376  4338    com.mysql.jdbc.ConnectionProperties$MemorySizeConnectionProperty

DataSource create
--------------------------------------------
com.mysql.jdbc.jdbc2.optional.MysqlDataSource nds = new com.mysql.jdbc.jdbc2.optional.MysqlDataSource();

nds.setUser(cs.username);
nds.setPassword(cs.password);
nds.setConnectTimeout(5*1000);
nds.setSocketTimeout(60*60*1000);
nds.setElideSetAutoCommits(true);
nds.setRewriteBatchedStatements(true);
nds.setCharacterEncoding("ascii");
nds.setServerName(SERVER_NAME);
nds.setDatabaseName(DB_NAME);

How to repeat:
unknown

Suggested fix:
unknown
[6 Sep 2006 2:09] Mark Matthews
Are you using "profileSQL=true"? If so, then probably the same bug, which is fixed for 3.1.14 (see nightly snapshots) and in the 5.0 GA release.

This bit looks suspicious to me, as the only way these things could be around (that I can think of) is if _something_ is holding on to connection instances:

25714200        189075 
com.mysql.jdbc.ConnectionProperties$BooleanConnectionProperty
24280176        505837  java.util.HashMap$Entry
22790016        474792  java.util.Hashtable$Entry
16519000        412975  java.util.LinkedList$Entry
12238040        5919    java.util.HashMap$Entry[]

You don't cache connections anywhere in your code?
[6 Sep 2006 2:25] P Eger
The datasource options above are the only ones we are setting beyond the defaults.

I have confirmed that "profileSQL" is definitely not enabled, nor should any kind of connection caching. Basically we open 1 mysql connection on each incoming request, then close when the request completes.  The app is fronted by Tomcat with a max pool size of 500 so there should be no way more that # of connections could be open at any one time.

I will triple check the app for connection closure but i'm almost certain we're using safe try/catch idiom on all connections.

Also please note, if it was not clear, that the char[], byte[], String, HashMap, Object[], data in the above heap is part of the app's normal heap (IE part of normal operations).
[6 Sep 2006 3:29] Mark Matthews
This number:

25714200        189075 
com.mysql.jdbc.ConnectionProperties$BooleanConnectionProperty

Roughly corresponds with this number:

3278832 2469    com.mysql.jdbc.Connection

(i.e. approx 2500 live instances of Connection, since there's approximately 84 BooleanConnectionProperties per Connection).

What I'm wondering is why you have 2500 live instances of Connections hanging around. I'll test this tomorrow, with what information you've given me, but I'd trying tracking down a couple of things. 

If you add "dontTrackOpenResources=true" to your JDBC URL (or by setting it on the DataSource), does this problem go away? (If so, you're leaking a statement or result set somewhere)

Does this problem go away if you use Connector/J 5.0.4 or a nightly snapshot of 3.1.14? If so, then I need to backtrack to see what bug you're actually hitting.
[6 Oct 2006 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[31 Mar 2014 11:52] Alexander Soklakov
I close this report as "Can't repeat" because there is no feedback for a long time and codebase is too old. Please, feel free to reopen it if the problem still exists in current driver.