| Bug #25073 | rewriteBatchedStatements memory leak on executeBatch | ||
|---|---|---|---|
| Submitted: | 14 Dec 2006 14:40 | Modified: | 22 Feb 2007 14:18 |
| Reporter: | Jason Winnebeck | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Connector / J | Severity: | S2 (Serious) |
| Version: | 3.1.13, 5.0.3, 5.0.4 | OS: | Any (all) |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | memory leak | ||
[14 Dec 2006 14:41]
Jason Winnebeck
Patch against 5.0.3 source
Attachment: patch (application/octet-stream, text), 489 bytes.
[14 Dec 2006 14:45]
Jason Winnebeck
I meant to mark this as S2.
[14 Dec 2006 14:47]
Jason Winnebeck
Sorry, patch is against 5.0.4 source and not 5.0.3 but the fix is the same only shifted a few lines.
[14 Dec 2006 18:19]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/16976
[22 Feb 2007 14:05]
MC Brown
A note has been added to the 5.0.5 changelog.
[22 Feb 2007 14:18]
Jason Winnebeck
Will this be put into the 3.1 series? The 5 series driver may be more functional but the 3.1 series is much faster in our application and the primary driver for our application is performance.
[4 Apr 2007 18:07]
Joseph Dunleavy
This is a HPROF output when I have &rewriteBatchedStatements=true.
percent live alloc'ed stack class
rank self accum bytes objs bytes objs trace name
1 30.97% 30.97% 131523792 6431956 154628992 7460048 330688 byte[]
2 16.37% 47.34% 69544416 859589 76181752 1002240 337650 byte[][]
Associated traces:
Responsible for 130MB of living objects after run is complete.
TRACE 330688:
com.mysql.jdbc.Buffer.getBytes(Buffer.java:198)
com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:318)
com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1366)
com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2333)
com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)
com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2040)
com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1443)
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1777)
Responsible for 70MB of living objects after run is complete.
TRACE 337650:
com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1361)
com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2333)
com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:435)
com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2040)
com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1443)
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1777)
com.mysql.jdbc.Connection.execSQL(Connection.java:3249)
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1268)
Description: When using parameters "?rewriteBatchedStatements=true&useServerPrepStmts=false", memory leak occurs when calling PreparedStatement.executeBatch() Analysis: When calling executeBatch, code flows to executeBatchedInserts, which creates a new PreparedStatement in PreparedStatement.executeBatchedInserts into variable batchedStatement. This statement is never closed and all references to it are lost when method ends. Apparently when a statement is created it registers itself into some map and this map holds a reference to the statement forever. Because of the client-side processing the PS object is very large, causing aggressive memory leak. How to repeat: Occurs always when enabling and actually using multi-insert statement rewriting Suggested fix: Close the temporary PrepStmt created patch against 5.0.3: (fix for 5.0.2 and 3.1.13 same, but different lines) diff -Naur src/com/mysql/jdbc/PreparedStatement.java src-rit/com/mysql/jdbc/PreparedStatement.java --- src/com/mysql/jdbc/PreparedStatement.java 2006-12-14 09:33:00.000000000 -0500 +++ src-rit/com/mysql/jdbc/PreparedStatement.java 2006-12-14 09:33:38.000000000 -0500 @@ -953,6 +953,8 @@ getBatchedGeneratedKeys(batchedStatement); } + batchedStatement.close(); + int[] updateCounts = new int[this.batchedArgs.size()]; for (int i = 0; i < this.batchedArgs.size(); i++) {