| Bug #41484 | NullPointerException in ResultSetImpl.findColumn() | ||
|---|---|---|---|
| Submitted: | 15 Dec 2008 23:58 | Modified: | 16 Sep 2009 16:02 |
| Reporter: | Boris Burtin | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Connector / J | Severity: | S2 (Serious) |
| Version: | 5.1.8 r.838 | OS: | Linux |
| Assigned to: | Mark Matthews | CPU Architecture: | Any |
[15 Dec 2008 23:58]
Boris Burtin
[1 Jan 2009 6:31]
abayi tay
Caused by: java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.buildIndexMapping(ResultSetImpl.java:722) at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1100) at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5531) ... 84 more i also got same problem , why.got any solution. and i also use jdbc 5.1.7 and database is 5.0.51a-community-nt
[17 Jan 2009 21:33]
Chris Lampley
This bug was introduced in Bug #39962. It appears that hasBuiltIndexMapping is never set back to false in the realClose method, even though columnToIndexCache is then set to null. So, the subsequent call to findColumn causes a NullPointerException. Here is a work around for this particular issue, but the real fix may be setting hasBuiltIndexMapping = false in the realClose method. One of the developers more familiar with this class would have to confirm if that would be correct behavior. --- ResultSetImpl.java 2009-01-17 15:20:41.000000000 -0600 +++ ResultSetImpl-1.java 2009-01-17 15:26:15.000000000 -0600 @@ -1100,10 +1100,12 @@ buildIndexMapping(); } - index = (Integer) this.columnToIndexCache.get(columnName); + if (this.columnToIndexCache != null) { + index = (Integer) this.columnToIndexCache.get(columnName); - if (index != null) { - return index.intValue() + 1; + if (index != null) { + return index.intValue() + 1; + } } index = (Integer) this.columnLabelToIndex.get(columnName); @@ -1117,7 +1119,9 @@ } if (index != null) { - this.columnToIndexCache.put(columnName, index); + if (columnToIndexCache != null) { + this.columnToIndexCache.put(columnName, index); + } return index.intValue() + 1; }
[12 Feb 2009 16:56]
Tonci Grgin
Test case
Attachment: TestBug41484.java (text/java), 2.22 KiB.
[12 Feb 2009 16:58]
Tonci Grgin
Hello all. Please review attached test case which does not fail on my machine. If I missed something, correct me and attach new test case. Reopen the report afterwards. Test case output: .Loading JDBC driver 'com.mysql.jdbc.Driver' Done. Done. Connected to 5.1.30-community-log java.vm.version : 1.5.0_17-b04 java.vm.vendor : Sun Microsystems Inc. java.runtime.version : 1.5.0_17-b04 os.name : Windows Server 2008 os.version : null sun.management.compiler : HotSpot Client Compiler 2008-05-09 2009-02-12 Time: 0,452 OK (1 test)
[12 Feb 2009 16:58]
Tonci Grgin
c/J used is from latest bzr sources, maybe you should try snapshot?
[12 Feb 2009 19:55]
Jess Balint
The only way I can see to reproduce this is:
rs.getString("abc");
rs.close();
rs.getString("abc");
However, this should throw a SQLException due to the ResultSet being closed, not a NPE.
[12 Feb 2009 20:04]
Jess Balint
=== modified file 'src/com/mysql/jdbc/ResultSetImpl.java'
--- src/com/mysql/jdbc/ResultSetImpl.java 2008-11-11 06:19:59 +0000
+++ src/com/mysql/jdbc/ResultSetImpl.java 2009-02-12 19:58:23 +0000
@@ -1100,6 +1100,8 @@
public synchronized int findColumn(String columnName) throws SQLException {
Integer index;
+ checkClosed();
+
if (!this.hasBuiltIndexMapping) {
buildIndexMapping();
}
=== modified file 'src/testsuite/regression/ResultSetRegressionTest.java'
--- src/testsuite/regression/ResultSetRegressionTest.java 2008-10-20 22:29:45 +0000
+++ src/testsuite/regression/ResultSetRegressionTest.java 2009-02-12 20:02:07 +0000
@@ -4790,4 +4790,22 @@
cal.setTime(ts);
assertEquals(797, cal.get(Calendar.MILLISECOND));
}
+
+ /**
+ * Bug #41484
+ * Accessing fields by name after the ResultSet is closed throws NullPointerException.
+ */
+ public void testBug41484() throws Exception {
+ try {
+ rs = stmt.executeQuery("select 1 as abc");
+ rs.next();
+ rs.getString("abc");
+ rs.close();
+ rs.getString("abc");
+ } catch(SQLException ex) {
+ /* expected */
+ } finally {
+ closeMemberJDBCResources();
+ }
+ }
}
[12 Feb 2009 20:10]
Chris Lampley
This block of code actually produced the issue: /* SNIP --- ResultSet rs is not null and is not closed */ CachedRowSet crs = new CachedRowSetImpl(); crs.populate(rs); <-- exception thrown here. From this behavior, either there is a bug in MySQL's implementation or Sun's CachedRowSetImpl. But, this bug does not appear when CacheResultSetMetadata is turned off, so that points the bug to the MySQL driver.
[12 Feb 2009 21:22]
Jess Balint
Chris, Thanks, but I'm still not able to reproduce the issue with what you've given. I've enabled "cacheResultSetMetadata", and used the same code you've showed. I hope your ResultSet isn't already closed? Is it possible you can attach a complete class that demonstrates this bug?
[1 Apr 2009 17:25]
Aaron Evans
I am also experiencing this bug in 5.1.7 of connector/j. I've attached a class that reproduces the issue for me.
[1 Apr 2009 17:26]
Aaron Evans
a test case to reproduce 41484
Attachment: Reproduce41484.java (application/octet-stream, text), 1.42 KiB.
[2 Apr 2009 18:12]
Jess Balint
Aaron, I'm not able to reproduce this with your testcase. Can you include: * The version of MySQL server you are using. * The schema for the table that causes the test to fail. * The stack trace when the test fails.
[3 Apr 2009 12:17]
Aaron Evans
fails both on my local machine: mysql> SHOW VARIABLES LIKE '%version%'; +-------------------------+---------------------+ | Variable_name | Value | +-------------------------+---------------------+ | protocol_version | 10 | | version | 5.1.31 | | version_comment | Source distribution | | version_compile_machine | i386 | | version_compile_os | apple-darwin9.6.0 | +-------------------------+---------------------+ and on a solaris box: mysql> show variables like '%version%'; +-------------------------+------------------------------+ | Variable_name | Value | +-------------------------+------------------------------+ | protocol_version | 10 | | version | 5.1.30-log | | version_comment | MySQL Community Server (GPL) | | version_compile_machine | i386 | | version_compile_os | pc-solaris2.10 | +-------------------------+------------------------------+ with a database with a single table like: mysql> show create table bar; +-------+--------------------------------------------------------------------------------------+ | Table | Create Table | +-------+--------------------------------------------------------------------------------------+ | bar | CREATE TABLE `bar` ( `x` int(11) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 | +-------+--------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) The stacktrace is: java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1103) at com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:4890) at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50) at com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4822) at com.mysql.jdbc.DatabaseMetaData$2.forEach(DatabaseMetaData.java:2395) at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:50) at com.mysql.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:2361) at org.aarone.bugs.Reproduce41484.testReproduce41484(Reproduce41484.java:36) at org.aarone.bugs.Reproduce41484.main(Reproduce41484.java:42)
[16 May 2009 21:09]
Alan Williamson
I can confirm this bug too. Driver: 5.1.7 Database: 5.0.51 java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1103) at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5531) at com.mysql.jdbc.Field.getCollation(Field.java:469) at com.mysql.jdbc.ResultSetMetaData.isCaseSensitive(ResultSetMetaData.java:558) I upgrade from 5.0.7 driver to fix the same problem. It *did* fix the problem for some of the SQL statements, but now it throws it for others!!!! This is for simple SELECT statements with named columns.
[18 May 2009 16:54]
Michael Watts
I too have this problem with the 5.1.7 connector with both 5.0.24a and 5.1.34 servers. I have tried to simplify my test case but even though I attempted to follow the problem code exactly ( there are 3 result sets open concurrently ) it works fine [the test code]. I suspect the bug is very subtle, not least because my code executes hundreds of similar calls ( getXXX(<columnName>) ) and only one so far has thrown up the issue. I tried adding the checkClosed call suggested by Jess: interestingly this does throw an error saying that you cannot use the resultset after it is closed but it should be open and previous driver versions agree with that! I also noted that the source I looked at didn't quite line up with the NPE exception and also I needed to delete a class ( Apple... ) in the regression testing package in order to be able to build the driver. Personally I think this is a more serious bug then 'non-critical' as it seems to be insidious: most of the time the code works but... I hope that's of use to someone.
[18 May 2009 21:13]
Mark Matthews
When this happens, can you try calling isClosed() on (1) the statement that creates the result set, and (2) on the connection that created the statement? I'm guessing that there's a communications exception thrown somewhere, that's getting swallowed by either c/j or your application code, but such exceptions force the connection closed, thus closing any statements they created, and any result sets those statements created (as required by the JDBC specification).
[18 May 2009 21:16]
Mark Matthews
It would also be of interest if "useDynamicCharsetInfo=false" in your JDBC url makes this issue go away. When you ask the driver for RSMD.isCaseSensitive(), it has to run out and do a query on the database to get collation metadata. If this fails for some reason in a way that would clobber the connection, it would clobber your result set as well. By telling the driver that you don't want dynamic character set information, it will use a built-in table to determine case-sensitivity (amongst other things), thus avoiding this round-trip to the database.
[19 May 2009 12:02]
Michael Watts
This is bizarre!
First off, as Mark suspected there is something closing before being used. Here is a snippet of the code:
<code>
long timestamp = rs.getLong("TimeStamp");
debugPrint(489, rs, offline, online);
while (checkForOffline && nextOfflineTS < timestamp) {
... some lines doing some ops with java.util.Lists ...
debugPrint(501, rs, offline, online);
byte ctrlRackID = offline.getByte("CtrlRackID");
</code>
debugPrint takes the line number and the 3 ResultSets and prints out the isClosed state of the connection and the 3 ResultSet.getStatement objects ( the statements themselves are out-of-scope ). 489 prints out ( as do 5 others before it, not shown here ) all 4 are 'not closed', 501 doesn't print out and an error message "Operation not allowed on closed ResultSet" is spit out.
I messed around with driver versions - 5.1.6 & 5.1.5 make no difference; 5.0.5 complained about the call to isClosed.
Returning the application code to the original, I tried with the 5.0.5 driver and to my surprise it still complains at that same getByte call albeit with different line numbers in ResultSetImpl ( 949 from 1627 ).
Finally I tried the useDynamicCharsetInfo=false on the connection URL with 5.1.7 driver - there was no difference i.e. NPE at 1103 in ResultSetImpl.
Any ideas what on earth could be going on?
[21 May 2009 6:53]
Alan Williamson
okay, just as a follow-up here, i can confirm that Mark's suggestion of adding: &useDynamicCharsetInfo=false to the JDBC URL string resolves the issue at least for the moment. I do not see this as a "fix" for the problem, merely a hack. I also vote for this bug to be upgraded to CRITICAL -- because code that was happily working before with much older drivers now don't. When the code doesn't change and the server doesn't change, then sadly, all fingers point at the driver.
[21 May 2009 9:36]
Michael Watts
Errm.. very sorry about this but I did some more investigation into my 'mystery' only to find that, in fact, the code explicitly closed the resultsets ( online and offline in my code same ) at the wrong level [of loop nesting]. Now it is true that it worked before ( since 2004! ) and so Jess's suggestion of putting in the checkClosed call gives the real reason ( at least in my case ) for the problem. Obviously the previous driver we were using didn't care about this client-code bug but the new one ( quite rightly ) does. Thanks to Mark for his suggestions - that at least pointed me in the right direction.
[2 Jun 2009 5:59]
Jess Balint
Pushed for release in 5.1.8
[2 Jun 2009 16:38]
Boris Burtin
Even if it is a problem with client code, the driver shouldn't throw NPE. It should detect the invalid state and throw a SQLException that tells the application developer what's going wrong.
[23 Jun 2009 13:54]
Tony Bedford
An entry was added to the 5.1.8 changelog: When accessing a result set column by name using ResultSetImpl.findColumn() an exception was generated: java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1103) at com.mysql.jdbc.ResultSetImpl.getShort(ResultSetImpl.java:5415) at org.apache.commons.dbcp.DelegatingResultSet.getShort(DelegatingResultSet.java:219) at com.zimbra.cs.db.DbVolume.constructVolume(DbVolume.java:297) at com.zimbra.cs.db.DbVolume.get(DbVolume.java:197) at com.zimbra.cs.db.DbVolume.create(DbVolume.java:95) at com.zimbra.cs.store.Volume.create(Volume.java:227) at com.zimbra.cs.store.Volume.create(Volume.java:189) at com.zimbra.cs.service.admin.CreateVolume.handle(CreateVolume.java:48) at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:428) at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:285)
[7 Aug 2009 17:33]
Boris Burtin
Still happening in the 5.1.8 version of the driver. Full Zimbra bug report here: http://bugzilla.zimbra.com/show_bug.cgi?id=39936 Can we please get this fixed, preferably with a turnaround that's less than 8 months long? We're still stuck on the 5.0.3 driver due to this bug. java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1111) at com.mysql.jdbc.ResultSetImpl.getShort(ResultSetImpl.java:5432) at org.apache.commons.dbcp.DelegatingResultSet.getShort(DelegatingResultSet.java:219) at com.zimbra.cs.db.DbVolume.constructVolume(DbVolume.java:309) at com.zimbra.cs.db.DbVolume.get(DbVolume.java:203) at com.zimbra.cs.db.DbVolume.create(DbVolume.java:96) at com.zimbra.cs.store.file.Volume.create(Volume.java:233) at com.zimbra.cs.store.file.Volume.create(Volume.java:194) at com.zimbra.cs.service.admin.CreateVolume.handle(CreateVolume.java:54) at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:417) at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:272) at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:156) at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:278) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
[10 Aug 2009 6:35]
Tonci Grgin
I must agree with Boris here... Using the test case I already attached earlier *and* cacheResultSetMetadata=true I get the error (5.1 rev.838): Connected to 5.1.31-log java.vm.version : 1.5.0_17-b04 java.vm.vendor : Sun Microsystems Inc. java.runtime.version : 1.5.0_17-b04 os.name : Windows Server 2008 os.version : null sun.management.compiler : HotSpot Client Compiler 2008-05-09 E Time: 0,36 There was 1 error: 1) testBug41484(testsuite.simple.TestBug41484)java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1111) at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5548) at testsuite.simple.TestBug41484.testBug41484(TestBug41484.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at testsuite.simple.TestBug41484.main(TestBug41484.java:69) Full connection string: -Xmx512M -XX:+UseParallelGC -Dcom.mysql.jdbc.java6.javac=C:\jvms\jdk1.6.0\bin\javac.exe -Dcom.mysql.jdbc.java6.rtjar=C:\jvms\jdk1.6.0\jre\lib\rt.jar -Dcom.mysql.jdbc.testsuite.url.default=jdbc:mysql://myserver:xxx/test?user=xxx&password=xxx&autoReconnect=false&connectTimeout=5000&socketTimeout=30000&useUnicode=true&characterSetResults=utf8&useInformationSchema=true&useServerPrepStmts=true&cacheResultSetMetadata=true
[10 Aug 2009 17:41]
Mark Matthews
There ended up being two different bugs mixed in this report. The first one (where an NPE would happen with cached result set metadata), is now fixed, and will be available in 5.1.9, as well as nightly snapshot builds after midnight UTC at http://downloads.mysql.com/snapshot.php#connector-j
[16 Sep 2009 13:59]
Mark Matthews
The second one was an issue where accessing result set columns by name after the result set had been closed resulted in a NullPointerException. The driver now throws a SQLException in this case.
[16 Sep 2009 16:02]
Tony Bedford
When accessing a result set column by name using ResultSetImpl.findColumn() an exception was generated: java.lang.NullPointerException at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1103) at com.mysql.jdbc.ResultSetImpl.getShort(ResultSetImpl.java:5415) at org.apache.commons.dbcp.DelegatingResultSet.getShort(DelegatingResultSet.java:219) at com.zimbra.cs.db.DbVolume.constructVolume(DbVolume.java:297) at com.zimbra.cs.db.DbVolume.get(DbVolume.java:197) at com.zimbra.cs.db.DbVolume.create(DbVolume.java:95) at com.zimbra.cs.store.Volume.create(Volume.java:227) at com.zimbra.cs.store.Volume.create(Volume.java:189) at com.zimbra.cs.service.admin.CreateVolume.handle(CreateVolume.java:48) at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:428) at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:285)
[12 Mar 2010 9:28]
Maurice G
Hi,
I have a NPE on ResultSetImpl.
I'm wondering what to do or what is the workaround. Is the presented workaround sufficient?
I'm using MySQL in combination with Tomcat. the MySQL driver that I'm using using is my-sql-connector-java-5.1.12 (latest as far as I know).
My application queries the database to create an intermediate result. The queries are relatively simple (basic select, with where clause that does not involve other tables), but there are a lot of them.
Quite frequent I get the following exception:
java.lang.NullPointerException
at com.mysql.jdbc.ResultSetImpl.buildIndexMapping(ResultSetImpl.java:732)
at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1069)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5488)
I know the data I'm querying is in the database. The exception does not occur always. It almost feels like it is a timing issue. However, I'm not querying concurrently; all queries are in sequence. I tried synchronizing the part of the code that does the final query, but this has no result.
The NPE always occurs in the same part of the code. I considered rewriting it, however, I'm working on this application form quite some time now. This part of the code is "old" and often used. The NPE surfaced now the application is getting bigger, or now that the number of queries in a session are increasing. Is there a limitation to the number of queries that can be executed within a session?
Any help how to remove this issue, or things to try is appreciated
Regards,
Maurice
[12 Mar 2010 9:32]
Tonci Grgin
Maurice, are you able to reliably produce the error without Tomcat? Cause we can not involve Tomcat in BugsDB...
[12 Mar 2010 9:36]
Tonci Grgin
Maurice, I will monitor this report as we should never NPE, as a rule. Please see if you can provide standalone test case along with connection string info and such.
[12 Mar 2010 15:14]
Mark Matthews
Maurice, The only way this could be happening, as far as I can see, is someone/something is calling ResultSet.close() right after ResultSet.getString() has been called, but before it returns. This could be your application (is it multi-threaded, or does it have a tricky processing path), or you could be losing the connection sometime during this time, which causes the driver to close the connection implicitly, which closes all open statements, which closes all open result sets. You shouldn't be getting an NPE though, we should raise a nicer SQLException. You might check the mysqld error log to see if anything is going on there that implies that the connection is being lost?
[15 Mar 2010 8:39]
Maurice G
Hi Tonci, Mark, Thanks for the quick response. Based on the suggestion from Mark I started diving into my code. Although the close of the resultset is not done, I found some other things that I was not happy with. What could happen is that within a connection and within a session, multiple queries were executed at the same time, but with different resultsets. This weekend I improved this. After the improvement, I cannot reproduce the NPE any more. If I encounter something again, I will isolate the error as good as I can and give more information. Thanks again for the suggestion.
[15 Mar 2010 8:42]
Tonci Grgin
Maurice, keep in mind, although things might be thread-safe that does not make them sharable across threads.
