Bug #41628 NullPointerException from Statement.execute() when closing from another thread
Submitted: 19 Dec 2008 8:35 Modified: 19 Dec 2008 10:27
Reporter: Lennart Schedin Email Updates:
Status: Not a Bug Impact on me:
None 
Category:Connector / J Severity:S3 (Non-critical)
Version:5.1.7 OS:Microsoft Windows
Assigned to:
Tags: NullPointerException, thread safe

[19 Dec 2008 8:35] Lennart Schedin
Description:
When a Statement has been closed/is closing there is a small time frame where calls to execute() can cause NullPointerException. In StatementImpl:
If thread A closes the Statement and the thread is at a point before this row in realClose():
		this.isClosed = true;
Thread B can then enter a execute() method and make it into the core of it, passing the checkClosed() check point.
Thread A then continues to execute and sets this.isClosed to true and then beginning to null the instance variables.
Thread B then continues to execute inside the execute method and tries to use the instance variables (the exact location of the NullPointerException may vary.

A Statement can be closed for a couple for reasons one example is that another thread invokes cancel(). Another might be that the Connection object is closed and close is propagates to all the opened Statement objects.

Here are some example stack traces:

With Mysql connector 5.0.8:
java.lang.NullPointerException
        at com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:1709)
        at com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:1685)
        at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1396)
        at se.lesc.database.DAOSql.getByKey(DAOSql.java:80)

Exception in thread "Timer-0" java.lang.NullPointerException
        at com.mysql.jdbc.Statement.execute(Statement.java:585)
        at se.lesc.database.DAOSql.getBySql(DAOSql.java:45)

java.lang.NullPointerException
	at com.mysql.jdbc.Statement.execute(Statement.java:742)
	at se.lesc.mysql_close_npe.CloseNPETest$1.run(CloseNPETest.java:34)
	at java.lang.Thread.run(Unknown Source)

With Mysql connector 5.1.7:
java.lang.NullPointerException
	at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:794)
	at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:587)
	at se.lesc.mysql_close_npe.CloseNPETest$1.run(CloseNPETest.java:34)
	at java.lang.Thread.run(Unknown Source)

How to repeat:
Run the following test program
public class CloseNPETest {

  @Test
  public void testNPE() throws ClassNotFoundException, SQLException, InterruptedException {

    //Dummy class to store an exception from a thread.
    class ExeceptionContainer {
      private Exception e;
    }
    final ExeceptionContainer exceptionContainer = new ExeceptionContainer();
    
    Class.forName("com.mysql.jdbc.Driver");
    Connection connection = DriverManager.getConnection("jdbc:mysql://localhost/test", "root", "");

    final Statement statement = connection.createStatement();
    
    Thread myThread = new Thread(new Runnable() {
      public void run() {
        try {
          for (int i = 0; i < 100000; i++) {
            try {
              //Execute a dummy statement
              statement.execute("SELECT 1+2");
            } catch (SQLException e) {
              //This is okay
            }
          }
        } catch (RuntimeException e) {
          //Store the exception
          exceptionContainer.e = e;
        }
      }
    });
    myThread.start();

    //Ensure that the thread has been started
    Thread.sleep(1000);
    
    //Perform "evil" close
    statement.close();
    
    myThread.join();
    
    if (exceptionContainer.e != null) {
      exceptionContainer.e.printStackTrace();
      fail("RuntimeException from execute");
    }
  }
}

The program must be modified to use correct database, user and password. The exact location of the NullPointerException might vary. It does not occur all the times. I executed the test on 
* Intel duel core CPU, Windows XP, Mysql server backend 5.0
* Intel single core CPU, Windows XP, Mysql server backend 5.1
The first two exceptions example in the bug report was seen on a production system running Linux with Intel Dual Core CPU (Mysql-server backend 5.0).

Suggested fix:
I guess a "closing"-state could be introduced in realClose. This closing state must probably be synchronized. This state can then be used to determine if other threads should enter the execute methods (via the checkClosed() method). This fix is probably not trivial since care must be taken because the isClosed instance variable is used in sub classes.
[19 Dec 2008 10:08] Tonci Grgin
Hi Lennart and thanks for your report.

As I repeated many times in BugsDB, Java might be thread-safe but that does *not* mean things are shareable across threads. It is my firm belief this is not a bug but I'll consult with devs before closing.
[19 Dec 2008 10:14] Tonci Grgin
Lennart, I've consulted and this is not expected to work at all. So, closing as !Bg.
[19 Dec 2008 10:27] Lennart Schedin
Hi Tonci,

I understand your point of view. The problem in my case is that we have a thread the periodically check if a connection is alive. If the connection is not alive we call Connection.close(). This call is then propagated to the Statement.close() and thus causing the NullPointerException for any other thread currently executing a Statement. I guess a real connection pooling would solve the problem.
[19 Dec 2008 10:42] Tonci Grgin
Lennart, just make sure you use pool that supports pooling API properly otherwise c/J will not have feedback that connection is pooled "above" it and you'll run into same trouble again.