MySQL Bugs: #15067: Thread Deadlock

Bug #15067	Thread Deadlock
Submitted:	19 Nov 2005 14:49	Modified:	31 Mar 2014 10:07
Reporter:	Jim Redman	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	Connector / J	Severity:	S1 (Critical)
Version:	mysql-connector-java-3.1.10-bin.jar	OS:	Linux (Linux)
Assigned to:	Assigned Account	CPU Architecture:	Any

Description:
$ uname -a
Linux charizard 2.6.11-1.1369_FC4 #1 Thu Jun 2 22:55:56 EDT 2005 i686 i686 i386 GNU/Linux
$ java -version
java version "1.4.2_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-b04)
Java HotSpot(TM) Client VM (build 1.4.2_05-b04, mixed mode)

Hot on the heels of 14972, which prevents me from using mysql-connector-java-3.1.11-bin.jar.  Here's another thread deadlock.

Thread [Timer for ExtremeServer "ExtremeServer50" - Value "0"] (Suspended)
	owns: Connection  (id=45339)
	owns: DatabaseRowArrayLookup  (id=45373)
	waiting for: ResultSet  (id=45372)
	ResultSet.close() line: 715
	Statement.realClose(boolean) line: 1597
	Connection.closeAllOpenStatements() line: 2112
	Connection.realClose(boolean, boolean, boolean, Throwable) line: 4379
	Connection.close() line: 2084
	DatabaseConnectionManager.closeConnection(ConnectionClosedListener) line: 291
	DatabaseRowArrayLookup(DatabaseConnection).closeConnection() line: 410
	DatabaseRowArrayLookup(DatabaseRawLookup).performQuery() line: 319
	DatabaseRowArrayLookup(DatabaseRawLookup).executeStatement() line: 205
	DatabaseRowArrayLookup(DatabaseRawLookup).trigger(ValueChangedEvent) line: 149
	DatabaseRowArrayLookup(DatabaseRawLookup).valueChanged(int, ValueChangedEvent) line: 133
	StaticConnector.validatedValueChanged(ValueChangedEvent) line: 101
	[.....]
	
	
Thread [Timer for SynchronizedClockTick "SynchronizedClockTick" - Value "0"] (Suspended)
	owns: ResultSet  (id=45372)
	waiting for: Connection  (id=45339)
	Connection.getCharsetConverter(String) line: 3161
	Field.getStringFromBytes(int, int) line: 539
	Field.getName() line: 456
	ResultSet.buildIndexMapping() line: 572
	ResultSet.findColumn(String) line: 886
	ResultSet.getBytes(String) line: 1580
	DatabaseMetaData.getColumns(String, String, String, String) line: 1906
	TriggeredHistorical.getType(String, String) line: 300
	TriggeredHistorical.update(long, Hashtable) line: 458
	TriggeredHistorical.trigger(ValueChangedEvent) line: 697
	TriggeredHistorical.valueChanged(int, ValueChangedEvent) line: 182
	StaticConnector.validatedValueChanged(ValueChangedEvent) line: 101
	StaticConnector(Connector).valueChanged(ValueChangedEvent) line: 567
       [....]

How to repeat:
Thread deadlocks are rarely reproducible.  I do not have a simple case for this.

Probably a similar reason 14972, so we'll look into it, however the stack trace does show that your application isn't closing the statement(s) it created, which is a resource leak, at least until the connection itself is closed, so you'll probably want to look into that.

Here's a quick overview of how we use connections, in case it's useful.

We open a connection to the database and share it between components on a number of threads.  Each component may create a number of statements which remain open for the life of the connection.  Any thread, when it detects any error will close the connection and notify the close. (OT - the lack of standardization of getErrorCode() means we have no way to judge the serverity of the error across different databases).  When the application is configured correctly, connections close infrequently and only when there's a terminal problem with the connection (loss of network, database down, etc.).  

In the particular example, one component was mis-configured so was closing the connection, about every 10-15 seconds, on a "soft" error (wrong table name or something similar) even while the connection was still valid and other threads were using it.  Hence the stack trace where one thread is asking for the type of a column while the other thread is closing the connection.

We're going to tackle this when we fix BUG#18719 in Connector/J 5.0, so I'm consolidating all thread deadlock reports into this one bug, as they're all related.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/4463

Please test propsed fixes in this nightly build:

http://downloads.mysql.com/snapshots/mysql-connector-java-5.0/mysql-connector-java-5.0-nig...

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Duplicate of Bug#18719