Bug #93590 javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
Submitted: 13 Dec 2018 10:37 Modified: 16 Apr 2019 16:59
Reporter: Arnolnt Spyros Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / J Severity:S2 (Serious)
Version:8.0.13 OS:Windows (windows 10)
Assigned to: CPU Architecture:Any

[13 Dec 2018 10:37] Arnolnt Spyros
Description:
I am using OpenJDK 11 and mysql-connector-java:8.0.13 in order to connect to a MySQL Server 8.0 which runs on localhost.

When I close the connection using Connection.close(), the TCP connection will remain in TIME_WAIT state on the server side, where it should actually be on the client side.

How to repeat:
Connect to a local MySQL Server 8.0 using JDBC.
[13 Dec 2018 10:41] Arnolnt Spyros
Related bug report: Bug#56979
[13 Dec 2018 13:32] MySQL Verification Team
Hello Arnold Spyros,

Thank you for the report.

regards,
Umesh
[14 Dec 2018 14:49] Cott Lang
This seems to impact any server version when using Java 11.
[31 Dec 2018 15:20] David Goate
I can confirm that I also see this on 

Spring Boot 2.1.1.RELEASE
Java 11 (adoptopenjdk/openjdk11:jdk-11.28-alpine-slim)
mysql-connector-java 8.0.13
HikariCP 3.2.0
Database: RDS Aurora MySql 5.7.12 (default param group)

This was also reported here: https://github.com/brettwooldridge/HikariCP/issues/1268
[13 Feb 2019 23:30] Clément Guillaume
I have the same issue with mysql-connector-java 8.0.15 and openjdk 11.0.1 (Ubuntu 18.04.2 LTS)
[13 Feb 2019 23:43] Domas Mituzas
is there an expectation that Java connectors will work with SSL servers in future?
[14 Feb 2019 16:01] John Casebolt
I have the same issue with mysql-connector-java 8.0.15 and ORACLE JDK 11.0.2 (RHEL 6.10).
I am using Jetty with DB sessions.

MySQL Connector/J connection params:

jdbc:mysql://user:password@host.io/sessions?useSSL=true&requireSSL=true&verifyServerCertificate=true&trustCertificateKeyStoreUrl=file:///opt/certs/MySQLTrustStore&trustCertificateKeyStorePassword=trustPass
[15 Feb 2019 15:02] Filipe Silva
I'm sorry, I have to ask. Why do you think TIME_WAIT should be on client side?
[18 Feb 2019 8:26] David Goate
I'm not personally sure about the TIME_WAIT aspect. Maybe this isn't the same bug I thought it was. I didn't check connection TCP states but this is the stacktrace I saw after changing to java 11 from 10.

Caught while disconnecting...
EXCEPTION STACK TRACE:

** BEGIN NESTED EXCEPTION ** 

javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify

STACKTRACE:

javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
	at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1312)
	at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
	at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
	at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
	at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:135)
	at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:441)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

See also: 

1) https://stackoverflow.com/questions/53988368/hikari-cp-ssl-exception-closing-inbound-befor...
2) https://github.com/brettwooldridge/HikariCP/issues/1268
[18 Feb 2019 11:34] Filipe Silva
Yes, the exception stack trace is an issue, but it's just clutter in the log. Java 11 just implemented SSL support differently and now this exception is being caught on a different place, and thus being logged while it wasn't before.

The real problem is how to control where sockets stay in TIME_WAIT state. This is being investigated at the moment.
[26 Feb 2019 0:44] Douglas Pope
I am having the same problem with the following test case.  URL, username, and password have already been input and are being stored in an object called dbcon.

LOGGER.info("Connecting to the DB...");
		
try (Connection connection = DriverManager.getConnection(dbcon.getUrl(), dbcon.getUser(), dbcon.getPasswd())) {
	LOGGER.info("DB is connected!");
} catch (SQLException e) {
	LOGGER.severe("DB connection didn't work");
	LOGGER.severe(e.getMessage());
}

I get two info log entries confirming that the DB connection works, and then the SSLException that is this bug.  Looking at my MySQL Workbench, I can sometimes see a new connection in the Sleep state, even after the test program has ended.
[11 Mar 2019 14:06] Rusu Robert
Same problem when using Java 11
[22 Mar 2019 9:32] Nicolae Iotu
I was struggling hard with the same issue.
My rig:
- CentOS 7 Linux 3.10.0-957.10.1.el7.x86_64
- mysql  Ver 8.0.15 for Linux on x86_64 (MySQL Community Server - GPL)
- mysql-connector-java-8.0.15
- Eclipse JEE

Finally tried the instructions given at:
https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-usagenotes-statements.html#connec...

Just run or adapt the example given. It works like a charm with no other tweaks. In other words, you need to use the correct syntax to the letter.

Hope it helps. Good luck!
[22 Mar 2019 11:45] Nicolae Iotu
Did a couple more tests and found out that actually the error is caused by the usage of try-with-resources.

Initially I was thinking that virtual destructors checking for null values for statement and/or ResultSet, are causing the bug. When commenting 'finally' block I found that actually try-with-resources is to blame.

Below works just fine for me:
========================================================
try {
        connection = DatabaseConfiguration.getConnection();
					
	if (connection != null) {
		PreparedStatement preparedStatement = null;
		ResultSet resultSet = null;

		String query = "SELECT * FROM mytable WHERE name = ?;";

		try {
			preparedStatement = connection.prepareStatement(query);
			preparedStatement.setString(1, name);
....
========================================================

While try-with-resources fails producing subject error and stack trace:
========================================================
try (Connection connection = DatabaseConfiguration.getConnection()) {
	String query = "SELECT * FROM mytable WHERE name = ?;";

	if (connection != null) {
		PreparedStatement preparedStatement = null;
		ResultSet resultSet = null;
...
========================================================

A detailed explanation is given here:
https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
[22 Mar 2019 11:51] David Goate
@Nicolae Iotu

Which version of the JRE are you using specifically?

Is the stacktrace you mention the same as:

EXCEPTION STACK TRACE:

** BEGIN NESTED EXCEPTION ** 

javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify

STACKTRACE:

javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
	at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1312)
	at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
	at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
	at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
	at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:135)
	at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:441)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

** END NESTED EXCEPTION **
[22 Mar 2019 12:25] Nicolae Iotu
Using JDK 11.0.2.

Below my stacktrace:

Fri Mar 22 13:39:50 EET 2019 WARN: Caught while disconnecting...

EXCEPTION STACK TRACE:

** BEGIN NESTED EXCEPTION ** 

javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify

STACKTRACE:

javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
	at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
	at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1319)
	at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
	at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
	at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
	at
...
...

** END NESTED EXCEPTION **
[2 Apr 2019 15:05] David Goate
This seems to also be under investigation here: https://bugs.openjdk.java.net/browse/JDK-8215102
[3 Apr 2019 21:32] Eric Peters
I added a "fix" for this here: https://github.com/mysql/mysql-connector-j/pull/32

Inspired by what netty does, just ignore this error.
[10 Apr 2019 15:58] OCA Admin
Contribution submitted via Github - Fix for #93590 - ignore javax.net.ssl.SSLException: closing inbound before recei 
(*) Contribution by Eric Peters (Github er1c, mysql-connector-j/pull/32#issuecomment-481670029): I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.

Contribution: git_patch_267214351.txt (text/plain), 1.44 KiB.

[16 Apr 2019 16:59] Daniel So
Posted by developer:
 
Added the following entry to the Connector/J 8.0.16 changelog:

"For an SSL connection, after a client disconnected from a server by calling Connection.close(), the TCP connection remained in the TIME_WAIT state on the server side. With this fix, the connection remains in the TIME_WAIT state on the client side instead, in most cases."
[24 May 2019 14:01] Daniel So
Posted by developer:
 
Also added the changelog entry for Connector/J 5.1.48.