Bug #87640 MySQL server compiled with openSSL is not compatible with old yaSSL client
Submitted: 1 Sep 2017 13:46 Modified: 27 Oct 2017 16:51
Reporter: Zhao Jianwei Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Documentation Severity:S1 (Critical)
Version:5.7.18 OS:Red Hat
Assigned to: CPU Architecture:Any

[1 Sep 2017 13:46] Zhao Jianwei
Description:
Hi, pretty guys 

I have compiled MySQL 5.7.18 with -DWITH_SSL=system.  it used system openSSL 1.0.1e.

then I boot a MySQL instance:
[my.cnf]

ssl-cert=/home/server-cert.pem
ssl-key=/home/server-key.pem
tls-version=TLSv1,TLSv1.1,TLSv1.2

when I used MySQL 5.7.18 client to connect server, it successed:

$ mysql_5718/bin/mysql -h127.0.0.1 -usu -P3306 --ssl-ca=/home/ca.pem  -p
mysql> show status like '%ssl_version%';
+---------------+---------+
| Variable_name | Value   |
+---------------+---------+
| Ssl_version   | TLSv1.2 |
+---------------+---------+

But when I used MySQL 5.6 or before client to connect server, it failed:

#mysql_5616/bin/mysql -h127.0.0.1 -usu -P3306 --ssl-ca=/home/ca.pem  -psu
ERROR 2026 (HY000): SSL connection error: unknown error number

so, does it mean that MySQL server compiled with openSSL is not compatible with old yaSSL client?

How to repeat:
1. reboot a MySQL server 5.7.18 compiled with  -DWITH_SSL=system,
2. then use old yaSSL client to connect server with --ssl-ca.
[1 Sep 2017 13:48] Zhao Jianwei
these are the certificate files that I used.

Attachment: server-key.pem (application/x-x509-ca-cert, text), 1.64 KiB.

[1 Sep 2017 13:49] Zhao Jianwei
these are the certificate files that I used.

Attachment: ca.pem (application/x-x509-ca-cert, text), 1.38 KiB.

[1 Sep 2017 13:49] Zhao Jianwei
these are the certificate files that I used.

Attachment: server-cert.pem (application/x-x509-ca-cert, text), 1.26 KiB.

[8 Sep 2017 8:37] Umesh Shastry
Hi jianwei zhao,

Thank you for the report and feedback.
I just built 5.7.19 with -DWITH_SSL=system, using the certs provided here and confirmed that not just 5.7.19 client is able to connect but also 5.6.37/5.5.56 clients are able to connect without any issues. Could you please try with 5.7.19, and tell us what other options you are using while building server(for 5.7.18 src build) etc. I'm joining build and test logs shortly for your reference.

Thanks,
Umesh
[8 Sep 2017 8:38] Umesh Shastry
5.7.19 build and test details

Attachment: 87640.results (application/octet-stream, text), 213.67 KiB.

[10 Sep 2017 3:00] Zhao Jianwei
Hi, Umesh

My client that compiled with yaSSL is 5.6.16,  that yaSSL have a bug:

OpenSSL require blank certificate when one of ssl-ca, or ssl-cert or ssl-key is missing.
but yaSSL doesn't send.  so it report connection error.

My real problem is that: our customers all over the world has different old client version,
so I can't fix the bug, but have to cover the bug on the side of server.

so I disabled the SSL_VERIFY_PEER on the server explicitly when ssl file missed.

thank you very mush for you kind explain and test.
[11 Sep 2017 10:29] Umesh Shastry
Thank you for confirming.
I'll close this bug as can't repeat for now as none of us here seeing this on latest GA builds, and also we don't fix bug in old versions and suggest you to upgrade.

Thanks,
Umesh
[12 Sep 2017 6:28] Umesh Shastry
After discussing internally with our security lead, concluded that this is not documented and hence converting this to doc request for now.

## 
-- 5.5.16 client

[umshastr@hod03]/export/umesh/server/binaries/GABuilds: cd mysql-5.5.16-linux2.6-x86_64
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.5.16-linux2.6-x86_64: bin/mysql -ubug -S /tmp/mysql_ushastry.sock --ssl-ca=/export/umesh/server/source/bugs/87640/mysql-5.7/certs/ca.pem
ERROR 2026 (HY000): SSL connection error: unknown error number
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.5.16-linux2.6-x86_64:
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.5.16-linux2.6-x86_64: cd ../mysql-5.6.16-linux-glibc2.5-x86_64

-- 5.6.16 client

[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.6.16-linux-glibc2.5-x86_64:
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.6.16-linux-glibc2.5-x86_64: bin/mysql -ubug -S /tmp/mysql_ushastry.sock --ssl-ca=/export/umesh/server/source/bugs/87640/mysql-5.7/certs/ca.pem
ERROR 2026 (HY000): SSL connection error: unknown error number
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.6.16-linux-glibc2.5-x86_64:

-- 5.1.77 client

[umshastr@hod03]/export/umesh/server/binaries/GABuilds: cd mysql-5.1.77/
[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.1.77: bin/mysql -ubug -S /tmp/mysql_ushastry.sock --ssl-ca=/export/umesh/server/source/bugs/87640/mysql-5.7/certs/ca.pem
ERROR 2026 (HY000): SSL connection error

-- 5.0.96 client

[umshastr@hod03]/export/umesh/server/binaries/GABuilds/mysql-5.0.96: bin/mysql -ubug -S /tmp/mysql_ushastry.sock --ssl-ca=/export/umesh/server/source/bugs/87640/mysql-5.7/certs/ca.pem
ERROR 2026 (HY000): SSL connection error
[27 Oct 2017 16:51] Paul Dubois
Posted by developer:
 
Added the following note to
https://dev.mysql.com/doc/refman/5.6/en/openssl-versus-yassl.html:

If the server is compiled against OpenSSL, clients from MySQL 5.6
versions older than 5.6.17 are not able to connect to the server
using encrypted connections if the client library is compiled using
yaSSL. Either use a client and server compiled using the same SSL
package, or upgrade to clients compiled against a client library
version from MySQL 5.6.17 or higher.

Added similar note to
https://dev.mysql.com/doc/refman/5.5/en/openssl-versus-yassl.html
except that the version is 5.5.37.