MySQL Bugs: #14221: SSL support breaks going from OpenSSL 0.9.7i to 0.9.8a

Bug #14221	SSL support breaks going from OpenSSL 0.9.7i to 0.9.8a
Submitted:	21 Oct 2005 21:38	Modified:	9 Dec 2005 0:26
Reporter:	Kris Karas	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server	Severity:	S3 (Non-critical)
Version:	4.1.16-BK, 4.1.15, 5.0.15	OS:	Linux (Linux)
Assigned to:	Sergei Glukhov	CPU Architecture:	Any

Description:
When upgrading OpenSSL shared libraries from version 0.9.7i to 0.9.8a, MySQL will no longer honor SSL connection requests; grants with 'REQUIRE SSL' will fail to authenticate.

Recompiling MySQL against the new openssl-0.9.8a include files does not help.

Doing a tcpdump against the raw MySQL tcp traffic shows that no attempt is made to start the SSL handshake.

Recompiling MySQL with '--enable-debug' and tracing both the client and server shows the following possibly smoking gun amidst the server trace output:
"OpenSSL: error:140A90A1:SSL routines:SSL_CTX_new:library has no ciphers:ssl_lib.c 1424:"
Attempting to manually add a cipher list via '--ssl-cipher=...' has no effect.

How to repeat:
1. Compile mysql against a dynamically-linked version of openssl (e.g. libssl.so.0 and libcrypto.so.0), either openssl 0.9.7 or 0.9.8 is fine.

2. Install openssl 0.9.7 if it's not the current version, /sbin/ldconfig, then start mysql with /etc/my.cnf set up with certificates so as to enable SSL for both client and server.

3. mysql> \s
...
SSL:   Cipher in use is DHE-RSA-AES256-SHA

4. Shut down mysql.  Upgrade libssl.so.0 and libcrypto.so.0 to OpenSSL version 0.9.8a.  Restart mysqld.

5. mysql> \s
...
SSL:  Not in use

Suggested fix:
Unknown.  The SSL_CTX_new routine hints at not being able to initialize the list of ciphers.

Thank you for a problem report. Sorry, but I can't repeat the problem you had written about. Please, describe the exact steps you performed, in particular, how you installed openssl-0.9.7i and then how you upgraded it to openssl-0.9.8a. 

In fact, the problem I discovered is that 0.9.8a is not creating libcrypto.so.0.9.8 for me. So, either 0.9.7 is used (then I have SSL in place after restart), or I simply get:

/users/vkravchuk/dbs/4.1-nocona/libexec/mysqld: error while loading shared libraries: libcrypto.so.0.9.7: cannot open shared object file: No such file or directory

upon server startup.

What Linux had you used, by the way? I tried on:

-bash-2.05b$ uname -a
Linux nocona.mysql.com 2.4.21-32.0.1.ELsmp #1 SMP Tue May 17 17:46:36 EDT 2005 x86_64 x86_64 x86_64 GNU/Linux

For Valeriy Kravchuk:

This is a Slackware GNU/Linux system, version 10.2, with gcc 3.3.6 and glibc 2.3.5:
    "Linux gaslamp 2.6.13.4 #1 SMP Tue Oct 11 03:07:17 EDT 2005
     i686 unknown unknown GNU/Linux"
The Linux kernel, critical system libraries (such as OpenSSL) and servers/services (such as MySQL and Apache) are hand-compiled from the most recent public source without vendor patches, so as to maintain a pristine environment faithful to the authors' wishes.

To build OpenSSL 0.9.8a on your particular Linux system, do the following:
    tar xzf openssl-0.9.8a.tar.gz
    cd openssl-0.9.8a
    # note: add any extra args to Configure here, such as -DPERL5
    # or --with-krb5-flavor.  This is just the basics:
    ./Configure shared linux-elf
    make && make install
    # Now, make sure this library also supports OpenSSL 0.9.7
    cd /usr/local/ssl/lib
    ln -s libssl.so.0.9.8 libssl.so.0.9.7
    ln -s libcrypto.so.0.9.8 libcrypto.so.0.9.7
    # Make sure our linker finds it!
    cd /etc
    grep -q usr/local/ssl/lib ld.so.conf || echo "/usr/local/ssl/lib" >> ld.so.conf
    ldconfig

Of course, if you wish to use your vendor's pre-compiled openssl-0.9.8a, then you will have to create the symbolic links just as I did above, but in whatever directory the vendor places the files (probably /usr/lib).  If you forget this step, you will get the error message you saw earlier, "error loading shared libraries: libcrypto.so.0.9.7: no such file".

Additional information: http://lists.mysql.com/internals/31769

Verified just as described on Fedora Core 1 with 4.1.16-BK (ChangeSet@1.2462, 2005-11-03 13:02:05+02:00, jani@ua141d10.elisa.omakaista.fi).

With 0.9.7h we got:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.16

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> \s
--------------
bin/mysql  Ver 14.7 Distrib 4.1.16, for redhat-linux-gnu (i686) using  EditLine
wrapper

Connection id:          1
Current database:
Current user:           root@localhost
SSL:                    Cipher in use is DHE-RSA-AES256-SHA
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         4.1.16
...

After installation of 0.9.8a and making symbolic links:

[openxs@Fedora openxs]$ ls -l /tmp/lib
total 3928
drwxrwxr-x    2 openxs   openxs       4096 Nov  4 17:54 engines
-rw-r--r--    1 openxs   openxs    2041826 Nov  4 17:54 libcrypto.a
lrwxrwxrwx    1 openxs   openxs         18 Nov  4 17:54 libcrypto.so -> libcrypt
o.so.0.9.8
lrwxrwxrwx    1 openxs   openxs         18 Oct 28 16:16 libcrypto.so.0 -> libcry
pto.so.0.9.7
lrwxrwxrwx    1 openxs   openxs         18 Nov  4 18:04 libcrypto.so.0.9.7 -> li
bcrypto.so.0.9.8
-r-xr-xr-x    1 openxs   openxs    1327191 Nov  4 17:54 libcrypto.so.0.9.8
-rw-r--r--    1 openxs   openxs     359446 Nov  4 17:54 libssl.a
lrwxrwxrwx    1 openxs   openxs         15 Nov  4 17:54 libssl.so -> libssl.so.0
.9.8
lrwxrwxrwx    1 openxs   openxs         15 Oct 28 16:16 libssl.so.0 -> libssl.so
.0.9.7
lrwxrwxrwx    1 openxs   openxs         15 Nov  4 18:03 libssl.so.0.9.7 -> libss
l.so.0.9.8
-r-xr-xr-x    1 openxs   openxs     261865 Nov  4 17:54 libssl.so.0.9.8
drwxrwxr-x    2 openxs   openxs       4096 Nov  4 17:54 pkgconfig
[openxs@Fedora openxs]$ cd dbs/4.1/
[openxs@Fedora 4.1]$ bin/mysqld_safe --defaults-file=/home/openxs/dbs/4.1/my.cn
f &
[1] 13069
[openxs@Fedora 4.1]$ Starting mysqld daemon with databases from /home/openxs/dbs/4.1/var

[openxs@Fedora 4.1]$ bin/mysql --defaults-file=/home/openxs/dbs/4.1/my.cnf -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.16

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> \s
--------------
bin/mysql  Ver 14.7 Distrib 4.1.16, for redhat-linux-gnu (i686) using  EditLine
wrapper

Connection id:          1
Current database:
Current user:           root@localhost
SSL:                    Not in use
...

[openxs@Fedora 4.1]$ uname -a
Linux Fedora 2.4.22-1.2115.nptl #1 Wed Oct 29 15:42:51 EST 2003 i686 i686 i386 GNU/Linux

Addendum:  This also affects MySQL 5.0.15 (and presumably earlier 5.0.x).
KTK

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/32403

The recently-submitted patch has been verified here in the field against MySQL 5.0.16

Warning: this might be a good entry for the FAQ or release-notes:
Assume that a client connects to a remote server via credentials that
do not require SSL (mysql.user.ssl_type='') but SSL has been requested (--ssl-cert=somecert.pem) by the client:
If both client and server are patched, the connection succeeds with SSL.
If both client and server are unpatched, the connection also succeeds, without SSL.
If the server is patched and the client unpatched, the connection fails with ERROR 2026.
This behavior is probably a good thing if the client is interactive, but might cause problems for unattended operation (e.g. replication farms where one machine is upgraded before another).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/32816

Fixed in 4.1.16

Noted in 4.1.16 changelog.