Bug #33050 5.0.50 fails many SSL testcases
Submitted: 6 Dec 2007 22:10 Modified: 20 Mar 2008 18:31
Reporter: Robin Johnson Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Tests Severity:S7 (Test Cases)
Version:5.0.50, 5.0.51, 5.0.52 OS:Linux
Assigned to: Magnus Blåudd CPU Architecture:Any
Tags: openssl ssl test 5.0.50 5.0.51
Triage: D3 (Medium)

[6 Dec 2007 22:10] Robin Johnson
Description:
5.0.50 fails a lot of the SSL testcases.
openssl_1 rpl_openssl rpl_ssl ssl ssl_8k_key ssl_compress ssl_connect

See the attached log for the full failure output.
I can reproduce on x86_64, PPC64 and x86.

./configure --prefix=/usr --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libexecdir=/usr/sbin --sysconfdir=/etc/mysql --localstatedir=/var/lib/mysql --sharedstatedir=/usr/share/mysql --libdir=/usr/lib64/mysql --includedir=/usr/include/mysql --with-low-memory --with-client-ldflags=-lstdc++ --enable-thread-safe-client --with-comment=Gentoo Linux mysql-5.0.50 --without-docs --without-big-tables --enable-local-infile --with-extra-charsets=all --with-mysqld-user=mysql --with-server --with-unix-socket-path=/var/run/mysqld/mysqld.sock --without-libwrap --enable-shared --enable-static --without-debug --without-ndb-debug --with-charset=utf8 --with-collation=utf8_general_ci --with-embedded-privilege-control --with-embedded-server --with-bench --enable-assembler --with-extra-tools --with-innodb --without-readline --with-openssl --without-berkeley-db --with-geometry --with-ndbcluster --with-archive-storage-engine --with-csv-storage-engine --with-blackhole-storage-engine --with-federated-storage-engine --build=x86_64-pc-linux-gnu

System information:
Portage 2.1.4_rc4 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.7-r0, 2.6.24-rc2-pmp-g28e80f62 x86_64)
=================================================================
System uname: 2.6.24-rc2-pmp-g28e80f62 x86_64 Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz
Timestamp of tree: Unknown
ccache version 2.4 [enabled]
app-shells/bash:     3.2_p17-r1
dev-lang/python:     2.5.1-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 2.0.0_rc6
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.24
virtual/os-headers:  2.6.23-r2
dev-libs/openssl:    0.9.8g
ACCEPT_KEYWORDS="amd64 ~amd64"
CFLAGS="-march=nocona -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CXXFLAGS="-march=nocona -O2 -pipe"

How to repeat:
build + run testcases
[6 Dec 2007 22:11] Robin Johnson
Complete build and test output.

Attachment: 20071206-mysql-5.0.50-openssl-failures.txt.gz (application/gzip, text), 98.42 KiB.

[7 Dec 2007 0:01] Robin Johnson
5.0.51 also fails the same SSL testcases (plus one more, mysqlcheck).
[7 Dec 2007 9:57] Norbert Tretkowski
I don't get the SSL failures on Debian, but the mysqlcheck failure as well:

--- /tmp/buildd/mysql-dfsg-5.0-5.0.51/mysql-test/r/mysqlcheck.result    2007-11-15 17:28:37.000000000 +0300
+++ /tmp/buildd/mysql-dfsg-5.0-5.0.51/mysql-test/var/log/mysqlcheck.reject      2007-12-07 12:56:29.000000000 +0300
@@ -49,7 +49,8 @@
 flush tables;
 removing and creating
 d_bug25347.t_bug25347
-error    : Incorrect file format 't_bug25347'
+Error    : Incorrect file format 't_bug25347'
+error    : Corrupt
 insert into t_bug25347 values (4),(5),(6);
 ERROR HY000: Incorrect file format 't_bug25347'
 d_bug25347.t_bug25347

mysqltest: Result content mismatch
[7 Dec 2007 10:20] Norbert Tretkowski
Full build log of 5.0.51 on Debian 4.0/amd64.

Attachment: buildd.log.gz (application/x-gzip, text), 104.91 KiB.

[8 Dec 2007 7:29] Robin Johnson
Norbert: your Debian build log shows that you had SSL disabled via --without-openssl. Could you please set it up to build with openssl, and also mention what version of OpenSSL is on your system?
[8 Dec 2007 18:44] Norbert Tretkowski
Indeed, the tests also fail with OpenSSL 0.9.8g on Debian when using --with-openssl.
[10 Dec 2007 9:48] Norbert Tretkowski
The mysqlcheck test doesn't fail in 5.0.52.
[10 Dec 2007 10:06] Norbert Tretkowski
SSL tests still fail with 5.0.52.
[10 Dec 2007 17:53] David Tonhofer
SSL tests fail for self-compiled 5.0.51 on Red Hat ES 4.6 (other tests pass except for "mysqlcheck" - see bug #33104.)

E.g. testing "openssl_1" works up and to the connection. It is likely that the behaviour of the client is not as expected in this environment:

---------- This gets executed ok: -----------

drop table if exists t1;
create table t1(f1 int);
insert into t1 values (5);
grant select on test.* to ssl_user1@localhost require SSL;
grant select on test.* to ssl_user2@localhost require cipher "DHE-RSA-AES256-SHA";
grant select on test.* to ssl_user3@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB/emailAddress=abstract.mys
ql.developer@mysql.com";
grant select on test.* to ssl_user4@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB/emailAddress=abstract.mys
ql.developer@mysql.com" ISSUER "/C=SE/ST=Uppsala/L=Uppsala/O=MySQL AB";
grant select on test.* to ssl_user5@localhost require cipher "DHE-RSA-AES256-SHA" AND SUBJECT "xxx";
flush privileges;

---------- The next instruction is: -----------

connect(localhost,ssl_user5,,test,MASTER_PORT,MASTER_SOCKET);

---------- With expected output: -----------

ERROR 28000: Access denied for user 'ssl_user5'@'localhost' (using password: NO)

---------- But what actually comes out is: -----------

mysqltest: At line 19: query 'connect  con1,localhost,ssl_user1,,,,,SSL' failed: 2026: SSL connection error
[20 Dec 2007 23:04] Miguel Solorzano
Thank you for the bug report. Verified on FC 6.0:

[miguel@amanhecer mysql-test]$ ./mysql-test-run.pl openssl_1
Logging: ./mysql-test-run.pl openssl_1
071220 20:55:42 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
071220 20:55:42 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295
MySQL Version 5.0.56
Skipping ndbcluster, mysqld not compiled with ndbcluster
Setting mysqld to support SSL connections
Using MTR_BUILD_THREAD      = 0
Using MASTER_MYPORT         = 9306
Using MASTER_MYPORT1        = 9307
Using SLAVE_MYPORT          = 9308
Using SLAVE_MYPORT1         = 9309
Using SLAVE_MYPORT2         = 9310
Using IM_PORT               = 9312
Using IM_MYSQLD1_PORT       = 9313
Using IM_MYSQLD2_PORT       = 9314
Killing Possible Leftover Processes
Removing Stale Files
Creating Directories
Installing Master Database
=======================================================
Starting Tests in the 'main' suite

TEST                           RESULT         TIME (ms)
-------------------------------------------------------

openssl_1                      [ fail ]
[22 Dec 2007 3:00] Kent Boortz
Configuring with --with-debug and running the test case like

  % ./mysql-test-run.pl --debug openssl_1

will show in "var/log/mysqltest.trace"

  error: SSL_do_handshake failure
  error: OpenSSL: error:140B4090:SSL routines:SSL_do_handshake:connection type not set:ssl_lib.c:2034:

As it turns out, OpenSSL contrary to yaSSL can't guess in
SSL_do_handshake() if called by a client or a server, you
need to call one of SSL_set_connect_state() or 
SSL_set_accept_state() before calling SSL_do_handshake().
[5 Jan 2008 2:20] Timothy Smith
I have a slight preference for Magnus' version of the patch, which passes a pointer to SSL_accept or SSL_connect to the helper function, instead of passing a flag.

It's frustrating that the api has this SSL_set_{connect,accept}_state() call, which isn't orthoganal with the separate SSL_accept and SSL_connect calls.  The whole SSL_do_handshake() thing is less user-friendly that it first appears.  Avoiding it makes the code clearer, I think.
[7 Feb 2008 7:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/41851

ChangeSet@1.2573, 2008-02-07 08:48:28+01:00, msvensson@pilot.mysql.com +1 -0
  Bug#33050 5.0.50 fails many SSL testcases
[25 Feb 2008 15:59] Bugs System
Pushed into 5.1.24-rc
[25 Feb 2008 16:05] Bugs System
Pushed into 5.0.58
[25 Feb 2008 16:05] Bugs System
Pushed into 6.0.5-alpha
[26 Feb 2008 11:31] Magnus Blåudd
When MySQL was build with OpenSSL the SSL library was not properly initialized with information of which endpoint(server or client) it was, this failing to connect.
[20 Mar 2008 18:31] Paul Dubois
Noted in 5.0.58, 5.1.24, 6.0.5 changelogs.
[31 Mar 2008 19:59] Jon Stephens
Pushed into 5.1-telco-6.3.
[31 Mar 2008 20:00] Jon Stephens
Also documented in the 5.1.23-ndb-6.3.11 changelog.
[21 Oct 2008 9:10] Valeriy Kravchuk
Bug #40141 was marked as a duplicate of this one.