Bug #21868 Server crashes if encrypted certificate key provided
Submitted: 28 Aug 2006 11:17 Modified: 9 Nov 2006 15:13
Reporter: Domas Mituzas Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S3 (Non-critical)
Version:5.1.12-bk@1.2285 OS:Linux (Linux, MacOSX)
Assigned to: Magnus Blåudd CPU Architecture:Any
Tags: yassl

[28 Aug 2006 11:17] Domas Mituzas
Description:
If certificate file is DES-encrypted, MySQL crashes with assertion:

coding.cpp:243: failed assertion `endLine == '\n''

certificate files attached to issue

How to repeat:
on fresh mysql_install_db environment:

(gdb) run
Starting program: /opt/local/bin/mysqld --datadir=/Users/midom/Tests/yassl --ssl --ssl-ca=/Users/midom/Tests/yassl/certs/ca-cert.pem --ssl-cert=/Users/midom/Tests/yassl/certs/server-cert.pem --ssl-key=/Users/midom/Tests/yassl/certs/server-key.pem
Reading symbols for shared libraries .+ done
060828 14:11:55 [Warning] Setting lower_case_table_names=2 because file system for /Users/midom/Tests/yassl/ is case insensitive
coding.cpp:243: failed assertion `endLine == '\n''

Program received signal SIGABRT, Aborted.
0x9003dacc in kill ()
(gdb) bt
#0  0x9003dacc in kill ()
#1  0x9010f22f in raise ()
#2  0x9010dd82 in abort ()
#3  0x0046caaa in __eprintf () at TCP_Transporter.cpp:292
#4  0x004955eb in TaoCrypt::Base64Decoder::Decode (this=0xbffff6f0) at coding.cpp:243
#5  0x004749ec in yaSSL::PemToDer (file=0xa000bd00, type=PrivateKey) at ../taocrypt/include/coding.hpp:87
#6  0x0046cc4d in yaSSL::read_file (ctx=0x4b012a0, file=0xbffffacb "/Users/midom/Tests/yassl/certs/server-key.pem", format=11, type=PrivateKey) at ssl.cpp:96
#7  0x0046d561 in yaSSL_CTX_use_PrivateKey_file (ctx=0x4b012a0, file=0xbffffacb "/Users/midom/Tests/yassl/certs/server-key.pem", format=11) at ssl.cpp:515
#8  0x0042a4b2 in vio_set_cert_stuff (ctx=0x4b012a0, cert_file=0xbffffa92 "/Users/midom/Tests/yassl/certs/server-cert.pem", key_file=0xbffffacb "/Users/midom/Tests/yassl/certs/server-key.pem") at viosslfactories.c:100
#9  0x0042aaee in new_VioSSLFd (key_file=0xbffffacb "/Users/midom/Tests/yassl/certs/server-key.pem", cert_file=0xbffffa92 "/Users/midom/Tests/yassl/certs/server-cert.pem", ca_file=0xbffffa5c "/Users/midom/Tests/yassl/certs/ca-cert.pem", ca_path=0x0, cipher=0x0, method=0x4b00a30) at viosslfactories.c:268
#10 0x0042ac5c in new_VioSSLAcceptorFd (key_file=0xbffffacb "/Users/midom/Tests/yassl/certs/server-key.pem", cert_file=0xbffffa92 "/Users/midom/Tests/yassl/certs/server-cert.pem", ca_file=0xbffffa5c "/Users/midom/Tests/yassl/certs/ca-cert.pem", ca_path=0x0, cipher=0x0) at viosslfactories.c:321
#11 0x0009bddf in main (argc=6, argv=0xbffff98c) at mysqld.cc:2974

Suggested fix:
-
[28 Aug 2006 11:17] Domas Mituzas
public key

Attachment: server-cert.pem (application/octet-stream, text), 956 bytes.

[28 Aug 2006 11:17] Domas Mituzas
private key

Attachment: server-key.pem (application/octet-stream, text), 963 bytes.

[23 Sep 2006 3:37] Craig Howland
Did anyone find a fix for this?  I have the same problem on mysql 5.0.24a, openssl 0.9.7a., Linux 2.6.9-42.0.2.ELsmp
[25 Sep 2006 14:44] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12476

ChangeSet@1.2284, 2006-09-25 16:44:15+02:00, msvensson@neptunus.(none) +5 -0
  Bug#21868 Server crashes if encrypted certificate key provided
   - Add test case which runs the same tests as ssl.test but with a different DES encrypted key
[26 Sep 2006 10:46] Magnus Blåudd
Imported latest version of yaSSL and then run the test for this bug. Server doesn't crash but it doesn't seem be able to read the cert and key either.

Best workaround is to use a different encryption for the certificates.
[8 Oct 2006 3:31] Jeanne Rash
We also ran into this problem with a PEM encoded SSL CERT that was sent to us via Email by a commercial Certificate Authority.  

mysqld: coding.cpp:239: void TaoCrypt::Base64Decoder::Decode(): Assertion `endLine == '\n'' failed.
mysqld got signal 6;

When we looked at the format of the cert we discovered that there was a single space at the beginning of each line.  Once we deleted the extra spaces mysql came up fine.
[12 Oct 2006 17:50] Magnus Blåudd
No crash anymore after importing latest version of yaSSL
[9 Nov 2006 15:13] Paul DuBois
Noted in 5.0.30 (not 5.0.29), 5.1.13 changelogs.

Use of a DES-encrypted SSL certificate file caused a server crash.