MySQL Bugs: #25189: mysqld: coding.cpp:243: void TaoCrypt::Base64Decoder::Decode(): Assertion `endLi

Bug #25189	mysqld: coding.cpp:243: void TaoCrypt::Base64Decoder::Decode(): Assertion `endLi
Submitted:	19 Dec 2006 22:10	Modified:	15 Feb 2007 4:16
Reporter:	Colin Keith	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Errors	Severity:	S2 (Serious)
Version:	mysql-standard-5.0.27-linux-i686-glibc23	OS:	Linux (Fedora Core 5, Linux)
Assigned to:	Magnus Blåudd	CPU Architecture:	Any
Tags:	assertion failed, crash, SSL, TaoCrypt::Base64Decoder::Decode

Description:
Issue:
------

Mysql 5.0.27 crashes on start up with a failed assertion error when I used an SSL certificate which has a trailing white space at the end of each line of a PEM encoded certificate This certificate is happily used by Apache, so its not particularly strange. While the assertion is 'technically' correct none of the other apps that I've come across (Apache, Openssl etc.) seem to be bothered by trailing white spaces.

Crash/Stack trace:
------------------

061219 16:00:15  mysqld started
mysqld: coding.cpp:243: void TaoCrypt::Base64Decoder::Decode(): Assertion `endLine == '\n'' failed.
mysqld got signal 6;

key_buffer_size=0
read_buffer_size=258048
max_used_connections=0
max_connections=100
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 76399 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xbff1d7e8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x817adf8
0x4b32d402
0x4b3744f1
0x4b36c859
0x83e1bd4
0x83d6793
0x83c0e8e
0x83c1bf1
0x838783c
0x8387759
0x8387567
0x817d1d6
0x817b24a
0x4b3604e4
0x8103981
New value of fp=(nil) failed sanity check, terminating stack trace!

pitr# nm -n bin/mysqld > bin/mysqld.sym
pitr# resolve_stack_dump -s bin/mysqld.sym -n mysqld.stack
Cannot determine thread, fp=0xbff1d7e8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x817adf8 handle_segfault + 356
0x4b32d402 _end + 1121342994
0x4b3744f1 _end + 1121634049
0x4b36c859 _end + 1121602153
0x83e1bd4 _ZN8TaoCrypt13Base64Decoder6DecodeEv + 628
0x83d6793 _ZN5yaSSL8PemToDerEP8_IO_FILENS_8CertTypeE + 611
0x83c0e8e _ZN5yaSSL9read_fileEPNS_7SSL_CTXEPKciNS_8CertTypeE + 126
0x83c1bf1 yaSSL_CTX_use_certificate_file + 33
0x838783c vio_set_cert_stuff + 32
0x8387759 new_VioSSLFd + 293
0x8387567 new_VioSSLAcceptorFd + 35
0x817d1d6 _Z8init_sslv + 90
0x817b24a main + 230
0x4b3604e4 _end + 1121552116
0x8103981 _start + 33
New value of fp=(nil) failed sanity check, terminating stack trace!

My SSL settings:

ssl-ca = /usr/local/apache/conf/ssl
ssl-capath = /usr/local/openssl/certs
ssl-cert = /usr/local/apache/conf/ssl/XX.crt
ssl-key = /usr/local/apache/conf/ssl/XX.key
ssl-cipher = ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM

How to repeat:

Place a white space at the end of the first b64 encoded line of the cert data, E.g.:

"-----BEGIN CERTIFICATE-----
 MIIFbDCCBFSgAwIBAgIRAJ4ELJDAe1M1hVUt5m22ghEwDQYJKoZIhvcNAQEFBQAw "

It causes the following crash/stack trace.

Suggested fix:
The trailing white space character(s) should be stripped before the assertion of ending in a \n sequence.

Thank you for the report.

Verified as described on Linux using last BK sources.

Patch available in upstream yaSSL CVS repo. Will be fixed by next import.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/18554

ChangeSet@1.2390, 2007-01-22 16:54:40+01:00, msvensson@neptunus.(none) +17 -0
  Import yaSSL version 1.5.8
   - Includes patch for Bug#25189

Available in 5.0.36 and 5.1.16-beta.

Noted in 5.0.36, 5.1.16 changelogs.

yaSSL was sensitive to the presence of whitespace at the ends of
lines in PEM-encoded certificates, causing a server crash.