MySQL Bugs: #67694: Illegal SSL/TLS record from MySQL server

Bug #67694	Illegal SSL/TLS record from MySQL server
Submitted:	24 Nov 2012 17:11	Modified:	29 Jan 2013 12:59
Reporter:	Peter-Jan Roes	Email Updates:
Status:	Won't fix	Impact on me:	None
Category:	MySQL Server: Options	Severity:	S2 (Serious)
Version:	5.5.28, 5.0.97, 5.1.68, 5.5.30, 5.7.1	OS:	Any (MS Windows 8, Linux)
Assigned to:		CPU Architecture:	Any
Triage:	Needs Triage: D3 (Medium)

Description:
I am connecting directly to a MySQL server installed on Windows 8 with a custom developed connector using the client/server protocol. During the SSL/TLS handshake (after the SSL Request Packet) the MySQL server returns an invalid SSL record. The SSL/TLS implementation I use is able to connect to other SSL/TLS servers without any problems.

The illegal record (received in the record layer) starts with the following bytes:

16 00 00 02 FF ...

The first byte (0x16) indicates that the record content type is Handshake, which seems correct. However, the next two bytes should indicate a protocol version ("03 00" for SSL 3.0, "03 01" for TLS 1.0, for instance). The "00 00" is clearly wrong (searching the bug database also shows a number of "Unknown protocol version 0.0 errors"). Finally the bytes "02 FF" indicate a record payload length of 767 bytes which are not present.

The connector I use might be doing something wrong during the SSL/TLS handshake, however, this should not result in MySQL returning illegal records. Also, the problem occurs when connecting using SSL 3.0 right after the ClientHello message. Using TLS 1.0 the handshake can be completed correctly, provided that no Certificate message is returned (the standard indicates that one should return an empty Certificate message instead). Finally, using TLS 1.1 the same erroneous record is returned at an other point during the handshake.

Reading through some other bug reports I started to think that the problem is caused by yaSSL as this is the SSL/TLS server used in the MySQL server for Windows. Also, similar problems have been reported concerning OpenSSL and yaSSL interoperability.

How to repeat:
It is hard to describe how exactly this problem can be repeated. It comes down to sending a ClientHello message using the SSL 3.0 protocol after the SSL Request Packet and reading the first bytes returned by the MySQL server. The ClientHello message contains the SSL_RSA_WITH_RC4_128_SHA and SSL_RSA_WITH_RC4_128_MD5 cipher suites. Please let me know if more details are required.

Thank you for the report.

Could you please create minimal version of your custom connector, demonstrating this issue?

A Python script performing a very basic SSL 3.0 handshake with MySQL/yaSLL

Attachment: ProtocolError.py (text/plain), 2.31 KiB.

After executing the Python script one can see the illegal SSL record from MySQL/yaSSL under "Receiving ServerHello: ". After changing the protocol version to 3.1 (which is TLS 1.0) in the script, the returned SSL record is correct. Please note that, contrary to what I mentioned in the original description, TLS 1.1 does work correctly.

Thank you for the script.

Verified as described in versions 5.0 and 5.1:

[sveta@delly bug67694]$ python ProtocolError.py 
Receiving Initial handshake packet:
3e 00 00 00 0a 35 2e 30 2e 39 37 2d 64 65 62 75 67 2d 6c 6f 67 00 01 00 00 00 2c 7e 58 7b 25 43 60 3a 00 2c aa 08 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3f 6c 24 69 69 51 3c 76 60 5d 34 29 00

Sending SSL Request packet:
20 00 00 01 08 8a 08 00 ff ff ff 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Sending ClientHello:
16 03 00 00 2f 01 00 00 2b 03 00 50 b5 e1 d5 85 a5 56 20 25 42 84 ae 42 21 4f b1 b9 d9 17 ef 3f 70 88 46 8b 0f 29 fb 41 f6 9d 0c 00 00 04 00 05 00 04 01 00

Receiving ServerHello:
16 00 00 02 ff 13 04 23 30 38 53 30 31 42 61 64 20 68 61 6e 64 73 68 61 6b 65
[sveta@delly bug67694]$ vim ProtocolError.py <change to 3.1>
[sveta@delly bug67694]$ python ProtocolError.py 
Receiving Initial handshake packet:
3e 00 00 00 0a 35 2e 30 2e 39 37 2d 64 65 62 75 67 2d 6c 6f 67 00 02 00 00 00 77 5c 44 23 7c 6a 7d 56 00 2c aa 08 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 76 70 4d 70 6a 47 61 33 76 7b 2a 59 00

Sending SSL Request packet:
20 00 00 01 08 8a 08 00 ff ff ff 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Sending ClientHello:
16 03 01 00 2f 01 00 00 2b 03 01 50 b5 e1 d5 85 a5 56 20 25 42 84 ae 42 21 4f b1 b9 d9 17 ef 3f 70 88 46 8b 0f 29 fb 41 f6 9d 0c 00 00 04 00 05 00 04 01 00

Receiving ServerHello:
16 03 01 00 4a 02 00 00 46 03 01 c1 ff 25 c1 ae 16 ac ef 49 6a ab 92 af ee 53 0e bf 87 92 e0 ee 1a 8b 98 68 c9 c3 6e 66 d4 c0 ae 20 89 a1 e9 f7 1f e6 29 47 9a 2b 6a 45 b6 31 11 11 9d 25 8b 7c 58 25 18 52 6b ed 0d fa d4 f6 7f 78 00 05 00 16 03 01 03 0b 0b 00 03 07 00 03 04 00 03 01 30 82 02 fd 30 82 02 66 a0 03 02 01 02 02 01 01 30 0d 06 09 2a 86 48 86 f7 0d 01 01 05 05 00 30 81 85 31 0b 30 09 06 03 55 04 06 13 02 55 53 31 0b 30 09 06 03 55 04 08 0c 02 43 41 31 14 30 12 06 03 55 04 07 0c 0b 53 61 6e 74 61 20 43 6c 61 72 61 31 0f 30 0d 06 03 55 04 0a 0c 06 4f 72 61 63 6c 65 31 0e 30 0c 06 03 55 04 0b 0c 05 4d 79 53 51 4c 31 0d 30 0b 06 03 55 04 03 0c 04 62 75 67 73 31 23 30 21 06 09 2a 86 48 86 f7 0d 01 09 01 16 14 73 76 65 74 61 40 62 75 67 73 2e 6d 79 73 71 6c 2e 63 6f 6d 30 1e 17 0d 31 32 31 32 32 35 31 36 30 34 33 34 5a 17 0d 31 33 31 32 32 35 31 36 30 34 33 34 5a 30 81 85 31 0b 30 09 06 03 55 04 06 13 02 55 53 31 0b 30 09 06 03 55 04 08 0c 02 43 41 31 14 30 12 06 03 55 04 07 0c 0b 53 61 6e 74 61 20 43 6c 61 72 61 31 0f 30 0d 06 03 55 04 0a 0c 06 4f 72 61 63 6c 65 31 0e 30 0c 06 03 55 04 0b 0c 05 4d 79 53 51 4c 31 0d 30 0b 06 03 55 04 03 0c 04 62 75 67 73 31 23 30 21 06 09 2a 86 48 86 f7 0d 01 09 01 16 14 73 76 65 74 61 40 62 75 67 73 2e 6d 79 73 71 6c 2e 63 6f 6d 30 81 9f 30 0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 81 8d 00 30 81 89 02 81 81 00 c3 ff f3 d5 42 15 0b 87 21 49 b3 82 d6 f2 b7 dd 26 f3 76 28 f3 03 92 54 31 31 68 21 67 25 a2 5f 4f fd bb 35 28 bd 02 76 2f 81 45 47 0d 4f 27 0c 2d 3a ec d1 db fe bb 80 e8 45 d6 08 7f 0b df 49 9e d1 cf 94 9f 89 ca 3d 65 25 57 00 ca 9b ac 7d c9 06 45 64 5f 8f 01 42 fd e2 8f b7 4d 62 ac a2 31 8c 10 32 aa 70 fd d8 64 9c 0e 6f 4e 15 bc ba ef 27 ae 6e a7 15 d5 bd f3 e5 4d ab 76 c9 84 cd 02 03 01 00 01 a3 7b 30 79 30 09 06 03 55 1d 13 04 02 30 00 30 2c 06 09 60 86 48 01 86 f8 42 01 0d 04 1f 16 1d 4f 70 65 6e 53 53 4c 20 47 65 6e 65 72 61 74 65 64 20 43 65 72 74 69 66 69 63 61 74 65 30 1d 06 03 55 1d 0e 04 16 04 14 34 1a ea 1e 0b 05 26 81 c4 61 fa 40 57 34 8b e6 67 d1 0a 86 30 1f 06 03 55 1d 23 04 18 30 16 80 14 eb 95 77 6c 2f bd fe d8 bd 6a 30 3c a4 ea db 7d e8 0f 7b 2e 30 0d 06 09 2a 86 48 86 f7 0d 01 01 05 05 00 03 81 81 00 88 92 d3 42 ad 50 78 4b 34 85 93 d6 da cb 26 4e ed af 31 ee 2b 0e f7 b9 0c f4 95 9f ec ea 77 4a

With newer versions I get strange answer:

python ProtocolError.py 
Receiving Initial handshake packet:
57 00 00 00 0a 35 2e 37 2e 31 2d 6d 31 31 2d 64 65 62 75 67 2d 6c 6f 67 00 01 00 00 00 42 7e 74 4b 5a 62 5c 2b 00 ff ff 08 02 00 7f c0 15 00 00 00 00 00 00 00 00 00 00 5b 69 23 6a 6f 6f 60 71 38 61 62 49 00 6d 79 73 71 6c 5f 6e 61 74

Sending SSL Request packet:
20 00 00 01 08 8a 08 00 ff ff ff 00 21 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Sending ClientHello:
16 03 01 00 2f 01 00 00 2b 03 01 50 b5 e1 d5 85 a5 56 20 25 42 84 ae 42 21 4f b1 b9 d9 17 ef 3f 70 88 46 8b 0f 29 fb 41 f6 9d 0c 00 00 04 00 05 00 04 01 00

Receiving ServerHello:
69 76 65 5f 70 61 73 73 77 6f 72 64 00

Thanks for the detailed report. Unfortunately we don't officially support client libraries that we don't distribute. So please look into using supported client libraries or making your library behave as closely to the official ones as possible.