Bug #107037 mysql.connector changes python ssl CA store paths, breaking ssl in other libs
Submitted: 15 Apr 2022 13:06 Modified: 8 Aug 2022 16:29
Reporter: Vincent Huisman Email Updates:
Status: Verified Impact on me:
Category:Connector / Python Severity:S2 (Serious)
Version:8.0.28 OS:Ubuntu
Assigned to: CPU Architecture:Any

[15 Apr 2022 13:06] Vincent Huisman
Importing mysql.connector anywhere will change some defaults in the ssl module, making it unusable for most other uses. Most notably the CA search paths are changed to a non-existent path on my system. The problem seems to originate in the C extension.

Kubuntu 21.04 amd64
Python 3.9.5
mysql-connector-python 8.0.28
(urllib3 1.26.9 for testing but other libraries using the ssl module are similarly affected)

This was a particularly annoying one to trace, using an SDK to connect to an unrelated third-party service I was presented with SSL certificate verification errors that should otherwise not be there. A stand-alone test case revealed no problem. After a lot of tracing and commenting-out various things, commenting out the mysql.connector import resolved the problem. That allowed me to write a minimal test case.

It seems that merely the act of importing the ssl module before importing mysql.connector is a workaround that solves the problem. Printing the default CA verification paths in both cases reveals that the mysql.connector module changes those paths (or at least sets its own defaults). These other paths (/usr/local/mysql/*) do not exist on my machine, so I can only assume the ssl module is going to continue working without a certificate store and that this affects mysql.connector similarly. I cannot even imagine that this could lead to successful mysql connections using SSL, so it seems like a bad default.

How to repeat:
I've created two test files, differing only in the order of importing mysql.connector and ssl.

import mysql.connector
import ssl
import urllib3
	urllib3.PoolManager().request("GET", "https://mysql.com")
except urllib3.exceptions.MaxRetryError as e:
	print(f"fail {e}")

import ssl
import mysql.connector
import urllib3
	urllib3.PoolManager().request("GET", "https://mysql.com")
except urllib3.exceptions.MaxRetryError as e:
	print(f"fail {e}")

Then, running the following for a clean environment:
$ python3 -m venv mysqlvenv
$ mysqlvenv/bin/pip3 install mysql-connector-python urllib3

Actual result:
$ mysqlvenv/bin/python3 test1.py
DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/local/mysql/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/local/mysql/ssl/certs')
fail HTTPSConnectionPool(host='mysql.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))

Expected result:
$ mysqlvenv/bin/python3 test2.py
DefaultVerifyPaths(cafile=None, capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')

Suggested fix:
A workaround is importing ssl before importing mysql.connector. However, that is not always possible or obvious, especially when using third-party libraries. A coding standard dictating alphabetical order of imports is also not helpful in this case.

I have done some experiments in the mysql-connector-python package. Placing "import ssl" at the top of lib/mysql/connector/__init__.py (https://github.com/mysql/mysql-connector-python/blob/90eaeca65a6bbfc1fd9218aad530395779821...) resolves the problem for me. Going a bit further, it seems that the error is solely in the C extension. Placing the "import ssl" anywhere below line 34 makes the problem reapppear, so it must be caused by the import on line 34. Intentionally breaking the import by importing a non-existent module between lines 33 and 34 also fixes the problem by not using the C extension.

Since I do not know what code in the C extension causes this, my suggested fix is to just ensure the ssl module is imported in the above-mentioned __init__.py, before importing the C module. This should load the correct defaults being used throughout each project this connector is used in.
[20 Apr 2022 14:51] MySQL Verification Team
Hello Vincent Huisman,

Thank you for the report and test case.
Verified as described.

[27 May 2022 16:02] Nuno Mariz
Posted by developer:
Please read the explanation from Johannes in https://bugs.mysql.com/bug.php?id=97220 about this issue.
We still don't have a clear solution for it.
[8 Aug 2022 16:29] Vincent Huisman
Well, that other ticket does make it look problematic to implement a portable blanket fix but I think there might be two separate issues going on in this report. On the one hand is the "conflicting" in loading shared libraries, on the other hand is the ssl library that was shipped with Connector looks broken in that it always points to a non-existent cert store and does not ship with one either, so even without using ssl elsewhere in the project this one seems unable to establish any trusted secure connection, unless I am extremely mistaken. Granted, the import problem may be hard to fix but I think the missing cert store is the root cause of this problem. And I think fixing that also fixes e.g. urllib if it's imported after Connector.