MySQL Bugs: #113427: MySQL APT for Debian is signed with an expired key

Bug #113427	MySQL APT for Debian is signed with an expired key
Submitted:	14 Dec 2023 19:43	Modified:	15 Dec 2023 15:00
Reporter:	Sebastian Orellana	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Package Repos	Severity:	S2 (Serious)
Version:	8.0	OS:	Debian
Assigned to:	Sreedhar Sreedhargadda	CPU Architecture:	Any

Description:
MySQL APT for Debian is currently signed using an expired key (3A79BD29) and is not available for the new key (A8D3785C).

The key 3A79BD29 expires today: https://dev.mysql.com/doc/refman/8.0/en/checking-gpg-signature.html

The APT for Debian is signed with the expired key: https://repo.mysql.com/apt/debian/conf/distributions

How to repeat:
With a fresh server with Debian Buster:
1. Install mysql-apt-config_0.8.20-1_all.deb
2. Add MySQL GPG Public Key (apt-key adv --keyserver pgp.mit.edu --recv-keys A8D3785C)
3. Try to update APT (apt-get update -qq)

Expected result:

W: GPG error: http://repo.mysql.com/apt/debian buster InRelease: The following signatures were invalid: EXPKEYSIG 467B942D3A79BD29 MySQL Release Engineering <mysql-build@oss.oracle.com>
E: The repository 'http://repo.mysql.com/apt/debian buster InRelease' is not signed.

This is also the case for Ubuntu, potentially all releases.

I just wanted to post that this is affecting me on Ubuntu 22.04 as I have MySQL version 8.0.35 currently (8.0.35-1ubuntu22.04), so I was checking for system updates today via APT ($ sudo apt update), but I get the following errors:

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://repo.mysql.com/apt/ubuntu jammy InRelease: The following signatures were invalid: EXPKEYSIG 467B942D3A79BD29 MySQL Release Engineering <mysql-build@oss.oracle.com>
W: Failed to fetch http://repo.mysql.com/apt/ubuntu/dists/jammy/InRelease  The following signatures were invalid: EXPKEYSIG 467B942D3A79BD29 MySQL Release Engineering <mysql-build@oss.oracle.com>
W: Some index files failed to download. They have been ignored, or old ones used instead.

My attempts to resolve it on my own were the following:

I was able to download and install the new MySQL APT repo config (mysql-apt-config_0.8.28-1_all.deb) which went fine. For what it's worth, I was also able to verify mysql-apt-config_0.8.28-1_all.deb with the new GnuPG signing key (A8D3 785C) which I found via: http://pgp.mit.edu/pks/lookup?op=get&search=0xB7B3B788A8D3785C. Though, I kept having the same APT errors (listed above), so I also tried to remove the old APT signing key (3A79 BD29), then added the new key (A8D3 785C)... but this didn't help. I then re-added the old key back to APT, so now I have it so that both the old and new keys are available for APT:

$ sudo apt-key list
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
/etc/apt/trusted.gpg
--------------------
pub   rsa4096 2023-10-23 [SC] [expires: 2025-10-22]
      BCA4 3417 C3B4 85DD 128E  C6D4 B7B3 B788 A8D3 785C
uid           [ unknown] MySQL Release Engineering <mysql-build@oss.oracle.com>
sub   rsa4096 2023-10-23 [E] [expires: 2025-10-22]

pub   rsa4096 2021-12-14 [SC] [expired: 2023-12-14]
      859B E8D7 C586 F538 430B  19C2 467B 942D 3A79 BD29
uid           [ expired] MySQL Release Engineering <mysql-build@oss.oracle.com>

Yet, I still get those APT errors about the EXPKEYSIG, which led me to this bug report. So, I am hoping a fix or workaround is possible to help me out. Thanks!

This will be fixed when https://repo.mysql.com/apt/ubuntu/conf/distributions is updated with the new key.

Hello Sebastian Orellana,

Thank you for the report and feedback.

regards,
Umesh

It also affects all Airflow user. This is a serious problem for anyone who installs MySQL repo because the old key that the repo is signed with is expired.

We are seriously considering switching to MariaDB and suggest it to our users.

https://github.com/apache/airflow/issues/36231

New mysql-apt-config.deb is being uploaded with new gpg key.

Will you let us know when the process of resigning is complete? I guess it will take some time to resign and publish the files with the new key?

It is already working, with the latest apt-config, thanks!

Update: 

I can confirm that the latest `mysql-apt-config_0.8.29-1_all.deb` fixed the APT errors (EXPKEYSIG) for me on Ubuntu 22.04. So, I can now run `sudo apt update` without problems. Thank you!

Thanks for the bug report. We have rebuilt mysql-apt-config (mysql-apt-config_0.8.29-1_all.deb) with the latest GPG Key, refreshed the repo metadata and published them.

Thanks for the fix. I think however the policy of Oracle/MySQL to have expiry date for your software is deeply flawed.

We had to manually fix all ~50 images we released in the past of our for Apache Airflow because of the expiry date.

Nobody else does it. Postgres, MariaDB, even MsSQL put no expiry date on the keys that are used to sign repos.

By putting expiry key on your apt repository you basically put an expiry date on your software and this expiry date gets shorter and shorter.

A good example of that are your own images that are affected. We had a user asking us for help in Airflow repo https://github.com/apache/airflow/issues/36231#issuecomment-1858419966
`
to help to fix the same issue with `mysql:8.0.35-debian` image of yours and we sent them to your support (as well, you should deal with your own problems).

This image has been released just 25 days ago. And due to the flawed policy of having an expiry date on your key, effectively lifetime of this image was 24 days. Not much. And likely you have a number of those images (similarly as what we had 50 of ours). Now I guess you need to retroactively rebuild/patch your images - which is something the flawed policy of yours made us to get 36 hours of scrambling and and answering support issues of our users (which we did despite our team is made of volunteers, not paid staff as is in the case of MySQL/Oracle).

We kinda lost faith in Oracle being a good steward of MySQL apt repos and we decided in Apache Airflow in accelerated discussion and (currently running) lazy consensus, to switch to MariaDB clients for all our future releases (including the 2.8.0 release that was actually delayed by at least 2 days because of this bug).

Lazy consensus thread here: https://lists.apache.org/list.html?dev@airflow.apache.org

I hope - for the sake of your users loosing days due to such issues, you will reconsider your policies around signing your APT repos.