Description:
An in-place upgrade to 8.4.11 or later (also 9.7.2+ and 26.7.0+) aborts right at the start of the upgrade checks with only "[ERROR] [MY-010020] [Server] Data Dictionary initialization failed." when mysql.tables holds a row whose schema_id points at a schema that no longer exists. Nothing in the error log names the row, even with --log-error-verbosity=3. The same data directory upgrades fine to 8.4.10 and earlier.
Such rows are easy to create. Bug #119661 (also #113597, #119974). A later DROP DATABASE removes the schema row and the .ibd files but leaves the leaked aux table rows in mysql.tables. The server keeps running normally on 8.0 and 8.4, the only sign is "[Warning] [MY-012351] [InnoDB] Tablespace N, name 'db/fts_..._index_N' ... is missing!" at every start. Every physical backup carries the rows along. Bothing on 8.0 exists that removes them (more FTS DDL, OPTIMIZE, table rebuilds, recreating the schema by name, the gdb workaround from #119974).
Why it now fails: the rewrite of the upgrade checks for Bug#38031020 (upgrade takes too long with many tables) replaced the per-schema table scans with one scan of the whole of mysql.tables:
8.4.10 and earlier only scanned the tables of each existing schema (create_key_by_schema_id(schema->id())), so they never read these rows.
Impact: the server can't be upgraded in place at all, and the log gives no hint why. 8.0 to 8.4.11+ upgrades and 8.4.10 to 8.4.11+ patch upgrades both fail, and the leak itself still happens on 8.4. There is no way to move forward with upgrade. We hit this on production 8.0 services whose rows came from a database dropped long ago.
How to repeat:
1. Start 8.4.10 service.
2. create orphan rows
CREATE DATABASE yus;
CREATE TABLE yus.t (id INT PRIMARY KEY, a TEXT, b TEXT, c TEXT,
FULLTEXT KEY fa (a), FULLTEXT KEY fb (b), FULLTEXT KEY fc (c)) ENGINE=InnoDB;
ALTER TABLE yus.t DROP INDEX fa, DROP INDEX fb;
DROP TABLE yus.t;
DROP DATABASE yus;
SELECT NAME, FILE_SIZE FROM information_schema.INNODB_TABLESPACES WHERE NAME LIKE '%fts%';
3. do inplace upgrade to any version
4. observe"[ERROR] [MY-010020] [Server] Data Dictionary initialization failed."
Suggested fix:
1. Fix the leak so no new rows of this kind appear.
2. Make the upgrade check tolerate rows whose schema is missing.
3. If MY-010020 shows reason why it fails it would save us many hours debugging in production
4. Tooling to delete orphaned rows to fix corrupted schema
