Description:
An online InnoDB table rebuild can write an invalid fixed-length `CHAR` field
when all of the following apply:
* The source table uses `ROW_FORMAT=DYNAMIC` and `utf8mb3`.
* The target table uses `ROW_FORMAT=REDUNDANT` and `utf8mb4`.
* A concurrent insert is recorded in the online DDL row log.
I reproduced the case with a Debug build on Linux x86_64. The affected source
file, `storage/innobase/row/row0log.cc`, is unchanged from the current
`upstream/9.7` ref `008e09c2834b98143a8c067d4d225c90953050cf` (9.7.2
development source). The build used `CMAKE_BUILD_TYPE=Debug` and
`WITH_DEBUG=1`.
On an unmodified Debug build, the row-log apply first aborts at:
```
Assertion failure: row0log.cc:1465:
!((new_col->prtype ^ col->prtype) & ~DATA_NOT_NULL)
```
That assertion prevents the subsequent record conversion from being observed.
For diagnosis only, I temporarily removed the two `ut_ad()` checks immediately
after that location. This does not change the row-log conversion logic. With
the same test case, the server reports:
```
Field 3 len is 96, should be 128
```
and `CHECK TABLE` reports that the `PRIMARY` B-tree is corrupted. Thus this is
not only a Debug assertion problem: the row-log conversion constructs a record
that violates the target table's fixed-field length invariant.
Removing the assertions is diagnostic-only and is not a proposed production
fix. It merely lets the Debug build reach the same conversion code whose output
is invalid.
The explicit `ROW_FORMAT=REDUNDANT` change requires a rebuild, so a separate
optimization that makes a charset-only conversion metadata-only does not avoid
this case.
How to repeat:
Build a Debug server. The following is an MTR test because `DEBUG_SYNC` makes
the concurrent DML deterministic:
```text
--source include/have_debug.inc
--source include/have_debug_sync.inc
CREATE TABLE t1 (
id INT NOT NULL PRIMARY KEY,
c CHAR(32) NOT NULL DEFAULT ''
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC;
INSERT INTO t1 VALUES (1, 'before alter');
connect (con1,localhost,root,,);
connection con1;
SET DEBUG_SYNC =
'alter_table_inplace_after_lock_downgrade SIGNAL alter_ready WAIT_FOR dml_done';
--send ALTER TABLE t1 CONVERT TO CHARACTER SET utf8mb4, ROW_FORMAT=REDUNDANT, ALGORITHM=INPLACE, LOCK=NONE
connection default;
SET DEBUG_SYNC='now WAIT_FOR alter_ready';
INSERT INTO t1 VALUES (2, 'during alter');
SET DEBUG_SYNC='now SIGNAL dml_done';
connection con1;
--reap
connection default;
SET DEBUG_SYNC='RESET';
CHECK TABLE t1;
DROP TABLE t1;
```
With an unmodified Debug build, the test deterministically aborts in
`row_log_table_apply_convert_mrec()` at the assertion shown above.
To expose the subsequent behavior in a Debug build, temporarily comment out
only these two Debug assertions in `row0log.cc` after `new_col` is obtained.
This diagnostic change is intentionally separate from the fix direction
suggested below:
```c++
ut_ad(!((new_col->prtype ^ col->prtype) & ~DATA_NOT_NULL));
ut_ad(!((new_col->prtype ^ dfield_get_type(dfield)->prtype) &
~DATA_NOT_NULL));
```
Rebuild and run the same test. The actual result is:
```text
[ERROR] [InnoDB] Field 3 len is 96, should be 128; RECORD(...)
test.t1 check Warning InnoDB: The B-tree of index PRIMARY is corrupted.
test.t1 check error Corrupt
```
Expected result: the online ALTER completes without a Debug assertion, without
an invalid record, and `CHECK TABLE t1` returns `OK`.
Description: An online InnoDB table rebuild can write an invalid fixed-length `CHAR` field when all of the following apply: * The source table uses `ROW_FORMAT=DYNAMIC` and `utf8mb3`. * The target table uses `ROW_FORMAT=REDUNDANT` and `utf8mb4`. * A concurrent insert is recorded in the online DDL row log. I reproduced the case with a Debug build on Linux x86_64. The affected source file, `storage/innobase/row/row0log.cc`, is unchanged from the current `upstream/9.7` ref `008e09c2834b98143a8c067d4d225c90953050cf` (9.7.2 development source). The build used `CMAKE_BUILD_TYPE=Debug` and `WITH_DEBUG=1`. On an unmodified Debug build, the row-log apply first aborts at: ``` Assertion failure: row0log.cc:1465: !((new_col->prtype ^ col->prtype) & ~DATA_NOT_NULL) ``` That assertion prevents the subsequent record conversion from being observed. For diagnosis only, I temporarily removed the two `ut_ad()` checks immediately after that location. This does not change the row-log conversion logic. With the same test case, the server reports: ``` Field 3 len is 96, should be 128 ``` and `CHECK TABLE` reports that the `PRIMARY` B-tree is corrupted. Thus this is not only a Debug assertion problem: the row-log conversion constructs a record that violates the target table's fixed-field length invariant. Removing the assertions is diagnostic-only and is not a proposed production fix. It merely lets the Debug build reach the same conversion code whose output is invalid. The explicit `ROW_FORMAT=REDUNDANT` change requires a rebuild, so a separate optimization that makes a charset-only conversion metadata-only does not avoid this case. How to repeat: Build a Debug server. The following is an MTR test because `DEBUG_SYNC` makes the concurrent DML deterministic: ```text --source include/have_debug.inc --source include/have_debug_sync.inc CREATE TABLE t1 ( id INT NOT NULL PRIMARY KEY, c CHAR(32) NOT NULL DEFAULT '' ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC; INSERT INTO t1 VALUES (1, 'before alter'); connect (con1,localhost,root,,); connection con1; SET DEBUG_SYNC = 'alter_table_inplace_after_lock_downgrade SIGNAL alter_ready WAIT_FOR dml_done'; --send ALTER TABLE t1 CONVERT TO CHARACTER SET utf8mb4, ROW_FORMAT=REDUNDANT, ALGORITHM=INPLACE, LOCK=NONE connection default; SET DEBUG_SYNC='now WAIT_FOR alter_ready'; INSERT INTO t1 VALUES (2, 'during alter'); SET DEBUG_SYNC='now SIGNAL dml_done'; connection con1; --reap connection default; SET DEBUG_SYNC='RESET'; CHECK TABLE t1; DROP TABLE t1; ``` With an unmodified Debug build, the test deterministically aborts in `row_log_table_apply_convert_mrec()` at the assertion shown above. To expose the subsequent behavior in a Debug build, temporarily comment out only these two Debug assertions in `row0log.cc` after `new_col` is obtained. This diagnostic change is intentionally separate from the fix direction suggested below: ```c++ ut_ad(!((new_col->prtype ^ col->prtype) & ~DATA_NOT_NULL)); ut_ad(!((new_col->prtype ^ dfield_get_type(dfield)->prtype) & ~DATA_NOT_NULL)); ``` Rebuild and run the same test. The actual result is: ```text [ERROR] [InnoDB] Field 3 len is 96, should be 128; RECORD(...) test.t1 check Warning InnoDB: The B-tree of index PRIMARY is corrupted. test.t1 check error Corrupt ``` Expected result: the online ALTER completes without a Debug assertion, without an invalid record, and `CHECK TABLE t1` returns `OK`.