MySQL Bugs: #75968: Mitigate the strictness of strict_* modes of innodb_checksum

Bug #75968	Mitigate the strictness of strict_* modes of innodb_checksum_algorithm
Submitted:	19 Feb 2015 12:51	Modified:	9 Apr 2015 17:55
Reporter:	Vasil Dimov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S3 (Non-critical)
Version:	5.6	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
Normally, with innodb_checksum_algorithm="innodb", "crc32" or "none" InnoDB will write the requested checksum on disk when writing pages, but will allow any of the 3 checksums when reading pages from disk and verifying their checksum. For a tablespace that has a mixture if checksums this could lead to some performance degradation since more than one checksum of the page is calculated in order to compare it with the checksum that is stored in the page. This was the reason why strict_ modes were introduced - "strict_innodb", "strict_crc32" and "strict_none". They behave like "innodb", "crc32" and "none" respectively, but only allow/recognize the corresponding checksum when reading a page from disk and verifying its checksum. This way the DBA knows, if he has set e.g. "strict_crc32", that InnoDB will never calculate checksums other than "crc32" when verifying pages on read.

Of course, if e.g. "strict_crc32" is used and a page contains a checksum that is other than "crc32" then InnoDB will act like what it does on unknown checksum - report it to the error log and call abort().

This is very well documented in http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_checksum_algor....

But apparently this action is too harsh and DBAs shoot themselves in the foot by changing innodb_checksum_algorithm to some strict_X mode while their files contain pages with (valid) checksums that are not X and then they think that something got corrupted or that there is a bug, or even a security bug in InnoDB. Here are some such reports:

Bug#13600064 INNODB_CHECKSUM_ALGORITHM IS A DYNAMIC PARAMETER, CAN LEAD TO DB CORRUPTION
Bug#13630849 CHANGING INNODB_CHECKSUM_ALGORITHM LEADS TO CORRUPTION/CRASHING
Bug#20031570 INNODB_CHECKSUM_ALGORITHM VARIABLE LEADS TO CRASHING

Those bug reports are like claiming that there is a (security) bug in InnoDB because it calls abort() if it cannot allocate memory or that it will crash/abort() if the DBA edits its data files with a hex editor changing the checksum values to some unknown ones.

How to repeat:
Set innodb_checksum_algorithm to X, write some pages, then change it to strict_Y and try to read those pages.

Suggested fix:
Change the behavior of strict_X when the page contains a checksum that is not X, but is a valid Y (where X != Y). For example if "strict_crc32" is specified but a page contains a "innodb" checksum. In such cases, instead of reporting a page corruption and calling abort() - print a message to the error log and accept the page. With this change the behavior of "strict_X" and "X" will differ only in that the "strict_X" mode will print a message if a valid, but !=X checksum is found on a page.

Posted by developer:
 
Fixed as of the upcoming 5.6.25, 5.7.8, 5.8.0 releases, and here's the changelog entry:

The "strict_*" forms of "innodb_checksum_algorithm" settings
("strict_none", "strict_innodb", and "strict_crc32") caused the server to
halt when a non-matching checksum was encountered, even if a non-matching
checksum was valid. For example, with
"innodb_checksum_algorithm=strict_crc32", encountering a valid "innodb"
checksum would halt the server. Instead of halting the server, a message
is now printed to the error log and the page is accepted as valid if it
matches an "innodb", "crc32" or "none" checksum. 

The 5.6 and 5.7 innodb_checksum_algorithm description was also updated.

Posted by developer:
 
Post-push fix. No change to existing changelog entry.

Posted by developer:
 
The changelog entry was revised for clarity: 

The innodb_checksum_algorithm strict_* settings (strict_none,
strict_innodb, and strict_crc32) caused the server to halt when InnoDB
encountered a valid but non-matching checksum. For example, with
innodb_checksum_algorithm=strict_crc32, a valid innodb checksum would
cause the server to halt. Now, instead of halting the server, InnoDB only
prints an error message.