MySQL Bugs: #95183: redo logs may silently be ignored when encrypted table cannot be decrypted.

Bug #95183	redo logs may silently be ignored when encrypted table cannot be decrypted.
Submitted:	29 Apr 2019 11:02	Modified:	22 Jul 2019 18:30
Reporter:	Robert Golebiowski	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Security: Encryption	Severity:	S3 (Non-critical)
Version:	8.0.15	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
There is a difference in behavior between debug and release server when redo logs are to be applied to a table that cannot be decrypted. Debug server will abort the start and release will generate errors but will start up. This is because in log0recv.cc there is:

    if (space.first != TRX_SYS_SPACE &&
        !fil_tablespace_open_for_recovery(space.first)) {
      /* Tablespace was dropped. It should not have been scanned unless it
      is an undo space that was under construction. */
      ut_ad(!fil_tablespace_lookup_for_recovery(space.first) ||
            fsp_is_undo_tablespace(space.first));

      dropped = true;

the call to fil_tablespace_lookup_for_recover will add space id to missing_ids which in turn will trigger this from srv0start.cc:

    if (srv_force_recovery == 0 && fil_check_missing_tablespaces()) {
      ib::error(ER_IB_MSG_1139);

      /* Set the abort flag to true. */
      auto p = recv_recovery_from_checkpoint_finish(*log_sys, true);

      ut_a(p == nullptr);

      return (srv_init_abort(DB_ERROR));
    }
fil_check_missing_tablespaces will return true.

For release the fil_tablespace_lookup_for_recover will not be called as it is inside a debug assert. The release version should be fixed and abort to start.

How to repeat:
On release server:

1) create a couple of encrypted tables
2) restart the server so we would start with clean redo
3) do a bunch of inserts to the encrypted tables so the redo would have them
4) kill the server while those are not yet committed to disk
5) restart the server - it will start successfully, although redo will generate missing tables errors

Suggested fix:
abort the server the same way it is aborted for debug version.

Hello Robert,

Thank you for your bug report.

This bug report makes lots of sense to me. However, I need a couple of additional info in order to be able to repeat it and understand it.

First of all, can you provide just CREATE TABLE commands for those encrypted tables.

Second, I see that release server will start successfully. What happens with a debug server, exactly ???

Third, I see two snippets of code.  How do you think that the code in release server should be changed, or whether do you think that the particular ut_ad() should be converted to ut_a() ???

Fourth, are there any consequences with those tables or redo log if the restart is successful as in the release server ???

Simply, I would like to hear the reasoning behind your thinking ......

Thanks in advance.

Hi Mr. Golebiowski,

We are still waiting on your feedback. Can you, please, provide it, together with a full test case on how to repeat the behaviour ???

We are asking you this, because we are very much interested in fixing this behaviour !!!!!!

Thanks in advance !!!!!

Hi Sinisa,

Why so serious ? ;) Sorry, I have been busy lately. I have prepared the testcase. Please find it attached. Note, that for the debug build this test will pass and for the release it will hung on:
--error 1
--exec $MYSQLD_CMD --log-error=$error_log
This is because the test will not abort with error as it should have, but initialize successfully.

the testcase

Attachment: bug95183.test (application/octet-stream, text), 2.08 KiB.

master file for the testcase

Attachment: bug95183-master.opt (application/octet-stream, text), 134 bytes.

To answer your question - the redo log for encrypted tables will silently be lost for release server. This breaks ACID.

Hi, 

Thank you for your test case.

We shall run it and will let you know of the results.

Hi,

You have specified in your .opt file the following option pertaining  the key ring file data:

$MYSQLTEST_VARDIR/std_data/keys2.txt

I have on my machine several text files with encryption keys, but I do not know if those will work with your test case.

Also, if any of the key files in current MTR datadir can be used instead, just let us know.

Otherwise, please, upload your keys2.txt, just to be sure. You can use "Files" tab and upload it as a hidden file.

Thanks in advance.

Hi Sinisa,

Actually you can just replace it with $MYSQLTEST_VARDIR/tmp/keys.txt. It will create keyring with encryption keys needed. it should also work as it is. If the file does not exist it will create it and put the encryption keys in it.

Hopes it helps
Robert

HI,

I have managed to repeat a test case from your bug report.

Verified as reported.

Thanks a lot.

Posted by developer:
 
Fixed as of the upcoming 8.0.18 release, and here's the changelog entry:

During recovery, missing table errors that occur when applying redo logs
to encrypted tables could be ignored, permitting startup to proceed.
Startup should be halted in this case.

Thank you, Daniel !