Bug #75809 Error propagation does fully work in XA
Submitted: 6 Feb 2015 15:52 Modified: 7 Oct 2015 17:16
Reporter: Andrei Elkin Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Storage Engine API Severity:S3 (Non-critical)
Version:5.7.6 OS:Any
Assigned to: CPU Architecture:Any

[6 Feb 2015 15:52] Andrei Elkin
Description:
In rare (extremely) circumstances of an error in preparing an XA transaction, say by binlog hton, e.g due to some resources
limitation the error causes an assert.
One part of the reason must be in that the following function does some resetting
for XA transaction which could be done only to successfully committed one.

Consider 

Sql_cmd_xa_commit::trans_xa_commit(
{
  ...
  if (xid_state->has_state(XID_STATE::XA_PREPARED) 
  {
    ...
    if (tc_log)
         res= MY_TEST(tc_log->commit(thd, /* all */ true));

even though the error is prepared

    if (res)
        my_error(ER_XAER_RMERR, MYF(0));

there's early return unlike in other failing branches in this function.
Execution proceeds to

  thd->variables.option_bits&= ~OPTION_BEGIN;
  thd->get_transaction()->reset_unsafe_rollback_flags(
    Transaction_ctx::SESSION);
  thd->server_status&=
    ~(SERVER_STATUS_IN_TRANS | SERVER_STATUS_IN_TRANS_READONLY);

etc, only hit as assert through this stack:

#4  0xb7c8e777 in __GI___assert_fail (assertion=0x916bcf8 "thd->in_active_multi_stmt_transaction() || thd->m_transaction_psi == __null", 
    file=0x916b9c0 "/home/andrei/MySQL/GIT/mysql/sql/transaction.cc", line=443, 
    function=0x916bf60 <trans_rollback_stmt(THD*)::__PRETTY_FUNCTION__> "bool trans_rollback_stmt(THD*)") at assert.c:101
#5  0x0897748f in trans_rollback_stmt (thd=0xac4008a0) at /home/andrei/MySQL/GIT/mysql/sql/transaction.cc:442
#6  0x0889a626 in mysql_execute_command (thd=0xac4008a0) at /home/andrei/MySQL/GIT/mysql/sql/sql_parse.cc:4832

Straightforward implementing the early return seem to have made the current (simulated as) failing XA COMMIT to succeed,
but a similar assert was hit in execution of a following query.

How to repeat:
Check sources.
Simulate a failure in tc_log->commit() at execution on XA COMMIT,
to get the description assert.

Suggested fix:
State of the failed at COMMIT,ROLLBACK or PREPARE transactions should be
made sane.
[13 Feb 2015 18:51] Andrei Elkin
Consider this patch to simulate the described scenario:

--- a/sql/transaction.cc
+++ b/sql/transaction.cc
@@ -446,8 +446,14 @@ bool trans_rollback_stmt(THD *thd)
     tc_log->rollback(thd, false);
 
   /* In autocommit=1 mode the transaction should be marked as complete in P_S */
+#ifndef DBUG_OFF
+    bool error_simul= false;
+    DBUG_EXECUTE_IF("simulate_xa_commit_log_failure", { error_simul= true; });
+#endif
[7 Oct 2015 17:16] Paul DuBois
Noted in 5.8.0 changelog.

For some instances of failure to prepare an XA transaction,
incomplete transaction cleanup could raise an assertion.