Bug #112425 trx_t might be Use-After-Free in innobase_commit_by_xid
Submitted: 22 Sep 2023 5:39 Modified: 22 Sep 2023 11:32
Reporter: Zhang JiYang Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: InnoDB storage engine Severity:S3 (Non-critical)
Version:8.0 OS:Any
Assigned to: CPU Architecture:Any

[22 Sep 2023 5:39] Zhang JiYang
Description:
See innobase_commit_by_xid in storage/innobase/handler/ha_innodb.cc

1. TrxInInnoDB::enter
  save trx into TrxInInnoDB::m_trx
  access and modify TrxInInnoDB::m_trx

2. trx_free_for_background(trx);
  reinit trx and free to trx_pools

3. TrxInInnoDB::exit
  access and modify TrxInInnoDB::m_trx which has already free to trx_pools

After step-2 and before step-3, the trx_t object might be allocated and used by other thread, which is really dangerous.

In our test, finally we crash in:

storage/innobase/include/trx0trx.h:1504
ut_ad((trx->in_innodb & TRX_FORCE_ROLLBACK_MASK) == 0);

How to repeat:
Run XA transactions in parallel for a long time.

Suggested fix:
An immature fix:

diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc
index 4aeacf071b7..4591845ad5d 100644
--- a/storage/innobase/handler/ha_innodb.cc
+++ b/storage/innobase/handler/ha_innodb.cc
@@ -20076,13 +20076,15 @@ static xa_status_code innobase_commit_by_xid(
   trx_t *trx = trx_get_trx_by_xid(xid);

   if (trx != nullptr) {
-    TrxInInnoDB trx_in_innodb(trx);
+    {
+      TrxInInnoDB trx_in_innodb(trx);

-    innobase_commit_low(trx);
-    ut_ad(trx->mysql_thd == nullptr);
-    /* use cases are: disconnected xa, slave xa, recovery */
-    trx_deregister_from_2pc(trx);
-    ut_ad(!trx->will_lock); /* trx cache requirement */
+      innobase_commit_low(trx);
+      ut_ad(trx->mysql_thd == nullptr);
+      /* use cases are: disconnected xa, slave xa, recovery */
+      trx_deregister_from_2pc(trx);
+      ut_ad(!trx->will_lock); /* trx cache requirement */
+    }
     trx_free_for_background(trx);

     return (XA_OK);
@@ -20104,12 +20106,15 @@ static xa_status_code innobase_rollback_by_xid(
   trx_t *trx = trx_get_trx_by_xid(xid);

   if (trx != nullptr) {
-    TrxInInnoDB trx_in_innodb(trx);
+    int ret;
+    {
+      TrxInInnoDB trx_in_innodb(trx);

-    int ret = innobase_rollback_trx(trx);
+      ret = innobase_rollback_trx(trx);

-    trx_deregister_from_2pc(trx);
-    ut_ad(!trx->will_lock);
+      trx_deregister_from_2pc(trx);
+      ut_ad(!trx->will_lock);
+    }
     trx_free_for_background(trx);

     return (ret != 0 ? XAER_RMERR : XA_OK);
[22 Sep 2023 7:49] zhai weixiang
duplicate of bug#99643 ?
[22 Sep 2023 11:32] MySQL Verification Team
Hi Mr. Jiyang,

Thank you for your bug report.

We have analysed it properly and we consider that you are correct.

This is now a verified bug.

We also do not think that this bug is a duplicate of:

https://bugs.mysql.com/bug.php?id=99643
[22 Sep 2023 11:33] MySQL Verification Team
Hi,

We were not able to repeat the crash, since the test case is not complete nor detailed, hence it's severity is lower.