Bug #51342 more xid crashing
Submitted: 20 Feb 2010 7:47 Modified: 18 Jun 2010 1:42
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: General Severity:S1 (Critical)
Version:5.0.90, 5.1.44, 5.1.45-bzr OS:Any
Assigned to: Sergey Vojtovich CPU Architecture:Any
Tags: crash, xa, xid

[20 Feb 2010 7:47] Shane Bester
Description:
during a test with transactions and load data, 5.1.44 crashed:

 mysqld.exe!movelink()[hash.c:294]
 mysqld.exe!my_hash_insert()[hash.c:456]
 mysqld.exe!xid_cache_insert()[sql_class.cc:3317]
 mysqld.exe!mysql_execute_command()[sql_parse.cc:4723]
 mysqld.exe!mysql_parse()[sql_parse.cc:5975]
 mysqld.exe!dispatch_command()[sql_parse.cc:1235]
 mysqld.exe!do_command()[sql_parse.cc:874]
 mysqld.exe!handle_one_connection()[sql_connect.cc:1127]
 mysqld.exe!pthread_start()[my_winthread.c:85]
 mysqld.exe!_callthreadstart()[thread.c:295]
 mysqld.exe!_threadstart()[thread.c:275]
 kernel32.dll!BaseThreadStart()
variables.
e invalid and cause the dump to abort...
00025CD30F0=xa start 'xid1'

How to repeat:
give me time..
[20 Feb 2010 10:53] MySQL Verification Team
another 5.1.44 stack trace, crash after a thread disconnected:

mysqld.exe!xid_get_hash_key()[sql_class.cc:3254]
mysqld.exe!my_hash_delete()[hash.c:508]
mysqld.exe!xid_cache_delete()[sql_class.cc:3326]
mysqld.exe!THD::cleanup()[sql_class.cc:923]
mysqld.exe!unlink_thd()[mysqld.cc:1858]
mysqld.exe!one_thread_per_connection_end()[mysqld.cc:1944]
mysqld.exe!handle_one_connection()[sql_connect.cc:1134]
mysqld.exe!pthread_start()[my_winthread.c:85]
mysqld.exe!_callthreadstart()[thread.c:295]
mysqld.exe!_threadstart()[thread.c:275]
kernel32.dll!BaseThreadStart()
[20 Feb 2010 11:56] MySQL Verification Team
testcase!

Attachment: bug51342.c (text/plain), 6.46 KiB.

[20 Feb 2010 12:01] MySQL Verification Team
bugs like this have been pestering me for a long time:

bug #40437 (mysqld segfault on my_hash_insert and XA)
bug #28323 (Server crashed in xid cache operations)
bug #36642 (server hang on shutdown after some XA transaction tests)
bug #43171 (Assertion failed: thd->transaction.xid_state.xid.is_null())
bug #27871 (MySQL server crashed with damaged stack trace)
[20 Feb 2010 12:38] Valeriy Kravchuk
Verified just as described with recent 5.1.45 from bzr on Mac OS X:

77-52-24-143:5.1 openxs$ ./bug51342 2>&1 | more
running initializations..
client version=50145
server version=50145
about to spawn 4 threads
....
completed spawning new database worker threads
testcase is now running, so watch for error output
query failed 'xa commit 'x'' : 1397 (XAER_NOTA: Unknown XID) (XAE04)
query failed 'rollback' : 1399 (XAER_RMFAIL: The command cannot be executed when
 global transaction is in the  ACTIVE state) (XAE07)
query failed 'xa start 'x'' : 1399 (XAER_RMFAIL: The command cannot be executed 
when global transaction is in the  ACTIVE state) (XAE07)
query failed 'xa commit 'x'' : 1399 (XAER_RMFAIL: The command cannot be executed
 when global transaction is in the  ACTIVE state) (XAE07)
query failed 'rollback' : 1399 (XAER_RMFAIL: The command cannot be executed when
 global transaction is in the  ACTIVE state) (XAE07)
query failed 'rollback' : 1399 (XAER_RMFAIL: The command cannot be executed when
 global transaction is in the  ACTIVE state) (XAE07)
query failed 'xa start 'x'' : 1399 (XAER_RMFAIL: The command cannot be executed 
when global transaction is in the  ACTIVE state) (XAE07)
query failed 'xa start 'x'' : 1399 (XAER_RMFAIL: The command cannot be executed 
when global transaction is in the  ACTIVE state) (XAE07)
query failed 'xa start 'x'' : 1440 (XAER_DUPID: The XID already exists) (XAE08)
^C
77-52-24-143:5.1 openxs$ 100220 14:34:50 mysqld_safe mysqld restarted

and in the error log I've got:

...
stack_bottom = 0xb026af64 thread_stack 0x30000
0   mysqld                              0x0057d5f0 my_print_stacktrace + 44
1   mysqld                              0x00101004 handle_segfault + 836
2   libSystem.B.dylib                   0x940472bb _sigtramp + 43
3   ???                                 0xffffffff 0x0 + 4294967295
4   mysqld                              0x000e1bd7 _Z16xid_cache_deleteP12st_xid_state + 73
5   mysqld                              0x000e7ecc _ZN3THD7cleanupEv + 190
6   mysqld                              0x0010151f _Z10unlink_thdP3THD + 137
7   mysqld                              0x0010166f _Z29one_thread_per_connection_endP3THDb + 101
8   mysqld                              0x0010b7cd handle_one_connection + 495
9   libSystem.B.dylib                   0x9400c095 _pthread_start + 321
10  libSystem.B.dylib                   0x9400bf52 thread_start + 34
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x0 is an invalid pointer
thd->thread_id=2
thd->killed=KILL_CONNECTION
...
[4 Mar 2010 14:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/102330

2847 Sergey Vojtovich	2010-03-04
      BUG#51342 - more xid crashing
      
      SET autocommit=1 while XA transaction is active may
      cause various side effects, including memory corruption
      and server crash.
      
      The problem is that SET autocommit=1 and further queries
      attempt to commit local transaction, whereas XA transaction
      is still active.
      
      As local and XA transactions are mutually exclusive, this
      patch forbids enabling autocommit mode while XA transaction
      is active.
     @ mysql-test/r/xa.result
        A test case for BUG#51342.
     @ mysql-test/t/xa.test
        A test case for BUG#51342.
     @ sql/set_var.cc
        Forbid enabling autocommit mode while XA transaction is
        active.
[4 Mar 2010 18:26] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/102367

2847 Sergey Vojtovich	2010-03-04
      BUG#51342 - more xid crashing
      
      SET autocommit=1 while XA transaction is active may
      cause various side effects, including memory corruption
      and server crash.
      
      The problem is that SET autocommit=1 and further queries
      attempt to commit local transaction, whereas XA transaction
      is still active.
      
      As local and XA transactions are mutually exclusive, this
      patch forbids enabling autocommit mode while XA transaction
      is active.
     @ mysql-test/r/xa.result
        A test case for BUG#51342.
     @ mysql-test/t/xa.test
        A test case for BUG#51342.
     @ sql/set_var.cc
        Forbid enabling autocommit mode while XA transaction is
        active.
[10 Mar 2010 11:05] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/102858

2847 Sergey Vojtovich	2010-03-10
      BUG#51342 - more xid crashing
      
      SET autocommit=1 while XA transaction is active may
      cause various side effects, including memory corruption
      and server crash.
      
      The problem is that SET autocommit=1 and further queries
      attempt to commit local transaction, whereas XA transaction
      is still active.
      
      As local and XA transactions are mutually exclusive, this
      patch forbids enabling autocommit mode while XA transaction
      is active.
     @ mysql-test/r/xa.result
        A test case for BUG#51342.
     @ mysql-test/t/xa.test
        A test case for BUG#51342.
     @ sql/set_var.cc
        Forbid enabling autocommit mode while XA transaction is
        active.
[26 Mar 2010 8:21] Bugs System
Pushed into 5.5.4-m3 (revid:alik@sun.com-20100326080914-2pz8ns984e0spu03) (version source revid:alexey.kopytov@sun.com-20100312095153-t4rtoqc7p96lmxvh) (merge vers: 5.5.3-m2) (pib:16)
[26 Mar 2010 8:25] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100326081116-m3v4l34yhr43mtsv) (version source revid:alik@sun.com-20100325072612-4sds00ix8ajo1e84) (pib:16)
[26 Mar 2010 8:29] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100326081944-qja07qklw1p2w7jb) (version source revid:alik@sun.com-20100325073410-4t4i9gu2u1pge7xb) (merge vers: 6.0.14-alpha) (pib:16)
[6 Apr 2010 7:53] Bugs System
Pushed into 5.0.91 (revid:joro@sun.com-20100406075152-flz4btqirl9hly31) (version source revid:svoj@sun.com-20100310152849-pkpn5fkl335fhn6x) (merge vers: 5.0.91) (pib:16)
[6 Apr 2010 7:59] Bugs System
Pushed into 5.1.46 (revid:sergey.glukhov@sun.com-20100405111026-7kz1p8qlzglqgfmu) (version source revid:svoj@sun.com-20100310153122-00nv55acfukokfc7) (merge vers: 5.1.45) (pib:16)
[12 Apr 2010 22:45] Paul Dubois
Noted in 5.0.91, 5.1.46, 5.5.5, 6.0.14.

With an XA transaction active, SET autocommit = 1 could cause side
effects such as memory corruption or a server crash.
[13 Apr 2010 12:00] Valeriy Kravchuk
Bug #52779 was marked as a duplicate of this one.
[30 May 2010 9:16] Vitaly Karasik
I still have the same crashes  even after upgrading to 5.0.91.

Vitaly
[31 May 2010 10:56] Vitaly Karasik
is there a test case I can use for reproducing this bug?
[31 May 2010 12:29] MySQL Verification Team
Vitaly, see the attached file on my comment [20 Feb 12:56].

I guess you could add 'start transaction', 'commit', 'set transaction isolation ...' to the worker_thread() function for more testing...
[2 Jun 2010 14:23] Vitaly Karasik
Shane , thank you, I'll run this testcase.

BTW, I'm still not sure that "my" (#52779) bug is duplicate for yours. In our application we don't use distributed transactions and "autocimmit=1" in the same connection pool.
[17 Jun 2010 11:50] Bugs System
Pushed into 5.1.47-ndb-7.0.16 (revid:martin.skold@mysql.com-20100617114014-bva0dy24yyd67697) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)
[17 Jun 2010 12:27] Bugs System
Pushed into 5.1.47-ndb-6.2.19 (revid:martin.skold@mysql.com-20100617115448-idrbic6gbki37h1c) (version source revid:martin.skold@mysql.com-20100609211156-tsac5qhw951miwtt) (merge vers: 5.1.46-ndb-6.2.19) (pib:16)
[17 Jun 2010 13:15] Bugs System
Pushed into 5.1.47-ndb-6.3.35 (revid:martin.skold@mysql.com-20100617114611-61aqbb52j752y116) (version source revid:vasil.dimov@oracle.com-20100331130613-8ja7n0vh36a80457) (merge vers: 5.1.46) (pib:16)