| Bug #50157 | Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx | ||
|---|---|---|---|
| Submitted: | 7 Jan 2010 20:40 | Modified: | 7 Mar 2010 1:18 |
| Reporter: | Elena Stepanova | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Replication | Severity: | S2 (Serious) |
| Version: | 5.5.1-m2 | OS: | Linux |
| Assigned to: | Libing Song | CPU Architecture: | Any |
[7 Jan 2010 20:40]
Elena Stepanova
[7 Jan 2010 21:24]
Elena Stepanova
All threads bt
Attachment: all_threads_bug50157.out (application/octet-stream, text), 15.90 KiB.
[7 Jan 2010 21:24]
Elena Stepanova
Stress test
Attachment: stress_test_basedir.tar.gz (application/gzip, text), 21.21 KiB.
[7 Jan 2010 21:24]
Elena Stepanova
See also bug#50160 and bug#50163. The attached stress test is the same for all three bugs, and it randomly ends with one of these crashes. All three problems might be the same, but since the produced stack trace is noticeably different in each case, I'm logging each one separately.
[20 Jan 2010 8:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/97490 3127 Li-Bing.Song@sun.com 2010-01-20 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is created when each time some log events are written into binlog file and is synchronized. TranxNodes' memories are allocted from mem_root of the current thread, and will be freed immediately after current statement ending. Sometimes, a statement ends and its TranxNode is freed before it is cleared from TranxNode list. So the Pointer of the TranxNode in TranxNode list becomes a wild pointer. After this patch, One statement always waits until its log events has been replicated completely if it has been binlogged and synchronized. For the TranxNode will be cleared from the TranxNode list after the log event has been replicated. @ sql/rpl_handler.cc params are not initialized.
[22 Jan 2010 14:59]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/97891 3128 Li-Bing.Song@sun.com 2010-01-22 Bug #50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is created when each time some log events are written into binlog file and is synchronized. TranxNodes' memories are allocted from mem_root of the current thread, and will be freed immediately after current statement ending. from TranxNode list. So the Pointer of the TranxNode in TranxNode list becomes a wild pointer. After this patch, TranxNodes are not allocated from mem_root.
[25 Jan 2010 9:04]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98017 3127 Li-Bing.Song@sun.com 2010-01-25 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is allocated and insertted into the active list each time when some log events are written into binlog file and is flushed. The memory for TranxNode is allocted with thd_alloc and will be freed after at the end of the statement. The after_commit/after_rollback callback was supposed to be call before the end of each statement and remove the node from the active list. However this assumption is not correct in all cases(e.g. CREATE TEMPORARY ... SELECT), and can cause the memory allocated for TranxNode be freed before it was removed from the active list. So The TranxNode pointer in the active list would become a wild pointer and cause the crash. After this patch, The memory for TranxNode will be allocated by my_malloc. @ sql/rpl_handler.cc params are not initialized.
[27 Jan 2010 7:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98279 3127 Li-Bing.Song@sun.com 2010-01-27 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is allocated and insertted into the active list each time when some log events are written into binlog file and is flushed. The memory for TranxNode is allocted with thd_alloc and will be freed after at the end of the statement. The after_commit/after_rollback callback was supposed to be call before the end of each statement and remove the node from the active list. However this assumption is not correct in all cases(e.g. CREATE TEMPORARY ... SELECT), and can cause the memory allocated for TranxNode be freed before it was removed from the active list. So The TranxNode pointer in the active list would become a wild pointer and cause the crash. After this patch, The memory for TranxNode will be allocated by my_malloc. @ sql/rpl_handler.cc params are not initialized.
[29 Jan 2010 6:39]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98535 3127 Li-Bing.Song@sun.com 2010-01-29 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is allocated and insertted into the active list each time a log events is written and flushed into the binlog file. The memory for TranxNode is allocated with thd_alloc and will be freed at the end of the statement. The after_commit/after_rollback callback was supposed to be called before the end of each statement and remove the node from the active list. However this assumption is not correct in all cases(e.g. call 'CREATE TEMPORARY TABLE myisam_t SELECT * FROM innodb_t' in a transaction and delete all temporary tables automatically when a session closed), and can cause the memory allocated for TranxNode be freed before it was removed from the active list. So The TranxNode pointer in the active list would become a wild pointer and cause the crash. After this patch, We had a class called a TranxNodeAllocate which manages the memory for allocating and freeing TranxNode. It uses my_malloc to allocate memory. @ sql/rpl_handler.cc params are not initialized.
[30 Jan 2010 15:44]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98715 3127 Li-Bing.Song@sun.com 2010-01-30 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is allocated and inserted into the active list each time a log event is written and flushed into the binlog file. The memory for TranxNode is allocated with thd_alloc and will be freed at the end of the statement. The after_commit/after_rollback callback was supposed to be called before the end of each statement and remove the node from the active list. However this assumption is not correct in all cases(e.g. call 'CREATE TEMPORARY TABLE myisam_t SELECT * FROM innodb_t' in a transaction and delete all temporary tables automatically when a session closed), and can cause the memory allocated for TranxNode be freed before it was removed from the active list. So The TranxNode pointer in the active list would become a wild pointer and cause the crash. After this patch, We have a class called a TranxNodeAllocate which manages the memory for allocating and freeing TranxNode. It uses my_malloc to allocate memory. @ sql/rpl_handler.cc params are not initialized.
[30 Jan 2010 18:29]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98725 3127 Li-Bing.Song@sun.com 2010-01-31 BUG#50157 Assertion !active_tranxs_->is_tranx_end_pos(..) in ReplSemiSyncMaster::commitTrx The root cause of the crash is that a TranxNode is freed before it is used. A TranxNode is allocated and inserted into the active list each time a log event is written and flushed into the binlog file. The memory for TranxNode is allocated with thd_alloc and will be freed at the end of the statement. The after_commit/after_rollback callback was supposed to be called before the end of each statement and remove the node from the active list. However this assumption is not correct in all cases(e.g. call 'CREATE TEMPORARY TABLE myisam_t SELECT * FROM innodb_t' in a transaction and delete all temporary tables automatically when a session closed), and can cause the memory allocated for TranxNode be freed before it was removed from the active list. So The TranxNode pointer in the active list would become a wild pointer and cause the crash. After this patch, We have a class called a TranxNodeAllocate which manages the memory for allocating and freeing TranxNode. It uses my_malloc to allocate memory. @ sql/rpl_handler.cc params are not initialized.
[30 Jan 2010 19:17]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98733 2932 Li-Bing.Song@sun.com 2010-01-31 [merge] Auto Merge fix for bug#50157
[31 Jan 2010 13:44]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98778 2979 Li-Bing.Song@sun.com 2010-01-31 [merge] Manual Merge for bug#50157
[31 Jan 2010 13:53]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/98779 3859 Li-Bing.Song@sun.com 2010-01-31 [merge] Manual merge for bug#50157
[1 Feb 2010 7:29]
Libing Song
Pushed to mysql-5.1-rep-semisync and merged to mysql-trunk-bugfixing, mysql-next-mr-bugfixing and mysql-6.0-codebase-bugfixing
[4 Feb 2010 14:23]
Zhenxing He
Bug#50060 is marked as dup of this bug.
[12 Feb 2010 17:40]
Bugs System
Pushed into 5.5.2-m2 (revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (version source revid:joerg@mysql.com-20100212164100-jnurxdw5z88m472s) (merge vers: 5.5.2-m2) (pib:16)
[13 Feb 2010 8:37]
Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100213083436-9pesg4h55w1mekxc) (version source revid:luis.soares@sun.com-20100211135109-t63avry9fqpgyh78) (merge vers: 6.0.14-alpha) (pib:16)
[13 Feb 2010 8:39]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100213083327-cee4ao3jpg33eggv) (version source revid:luis.soares@sun.com-20100211135018-1f9dbghg0itszigo) (pib:16)
[18 Feb 2010 18:28]
Paul DuBois
Noted in 5.5.2, 6.0.14 changelogs. With semisynchronous replication, memory allocated for handling transactions could be freed while still in use, resulting in a server crash. Setting report to Need Merge pending push of Celosia to release tree.
[6 Mar 2010 11:06]
Bugs System
Pushed into 5.5.3-m3 (revid:alik@sun.com-20100306103849-hha31z2enhh7jwt3) (version source revid:vvaintroub@mysql.com-20100213160132-nx1vlocxuta76txh) (merge vers: 5.5.99-m3) (pib:16)
[7 Mar 2010 1:18]
Paul DuBois
Already fixed in earlier 5.5.x.
