Bug #54583 | InnoDB: Assertion failure in thread <..> in file sync/sync0sync.c line 1226 | ||
---|---|---|---|
Submitted: | 17 Jun 2010 12:14 | Modified: | 15 Oct 2010 13:22 |
Reporter: | Alexey Stroganov | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: InnoDB storage engine | Severity: | S3 (Non-critical) |
Version: | 5.5.5pre-debug | OS: | Any |
Assigned to: | Sunny Bains | CPU Architecture: | Any |
[17 Jun 2010 12:14]
Alexey Stroganov
[17 Jun 2010 12:44]
Alexey Stroganov
To reproduce isssue it is enough to use fresh datadir created by mysql_install_db with server/InnoDB <1.1.
[17 Jun 2010 13:27]
Mikhail Izioumtchenko
happens in the extra rseg creation. Everything works fine unless UNIV_SYNC_DEBUG is defined. The question is, if it's not some form of overasserting, can it happen outside of this once-per-upgrade codepath? My testing said no, so far.
[24 Jun 2010 1:14]
Sunny Bains
I'm not sure this is actually a bug even. One of the requirements when upgrading is that the database must have been shutdown cleanly. Implicit in this is that there should be no purge and insert buffer operations pending. I think the documentation should explicitly state what a clean shutdown means, at least in this context, if it doesn't do so already. The problem in this bug report is that when the page is accessed it triggers an insert buffer merge operation and that cause the problem.
[24 Jun 2010 1:22]
Sunny Bains
http://dev.mysql.com/doc/innodb-plugin/1.1/en/innodb-multiple-rollback-segments.html Looks like the documentation already mentions "slow shutdown". The slow shutdown should ensure that there are no purge or insert buffer merge operations at the next startup. Secondly, the documentation says: "Each transaction is assigned to one of the rollback segments using a hashing function, and remains tied to that rollback segment for the duration." -- there is no hashing function, InnoDB uses a round robin scheme. The first perhaps needs some further elaboration and the second needs to be corrected.
[28 Jun 2010 14:53]
Mikhail Izioumtchenko
It's still a bug because slow shutdown still ends up in an assert. We need to execute insert buffer merge even if to find out there's nothing to merge, and there we assert somewhere. I'll reassign it to Marko to look at. The initial assesment of a serious bug was to get someone to look at a.s.a.p. At this moment the bug does not look serious, so it needs to be retriaged. Regarding why we didn't see it in our testing, I haven't run the compatibility test suite which was supposed to catch it, with UNIV_DEBUG or UNIV_SYNC_DEBUG, until lately. I found many complications and some noise there. So far I've had about one run with UNIV*DEBUG where I don't remember seeing this. However the expectation is that UNIV_SYNC_DEBUG works always so we should fix this bug unless I'll never be able to run the compatibility tests with UNIV_SYNC_DEBUG. I should retry it eventually to see if and why this bug isn't showing there but for now the bug doesn't look too dangerous anyway so it may take time.
[30 Jun 2010 7:10]
Marko Mäkelä
At the time of the assertion failure, following InnoDB latching orders are in use: SYNC_NO_ORDER_CHECK (8 occurrences) SYNC_FSP_PAGE (5 occurrences) SYNC_FSP (3 occurrences) These are OK, because they are above SYNC_IBUF_BITMAP. The following are not (1 occurrence each): SYNC_KERNEL SYNC_TRX_SYS_HEADER SYNC_BUF_POOL SYNC_BUF_BLOCK Sunny is right that this assertion failure does not indicate a real problem, because InnoDB is running in a single-threaded mode. Michael is right that we should keep UNIV_SYNC_DEBUG clean of bogus failures. I will try to figure out an elegant fix for this.
[30 Jun 2010 8:00]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/112524
[30 Jun 2010 8:00]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/112525
[30 Jun 2010 8:32]
Marko Mäkelä
Sunny, this occurs even without any pending insert buffer merges. I repeated this as follows, with an empty insert buffer bitmap: (1) delete any data files (2) start MySQL 5.1, let InnoDB create the system tablespace (3) send SIGQUIT (ctrl-\) to mysqld (4) start MySQL 5.5 compiled with -DUNIV_DEBUG -DUNIV_SYNC_DEBUG I did not test what happens when an insert buffer merge (discarding buffered entries to a table that was dropped from the system tablespace) is needed. Testing that would require populating a large table with innodb_file_per_table=0, dropping it and doing a fast shutdown. (The upgrade advice of doing a slow shutdown would imply an insert buffer merge.) Also, note that trx_sys_create_rsegs() does nothing if srv_force_recovery == TRUE or recv_needed_recovery == TRUE.
[30 Jun 2010 9:06]
Marko Mäkelä
I would welcome a better fix that avoids relaxing the latching order checks. I believe that we should try to obey the latching order even during single-threaded startup, because we might want to defer some tasks to a background thread, such as the rollback of incomplete transactions currently is. You may want to reassign this to someone else while I am on vacation.
[19 Jul 2010 14:37]
Bugs System
Pushed into 5.1.49 (revid:build@mysql.com-20100719143034-omcma40sblwmay3x) (version source revid:vasil.dimov@oracle.com-20100704071244-3lo4okzels3kvy1p) (merge vers: 5.1.49) (pib:16)
[23 Jul 2010 12:28]
Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100723121820-jryu2fuw3pc53q9w) (version source revid:alik@sun.com-20100723121820-jryu2fuw3pc53q9w) (merge vers: 5.5.6-m3) (pib:18)
[23 Jul 2010 12:33]
Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100723121929-90e9zemk3jkr2ocy) (version source revid:alik@sun.com-20100723121827-3bsh51m5sj6g4oma) (pib:18)
[4 Aug 2010 7:52]
Bugs System
Pushed into mysql-trunk 5.5.6-m3 (revid:alik@sun.com-20100731131027-1n61gseejyxsqk5d) (version source revid:alik@sun.com-20100731074942-o840woifuqioxxe4) (merge vers: 5.5.6-m3) (pib:18)
[4 Aug 2010 8:10]
Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804080001-bny5271e65xo34ig) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (merge vers: 5.6.99-m4) (pib:18)
[4 Aug 2010 8:26]
Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alik@ibmvm-20100804081533-c1d3rbipo9e8rt1s) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (merge vers: 5.6.99-m4) (pib:18)
[4 Aug 2010 9:05]
Bugs System
Pushed into mysql-next-mr (revid:alik@ibmvm-20100804081630-ntapn8bf9pko9vj3) (version source revid:alik@sun.com-20100731075120-qz9z8c25zum2wgmm) (pib:20)
[10 Aug 2010 18:30]
John Russell
Closing without adding anything to the doc changelog. I did correct the reference to the 'hashing function' for the transaction <=> rollback segment mapping.
[14 Oct 2010 8:38]
Bugs System
Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.20 (revid:martin.skold@mysql.com-20101014082627-jrmy9xbfbtrebw3c) (version source revid:martin.skold@mysql.com-20101014082627-jrmy9xbfbtrebw3c) (merge vers: 5.1.51-ndb-7.0.20) (pib:21)
[14 Oct 2010 8:53]
Bugs System
Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.39 (revid:martin.skold@mysql.com-20101014083757-5qo48b86d69zjvzj) (version source revid:martin.skold@mysql.com-20101014083757-5qo48b86d69zjvzj) (merge vers: 5.1.51-ndb-6.3.39) (pib:21)
[14 Oct 2010 9:11]
Bugs System
Pushed into mysql-5.1-telco-6.2 5.1.51-ndb-6.2.19 (revid:martin.skold@mysql.com-20101014084420-y54ecj85j5we27oa) (version source revid:martin.skold@mysql.com-20101014084420-y54ecj85j5we27oa) (merge vers: 5.1.51-ndb-6.2.19) (pib:21)
[15 Oct 2010 13:22]
Jon Stephens
Setting back to Closed -- see above.
[7 Feb 2011 7:24]
Marko Mäkelä
This is a case of Bug #59733. When Bug #59733 is fixed, this "fix" of suppressing the check should be reverted.
[4 Aug 2011 13:57]
Marko Mäkelä
It now looks like Bug#59733 is overasserting, and suppressing this assertion was OK after all.