| Bug #61132 | Infinite loop in buf_page_get_gen() when handling compressed pages | ||
|---|---|---|---|
| Submitted: | 11 May 2011 11:21 | Modified: | 21 Aug 2013 14:51 |
| Reporter: | Alexey Kopytov | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: InnoDB Plugin storage engine | Severity: | S3 (Non-critical) |
| Version: | 5.1, 5.5 | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[16 May 2011 19:18]
Marko Mäkelä
While I have not repeated this bug, it is a plausible scenario. We can prevent the eviction of the block by buffer-fixing it before releasing the buffer pool mutex. I/O-fixing the block earlier should not buy us anything. For example, buf_page_get_zip() can still interfere with buf_page_get_gen(). This should be extremely unlikely, because buf_page_get_zip() should only be used for accessing compressed BLOB pages, and compressed BLOB pages should never be accessed by buf_page_get_gen(), except when inserting the BLOB.
[16 May 2011 19:28]
Marko Mäkelä
Tentative patch for mitigating the issue
Attachment: bug61132.patch (text/x-diff), 2.32 KiB.
[29 May 2013 8:22]
Marko Mäkelä
Inaam Rana brought up an interesting point. When running on a very small buffer pool, my fix could make the situation worse by buffer-fixing the compressed-only page while allocating an uncompressed page. If multiple threads are doing the same (for different blocks), we could run out of buffer pool. However, I am not sure if this is a practical problem. The original hang should be possible with a reasonably-sized buffer pool as well, if the table sits idle long enough, so that the uncompressed pages will be evicted but the compressed pages will be retained in the buffer pool. Without the fix, the very same page that buf_page_get_gen() is trying to decompress so that it can be accessed, could be evicted from the buffer pool, and buf_page_get_gen() would end up re-reading the same page from the data file.
[21 Aug 2013 14:51]
Bugs System
Noted in 5.1.72, 5.5.34, 5.6.14, 5.7.3 changelog: An infinite loop could occur in "buf_page_get_gen" when handling compressed-only pages. Thank you for the bug report.
[27 Sep 2013 7:28]
Laurynas Biveinis
5.1$ bzr log -r 4038 -n0
------------------------------------------------------------
revno: 4038 [merge]
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.1-current
timestamp: Wed 2013-08-21 11:54:09 +0300
message:
Merge working copy to mysql-5.1.
------------------------------------------------------------
revno: 4036.1.2 [merge]
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.1
timestamp: Wed 2013-08-21 10:03:31 +0300
message:
Merge mysql-5.1 to working copy.
------------------------------------------------------------
revno: 4036.1.1
committer: Marko Mäkelä <marko.makela@oracle.com>
branch nick: mysql-5.1
timestamp: Wed 2013-08-21 08:22:05 +0300
message:
Bug#12560151 61132: infinite loop in buf_page_get_gen() when handling
compressed pages
After loading a compressed-only page in buf_page_get_gen() we allocate a new
block for decompression. The problem is that the compressed page is neither
buffer-fixed nor I/O-fixed by the time we call buf_LRU_get_free_block(),
so it may end up being evicted and returned back as a new block.
buf_page_get_gen(): Temporarily buffer-fix the compressed-only block
while allocating memory for an uncompressed page frame.
This should prevent this form of the infinite loop, which is more likely
with a small innodb_buffer_pool_size.
rb#2511 approved by Jimmy Yang, Sunny Bains
Description: After loading a compressed page in buf_page_get_gen() we allocate a new block for decompression. The problem is that the compressed page is neither buffer-fixed nor I/O-fixed by the time we call buf_LRU_get_free_block(), so it may end up being evicted and returned back as a new block. We then call buf_page_hash_get() to see if the page hash has been modified, and indeed it returns NULL because the original page in not in the buffer pool anymore. In which case we go back to read the page again. Here's the relevant code from buf_page_get_gen() in 5.1 plugin: /* Allocate an uncompressed page. */ buf_pool_mutex_exit(); mutex_exit(&buf_pool_zip_mutex); block = buf_LRU_get_free_block(); ut_a(block); buf_pool_mutex_enter(); mutex_enter(&block->mutex); { buf_page_t* hash_bpage = buf_page_hash_get(space, offset); if (UNIV_UNLIKELY(bpage != hash_bpage)) { /* The buf_pool->page_hash was modified while buf_pool_mutex was released. Free the block that was allocated. */ buf_LRU_block_free_non_file_page(block); mutex_exit(&block->mutex); block = (buf_block_t*) hash_bpage; goto loop2; } } We had a reproducible case when the above loop becomes infinite with low innodb_buffer_pool_size. Temporarily buffer-fixing the compressed page to protect it from buf_LRU_get_free_block() solved the problem. How to repeat: Examine the code in buf_page_get_gen().