Bug #59331 filesort with priority queue: handling of out-of-memory

• Submitted: 6 Jan 2011 14:27
• Modified: 10 Jan 2011 16:48
• Reporter: Tor Didriksen
• Status: Closed
• Category: MySQL Server: Optimizer
• Severity: S3 (Non-critical)
• Version: 5.6
• OS: Any
• Assigned to: Tor Didriksen
• CPU Architecture: Any

[6 Jan 2011 14:27] Tor Didriksen

Description:
Comments in filesort.cc indicate that we are able to fallback
to ordinary sort/merge if we fail to initialize the priority queue.
This is not the case, since my_malloc will call my_error, which sets
the THD in error state.

How to repeat:
Simulate out-of-memory in init_queue()

Suggested fix:
Update code/comments to reflect failure.

[6 Jan 2011 14:38] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128085

3465 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.

[10 Jan 2011 8:42] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128252

3471 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.

[10 Jan 2011 9:35] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128256

3472 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.

[10 Jan 2011 9:58] Tor Didriksen

Pushed to trunk.

[10 Jan 2011 9:58] Bugs System

Pushed into mysql-trunk 5.6.2 (revid:[e-mail]-20110106143838-tcimyx8pvf4hxhxs) (version source revid:[e-mail]-20110106143838-tcimyx8pvf4hxhxs) (merge vers: 5.6.2) (pib:24)

[10 Jan 2011 13:23] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128291

3475 Tor Didriksen	2011-01-10
      Bug#59331 filesort with priority queue: handling of out-of-memory
      Post-push fix: the .result file was wrong on 32-bit platforms.

[10 Jan 2011 13:24] Bugs System

Pushed into mysql-trunk 5.6.2 (revid:[e-mail]-20110110132311-mnj4kdllluk3wu80) (version source revid:[e-mail]-20110110132311-mnj4kdllluk3wu80) (merge vers: 5.6.2) (pib:24)

[10 Jan 2011 16:48] Paul DuBois

Noted in 5.6.2 changelog.

If filesort fell back to an ordinary sort/merge, it could fail to
handle memory correctly.