Bug #59331 filesort with priority queue: handling of out-of-memory
Submitted: 6 Jan 2011 14:27 Modified: 10 Jan 2011 16:48
Reporter: Tor Didriksen Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Optimizer Severity:S3 (Non-critical)
Version:5.6 OS:Any
Assigned to: Tor Didriksen CPU Architecture:Any

[6 Jan 2011 14:27] Tor Didriksen
Description:
Comments in filesort.cc indicate that we are able to fallback
to ordinary sort/merge if we fail to initialize the priority queue.
This is not the case, since my_malloc will call my_error, which sets
the THD in error state.

How to repeat:
Simulate out-of-memory in init_queue()

Suggested fix:
Update code/comments to reflect failure.
[6 Jan 2011 14:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128085

3465 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.
[10 Jan 2011 8:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128252

3471 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.
[10 Jan 2011 9:35] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128256

3472 Tor Didriksen	2011-01-06
      Bug#59331 filesort with priority queue: handling of out-of-memory
      
      Verifies that we have an error if init_queue() fails.
     @ mysql-test/r/filesort_debug.result
        New test case.
     @ mysql-test/t/filesort_debug.test
        New test case.
     @ sql/bounded_queue.h
        Add DBUG code to simulate out-of-memory.
     @ sql/filesort.cc
        Terminate filesort if pq.init() fails.
        Also: don't allocate more memory than we actually need for the 'strip-off-addon-fields' case.
[10 Jan 2011 9:58] Tor Didriksen
Pushed to trunk.
[10 Jan 2011 9:58] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:tor.didriksen@oracle.com-20110106143838-tcimyx8pvf4hxhxs) (version source revid:tor.didriksen@oracle.com-20110106143838-tcimyx8pvf4hxhxs) (merge vers: 5.6.2) (pib:24)
[10 Jan 2011 13:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/128291

3475 Tor Didriksen	2011-01-10
      Bug#59331 filesort with priority queue: handling of out-of-memory
      Post-push fix: the .result file was wrong on 32-bit platforms.
[10 Jan 2011 13:24] Bugs System
Pushed into mysql-trunk 5.6.2 (revid:tor.didriksen@oracle.com-20110110132311-mnj4kdllluk3wu80) (version source revid:tor.didriksen@oracle.com-20110110132311-mnj4kdllluk3wu80) (merge vers: 5.6.2) (pib:24)
[10 Jan 2011 16:48] Paul Dubois
Noted in 5.6.2 changelog.

If filesort fell back to an ordinary sort/merge, it could fail to
handle memory correctly.