Bug #34799 crash or/and memory overrun with dependant subquery and some joins

Description:
I had some crashing queries which I discovered are due to memory overruns.
On 6.0.3-debug the error log shows this:

Version: '6.0.3-alpha-community-debug'  socket: ''  port: 3306  MySQL Community Server - Debug (GPL)
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mf_iocache.c:238'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mi_open.c:713'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\sql_select.cc:8279'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\sql_select.cc:8279'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mf_iocache.c:1825'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\sql_select.cc:8279'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mf_iocache.c:1825'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\sql_select.cc:8279'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\mf_iocache.c:1825'
Error: Memory allocated at .\sql_select.cc:16223 was overrun, discovered at '.\sql_select.cc:8279'

release version might crash, or it might not :D

How to repeat:
run under valgrind.
or, run debug binary and check safemalloc errors.
see attached .sql file..

testcase. try it multiple times with release binary, crash may occur. else check savemalloc errors from debug binary.

Attachment: bug34799_testcase.sql (application/unknown, text), 8.50 KiB.

you might get a crash, with stack trace similar to this

mysqld.exe!free
mysqld.exe!st_join_table::cleanup
mysqld.exe!JOIN::cleanup
mysqld.exe!JOIN::join_free
mysqld.exe!do_select
mysqld.exe!JOIN::exec
mysqld.exe!mysql_select
mysqld.exe!handle_select
mysqld.exe!execute_sqlcom_select
mysqld.exe!mysql_execute_command
mysqld.exe!mysql_parse
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_threadstart

Thank you for a bug report. Verified just as described. Non-debug binaries crashes at 10th attempt or so for me. Stack trace on Windows was weird enough:

 	ntdll.dll!7c911e58() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
 	ntdll.dll!7c918251() 	
 	ntdll.dll!7c911c76() 	
 	mysqld.exe!best_extension_by_limited_search(JOIN * join=, unsigned __int64 remaining_tables=, unsigned int idx=, double record_count=, double read_time=, unsigned int search_depth=, unsigned int prune_level=)  Line 6443 + 0x24 bytes	C++
 	mysqld.exe!JOIN::alloc_func_list()  Line 17327 + 0x1d bytes	C++
>	mysqld.exe!_heap_alloc(unsigned int size=0)  Line 212	C

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46257

ChangeSet@1.2629, 2008-05-01 12:45:57+04:00, sergefp@mysql.com +3 -0
  BUG#34799: crash or/and memory overrun with dependant subquery and some joins
  - make join_init_cache() take into account space occupied by table rowid when 
    calculating join cache record length (the relationship with subqueries is 
    that the rowid is used by semi-join duplicate elimination strategy)

Pushed into 6.0.6-alpha

Noted in 6.0.6 changelog.

A server crash or memory overrun could occur with a dependent
subquery and joins.

Pushed into mysql-next-mr (revid:alik@sun.com-20100816062819-bluwgdq8q4xysmlg) (version source revid:alik@sun.com-20100816062612-enatdwnv809iw3s9) (pib:20)

Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (merge vers: 5.6.99-m4) (pib:21)

Noted in 5.6.1 changelog.