Bug #34161 freeing maria tmp table after fetching rows using prepared statements - crashes
Submitted: 30 Jan 2008 11:24 Modified: 5 Mar 2008 9:55
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Maria storage engine Severity:S1 (Critical)
Version:5.1.23-maria-alpha-pb297 OS:Any
Assigned to: Oleksandr Byelkin CPU Architecture:Any

[30 Jan 2008 11:24] Shane Bester
Description:
mysqld.exe!unlink_block
mysqld.exe!free_block
mysqld.exe!flush_pagecache_blocks_int
mysqld.exe!flush_pagecache_blocks_with_filter
mysqld.exe!_ma_once_end_block_record
mysqld.exe!maria_close
mysqld.exe!ha_maria::close
mysqld.exe!handler::drop_table
mysqld.exe!free_tmp_table
mysqld.exe!Materialized_cursor::close
mysqld.exe!Materialized_cursor::fetch
mysqld.exe!mysql_stmt_fetch
mysqld.exe!dispatch_command
mysqld.exe!do_command
mysqld.exe!handle_one_connection
mysqld.exe!pthread_start
mysqld.exe!_callthreadstart

How to repeat:
Will upload standalone C testcase later.
[30 Jan 2008 11:24] MySQL Verification Team
Built RelDebInfo project on Windows using the pushbuild sources
mysql-5.1.23-rc-pb260.tar.gz

Ran my application testsuite:

gypsy --host=127.0.0.1:3307 --queryfile=qa.sql --threads=10 --shuffle-queries=1 --duration=3000 --mysql_store_result=1
[30 Jan 2008 11:45] MySQL Verification Team
some debug info. this is a race condition which happens with >= 2 threads

Attachment: bug34161_debug_info.txt (text/plain), 6.05 KiB.

[1 Feb 2008 10:26] MySQL Verification Team
the qa.sql file...

Attachment: qa.sql (application/unknown, text), 12.80 KiB.

[1 Feb 2008 21:50] Guilhem Bichot
I'll try to reproduce it as I have a Windows machine.
[8 Feb 2008 17:00] Guilhem Bichot
Hi Shane, I ran the gypsy testcase, hit an assertion in Maria's ma_pagecache.c (so, not same crash as yours) and after fixing this misplaced assertion, it has been running for one hour without issue. Could you please re-test with latest sources (like from pushbuild) ?
[11 Feb 2008 11:41] MySQL Verification Team
crashes with build of pb321 - attached is stack traces of all threads. takes ~4 minutes to crash with 15 threads.

Attachment: bug34161_debug_info_mysql-5.1.24-maria-alpha-pb321.txt (text/plain), 50.14 KiB.

[14 Feb 2008 17:16] Guilhem Bichot
Ouf, I finally managed to see what line of ma_pagecache.c looks wrong. I am assigning to Sanja who inserted this suspicious line and knows the pagecache by heart :)
A small testcase which works on my Linux is in the "Files" section.
[14 Feb 2008 17:18] Guilhem Bichot
short test to provoke problem

Attachment: bug34161_small_testcase_linux.tar.bz2 (application/x-bzip2, text), 3.08 KiB.

[14 Feb 2008 17:24] Guilhem Bichot
So, please download bug34161_small_testcase_linux.tar.bz2 from the "Files" section; apply diff to ma_pagecache.c; replace ma_pagecache_single.c of your tree with the one from the tar.bz2, recompile, run "ma_pagecache_single_1k-t --debug". It should segfault. Look at the debug trace it contains tags "BUGINFO" which show what is going wrong. Also look at the comment at the start of ma_pagecache_single.c it explains what the test does. The same scenario happened in Shane's original gypsy testcase though it took longer to crash and crashed only on Windows (must be due to thread scheduling).
The suspicious line is 
    block->status&= ~PCBLOCK_IN_SWITCH;
in ma_pagecache.c, it was added in
ChangeSet 1.2303.214.1 2007/02/02 09:41:32 bell@desktop.sanja.is.com.ua
  postreview changes for page cache and pre review commit for loghandler
apparently to fix a bug: the comment says "fixed bug in unregistering block during write".
[3 Mar 2008 21:24] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/43340

ChangeSet@1.2611, 2008-03-03 23:23:21+02:00, bell@desktop.sanja.is.com.ua +1 -0
  Removed incorrect clearing IN_SWITCH status. (BUG#34161)