Bug #21476 stack overflow crashes server; error-message stack reservation too small
Submitted: 7 Aug 2006 11:22 Modified: 22 Jun 2011 17:08
Reporter: Valery Dachev Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:5.0.22, 5.0.32, 5.0.46, 6.0.6 OS:Other (several)
Assigned to: Ramil Kalimullin CPU Architecture:Any
Tags: bfsm_2007_10_25, bfsm_2007_12_06
Triage: Needs Triage: D1 (Critical)

[7 Aug 2006 11:22] Valery Dachev
Description:
I want to upgrade one of my SMF forums. However, the upgrade script failed with "Lost connection to MySQL server during query". I dumped the table and executed the same query in the command line and got "ERROR 2013 (HY000) at line 1: Lost connection to MySQL server during query", so it doesn't seem like a problem in the PHP MySQL extension. I tried to executing the query with an empty table, but I got the same result. I tried restarting the MySQL server, but with no result.

I'm running MySQL 5.0.22. Debian package versions:
==
libmysqlclient15off - 5.0.22-3
mysql-client-5.0 - 5.0.22-3
mysql-common - 5.0.22-3
mysql-server-5.0 - 5.0.22-3
==
I suppose it's some kind of stack overflow in the IF() function. I tried reproducing the same bug in 4.1.11a-4sarge5, but the server didn't fail.

How to repeat:
Here is the table definition:

CREATE TABLE `smf_metal_log_boards` (
  `ID_BOARD` smallint(5) unsigned NOT NULL default '0',
  `ID_MEMBER` mediumint(8) unsigned NOT NULL default '0',
  `logTime` int(10) unsigned NOT NULL default '0',
  `ID_MSG` mediumint(8) unsigned NOT NULL default '0',
  PRIMARY KEY  (`ID_MEMBER`,`ID_BOARD`),
  KEY `logTime` (`logTime`)
) ENGINE=MyISAM DEFAULT CHARSET=cp1251 COLLATE=cp1251_bulgarian_ci;

The query that crashes the server seems to be too long to write it here so I'll do it after the bug is opened.
[7 Aug 2006 11:23] Valery Dachev
Table definition

Attachment: smf_metal_log_boards.sql (application/octet-stream, text), 2.46 KiB.

[7 Aug 2006 11:24] Valery Dachev
Crash Script

Attachment: smf_metal_log_boards_crash.sql (application/octet-stream, text), 14.89 KiB.

[7 Aug 2006 11:51] Sveta Smirnova
Thank you for the report.

Verified on Linux using BK sources.

It is only repeatable with thread_stack value closed to really needed:

$mysqld_safe -O thread_stack=131072 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
ERROR 1436 (HY000): Thread stack overrun:  122988 bytes used of a 131072 byte stack, and 8192 bytes needed.  Use 'mysqld -O thread_stack=#' to specify a bigger stack.
mysql> \q
Bye

$mysqld_safe -O thread_stack=145000 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> \q
Bye
[1]+  Segmentation fault      libexec/mysqld -O thread_stack=145000

$mysqld_safe -O thread_stack=196608 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
Query OK, 0 rows affected (0.04 sec)
Rows matched: 0  Changed: 0  Warnings: 0

mysql> \q
Bye
[26 Sep 2006 22:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12577

ChangeSet@1.2284, 2006-09-26 18:16:37-04:00, cmiller@zippy.cornsilk.net +1 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 500B of wiggle-room for sundry
  word sizes and stack behaviors.
[27 Sep 2006 17:25] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12635

ChangeSet@1.2284, 2006-09-27 13:27:53-04:00, cmiller@zippy.cornsilk.net +7 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 100B of wiggle-room for sundry
  word sizes and stack behaviors.
  
  The added test verifies in a cross-platform way that there are no gaps 
  between the space that we think we need and what we actually need to report 
  an error.
  
  DOCUMENTERS:  This also adds "let" to the mysqltest commands that evaluate
  an argument to expand variables therein.  (Only right of the "=", of course.)
[27 Sep 2006 18:40] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12643

ChangeSet@1.2284, 2006-09-27 14:42:56-04:00, cmiller@zippy.cornsilk.net +7 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 100B of wiggle-room for sundry
  word sizes and stack behaviors.
  
  The added test verifies in a cross-platform way that there are no gaps 
  between the space that we think we need and what we actually need to report 
  an error.
  
  DOCUMENTERS:  This also adds "let" to the mysqltest commands that evaluate
  an argument to expand variables therein.  (Only right of the "=", of course.)
[28 Sep 2006 13:48] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12719

ChangeSet@1.2288, 2006-09-28 09:51:06-04:00, cmiller@zippy.cornsilk.net +1 -0
  Additional patch to Bug#21476:  Free newly-allocated memory in mysqltest.
[3 Oct 2006 20:01] Chad MILLER
Available in 5.0.26.
[3 Oct 2006 20:15] Chad MILLER
Available in 5.1.12-beta.
[6 Oct 2006 2:25] Paul Dubois
No changelog entry needed, but I've updated the
description of the "let" command in the mysqltest
manual to indicate that the value assigned to a
variable now can contain variable references.
[10 Oct 2006 18:30] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13422

ChangeSet@1.2347, 2006-10-10 20:30:33+02:00, istruewing@chilla.local +1 -0
  Bug#21476 - Lost Database Connection During Query
  Raised STACK_MIN_SIZE for Debian GNU/Linux Sid,
  Linux kernel 2.6.16,
  gcc version 3.3.6 (Debian 1:3.3.6-13),
  libc6-dbg 2.3.6.ds1-4,
  Pentium4 (x86),
  BUILD/compile-pentium-debug-max
  Raised about 100 Bytes above the required minimum.
[19 Oct 2006 11:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13973

ChangeSet@1.2284, 2006-10-19 13:42:26+02:00, istruewing@chilla.local +1 -0
  Bug#21476 - Lost Database Connection During Query
  Backport from 5.1.
  Raised STACK_MIN_SIZE for Debian GNU/Linux Sid,
  Linux kernel 2.6.16,
  gcc version 3.3.6 (Debian 1:3.3.6-13),
  libc6-dbg 2.3.6.ds1-4,
  Pentium4 (x86),
  BUILD/compile-pentium-debug-max
  Raised about 100 Bytes above the required minimum.
[21 Dec 2006 11:46] Daniel Fischer
Re-opening this bug because of reproducible failure of the test case that tests this bug (execution_constants) on HP/UX 11 and several flavours of Linux on IA64 with 5.0.32.
[21 Dec 2006 11:47] Daniel Fischer
Additional test suite output:

execution_constants            [ fail ]

Errors are (from /usr/share/mysql-test/var/log/mysqltest-time) :
mysqltest: At line 65: query '$query_head 0 $query_tail' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...
[7 May 2007 6:19] Shane Bester
Seems to occur on Netware binaries too.
[14 May 2007 6:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/26566

ChangeSet@1.2478, 2007-05-14 11:55:55+05:00, ramil@mysql.com +1 -0
  Fix for bug #21476: stack overflow crashes server; error-message stack reservation too small
  
  Check for possible stack overflow in the Item_func_if::fix_fields().
[22 May 2007 6:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27110

ChangeSet@1.2496, 2007-05-22 11:16:29+05:00, ramil@mysql.com +1 -0
  Fix for bug #21476: stack overflow crashes server; error-message stack reservation too small
  
  Increase STACK_BUFF_ALLOC to avoid execution_constants test failure on the hpita2.
[6 Jun 2007 16:54] Bugs System
Pushed into 5.1.20-beta
[6 Jun 2007 16:58] Bugs System
Pushed into 5.0.44
[12 Jun 2007 18:29] Paul Dubois
Noted in 5.0.44, 5.1.20 changelogs.

Stack overflow caused server crashes.
[18 Oct 2007 14:17] Axel Schwenke
I reopen this bug because IMHO the fix it not sufficient. It assumes the neccessary stack size to avoid a crash is independent from the platform. However I see the execution_constants test from test suite failing on mips64 (platform details in support issue #19871).

There is also related bug #28271 for the Netware platform.

We should either increase the STACK_MIN_SIZE limit or employ better guesswork to tune it for different platforms.
[4 Mar 2008 5:07] [ name withheld ]
Please see bug #35019, which identifies a particular cross-platform difference that is likely a big part of what you are fighting here.
[3 Jul 2008 19:22] Joerg Bruehe
Same crash in 5.0.64 and in 5.1.26-rc
on HP-UX 11.31 (32 bit only),
reproducible in all builds/runs on that platform.
[9 Jul 2008 17:05] Joerg Bruehe
I suspect the following test failure, found in the 5.1.26-rc build
(specific to HP-UX 11.31, 32 bit, a "debug" build"),
may have the same cause:

=====
main.subselect_notembedded     [ fail ]

mysqltest: At line 52: query '$start $end' failed with wrong errno 2013: 'Lost connection to MySQL server during query', instead of 0...

The result from queries just before the failure was:
< snip >
create table t1(a int,b int,key(a),key(b));
insert into t1(a,b) values (1,2),(2,1),(2,3),(3,4),(5,4),(5,5),
(6,7),(7,4),(5,3);
5
4
3
2
1
26
25
24
23
22
21
20
19
18
17
16
15

More results from queries before failure can be found in /PATH/mysql-test/var/log/subselect_notembedded.log
=====
[9 Sep 2008 8:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/53578

2676 Ramil Kalimullin	2008-09-09
      Fix for bug#21476: stack overflow crashes server; error-message
      stack reservation too small
      
      Problem: some tests fail on HP-UX due to insufficient stack reservation.
      
      Fix: increase stack reservation.
[14 Feb 2009 10:40] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/66324

2799 Ramil Kalimullin	2009-02-14
      Fix for bug#21476: stack overflow crashes server; error-message
      stack reservation too small
      
      Problem: some tests fail on HP-UX due to insufficient stack reservation.
      
      Fix: increase stack reservation.
      modified:
        sql/mysql_priv.h
[19 Feb 2009 13:04] Bugs System
Pushed into 6.0.10-alpha (revid:sergey.glukhov@sun.com-20090218125737-5y5b2xo3duo1wlvo) (version source revid:ramil@mysql.com-20090214104225-yrr6nj3ze9rdee42) (merge vers: 6.0.10-alpha) (pib:6)
[27 Feb 2009 21:03] Paul Dubois
Noted in 6.0.10 changelog.

Allocation of stack space for error messages could be too small on
HP-UX, leading to stack overflow crashes.

Setting report to NDI pending push into 5.0.x/5.1.x.
[13 Mar 2009 19:03] Bugs System
Pushed into 5.1.33 (revid:joro@sun.com-20090313111355-7bsi1hgkvrg8pdds) (version source revid:vvaintroub@mysql.com-20090218093153-sjzxf01i4ezte0xp) (merge vers: 5.1.33) (pib:6)
[13 Mar 2009 20:05] Paul Dubois
Noted in 5.1.33.

Setting report to NDI pending push into 5.0.x.
[9 May 2009 16:40] Bugs System
Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508100057-30ote4xggi4nq14v) (merge vers: 5.1.33-ndb-6.2.18) (pib:6)
[9 May 2009 17:38] Bugs System
Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090508175813-s6yele2z3oh6o99z) (merge vers: 5.1.33-ndb-6.3.25) (pib:6)
[9 May 2009 18:35] Bugs System
Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509073226-09bljakh9eppogec) (merge vers: 5.1.33-ndb-7.0.6) (pib:6)
[11 May 2009 17:00] Paul Dubois
Setting report to NDI pending push into 5.0.x.
[3 Dec 2009 7:36] Sunanda Menon
I have tried increasing the STACK_MIZ_SIZE=16000 and see that execution_constants test still fails.Will try increasing to 18000 as my last try,but is there any basis by which this needs to increase?

I still see the failure on Opensolaris(X86)
[5 Dec 2009 6:29] James Day
Sunanda, try 160000 (ten times the new STACK_MIN_SIZE). If that solves it you can then reduce it to find out what the real minimum size is for that environment.
[22 Jun 2011 17:08] Paul Dubois
Closing bug. This will not have a 5.0.x push.