MySQL Bugs: #21476: stack overflow crashes server; error-message stack reservation too small

Bug #21476	stack overflow crashes server; error-message stack reservation too small
Submitted:	7 Aug 2006 13:22	Modified:	11 Aug 10:43
Reporter:	Valery Dachev
Status:	In progress
Category:	Server	Severity:	S2 (Serious)
Version:	5.0.22, 5.0.32, 5.0.46, 6.0.6	OS:	Other (several)
Assigned to:	Ramil Kalimullin	Target Version:	5.0+
Tags:	bfsm_2007_10_25, bfsm_2007_12_06
Triage:	D1 (Critical)

Description:
I want to upgrade one of my SMF forums. However, the upgrade script failed with "Lost
connection to MySQL server during query". I dumped the table and executed the same query
in the command line and got "ERROR 2013 (HY000) at line 1: Lost connection to MySQL server
during query", so it doesn't seem like a problem in the PHP MySQL extension. I tried to
executing the query with an empty table, but I got the same result. I tried restarting the
MySQL server, but with no result.

I'm running MySQL 5.0.22. Debian package versions:
==
libmysqlclient15off - 5.0.22-3
mysql-client-5.0 - 5.0.22-3
mysql-common - 5.0.22-3
mysql-server-5.0 - 5.0.22-3
==
I suppose it's some kind of stack overflow in the IF() function. I tried reproducing the
same bug in 4.1.11a-4sarge5, but the server didn't fail.

How to repeat:
Here is the table definition:

CREATE TABLE `smf_metal_log_boards` (
  `ID_BOARD` smallint(5) unsigned NOT NULL default '0',
  `ID_MEMBER` mediumint(8) unsigned NOT NULL default '0',
  `logTime` int(10) unsigned NOT NULL default '0',
  `ID_MSG` mediumint(8) unsigned NOT NULL default '0',
  PRIMARY KEY  (`ID_MEMBER`,`ID_BOARD`),
  KEY `logTime` (`logTime`)
) ENGINE=MyISAM DEFAULT CHARSET=cp1251 COLLATE=cp1251_bulgarian_ci;

The query that crashes the server seems to be too long to write it here so I'll do it
after the bug is opened.

Table definition

Attachment: smf_metal_log_boards.sql (application/octet-stream, text), 2.46 KiB.

Crash Script

Attachment: smf_metal_log_boards_crash.sql (application/octet-stream, text), 14.89 KiB.

Thank you for the report.

Verified on Linux using BK sources.

It is only repeatable with thread_stack value closed to really needed:

$mysqld_safe -O thread_stack=131072 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
ERROR 1436 (HY000): Thread stack overrun:  122988 bytes used of a 131072 byte stack, and
8192 bytes needed.  Use 'mysqld -O thread_stack=#' to specify a bigger stack.
mysql> \q
Bye

$mysqld_safe -O thread_stack=145000 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> \q
Bye
[1]+  Segmentation fault      libexec/mysqld -O thread_stack=145000

$mysqld_safe -O thread_stack=196608 &
$mysql bug21476
mysql> \. smf_metal_log_boards_crash.sql
Query OK, 0 rows affected (0.04 sec)
Rows matched: 0  Changed: 0  Warnings: 0

mysql> \q
Bye

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12577

ChangeSet@1.2284, 2006-09-26 18:16:37-04:00, cmiller@zippy.cornsilk.net +1 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 500B of wiggle-room for sundry
  word sizes and stack behaviors.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12635

ChangeSet@1.2284, 2006-09-27 13:27:53-04:00, cmiller@zippy.cornsilk.net +7 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 100B of wiggle-room for sundry
  word sizes and stack behaviors.
  
  The added test verifies in a cross-platform way that there are no gaps 
  between the space that we think we need and what we actually need to report 
  an error.
  
  DOCUMENTERS:  This also adds "let" to the mysqltest commands that evaluate
  an argument to expand variables therein.  (Only right of the "=", of course.)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12643

ChangeSet@1.2284, 2006-09-27 14:42:56-04:00, cmiller@zippy.cornsilk.net +7 -0
  Bug#21476: (Thread stack overrun not caught, causing SEGV)
  
  The STACK_MIN_SIZE is currently set to 8192, when we actually need 
  (emperically discovered) 9236 bytes to raise an fatal error, on Ubuntu 
  Dapper Drake, libc6 2.3.6-0ubuntu2, Linux kernel 2.6.15-27-686, on x86.
  
  I'm taking that as a new lower bound, plus 100B of wiggle-room for sundry
  word sizes and stack behaviors.
  
  The added test verifies in a cross-platform way that there are no gaps 
  between the space that we think we need and what we actually need to report 
  an error.
  
  DOCUMENTERS:  This also adds "let" to the mysqltest commands that evaluate
  an argument to expand variables therein.  (Only right of the "=", of course.)

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/12719

ChangeSet@1.2288, 2006-09-28 09:51:06-04:00, cmiller@zippy.cornsilk.net +1 -0
  Additional patch to Bug#21476:  Free newly-allocated memory in mysqltest.

Available in 5.0.26.

Available in 5.1.12-beta.

No changelog entry needed, but I've updated the
description of the "let" command in the mysqltest
manual to indicate that the value assigned to a
variable now can contain variable references.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13422

ChangeSet@1.2347, 2006-10-10 20:30:33+02:00, istruewing@chilla.local +1 -0
  Bug#21476 - Lost Database Connection During Query
  Raised STACK_MIN_SIZE for Debian GNU/Linux Sid,
  Linux kernel 2.6.16,
  gcc version 3.3.6 (Debian 1:3.3.6-13),
  libc6-dbg 2.3.6.ds1-4,
  Pentium4 (x86),
  BUILD/compile-pentium-debug-max
  Raised about 100 Bytes above the required minimum.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13973

ChangeSet@1.2284, 2006-10-19 13:42:26+02:00, istruewing@chilla.local +1 -0
  Bug#21476 - Lost Database Connection During Query
  Backport from 5.1.
  Raised STACK_MIN_SIZE for Debian GNU/Linux Sid,
  Linux kernel 2.6.16,
  gcc version 3.3.6 (Debian 1:3.3.6-13),
  libc6-dbg 2.3.6.ds1-4,
  Pentium4 (x86),
  BUILD/compile-pentium-debug-max
  Raised about 100 Bytes above the required minimum.

Re-opening this bug because of reproducible failure of the test case that tests this bug
(execution_constants) on HP/UX 11 and several flavours of Linux on IA64 with 5.0.32.

Additional test suite output:

execution_constants            [ fail ]

Errors are (from /usr/share/mysql-test/var/log/mysqltest-time) :
mysqltest: At line 65: query '$query_head 0 $query_tail' failed with wrong errno 2013:
'Lost connection to MySQL server during query', instead of 0...

Seems to occur on Netware binaries too.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/26566

ChangeSet@1.2478, 2007-05-14 11:55:55+05:00, ramil@mysql.com +1 -0
  Fix for bug #21476: stack overflow crashes server; error-message stack reservation too
small
  
  Check for possible stack overflow in the Item_func_if::fix_fields().

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/27110

ChangeSet@1.2496, 2007-05-22 11:16:29+05:00, ramil@mysql.com +1 -0
  Fix for bug #21476: stack overflow crashes server; error-message stack reservation too
small
  
  Increase STACK_BUFF_ALLOC to avoid execution_constants test failure on the hpita2.

Pushed into 5.1.20-beta

Pushed into 5.0.44

Noted in 5.0.44, 5.1.20 changelogs.

Stack overflow caused server crashes.

I reopen this bug because IMHO the fix it not sufficient. It assumes the neccessary stack
size to avoid a crash is independent from the platform. However I see the
execution_constants test from test suite failing on mips64 (platform details in support
issue #19871).

There is also related bug #28271 for the Netware platform.

We should either increase the STACK_MIN_SIZE limit or employ better guesswork to tune it
for different platforms.

Please see bug #35019, which identifies a particular cross-platform difference that is
likely a big part of what you are fighting here.

Same crash in 5.0.64 and in 5.1.26-rc
on HP-UX 11.31 (32 bit only),
reproducible in all builds/runs on that platform.

I suspect the following test failure, found in the 5.1.26-rc build
(specific to HP-UX 11.31, 32 bit, a "debug" build"),
may have the same cause:

=====
main.subselect_notembedded     [ fail ]

mysqltest: At line 52: query '$start $end' failed with wrong errno 2013: 'Lost connection
to MySQL server during query', instead of 0...

The result from queries just before the failure was:
< snip >
create table t1(a int,b int,key(a),key(b));
insert into t1(a,b) values (1,2),(2,1),(2,3),(3,4),(5,4),(5,5),
(6,7),(7,4),(5,3);
5
4
3
2
1
26
25
24
23
22
21
20
19
18
17
16
15

More results from queries before failure can be found in
/PATH/mysql-test/var/log/subselect_notembedded.log
=====