MySQL Bugs: #35019: Use pthread_attr_getguardsize for better robustness of stack thread sizes

Bug #35019	Use pthread_attr_getguardsize for better robustness of stack thread sizes
Submitted:	4 Mar 2008 1:29	Modified:	6 Oct 19:28
Reporter:	[ name withheld ]
Status:	Verified
Category:	Server: General	Severity:	S2 (Serious)
Version:	5.0.45	OS:	Any
Assigned to:		Target Version:
Tags:	Contribution
Triage:	Triaged: D2 (Serious)

Description:
mysql dumps core in the execution_constants regression test when running on PPC64
hardware that is using 64KB page size (as is the case with current RHEL5 kernels, for
example).  This failure is evidence that it is failing to guard adequately against stack
overflow, and therefore that it is vulnerable to crashes from overly-complex queries.

How to repeat:
Run regression tests on RHEL5 on PPC64.

Suggested fix:
The problem is that historically Linux threading libraries have allocated thread stacks
as the requested stack size minus the "guard area" (the unmapped address space placed at
the bottom of the stack to turn a stack overflow into SIGSEGV).  The default size of the
guard area is typically one hardware page, and therefore varies across platforms.  Mysql
has not bothered to account for this behavior explicitly, but has assumed that the guard
area requirement can be treated as part of the STACK_MIN_SIZE constant.  But this does
not work when the guard area is 4x the size of STACK_MIN_SIZE...

What I recommend doing about this is using pthread_attr_getguardsize(), where available,
to determine
the actual guard area size, and adding that onto the stack size request, so as to be sure
of getting usable stack space as large as requested.  (You might say that this is working
around a libpthread bug, and I wouldn't argue with you --- but given the chilly response
I got at Red Hat bug #435337, I think the odds of getting it changed are nil.)

As proof of concept I'll attach the patch I'll be using in RHEL and Fedora until
something is done about this upstream.  This patch is a bit quick-and-dirty; there are
several things you'll want to do before adopting it:

1. It needs autoconf support for testing whether pthread_attr_getguardsize() is actually
available.

2. I've only patched the two pthread_attr_setstacksize() calls in mysqld.cc.  There are
quite a few others in mysql, but I don't feel qualified to evaluate whether they need
adjustment.  (The patch is set up
so that the heavy lifting is in a subroutine that could be exported.)

3. I left the #ifdef __NETWARE__ hack in mysqld.cc alone, since I don't have Netware to
test on, but I suspect it should be folded into the subroutine.  Or maybe it could even
go away entirely; perhaps the problem it is hacking around is really a larger guard area
on Netware than elsewhere.

4. You could likely reduce the value of STACK_MIN_SIZE by 4K or even more, since it no
longer needs to try to include guard area.

RHEL/Fedora patch to increase stack size request by guard area size

Attachment: mysql-stack-guard.patch (, text), 4.93 KiB.

Verified as described by looking into bk-source code.

In order for us to continue the process of reviewing your contribution to MySQL - We need
you to review and sign the Sun|MySQL contributor agreement (the "SCA")

The process is explained here: 
http://forge.mysql.com/wiki/Sun_Contributor_Agreement

Getting a signed/approved SCA on file will help us facilitate your contribution-- this
one, and others in the future.

Thank you !

It would be really nice if you were to resolve this. Otherwise, everyone messing around
with RedHat/CentOS SRPMs must spend time on this.

In order to move forward on a fix , the contributor of the fix (appears that the name is
"withheld")  needs to submit the SCA for review and approval.

 The process is explained here: 
http://forge.mysql.com/wiki/Sun_Contributor_Agreement 

This is a basic legal requirement to enable us to accept contributions from the
community. 

thanks