Bug #35019 Use pthread_attr_getguardsize for better robustness of stack thread sizes

Description:
mysql dumps core in the execution_constants regression test when running on PPC64 hardware that is using 64KB page size (as is the case with current RHEL5 kernels, for example).  This failure is evidence that it is failing to guard adequately against stack overflow, and therefore that it is vulnerable to crashes from overly-complex queries.

How to repeat:
Run regression tests on RHEL5 on PPC64.

Suggested fix:
The problem is that historically Linux threading libraries have allocated thread stacks as the requested stack size minus the "guard area" (the unmapped address space placed at the bottom of the stack to turn a stack overflow into SIGSEGV).  The default size of the guard area is typically one hardware page, and therefore varies across platforms.  Mysql has not bothered to account for this behavior explicitly, but has assumed that the guard area requirement can be treated as part of the STACK_MIN_SIZE constant.  But this does not work when the guard area is 4x the size of STACK_MIN_SIZE...

What I recommend doing about this is using pthread_attr_getguardsize(), where available, to determine
the actual guard area size, and adding that onto the stack size request, so as to be sure of getting usable stack space as large as requested.  (You might say that this is working around a libpthread bug, and I wouldn't argue with you --- but given the chilly response I got at Red Hat bug #435337, I think the odds of getting it changed are nil.)

As proof of concept I'll attach the patch I'll be using in RHEL and Fedora until something is done about this upstream.  This patch is a bit quick-and-dirty; there are several things you'll want to do before adopting it:

1. It needs autoconf support for testing whether pthread_attr_getguardsize() is actually available.

2. I've only patched the two pthread_attr_setstacksize() calls in mysqld.cc.  There are quite a few others in mysql, but I don't feel qualified to evaluate whether they need adjustment.  (The patch is set up
so that the heavy lifting is in a subroutine that could be exported.)

3. I left the #ifdef __NETWARE__ hack in mysqld.cc alone, since I don't have Netware to test on, but I suspect it should be folded into the subroutine.  Or maybe it could even go away entirely; perhaps the problem it is hacking around is really a larger guard area on Netware than elsewhere.

4. You could likely reduce the value of STACK_MIN_SIZE by 4K or even more, since it no longer needs to try to include guard area.

RHEL/Fedora patch to increase stack size request by guard area size

Attachment: mysql-stack-guard.patch (, text), 4.93 KiB.

Verified as described by looking into bk-source code.

In order for us to continue the process of reviewing your contribution to MySQL - We need
you to review and sign the Sun|MySQL contributor agreement (the "SCA")

The process is explained here: 
http://forge.mysql.com/wiki/Sun_Contributor_Agreement

Getting a signed/approved SCA on file will help us facilitate your contribution-- this
one, and others in the future.

Thank you !

It would be really nice if you were to resolve this. Otherwise, everyone messing around with RedHat/CentOS SRPMs must spend time on this.

In order to move forward on a fix , the contributor of the fix (appears that the name is "withheld")  needs to submit the SCA for review and approval.

 The process is explained here: 
http://forge.mysql.com/wiki/Sun_Contributor_Agreement 

This is a basic legal requirement to enable us to accept contributions from the community. 

thanks

Noted in 5.6.14, 5.7.2 changelogs.

For better robustness against stack overflow, the server now accounts
for the size of the guard area when making thread stack size
requests.