Bug #73979 wrong stack size calculation leads to stack overflow in pinbox allocator
Submitted: 19 Sep 2014 10:37 Modified: 20 Feb 2015 18:12
Reporter: Vlad Lesin Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: Performance Schema Severity:S3 (Non-critical)
Version:5.5, 5.6 OS:Any
Assigned to: Marc ALFF CPU Architecture:Any

[19 Sep 2014 10:37] Vlad Lesin
1) The value in pins->stack_ends_here is actually incorrect. This value is calculated when the thread is initialized in my_thread_init(). The problem is that the value doesn't take into account existing stack usage and just adds the thread stack size to calculate the beginning of the stack.

2) _lf_pinbox_real_free() is using alloca() in a very unsafe way. alloca() should only be used by leaf functions, since you cannot predict the amount of stack space that non-leaf functions will use. As pins->stack_ends_here is calculated in wrong way and there is no window for the qsort() which is invoked from _lf_pinbox_real_free() after alloca(), stack overflow can take place.

See this https://bugs.launchpad.net/percona-server/+bug/1351148 analysis for more detailed description.

How to repeat:
There is not test case to repeat it.

Suggested fix:
1) Take into accout the current stack offset when stack size is calculated in my_thread_init();

2) Don't allocate the whole free stack space in _lf_pinbox_real_free(), take some window for qsort();

See this https://code.launchpad.net/~vlad-lesin/percona-server/5.6-553-bug-1351148 patch.
[19 Sep 2014 13:48] MySQL Verification Team
Hello and thank you for your questions.

First of all, the function (my_thread_init) is called immediately upon the entry to the new thread, so there is no stack usage, or it is very, very small. Insignificantly small.

Regarding alloca(), the function that you are mentioning is not using it. But, yes, it is used in other places. We took care to use it only for pointers for which we know that can not be large. Where we were unsure, we inserted stack size checks. Also, the return from the function is ALWAYS checked and the appropriate action is taken. So far, we did not have any reports of problems with this setup. Where we had, we inserted stack size checks and users would increase the thread stack size. There are practically no current reports on the problem in this area. I am the one who was involved in writing this part of code (as well) and as I am with MySQL for 16 years, I would have remembered any problems in this area.

I am leaving this bug in "Need Feedback" status, as it is always possible that we missed something.
[24 Sep 2014 13:34] Laurynas Biveinis
Sinisa, are you saying that _lf_pinbox_real_free is _not_ calling alloca followed by qsort?
[24 Sep 2014 13:51] MySQL Verification Team
Actually, it does. Simply, I was not directed to the proper source.

Your bug is justified. I can not give it any higher priority, since the usage of concurrent hashes is rather scarce. MUCH more important then that, chances of entering into problems due to this miscalculation are minutely small.

Verified as described.
[24 Sep 2014 17:04] Marc ALFF
Thanks for the report.
[20 Feb 2015 18:12] Paul DuBois
Noted in 5.7.6 changelog.

Miscalculation of memory requirements for qsort operations could
result in stack overflow errors in situations with a large number of
concurrent server connections.