Bug #22806 MySQL-server-4.1.21-0.glibc23 fails on RedHat ES3 custom 2.4.33.3 kernel.
Submitted: 28 Sep 2006 22:53 Modified: 30 Jan 2007 17:09
Reporter: Marek Biela Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server Severity:S3 (Non-critical)
Version:MySQL-server-4.1.21-0.glibc23 OS:Linux (RedHat ES3)
Assigned to: Jani Tolonen CPU Architecture:Any

[28 Sep 2006 22:53] Marek Biela
Description:
Problem similar to MySQL Bugs: #8310

I also used RH .config file for custom 2.4.33.3 kernel compilation. MySQL would produce this error in its logs:

060920 13:43:21  mysqld started
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=131072
max_used_connections=0
max_connections=150
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2542334 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x4422678c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x812b880
0x40043f88
(nil)
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow
instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
use RH ES3 .config file for custom 2.4.33.3 kernel compilation, and MySQL fails to start. Works well with default RedHat 2.4.21-47.ELsmp kernel.
[29 Sep 2006 11:09] Valeriy Kravchuk
Thank you for a problem report. How much RAM do you have? Please, send you my,cnf content also.
[29 Sep 2006 16:34] Marek Biela
server has 4GB of RAM:
free
             total       used       free     shared    buffers     cached
Mem:       3857176    3774804      82372          0      57708    2614392
-/+ buffers/cache:    1102704    2754472
Swap:      1522096       1372    1520724

I tried starting MySQL with default my-medium.cnf and even without it, but that did not help. It current /etc/my.cnf is:

[client]
socket=/var/lib/mysql/mysql.sock

[mysqld]
#basedir                        = /var/lib
datadir                 = /var/lib/mysql
tmpdir                  = /var/tmp
socket                  = /var/lib/mysql/mysql.sock
skip-locking
skip-innodb
set-variable    = thread_cache_size=8k
set-variable    = key_buffer=64M
set-variable    = max_allowed_packet=4M
set-variable    = table_cache=256
set-variable    = sort_buffer=16M
set-variable    = net_buffer_length=16K
set-variable    = myisam_sort_buffer_size=32M
set-variable    = max_connections=150
#set-variable    = character_set_server=utf8
log-error       = /var/lib/mysql/mysqld.log
pid-file            = /var/lib/mysql/mysqld.pid
server-id       = 2
[10 Oct 2006 12:42] Mike .
I had the same problem with RH9 upgrade to latest 2.4.33.3 vanilla kernel using the same .config as the original 2.4.20-30.9 RH kernel. But also RPM and X had problems loading. I have discovered that everything works fine with 2.4.32, so something is likely borked with the 33.3 kernel.
[16 Oct 2006 9:27] Mike .
Update: 2.4.33 works but 2.4.33.1 doesn't. Not sure about 2.4.33.2
[20 Oct 2006 5:50] Marek Biela
We were able to properly start MySQL with custom 2.4.33.3 kernel:

LD_ASSUME_KERNEL=2.4.19 /etc/init.d/mysql start

It also works after /lib/tls is renamed to /lib/tls.old.

As our client's research shown, this problem is most likely related to RHEL3 kernels that backported NPTL features from 2.6 series. Glibc has been compiled to use NPTL by default. Stock 2.4.xx kernels have no NPTL support. More information at http://dag.wieers.com/howto/compatibility/
[20 Oct 2006 9:18] Valeriy Kravchuk
So, it is yet another NPTL-related problem. I hope, will be able to solve them all some day.
[5 Nov 2006 8:46] Valeriy Kravchuk
Bug #22910 was marked as a duplicate of this one.
[22 Jan 2007 15:20] Jani Tolonen
This is a bug in RedHat glibc in NPTL (version 0.6) backported
code part.

The fix is to use either a static MySQL binary or switch to
LinuxThreads by running "export LD_ASSUME_KERNEL=2.4.1" before
starting mysqld.

The bug has been reported to RedHat and can be viewed here:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=223793

We are adding a check in mysqld_safe for detection of Linux kernel.
If the kernel is 2.4.x series, mysqld_safe will set the above
environment variable automatically.
[30 Jan 2007 17:09] Jani Tolonen
According to RedHat 2.4.33.3 kernel should not be used with RHEL3. This comes with uncompatible version of NPTL for that system.