Bug #3069 SELECT segfaults server on linux
Submitted: 4 Mar 2004 18:59 Modified: 5 Mar 2004 9:49
Reporter: Erasmus Oblar Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Server: MyISAM storage engine Severity:S2 (Serious)
Version:4.0.18 OS:Linux (Redhat 8.0)
Assigned to: Dean Ellis CPU Architecture:Any

[4 Mar 2004 18:59] Erasmus Oblar
Description:
After creating two simple tables, running a query against them will cause the server to either segfault and restart, or alternately, the query thread to hang permanently.

System has 1GB physical RAM, 2x800 MHz processors.
Environment info (from mysqlbug):
>Release:       mysql-4.0.18-standard (Official MySQL RPM)
                                                                                        
>C compiler:    2.95.3
>C++ compiler:  2.95.3
>Environment:
        <machine, os, target, libraries (multiple lines)>
System: Linux testbox 2.4.18-19.8.0smp #1 SMP Thu Dec 12 04:36:25 EST 2002 i686 i686 i3\86 GNU/Linux
Architecture: i686
                                                                                        
Some paths:  /usr/bin/perl /usr/bin/make /usr/bin/gmake /usr/bin/gcc /usr/bin/cc
GCC: Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/shar\e/info --enable-shared --enable-threads=posix --disable-checking --host=i386-redhat-lin\ux --with-system-zlib --enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)
Compilation info: CC='gcc'  CFLAGS='-O2 -mcpu=i486 -fno-strength-reduce'  CXX='gcc'  CX\XFLAGS='-O2 -mcpu=i486 -fno-strength-reduce                -felide-constructors -fno-ex\ceptions -fno-rtti                  '  LDFLAGS=''  ASFLAGS=''
LIBC:
lrwxrwxrwx    1 root     root           14 Jan 24  2003 /lib/libc.so.6 -> libc-2.2.93.s\o
-rwxr-xr-x    1 root     root      1235468 Sep  5  2002 /lib/libc-2.2.93.so
-rw-r--r--    1 root     root      2233342 Sep  5  2002 /usr/lib/libc.a
-rw-r--r--    1 root     root          178 Sep  5  2002 /usr/lib/libc.so
Configure command: ./configure '--disable-shared' '--with-mysqld-ldflags=-all-static' '\--with-client-ldflags=-all-static' '--with-server-suffix=-standard' '--without-embedded\-server' '--without-berkeley-db' '--with-innodb' '--without-vio' '--without-openssl' '-\-enable-assembler' '--enable-local-infile' '--with-mysqld-user=mysql' '--with-unix-sock\et-path=/var/lib/mysql/mysql.sock' '--prefix=/' '--with-extra-charsets=complex' '--exec\-prefix=/usr' '--libexecdir=/usr/sbin' '--libdir=/usr/lib' '--sysconfdir=/etc' '--datad\ir=/usr/share' '--localstatedir=/var/lib/mysql' '--infodir=/usr/share/info' '--included\ir=/usr/include' '--mandir=/usr/share/man' '--enable-thread-safe-client' '--with-commen\t=Official MySQL RPM' 'CC=gcc' 'CFLAGS=-O2 -mcpu=i486 -fno-strength-reduce' 'CXXFLAGS=-\O2 -mcpu=i486 -fno-strength-reduce                 -felide-constructors -fno-exceptions\ -fno-rtti                  ' 'CXX=gcc'
                                                                                        

Crashdump from error log:

040304 21:41:50  mysqld started
/usr/sbin/mysqld: ready for connections.
Version: '4.0.18-standard-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
 
key_buffer_size=268435456
read_buffer_size=1044480
max_used_connections=0
max_connections=100
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 466543 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x886e008
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0xbff5f5b8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x8071f44
0x82a0e38
0x82bdac9
0x82bd963
0x8279bd6
0x806ca56
0x806b079
0x8071b70
0x807b1b2
0x829e5ec
0x82c7dea
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://www.mysql.com/doc/en/Using_stack_trace.html and follow instructions on how
to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil)  is invalid pointer
thd->thread_id=54
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

How to repeat:
Pipe 'crashdb' into mysql to create the tables.  Then pipe 'query' into mysql several times until MySQL dies or hangs. It can take anywhere from 1-50 attempts at this query for it to fail.  Generally, just whacking the up-arrow and hitting enter (to re-send query), at a reasonable clip will do the trick in under a minute.
[4 Mar 2004 19:00] Erasmus Oblar
pipe into mysql to create test tables

Attachment: crashdb (application/octet-stream, text), 1.11 KiB.

[4 Mar 2004 19:01] Erasmus Oblar
query that will eventually crash the server

Attachment: query (application/octet-stream, text), 217 bytes.

[5 Mar 2004 8:04] Erasmus Oblar
Since it's short, I'll append the table creates and query that crash the server:

-- MySQL dump 9.10
--
-- Host: localhost    Database: crash
-- ------------------------------------------------------
-- Server version	4.0.18-standard-log

DROP TABLE IF EXISTS Event;
CREATE TABLE Event (
  ID bigint(20) NOT NULL auto_increment,
  attraction bigint(20) NOT NULL default '-1',
  business bigint(20) NOT NULL default '-1',
  attractionListing bigint(20) NOT NULL default '-1',
  feed bigint(20) NOT NULL default '-1',
  PRIMARY KEY  (ID),
  KEY attraction (attraction),
  KEY business (business),
  KEY attractionListing (attractionListing),
  KEY feed (feed)
) TYPE=MyISAM;

INSERT INTO Event VALUES (1,1,1,1,1);

DROP TABLE IF EXISTS Listing;
CREATE TABLE Listing (
  ID bigint(20) NOT NULL auto_increment,
  business bigint(20) NOT NULL default '-1',
  feed bigint(20) NOT NULL default '-1',
  feedId varchar(255) NOT NULL default '',
  hashKey bigint(20) NOT NULL default '-1',
  PRIMARY KEY  (ID),
  KEY business (business),
  KEY feed (feed),
  KEY feedID (feedId),
  KEY hashKey (hashKey)
) TYPE=MyISAM;

INSERT INTO Listing VALUES (1,1,1,'11111',73732500);
INSERT INTO Listing VALUES (2,2,1,'22222',73732500);

Now, just run this query in a loop and MySQL should eventualy hang or segfault.
SELECT DISTINCT e.ID, e.attraction, e.business, 
                e.attractionListing, e.feed
FROM 
      Event e, Listing li 
WHERE 
      li.feed=1 AND e.feed=1 AND e.business=li.business
ORDER BY
      li.hashKey;
[5 Mar 2004 9:49] Dean Ellis
I cannot repeat this with 4.0.19, nor with our 4.0.18 binary.

Copy your stack trace to some file, and resolve the stack dump (using the mysqld.sym file included in the distribution):

resolve_stack_dump -s/path/to/mysqld.sym -n/path/to/stackdump
[5 Mar 2004 11:12] Erasmus Oblar
here's the dump:

0x8071f44 handle_segfault + 420
0x82a0e38 pthread_sighandler + 184
0x82bdac9 chunk_free + 297
0x82bd963 free + 147
0x8279bd6 my_no_flags_free + 22
0x806ca56 net_end + 22
0x806b079 _._3THD + 73
0x8071b70 end_thread__FP3THDb + 64
0x807b1b2 handle_one_connection + 946
0x829e5ec pthread_start_thread + 220
0x82c7dea thread_start + 4

some sort of threading issue?