Description:
We've recently tried to upgrade to 5.0.67 from 5.0.19, and we receive a crash every time a certain query or stored procedure is requested from a web page. When this even happened, We tried reverting back to 5.0.51a and that gives us a crash as well, but with a different error. So far I've gone all the way back to 5.0.22 and the problem has disappeared.
However one of the reasons why we went to 5.0.67 was because we had an issue on 5.0.19 replication, when a stored procedure insert would not appropriately write an insert statement to the binary log, thus breaking replication. Here is the error log we receive:
5.0.67:
----------------- Start ----------------
080902 13:19:02 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.0.67-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
080902 13:20:52 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=268435456
read_buffer_size=2097152
max_used_connections=17
max_connections=1000
threads_connected=11
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 18694144 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd=0x10acdb20
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x10f127c8, backtrace may not be correct.
Bogus stack limit or frame pointer, fp=0x10f127c8, stack_bottom=0x45390000, thread_stack=262144, aborting backtrace.
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x10aab710 = SELECT COUNT(t.trackDetailID) as 'TotalViews',
(SELECT COUNT(tu.trackDetailID)
FROM iqtrack.postedjobs pj
JOIN iqtrack.trackurlapplyattempt tu
ON tu.reqEmediaID=pj.reqEmediaID
WHERE
pj.requisitionID = NAME_CONST('reqID',1312934)
AND
pj.postedDateTime is not null
AND
tu.createdDateTime BETWEEN NAME_CONST('dateFrom',_binary'2000-01-01') and NAME_CONST('dateTo',_binary'2008-09-02')
AND
pj.emediaID = p.emediaID) as 'TotalApplies',
p.emediaName,
(SELECT
tu.referrer as 'referrer'
FROM
trackurlapplyattempt tu
INNER JOIN postedjobs ptj on ptj.reqemediaid = tu.reqemediaid
WHERE
ptj.requisitionID = NAME_CONST('reqID',1312934)
and
tu.emediaID=p.emediaID
thd->thread_id=65
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Number of processes running now: 0
080902 13:20:52 mysqld restarted
InnoDB: Log scan progressed past the checkpoint lsn 19 4254751035
080902 13:20:53 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 19 4257398680
080902 13:20:53 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 163, file name mysql-bin.000010
InnoDB: Last MySQL binlog file position 0 7955072, file name ./mysql-bin.000059
080902 13:20:56 InnoDB: Started; log sequence number 19 4257398680
080902 13:20:56 [Note] Recovering after a crash using mysql-bin
080902 13:20:56 [Note] Starting crash recovery...
080902 13:20:56 [Note] Crash recovery finished.
080902 13:20:56 [Note] /usr/local/mysql/bin/mysqld: ready for connections.
Version: '5.0.67-log' socket: '/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL)
-------------- END -----------------
At First I thought the error was due to my over assignment of connections, but after changing those parameters in my.cnf, the issue remains so I have determined it doesn't have anything to do with those parameters. as well as it being repeated on 5.0.51a
Here is my current my.cnf
--------------- START ------------
[client]
#password = your_password
port = 3306
socket = /tmp/mysql.sock
[mysqld]
basedir=/usr/local/mysql
datadir=/var/mysql/data
innodb_log_group_home_dir=/var/mysql/data
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer = 256M
max_allowed_packet = 128M
table_cache = 256
sort_buffer_size = 16M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
myisam_sort_buffer_size = 8M
thread_cache = 24
query_cache_size = 128M
query_cache_type = 1
max_connections = 200
thread_concurrency = 8
tmp_table_size=256M
max_heap_table_size=32M
innodb_buffer_pool_size = 1200M
innodb_flush_log_at_trx_commit = 2
innodb_log_files_in_group=3
innodb_log_file_size=50M
innodb_log_buffer_size=8M
innodb_file_io_threads=4
#sync-binlog=1
#innodb_safe_binlog
innodb_file_per_table
log-bin=mysql-bin
relay-log=iq-mysql-1-relay-bin
server-id = 1
log_bin_trust_routine_creators = 1
old-passwords
#skip-slave-start
[mysqldump]
quick
max_allowed_packet = 128M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 8M
sort_buffer_size = 8M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 8M
sort_buffer_size = 8M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
-------------------- END --------------------
Please let me know if there are any additional questions
How to repeat:
Error is repeated when running a query generated from a stored procedure call from a web site.