071005 09:36:33  mysqld started
071005  9:36:33 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
071005  9:36:34  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
071005  9:36:34  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 647029599.
InnoDB: Doing recovery: scanned up to log sequence number 3 647029599
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2 row operations to undo
InnoDB: Trx id counter is 0 20194560
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 144654091, file name slave-bin.000048
InnoDB: Last MySQL binlog file position 0 381496, file name /db01/mydata/master-bin.000051
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 20194242
071005  9:36:35  InnoDB: Rollback of non-prepared transactions completed
071005  9:36:35  InnoDB: Started; log sequence number 3 647029599
071005  9:36:35 [Note] /usr/local/software/platform/database/bin/mysqld: ready for connections.
Version: '5.0.46-enterprise-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Enterprise Server (Commercial)
071005  9:36:36  InnoDB: Warning: purge reached the head of the history list,
InnoDB: but its length is still reported as 7081387! Make a detailed bug
InnoDB: report, and post it to bugs.mysql.com
071005  9:36:38 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000051' at position 385655, relay log '/db01/mydata/slave-relay-bin.000001' position: 4
071005  9:36:38 [Note] Slave I/O thread: connected to master 'repl@5.5.5.1:3306',  replication started in log 'master-bin.000051' at position 385655
071005  9:36:44 [Note] /usr/local/software/platform/database/bin/mysqld: Normal shutdown

071005  9:36:44 [Note] Slave I/O thread killed while reading event
071005  9:36:44 [Note] Slave I/O thread exiting, read up to log 'master-bin.000051', position 394137
071005  9:36:44 [Note] Error reading relay log event: slave SQL thread was killed
071005  9:36:44  InnoDB: Starting shutdown...
071005  9:36:47  InnoDB: Shutdown completed; log sequence number 3 647036979
071005  9:36:47 [Note] /usr/local/software/platform/database/bin/mysqld: Shutdown complete

071005 09:36:47  mysqld ended

071005 09:37:03  mysqld started
071005  9:37:03 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2 row operations to undo
InnoDB: Trx id counter is 0 20195072
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 20194242
071005  9:37:03  InnoDB: Rollback of non-prepared transactions completed
071005  9:37:03  InnoDB: Started; log sequence number 3 647036979
071005  9:37:03 [Note] /usr/local/software/platform/database/bin/mysqld: ready for connections.
Version: '5.0.46-enterprise-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Enterprise Server (Commercial)
071005  9:37:03 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000051' at position 394137, relay log '/db01/mydata/slave-relay-bin.000002' position: 8718
071005  9:37:03 [Note] Slave I/O thread: connected to master 'repl@5.5.5.1:3306',  replication started in log 'master-bin.000051' at position 394137
071005  9:37:13  InnoDB: Warning: purge reached the head of the history list,
InnoDB: but its length is still reported as 7081400! Make a detailed bug
InnoDB: report, and post it to bugs.mysql.com
071005 10:52:10InnoDB: Assertion failure in thread 1781328816 in file fsp0fsp.c line 1569
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
071005 10:52:10 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=3
max_connections=250
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2368534 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xc5c17d0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6a2ccc48, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818a3dd
0x83383aa
0x83393a6
0x82e8390
0x82f4d60
0x82f405c
0x82c3f10
0x82c2d3d
0x82c35b4
0x82c310d
0x82c3354
0x82c7cbe
0x823bb54
0x81eb505
0x81ea96e
0x81a12ee
0x81a5cb3
0x8205026
0x8204c50
0x826e697
0x826c34b
0x98cbd4
0x8d14fe
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x6a1023eb  is invalid pointer
thd->thread_id=2
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
071005 10:52:11  mysqld restarted
071005 10:52:11 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
071005 10:52:11  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
071005 10:52:11  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 662588575.
InnoDB: Doing recovery: scanned up to log sequence number 3 667831296
InnoDB: Doing recovery: scanned up to log sequence number 3 672518328
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2 row operations to undo
InnoDB: Trx id counter is 0 20234752
071005 10:52:12  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 36927004, file name master-bin.000051
InnoDB: Last MySQL binlog file position 0 98, file name /db01/mydata/slave-bin.000002
071005 10:52:15  InnoDB: Started; log sequence number 3 672518328
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 20194242
071005 10:52:15  InnoDB: Rollback of non-prepared transactions completed
071005 10:52:15 [Note] Recovering after a crash using /db01/mydata/slave-bin
071005 10:52:15 [Note] Starting crash recovery...
071005 10:52:15 [Note] Crash recovery finished.
071005 10:52:15 [Note] /usr/local/software/platform/database/bin/mysqld: ready for connections.
Version: '5.0.46-enterprise-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Enterprise Server (Commercial)
071005 10:52:15 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000051' at position 36927031, relay log '/db01/mydata/slave-relay-bin.000004' position: 36533130
071005 10:52:15 [Note] Slave I/O thread: connected to master 'repl@5.5.5.1:3306',  replication started in log 'master-bin.000051' at position 36933636
071005 10:52:15InnoDB: Assertion failure in thread 1781558192 in file fsp0fsp.c line 1569
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
071005 10:52:15 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=0
max_connections=250
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2368534 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xda897d8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6a304c48, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818a3dd
0x83383aa
0x83393a6
0x82e8390
0x82f4d60
0x82f405c
0x82c3f10
0x82c2d3d
0x82c35b4
0x82c310d
0x82c3354
0x82c7cbe
0x823bb54
0x81eb505
0x81ea96e
0x81a12ee
0x81a5cb3
0x8205026
0x8204c50
0x826e697
0x826c34b
0x98cbd4
0x8d14fe
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x6a105963  is invalid pointer
thd->thread_id=2
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
071005 10:52:15  mysqld restarted
071005 10:52:15 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
071005 10:52:16  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
071005 10:52:16  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 662588575.
InnoDB: Doing recovery: scanned up to log sequence number 3 667831296
InnoDB: Doing recovery: scanned up to log sequence number 3 672518328
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2 row operations to undo
InnoDB: Trx id counter is 0 20234752
071005 10:52:16  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 36927004, file name master-bin.000051
InnoDB: Last MySQL binlog file position 0 98, file name /db01/mydata/slave-bin.000002
071005 10:52:19  InnoDB: Started; log sequence number 3 672518328
071005 10:52:19 [Note] Recovering after a crash using /db01/mydata/slave-bin
071005 10:52:19 [Note] Starting crash recovery...
071005 10:52:19 [Note] Crash recovery finished.
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 20194242
071005 10:52:19  InnoDB: Rollback of non-prepared transactions completed
071005 10:52:19 [Note] /usr/local/software/platform/database/bin/mysqld: ready for connections.
Version: '5.0.46-enterprise-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Enterprise Server (Commercial)
071005 10:52:19 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000051' at position 36927031, relay log '/db01/mydata/slave-relay-bin.000004' position: 36533130
071005 10:52:19InnoDB: Assertion failure in thread 1781836720 in file fsp0fsp.c line 1569
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
071005 10:52:19 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=0
max_connections=250
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2368534 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xbf457d0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6a348c48, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818a3dd
0x83383aa
0x83393a6
0x82e8390
0x82f4d60
0x82f405c
0x82c3f10
0x82c2d3d
0x82c35b4
0x82c310d
0x82c3354
0x82c7cbe
0x823bb54
0x81eb505
0x81ea96e
0x81a12ee
0x81a5cb3
0x8205026
0x8204c50
0x826e697
0x826c34b
0x98cbd4
0x8d14fe
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xbf4c463 = INSERT INTO QRTZ_FIRED_TRIGGERS (ENTRY_ID, TRIGGER_NAME, TRIGGER_GROUP, IS_VOLATILE, INSTANCE_NAME, FIRED_TIME, STATE, JOB_NAME, JOB_GROUP, IS_STATEFUL, REQUESTS_RECOVERY) VALUES('11191554845608', 'MT_ggsipm84hb113', 'MANUAL_TRIGGER', 0, '1', 1191561695331, 'EXECUTING', '2', '1787915', 1, 1)
thd->thread_id=2
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0
071005 10:52:19  mysqld restarted
071005 10:52:19 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
071005 10:52:20  mysqld started
071005 10:52:20 [Warning] Changed limits: max_open_files: 1024  max_connections: 250  table_cache: 382
071005 10:52:20  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
071005 10:52:20  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 3 662588575.
InnoDB: Doing recovery: scanned up to log sequence number 3 667831296
InnoDB: Doing recovery: scanned up to log sequence number 3 672518328
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2 row operations to undo
InnoDB: Trx id counter is 0 20234752
071005 10:52:21  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 36927004, file name master-bin.000051
InnoDB: Last MySQL binlog file position 0 98, file name /db01/mydata/slave-bin.000002
071005 10:52:23  InnoDB: Started; log sequence number 3 672518328
071005 10:52:23 [Note] Recovering after a crash using /db01/mydata/slave-bin
071005 10:52:23 [Note] Starting crash recovery...
071005 10:52:23 [Note] Crash recovery finished.
InnoDB: Starting in background the rollback of uncommitted transactions
InnoDB: Cleaning up trx with id 0 20194242
071005 10:52:23  InnoDB: Rollback of non-prepared transactions completed
071005 10:52:23 [Note] /usr/local/software/platform/database/bin/mysqld: ready for connections.
Version: '5.0.46-enterprise-log'  socket: '/tmp/mysql.sock'  port: 3306  MySQL Enterprise Server (Commercial)
071005 10:52:23 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000051' at position 36927031, relay log '/db01/mydata/slave-relay-bin.000004' position: 36533130
071005 10:52:23InnoDB: Assertion failure in thread 1781595056 in file fsp0fsp.c line 1569
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html
InnoDB: about forcing recovery.
071005 10:52:23 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=67108864
read_buffer_size=1044480
max_used_connections=0
max_connections=250
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2368534 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0xc0807d8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x6a30dc48, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818a3dd
0x83383aa
0x83393a6
0x82e8390
0x82f4d60
0x82f405c
0x82c3f10
0x82c2d3d
0x82c35b4
0x82c310d
0x82c3354
0x82c7cbe
0x823bb54
0x81eb505
0x81ea96e
0x81a12ee
0x81a5cb3
0x8205026
0x8204c50
0x826e697
0x826c34b
0x98cbd4
0x8d14fe
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do 
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xc08746b = INSERT INTO QRTZ_FIRED_TRIGGERS (ENTRY_ID, TRIGGER_NAME, TRIGGER_GROUP, IS_VOLATILE, INSTANCE_NAME, FIRED_TIME, STATE, JOB_NAME, JOB_GROUP, IS_STATEFUL, REQUESTS_RECOVERY) VALUES('11191554845608', 'MT_ggsipm84hb113', 'MANUAL_TRIGGER', 0, '1', 1191561695331, 'EXECUTING', '2', '1787915', 1, 1)
thd->thread_id=2
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

Number of processes running now: 0