100707  8:00:47 [Note] /opt/mysql/bin/mysqld: Normal shutdown

100707  8:00:47 [Note] Event Scheduler: Purging the queue. 0 events
100707  8:00:49  InnoDB: Starting shutdown...
100707  8:00:51  InnoDB: Shutdown completed; log sequence number 32 1051951320
100707  8:00:51 [Note] /opt/mysql/bin/mysqld: Shutdown complete

100707 08:00:51 mysqld_safe mysqld from pid file /var/lib/mysql/mysql.pid ended
100707 08:21:43 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100707  8:21:43 [Note] Plugin 'FEDERATED' is disabled.
100707  8:21:43 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
100707  8:21:45  InnoDB: Started; log sequence number 32 1051951320
100707  8:21:45 [Note] Event Scheduler: Loaded 0 events
100707  8:21:45 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707  8:26:40  InnoDB: Assertion failure in thread 1097144640 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707  8:26:40 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=3
max_threads=500
threads_connected=3
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x14ab4750
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41651118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function)+0x2c9) [0x8392c5]
/opt/mysql/bin/mysqld [0x74c6ac]
/opt/mysql/bin/mysqld(handler::index_read_idx_map(unsigned char*, unsigned int, unsigned char const*, unsigned long, ha_rkey_function)+0xca) [0x741f9c]
/opt/mysql/bin/mysqld [0x67bec2]
/opt/mysql/bin/mysqld [0x6788a3]
/opt/mysql/bin/mysqld(JOIN::optimize()+0x486) [0x671826]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x98) [0x66f7e8]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x14abf040 = SELECT `XmlContent` FROM `Application` WHERE systemid = 2083
thd->thread_id=10
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 08:26:40 mysqld_safe Number of processes running now: 0
100707 08:26:40 mysqld_safe mysqld restarted
100707  8:26:40 [Note] Plugin 'FEDERATED' is disabled.
100707  8:26:40 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707  8:26:41  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707  8:26:42  InnoDB: Started; log sequence number 32 1051951330
100707  8:26:42 [Note] Recovering after a crash using mysql-bin
100707  8:26:42 [Note] Starting crash recovery...
100707  8:26:42 [Note] Crash recovery finished.
100707  8:26:42 [Note] Event Scheduler: Loaded 0 events
100707  8:26:42 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707  9:18:30 [Note] /opt/mysql/bin/mysqld: Normal shutdown

100707  9:18:30 [Note] Event Scheduler: Purging the queue. 0 events
100707  9:18:32  InnoDB: Starting shutdown...
100707  9:18:35  InnoDB: Shutdown completed; log sequence number 32 1051951340
100707  9:18:35 [Note] /opt/mysql/bin/mysqld: Shutdown complete

100707 09:18:35 mysqld_safe mysqld from pid file /var/lib/mysql/mysql.pid ended
100707 12:21:31 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100707 12:21:31 [Note] Plugin 'FEDERATED' is disabled.
100707 12:21:31 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
100707 12:21:33  InnoDB: Started; log sequence number 32 1051951340
100707 12:21:33 [Note] Event Scheduler: Loaded 0 events
100707 12:21:33 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 12:41:24  InnoDB: Assertion failure in thread 1178569024 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 12:41:24 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x14093020
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x463f8118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1409d910 = select * from Application
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 12:41:24 mysqld_safe Number of processes running now: 0
100707 12:41:24 mysqld_safe mysqld restarted
100707 12:41:24 [Note] Plugin 'FEDERATED' is disabled.
100707 12:41:24 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
100707 12:41:26  InnoDB: Started; log sequence number 32 1051951340
100707 12:41:26 [Note] Recovering after a crash using mysql-bin
100707 12:41:26 [Note] Starting crash recovery...
100707 12:41:26 [Note] Crash recovery finished.
100707 12:41:26 [Note] Event Scheduler: Loaded 0 events
100707 12:41:26 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:10:14  InnoDB: Assertion failure in thread 1168132416 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:10:14 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=3
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x11858020
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45a04118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x11862910 = select * from Application limit 2000, 100
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:10:14 mysqld_safe Number of processes running now: 0
100707 13:10:14 mysqld_safe mysqld restarted
100707 13:10:14 [Note] Plugin 'FEDERATED' is disabled.
100707 13:10:14 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:10:15  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:10:16  InnoDB: Started; log sequence number 32 1051951350
100707 13:10:16 [Note] Recovering after a crash using mysql-bin
100707 13:10:16 [Note] Starting crash recovery...
100707 13:10:16 [Note] Crash recovery finished.
100707 13:10:16 [Note] Event Scheduler: Loaded 0 events
100707 13:10:16 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:12:15  InnoDB: Assertion failure in thread 1101547840 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:12:15 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x212d1cc0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41a84118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x2134e000 = select * from Application limit 2080, 20
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:12:15 mysqld_safe Number of processes running now: 0
100707 13:12:15 mysqld_safe mysqld restarted
100707 13:12:15 [Note] Plugin 'FEDERATED' is disabled.
100707 13:12:15 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:12:16  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:12:17  InnoDB: Started; log sequence number 32 1051951360
100707 13:12:17 [Note] Recovering after a crash using mysql-bin
100707 13:12:17 [Note] Starting crash recovery...
100707 13:12:17 [Note] Crash recovery finished.
100707 13:12:17 [Note] Event Scheduler: Loaded 0 events
100707 13:12:17 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:12:50  InnoDB: Assertion failure in thread 1098213696 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:12:50 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=1
max_threads=500
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x12327cc0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41756118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x123a4000 = select * from Application limit 2080, 10
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:12:50 mysqld_safe Number of processes running now: 0
100707 13:12:50 mysqld_safe mysqld restarted
100707 13:12:50 [Note] Plugin 'FEDERATED' is disabled.
100707 13:12:50 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:12:51  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:12:52  InnoDB: Started; log sequence number 32 1051951360
100707 13:12:52 [Note] Recovering after a crash using mysql-bin
100707 13:12:52 [Note] Starting crash recovery...
100707 13:12:52 [Note] Crash recovery finished.
100707 13:12:52 [Note] Event Scheduler: Loaded 0 events
100707 13:12:52 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:14:18  InnoDB: Assertion failure in thread 1097189696 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:14:18 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x1a43d210
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x4165c118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1a447b00 = select * from Application limit 2081, 1
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:14:18 mysqld_safe Number of processes running now: 0
100707 13:14:18 mysqld_safe mysqld restarted
100707 13:14:18 [Note] Plugin 'FEDERATED' is disabled.
100707 13:14:18 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:14:19  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:14:20  InnoDB: Started; log sequence number 32 1051951360
100707 13:14:20 [Note] Recovering after a crash using mysql-bin
100707 13:14:20 [Note] Starting crash recovery...
100707 13:14:20 [Note] Crash recovery finished.
100707 13:14:20 [Note] Event Scheduler: Loaded 0 events
100707 13:14:20 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:14:53  InnoDB: Assertion failure in thread 1103255872 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:14:53 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=1
max_threads=500
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x26f77cc0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41c25118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x26ff4000 = select * from Application limit 2080, 1
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:14:53 mysqld_safe Number of processes running now: 0
100707 13:14:53 mysqld_safe mysqld restarted
100707 13:14:53 [Note] Plugin 'FEDERATED' is disabled.
100707 13:14:53 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:14:54  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:14:55  InnoDB: Started; log sequence number 32 1051951360
100707 13:14:55 [Note] Recovering after a crash using mysql-bin
100707 13:14:55 [Note] Starting crash recovery...
100707 13:14:55 [Note] Crash recovery finished.
100707 13:14:55 [Note] Event Scheduler: Loaded 0 events
100707 13:14:55 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:15:52  InnoDB: Assertion failure in thread 1166944576 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:15:52 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0xe650210
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x458e2118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xe65ab00 = select * from Application limit 2080, 1
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:15:52 mysqld_safe Number of processes running now: 0
100707 13:15:52 mysqld_safe mysqld restarted
100707 13:15:52 [Note] Plugin 'FEDERATED' is disabled.
100707 13:15:52 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:15:53  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:15:54  InnoDB: Started; log sequence number 32 1051951360
100707 13:15:54 [Note] Recovering after a crash using mysql-bin
100707 13:15:54 [Note] Starting crash recovery...
100707 13:15:54 [Note] Crash recovery finished.
100707 13:15:54 [Note] Event Scheduler: Loaded 0 events
100707 13:15:54 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:17:16  InnoDB: Assertion failure in thread 1175898432 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:17:16 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x1d186210
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x4616c118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1d190b00 = select * from Application limit 2090, 1
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:17:17 mysqld_safe Number of processes running now: 0
100707 13:17:17 mysqld_safe mysqld restarted
100707 13:17:17 [Note] Plugin 'FEDERATED' is disabled.
100707 13:17:17 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:17:18  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:17:19  InnoDB: Started; log sequence number 32 1051951370
100707 13:17:19 [Note] Recovering after a crash using mysql-bin
100707 13:17:19 [Note] Starting crash recovery...
100707 13:17:19 [Note] Crash recovery finished.
100707 13:17:19 [Note] Event Scheduler: Loaded 0 events
100707 13:17:19 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 13:35:40  InnoDB: Assertion failure in thread 1091168576 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 13:35:40 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x1a484210
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x4109e118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1a48eb00 = select * from Application limit 2080, 1
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 13:35:40 mysqld_safe Number of processes running now: 0
100707 13:35:40 mysqld_safe mysqld restarted
100707 13:35:40 [Note] Plugin 'FEDERATED' is disabled.
100707 13:35:40 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 13:35:41  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 13:35:42  InnoDB: Started; log sequence number 32 1051951380
100707 13:35:42 [Note] Recovering after a crash using mysql-bin
100707 13:35:42 [Note] Starting crash recovery...
100707 13:35:42 [Note] Crash recovery finished.
100707 13:35:42 [Note] Event Scheduler: Loaded 0 events
100707 13:35:42 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 14:05:06  InnoDB: Assertion failure in thread 1173285184 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 14:05:06 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x1db05a70
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45eee118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1dafb220 = select * from Application
thd->thread_id=3
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 14:05:06 mysqld_safe Number of processes running now: 0
100707 14:05:06 mysqld_safe mysqld restarted
100707 14:05:06 [Note] Plugin 'FEDERATED' is disabled.
100707 14:05:06 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 14:05:07  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 14:05:08  InnoDB: Started; log sequence number 32 1051951390
100707 14:05:08 [Note] Recovering after a crash using mysql-bin
100707 14:05:08 [Note] Starting crash recovery...
100707 14:05:08 [Note] Crash recovery finished.
100707 14:05:08 [Note] Event Scheduler: Loaded 0 events
100707 14:05:08 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 14:06:20  InnoDB: Assertion failure in thread 1090943296 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 14:06:20 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x1d74dce0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x41067118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x1d7ce0f0 = select * from Application limit 2080, 1
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 14:06:20 mysqld_safe Number of processes running now: 0
100707 14:06:20 mysqld_safe mysqld restarted
100707 14:06:20 [Note] Plugin 'FEDERATED' is disabled.
100707 14:06:20 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100707 14:06:21  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Last MySQL binlog file position 0 430778, file name ./mysql-bin.000101
100707 14:06:22  InnoDB: Started; log sequence number 32 1051951400
100707 14:06:22 [Note] Recovering after a crash using mysql-bin
100707 14:06:22 [Note] Starting crash recovery...
100707 14:06:22 [Note] Crash recovery finished.
100707 14:06:22 [Note] Event Scheduler: Loaded 0 events
100707 14:06:22 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 14:49:49 [Note] /opt/mysql/bin/mysqld: Normal shutdown

100707 14:49:49 [Note] Event Scheduler: Purging the queue. 0 events
100707 14:49:51  InnoDB: Starting shutdown...
100707 14:49:53  InnoDB: Shutdown completed; log sequence number 32 1051951410
100707 14:49:53 [Note] /opt/mysql/bin/mysqld: Shutdown complete

100707 14:49:53 mysqld_safe mysqld from pid file /var/lib/mysql/mysql.pid ended
100707 15:03:20 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
100707 15:03:20 [Note] Plugin 'FEDERATED' is disabled.
100707 15:03:20 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
100707 15:03:22  InnoDB: Started; log sequence number 32 1051951410
100707 15:03:22 [Note] Event Scheduler: Loaded 0 events
100707 15:03:22 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100707 15:04:09  InnoDB: Assertion failure in thread 1167542592 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100707 15:04:09 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=2
max_threads=500
threads_connected=2
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x228bfe30
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45974118 thread_stack 0x30000
/opt/mysql/bin/mysqld(my_print_stacktrace+0x20) [0xa051cc]
/opt/mysql/bin/mysqld(handle_segfault+0x368) [0x603740]
/lib64/libpthread.so.0 [0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2) [0x841d8a]
/opt/mysql/bin/mysqld [0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a) [0x924bf4]
/opt/mysql/bin/mysqld(ha_innobase::rnd_next(unsigned char*)+0x3d7) [0x839b19]
/opt/mysql/bin/mysqld(rr_sequential(st_read_record*)+0x1a) [0x73cd28]
/opt/mysql/bin/mysqld(sub_select(JOIN*, st_join_table*, bool)+0xbc) [0x690c26]
/opt/mysql/bin/mysqld [0x690779]
/opt/mysql/bin/mysqld(JOIN::exec()+0x1b05) [0x68b365]
/opt/mysql/bin/mysqld(mysql_select(THD*, Item***, TABLE_LIST*, unsigned int, List<Item>&, Item*, unsigned int, st_order*, st_order*, Item*, st_order*, unsigned long long, select_result*, st_select_lex_unit*, st_select_lex*)+0x18b) [0x66f8db]
/opt/mysql/bin/mysqld(handle_select(THD*, st_lex*, select_result*, unsigned long)+0x16c) [0x69813c]
/opt/mysql/bin/mysqld [0x619c68]
/opt/mysql/bin/mysqld(mysql_execute_command(THD*)+0x4b30) [0x613e6c]
/opt/mysql/bin/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x20a) [0x61a1fe]
/opt/mysql/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned int)+0x1350) [0x60e7e8]
/opt/mysql/bin/mysqld(do_command(THD*)+0x114) [0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34) [0x608850]
/lib64/libpthread.so.0 [0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d) [0x3ab76d3c2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x22918a20 = select * from Application
thd->thread_id=1
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100707 15:04:09 mysqld_safe Number of processes running now: 0
100707 15:04:09 mysqld_safe mysqld restarted
100707 15:04:09 [Note] Plugin 'FEDERATED' is disabled.
100707 15:04:09 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
100707 15:04:35  InnoDB: Started; log sequence number 32 1051951410
100707 15:04:35 [Note] Recovering after a crash using mysql-bin
100707 15:04:35 [Note] Starting crash recovery...
100707 15:04:35 [Note] Crash recovery finished.
100707 15:04:35 [Note] Event Scheduler: Loaded 0 events
100707 15:04:35 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)
100708 11:45:30  InnoDB: Assertion failure in thread 1167599936 in file btr/btr0cur.c line 3814
InnoDB: Failing assertion: copied_len < local_len + extern_len
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
100708 11:45:30 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8384512
read_buffer_size=131072
max_used_connections=3
max_threads=500
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 8269273 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x11d83e30
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45982118 thread_stack 0x30000
*** glibc detected *** /opt/mysql/bin/mysqld: malloc(): memory corruption: 0x0000000011e3da40 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3ab7672f39]
/lib64/libc.so.6(__libc_malloc+0x6e)[0x3ab7674bee]
/lib64/libc.so.6(__backtrace_symbols+0x104)[0x3ab76e6094]
/opt/mysql/bin/mysqld(my_print_stacktrace+0x48)[0xa051f4]
/opt/mysql/bin/mysqld(handle_segfault+0x368)[0x603740]
/lib64/libpthread.so.0[0x3ab820e7c0]
/opt/mysql/bin/mysqld(btr_rec_copy_externally_stored_field+0x3b2)[0x841d8a]
/opt/mysql/bin/mysqld[0x9278dc]
/opt/mysql/bin/mysqld(row_search_for_mysql+0x1f6a)[0x924bf4]
/opt/mysql/bin/mysqld(_ZN11ha_innobase8rnd_nextEPh+0x3d7)[0x839b19]
/opt/mysql/bin/mysqld(_Z13rr_sequentialP14st_read_record+0x1a)[0x73cd28]
/opt/mysql/bin/mysqld[0x76ab8e]
/opt/mysql/bin/mysqld(_Z17mysql_alter_tableP3THDPcS1_P24st_ha_create_informationP10TABLE_LISTP10Alter_infojP8st_orderb+0x159d)[0x762e63]
/opt/mysql/bin/mysqld[0x760f40]
/opt/mysql/bin/mysqld(_Z20mysql_optimize_tableP3THDP10TABLE_LISTP15st_ha_check_opt+0x4a)[0x76bfd6]
/opt/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0x2988)[0x611cc4]
/opt/mysql/bin/mysqld(_Z11mysql_parseP3THDPKcjPS2_+0x20a)[0x61a1fe]
/opt/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1350)[0x60e7e8]
/opt/mysql/bin/mysqld(_Z10do_commandP3THD+0x114)[0x60d494]
/opt/mysql/bin/mysqld(handle_one_connection+0xd34)[0x608850]
/lib64/libpthread.so.0[0x3ab82064a7]
/lib64/libc.so.6(clone+0x6d)[0x3ab76d3c2d]
======= Memory map: ========
00400000-00cb7000 r-xp 00000000 fd:00 46727208                           /opt/mysql-advanced-gpl-5.1.40-linux-x86_64-icc-glibc23/bin/mysqld
00db6000-00f47000 rwxp 008b6000 fd:00 46727208                           /opt/mysql-advanced-gpl-5.1.40-linux-x86_64-icc-glibc23/bin/mysqld
00f47000-00f5e000 rwxp 00f47000 00:00 0 
07fb3000-1482a000 rwxp 07fb3000 00:00 0                                  [heap]
408e3000-408e4000 ---p 408e3000 00:00 0 
408e4000-412e4000 rwxp 408e4000 00:00 0 
4131a000-4131b000 ---p 4131a000 00:00 0 
4131b000-41d1b000 rwxp 4131b000 00:00 0 
41d1b000-41d1c000 ---p 41d1b000 00:00 0 
41d1c000-4271c000 rwxp 41d1c000 00:00 0 
4271c000-4271d000 ---p 4271c000 00:00 0 
4271d000-4311d000 rwxp 4271d000 00:00 0 
4311d000-4311e000 ---p 4311d000 00:00 0 
4311e000-43b1e000 rwxp 4311e000 00:00 0 
43b1e000-43b1f000 ---p 43b1e000 00:00 0 
43b1f000-4451f000 rwxp 43b1f000 00:00 0 
4451f000-44520000 ---p 4451f000 00:00 0 
44520000-44f20000 rwxp 44520000 00:00 0 
44f20000-44f21000 ---p 44f20000 00:00 0 
44f21000-45921000 rwxp 44f21000 00:00 0 
45921000-45922000 ---p 45921000 00:00 0 
45922000-45952000 rwxp 45922000 00:00 0 
45952000-45953000 ---p 45952000 00:00 0 
45953000-45983000 rwxp 45953000 00:00 0 
45983000-45984000 ---p 45983000 00:00 0 
45984000-459b4000 rwxp 45984000 00:00 0 
459b4000-459b5000 ---p 459b4000 00:00 0 
459b5000-459e5000 rwxp 459b5000 00:00 0 
3830600000-383060d000 r-xp 00000000 fd:00 55705603                       /lib64/libgcc_s-4.1.2-20080825.so.1
383060d000-383080d000 ---p 0000d000 fd:00 55705603                       /lib64/libgcc_s-4.1.2-20080825.so.1
383080d000-383080e000 rwxp 0000d000 fd:00 55705603                       /lib64/libgcc_s-4.1.2-20080825.so.1
3ab7200000-3ab721c000 r-xp 00000000 fd:00 55705898                       /lib64/ld-2.5.so
3ab741b000-3ab741c000 r-xp 0001b000 fd:00 55705898                       /lib64/ld-2.5.so
3ab741c000-3ab741d000 rwxp 0001c000 fd:00 55705898                       /lib64/ld-2.5.so
3ab7600000-3ab774d000 r-xp 00000000 fd:00 55705899                       /lib64/libc-2.5.so
3ab774d000-3ab794d000 ---p 0014d000 fd:00 55705899                       /lib64/libc-2.5.so
3ab794d000-3ab7951000 r-xp 0014d000 fd:00 55705899                       /lib64/libc-2.5.so
3ab7951000-3ab7952000 rwxp 00151000 fd:00 55705899                       /lib64/libc-2.5.so
3ab7952000-3ab7957000 rwxp 3ab7952000 00:00 0 
3ab7a00000-3ab7a02000 r-xp 00000000 fd:00 55705900                       /lib64/libdl-2.5.so
3ab7a02000-3ab7c02000 ---p 00002000 fd:00 55705900                       /lib64/libdl-2.5.so
3ab7c02000-3ab7c03000 r-xp 00002000 fd:00 55705900                       /lib64/libdl-2.5.so
3ab7c03000-3ab7c04000 rwxp 00003000 fd:00 55705900                       /lib64/libdl-2.5.so
3ab7e00000-3ab7e82000 r-xp 00000000 fd:00 55705885                       /lib64/libm-2.5.so
3ab7e82000-3ab8081000 ---p 00082000 fd:00 55705885                       /lib64/libm-2.5.so
3ab8081000-3ab8082000 r-xp 00081000 fd:00 55705885                       /lib64/libm-2.5.so
3ab8082000-3ab8083000 rwxp 00082000 fd:00 55705885                       /lib64/libm-2.5.so
3ab8200000-3ab8216000 r-xp 00000000 fd:00 55705905                       /lib64/libpthread-2.5.so
3ab8216000-3ab8415000 ---p 00016000 fd:00 55705905                       /lib64/libpthread-2.5.so
3ab8415000-3ab8416000 r-xp 00015000 fd:00 55705905                       /lib64/libpthread-2.5.so
3ab8416000-3ab8417000 rwxp 00016000 fd:00 55705905                       /lib64/libpthread-2.5.so
3ab8417000-3ab841b000 rwxp 3ab8417000 00:00 0 
3abae00000-3abae15000 r-xp 00000000 fd:00 55705922                       /lib64/libnsl-2.5.so
3abae15000-3abb014000 ---p 00015000 fd:00 55705922                       /lib64/libnsl-2.5.so
3abb014000-3abb015000 r-xp 00014000 fd:00 55705922                       /lib64/libnsl-2.5.so
3abb015000-3abb016000 rwxp 00015000 fd:00 55705922                       /lib64/libnsl-2.5.so
3abb016000-3abb018000 rwxp 3abb016000 00:00 0 
3ac9c00000-3ac9c09000 r-xp 00000000 fd:00 55705915                       /lib64/libcrypt-2.5.so
3ac9c09000-3ac9e08000 ---p 00009000 fd:00 55705915                       /lib64/libcrypt-2.5.so
3ac9e08000-3ac9e09000 r-xp 00008000 fd:00 55705915                       /lib64/libcrypt-2.5.so
3ac9e09000-3ac9e0a000 rwxp 00009000 fd:00 55705915                       /lib64/libcrypt-2.5.so
3ac9e0a000-3ac9e38000 rwxp 3ac9e0a000 00:00 0 
2aaaaaaae000-2aaaaaacf000 rwxp 2aaaaaaae000 00:00 0 
2aaaaaae0000-2aaaaaaea000 r-xp 00000000 fd:00 55705627                   /lib64/libnss_files-2.5.so
2aaaaaaea000-2aaaaace9000 ---p 0000a000 fd:00 55705627                   /lib64/libnss_files-2.5.so
2aaaaace9000-2aaaaacea000 r-xp 00009000 fd:00 55705627                   /lib64/libnss_files-2.5.so
2aaaaacea000-2aaaaaceb000 rwxp 0000a000 fd:00 55705627                   /lib64/libnss_files-2.5.so
2aaaaaceb000-2aabc0728000 rwxp 2aaaaaceb000 00:00 0 
2aabc4000000-2aabc48ee000 rwxp 2aabc4000000 00:00 0 
2aabc48ee000-2aabc8000000 ---p 2aabc48ee000 00:00 0 
2aabe2559000-2aabe3841000 rwxp 2aabe2559000 00:00 0 
2ab2e2b28000-2ab2e2b29000 rwxp 2ab2e2b28000 00:00 0 
2ab2e2b3a000-2ab2e2b3c000 rwxp 2ab2e2b3a000 00:00 0 
2ab2e2b3c000-2ab2e2bfd000 r-xp 00000000 fd:00 2497238                    /usr/lib64/libstdc++.so.5.0.7
2ab2e2bfd000-2ab2e2dfd000 ---p 000c1000 fd:00 2497238                    /usr/lib64/libstdc++.so.5.0.7
2ab2e2dfd000-2ab2e2e06000 rwxp 000c1000 fd:00 2497238                    /usr/lib64/libstdc++.so.5.0.7
2ab2e2e06000-2ab2e2e1a000 rwxp 2ab2e2e06000 00:00 0 
7fffbeb0b000-7fffbeb20000 rwxp 7ffffffea000 00:00 0                      [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0                  [vdso]
Fatal signal 6 while backtracing
100708 11:45:31 mysqld_safe Number of processes running now: 0
100708 11:45:31 mysqld_safe mysqld restarted
100708 11:45:31 [Note] Plugin 'FEDERATED' is disabled.
100708 11:45:31 [Warning] option 'innodb-autoextend-increment': unsigned value 104857600 adjusted to 1000
InnoDB: Log scan progressed past the checkpoint lsn 32 1051952088
100708 11:45:32  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 32 1057194496
InnoDB: Doing recovery: scanned up to log sequence number 32 1062437376
InnoDB: Doing recovery: scanned up to log sequence number 32 1067680256
InnoDB: Doing recovery: scanned up to log sequence number 32 1072923136
InnoDB: Doing recovery: scanned up to log sequence number 32 1078166016
InnoDB: Doing recovery: scanned up to log sequence number 32 1083408896
InnoDB: Doing recovery: scanned up to log sequence number 32 1088651776
InnoDB: Doing recovery: scanned up to log sequence number 32 1093894656
InnoDB: Doing recovery: scanned up to log sequence number 32 1099137536
InnoDB: Doing recovery: scanned up to log sequence number 32 1104380416
InnoDB: Doing recovery: scanned up to log sequence number 32 1109623296
InnoDB: Doing recovery: scanned up to log sequence number 32 1114866176
InnoDB: Doing recovery: scanned up to log sequence number 32 1120109056
InnoDB: Doing recovery: scanned up to log sequence number 32 1125351936
InnoDB: Doing recovery: scanned up to log sequence number 32 1130594816
InnoDB: Doing recovery: scanned up to log sequence number 32 1135837696
InnoDB: Doing recovery: scanned up to log sequence number 32 1141080576
InnoDB: Doing recovery: scanned up to log sequence number 32 1146323456
InnoDB: Doing recovery: scanned up to log sequence number 32 1151566336
InnoDB: Doing recovery: scanned up to log sequence number 32 1156809216
InnoDB: Doing recovery: scanned up to log sequence number 32 1162052096
InnoDB: Doing recovery: scanned up to log sequence number 32 1167294976
InnoDB: Doing recovery: scanned up to log sequence number 32 1172537856
InnoDB: Doing recovery: scanned up to log sequence number 32 1177780736
InnoDB: Doing recovery: scanned up to log sequence number 32 1183023616
InnoDB: Doing recovery: scanned up to log sequence number 32 1188266496
InnoDB: Doing recovery: scanned up to log sequence number 32 1193509376
InnoDB: Doing recovery: scanned up to log sequence number 32 1198752256
InnoDB: Doing recovery: scanned up to log sequence number 32 1203995136
InnoDB: Doing recovery: scanned up to log sequence number 32 1209238016
InnoDB: Doing recovery: scanned up to log sequence number 32 1214480896
InnoDB: Doing recovery: scanned up to log sequence number 32 1219723776
InnoDB: Doing recovery: scanned up to log sequence number 32 1224966656
InnoDB: Doing recovery: scanned up to log sequence number 32 1227966313
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 2072 row operations to undo
InnoDB: Trx id counter is 0 59046400
100708 11:45:36  InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 2881, file name ./mysql-bin.000117
InnoDB: Starting in background the rollback of uncommitted transactions
100708 11:47:33  InnoDB: Rolling back trx with id 0 59046011, 2072 rows to undo

InnoDB: Progress in percents: 1100708 11:47:33  InnoDB: Started; log sequence number 32 1227966313
100708 11:47:33 [Note] Recovering after a crash using mysql-bin
100708 11:47:33 [Note] Starting crash recovery...
100708 11:47:33 [Note] Crash recovery finished.
 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100
InnoDB: Rolling back of trx id 0 59046011 completed
100708 11:47:33  InnoDB: Rollback of non-prepared transactions completed
100708 11:47:34 [Note] Event Scheduler: Loaded 0 events
100708 11:47:34 [Note] /opt/mysql/bin/mysqld: ready for connections.
Version: '5.1.40-enterprise-gpl-advanced-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Enterprise Server - Advanced Edition (GPL)