Description:
090715 14:50:12 InnoDB: Started; log sequence number 116 1948302284
090715 14:50:12 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--relay-log=db6-relay-bin' to avoid this problem.
090715 14:50:12 [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES','ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_Z
090715 14:50:12 [ERROR] Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.
090715 14:50:12 [Note] /usr/local/mysql/libexec/mysqld: ready for connections.
Version: '5.1.36' socket: '/mysql/3306/mysql.sock' port: 3306 Source distribution
090715 14:50:12 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000003' at position 178882101, relay log './db6-relay-bin.001263' position: 13864587
090715 14:50:12 [Note] Slave I/O thread: connected to master 'repl@dbmaster:3306',replication started in log 'mysql-bin.000003' at position 179093704
090715 14:50:12 InnoDB: Assertion failure in thread 1158261088 in file row/row0mysql.c line 1534
InnoDB: Failing assertion: index->type & DICT_CLUSTERED
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
090715 14:50:12 - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=16777216
read_buffer_size=1048576
max_used_connections=0
max_threads=1800
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5564293 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd: 0x60660a0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x45099e68 thread_stack 0x30000
/usr/local/mysql/libexec/mysqld(my_print_stacktrace+0x26) [0x865626]
/usr/local/mysql/libexec/mysqld(handle_segfault+0x324) [0x5c2104]
/lib64/tls/libpthread.so.0 [0x30a640c5b0]
/lib64/tls/libc.so.6(gsignal+0x3d) [0x30a5d2e25d]
/lib64/tls/libc.so.6(abort+0xfe) [0x30a5d2fa5e]
/usr/local/mysql/libexec/mysqld(row_unlock_for_mysql+0x469) [0x7bc629]
/usr/local/mysql/libexec/mysqld(row_search_for_mysql+0x2b50) [0x7ca720]
/usr/local/mysql/libexec/mysqld(ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function)+0x128) [0x73e6e8]
/usr/local/mysql/libexec/mysqld(handler::read_range_first(st_key_range const*, st_key_range const*, bool, bool)+0xb7) [0x6b0697]
/usr/local/mysql/libexec/mysqld(handler::read_multi_range_first(st_key_multi_range**, st_key_multi_range*, unsigned int, bool, st_handler_buffer*)+0xa7) [0x6b03a7]
/usr/local/mysql/libexec/mysqld(QUICK_RANGE_SELECT::get_next()+0x129) [0x69f089]
/usr/local/mysql/libexec/mysqld [0x6a7abd]
/usr/local/mysql/libexec/mysqld(mysql_update(THD*, TABLE_LIST*, List<Item>&, List<Item>&, Item*, unsigned int, st_order*, unsigned long long, enum_duplicates, bool)+0xfd2) [0x653fc2]
/usr/local/mysql/libexec/mysqld(mysql_execute_command(THD*)+0x2dcd) [0x5d45ed]
/usr/local/mysql/libexec/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x171) [0x5d9cc1]
/usr/local/mysql/libexec/mysqld(Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int)+0x339) [0x685a99]
/usr/local/mysql/libexec/mysqld(apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*, bool)+0xf4) [0x6e7d44]
/usr/local/mysql/libexec/mysqld(handle_slave_sql+0x8ea) [0x6e8f8a]
/lib64/tls/libpthread.so.0 [0x30a6406137]
/lib64/tls/libc.so.6(__clone+0x73) [0x30a5dc7533]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x605baaa = update house_image set source_id = 753873 where source_id = 1247637589260
thd->thread_id=2
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
090715 14:50:12 mysqld_safe Number of processes running now: 0
090715 14:50:12 mysqld_safe mysqld restarted
InnoDB: Log scan progressed past the checkpoint lsn 116 1938104672
090715 14:50:13 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 116 1943347200
InnoDB: Doing recovery: scanned up to log sequence number 116 1948302284
090715 14:50:15 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
Is this a setting issue, or a bug?
the same table in anothers slave servers using myisam engine worked well.
How to repeat:
1. upgrade slave server from 5.1.26 to 5.1.36
2. upgrase master server from 5.1.26 to 5.1.36
3. slave server #start slave
4. using innodb engine slave servers crashed
Description: 090715 14:50:12 InnoDB: Started; log sequence number 116 1948302284 090715 14:50:12 [Warning] Neither --relay-log nor --relay-log-index were used; so replication may break when this MySQL server acts as a slave and has his hostname changed!! Please use '--relay-log=db6-relay-bin' to avoid this problem. 090715 14:50:12 [ERROR] Incorrect definition of table mysql.event: expected column 'sql_mode' at position 14 to have type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_ZERO_IN_DATE','NO_ZERO_DATE','INVALID_DATES','ERROR_FOR_DIVISION_BY_ZERO','TRADITIONAL','NO_AUTO_CREATE_USER','HIGH_NOT_PRECEDENCE','NO_ENGINE_SUBSTITUTION','PAD_CHAR_TO_FULL_LENGTH'), found type set('REAL_AS_FLOAT','PIPES_AS_CONCAT','ANSI_QUOTES','IGNORE_SPACE','NOT_USED','ONLY_FULL_GROUP_BY','NO_UNSIGNED_SUBTRACTION','NO_DIR_IN_CREATE','POSTGRESQL','ORACLE','MSSQL','DB2','MAXDB','NO_KEY_OPTIONS','NO_TABLE_OPTIONS','NO_FIELD_OPTIONS','MYSQL323','MYSQL40','ANSI','NO_AUTO_VALUE_ON_ZERO','NO_BACKSLASH_ESCAPES','STRICT_TRANS_TABLES','STRICT_ALL_TABLES','NO_Z 090715 14:50:12 [ERROR] Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler. 090715 14:50:12 [Note] /usr/local/mysql/libexec/mysqld: ready for connections. Version: '5.1.36' socket: '/mysql/3306/mysql.sock' port: 3306 Source distribution 090715 14:50:12 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000003' at position 178882101, relay log './db6-relay-bin.001263' position: 13864587 090715 14:50:12 [Note] Slave I/O thread: connected to master 'repl@dbmaster:3306',replication started in log 'mysql-bin.000003' at position 179093704 090715 14:50:12 InnoDB: Assertion failure in thread 1158261088 in file row/row0mysql.c line 1534 InnoDB: Failing assertion: index->type & DICT_CLUSTERED InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html InnoDB: about forcing recovery. 090715 14:50:12 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=16777216 read_buffer_size=1048576 max_used_connections=0 max_threads=1800 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 5564293 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd: 0x60660a0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x45099e68 thread_stack 0x30000 /usr/local/mysql/libexec/mysqld(my_print_stacktrace+0x26) [0x865626] /usr/local/mysql/libexec/mysqld(handle_segfault+0x324) [0x5c2104] /lib64/tls/libpthread.so.0 [0x30a640c5b0] /lib64/tls/libc.so.6(gsignal+0x3d) [0x30a5d2e25d] /lib64/tls/libc.so.6(abort+0xfe) [0x30a5d2fa5e] /usr/local/mysql/libexec/mysqld(row_unlock_for_mysql+0x469) [0x7bc629] /usr/local/mysql/libexec/mysqld(row_search_for_mysql+0x2b50) [0x7ca720] /usr/local/mysql/libexec/mysqld(ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function)+0x128) [0x73e6e8] /usr/local/mysql/libexec/mysqld(handler::read_range_first(st_key_range const*, st_key_range const*, bool, bool)+0xb7) [0x6b0697] /usr/local/mysql/libexec/mysqld(handler::read_multi_range_first(st_key_multi_range**, st_key_multi_range*, unsigned int, bool, st_handler_buffer*)+0xa7) [0x6b03a7] /usr/local/mysql/libexec/mysqld(QUICK_RANGE_SELECT::get_next()+0x129) [0x69f089] /usr/local/mysql/libexec/mysqld [0x6a7abd] /usr/local/mysql/libexec/mysqld(mysql_update(THD*, TABLE_LIST*, List<Item>&, List<Item>&, Item*, unsigned int, st_order*, unsigned long long, enum_duplicates, bool)+0xfd2) [0x653fc2] /usr/local/mysql/libexec/mysqld(mysql_execute_command(THD*)+0x2dcd) [0x5d45ed] /usr/local/mysql/libexec/mysqld(mysql_parse(THD*, char const*, unsigned int, char const**)+0x171) [0x5d9cc1] /usr/local/mysql/libexec/mysqld(Query_log_event::do_apply_event(Relay_log_info const*, char const*, unsigned int)+0x339) [0x685a99] /usr/local/mysql/libexec/mysqld(apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*, bool)+0xf4) [0x6e7d44] /usr/local/mysql/libexec/mysqld(handle_slave_sql+0x8ea) [0x6e8f8a] /lib64/tls/libpthread.so.0 [0x30a6406137] /lib64/tls/libc.so.6(__clone+0x73) [0x30a5dc7533] Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at 0x605baaa = update house_image set source_id = 753873 where source_id = 1247637589260 thd->thread_id=2 thd->killed=NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 090715 14:50:12 mysqld_safe Number of processes running now: 0 090715 14:50:12 mysqld_safe mysqld restarted InnoDB: Log scan progressed past the checkpoint lsn 116 1938104672 090715 14:50:13 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 116 1943347200 InnoDB: Doing recovery: scanned up to log sequence number 116 1948302284 090715 14:50:15 InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed Is this a setting issue, or a bug? the same table in anothers slave servers using myisam engine worked well. How to repeat: 1. upgrade slave server from 5.1.26 to 5.1.36 2. upgrase master server from 5.1.26 to 5.1.36 3. slave server #start slave 4. using innodb engine slave servers crashed