2018-07-29 12:06:34 0x7f241a655700 InnoDB: Assertion failure in thread 139793038399232 in file trx0trx.ic line 213 InnoDB: Failing assertion: !trx->has_search_latch InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 12:06:34 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. key_buffer_size=104857600 read_buffer_size=131072 max_used_connections=650 max_threads=1500 thread_count=25 connection_count=22 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 698439 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f23140c1cd0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f241a654e30 thread_stack 0x40000 /usr/sbin/mysqld(my_print_stacktrace+0x3b)[0xefd8bb] /usr/sbin/mysqld(handle_fatal_signal+0x461)[0x7b14c1] /lib64/libpthread.so.0(+0xf6d0)[0x7f2ebc0356d0] /lib64/libc.so.6(gsignal+0x37)[0x7f2ebaa1e277] /lib64/libc.so.6(abort+0x148)[0x7f2ebaa1f968] /usr/sbin/mysqld[0x781aa4] /usr/sbin/mysqld[0x777b6c] /usr/sbin/mysqld[0xf38e15] /usr/sbin/mysqld(close_thread_table(THD*, TABLE**)+0x27a)[0xc6dc2a] /usr/sbin/mysqld(close_thread_tables(THD*)+0x33b)[0xc6e2db] /usr/sbin/mysqld(Prepared_statement::prepare(char const*, unsigned long)+0x72b)[0xcf652b] /usr/sbin/mysqld(mysql_sql_stmt_prepare(THD*)+0x3a1)[0xcf7a41] /usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0x199d)[0xcc7dad] /usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x50)[0xc4d5f0] /usr/sbin/mysqld(sp_lex_instr::reset_lex_and_exec_core(THD*, unsigned int*, bool)+0x454)[0xc4f1c4] /usr/sbin/mysqld(sp_lex_instr::validate_lex_and_execute_core(THD*, unsigned int*, bool)+0xbb)[0xc4fb5b] /usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0x128)[0xc50d08] /usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x4f4)[0xc48d04] /usr/sbin/mysqld(sp_head::execute_procedure(THD*, List*)+0x777)[0xc4c867] /usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0x1c42)[0xcc8052] /usr/sbin/mysqld(mysql_parse(THD*, Parser_state*)+0x3ad)[0xcccdad] /usr/sbin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0xa7d)[0xccd90d] /usr/sbin/mysqld(do_command(THD*)+0x19f)[0xccf2ff] /usr/sbin/mysqld(handle_connection+0x288)[0xd8eae8] /usr/sbin/mysqld(pfs_spawn_thread+0x1b4)[0x1273654] /lib64/libpthread.so.0(+0x7e25)[0x7f2ebc02de25] /lib64/libc.so.6(clone+0x6d)[0x7f2ebaae6bad] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7f2314a70210): [redacted] Connection ID (thread ID): 132831777 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 2018-07-29T12:06:54.401589Z 0 [Warning] The syntax '--log_warnings/-W' is deprecated and will be removed in a future release. Please use '--log_error_verbosity' instead. 2018-07-29T12:06:54.419853Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.21-log) starting as process 10989 ... 2018-07-29T12:06:54.454207Z 0 [Note] InnoDB: PUNCH HOLE support available 2018-07-29T12:06:54.454283Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-07-29T12:06:54.454311Z 0 [Note] InnoDB: Uses event mutexes 2018-07-29T12:06:54.454320Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2018-07-29T12:06:54.454329Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3 2018-07-29T12:06:54.454340Z 0 [Note] InnoDB: Using Linux native AIO 2018-07-29T12:06:54.455796Z 0 [Note] InnoDB: Number of pools: 1 2018-07-29T12:06:54.456004Z 0 [Note] InnoDB: Not using CPU crc32 instructions 2018-07-29T12:06:54.459415Z 0 [Note] InnoDB: Initializing buffer pool, total size = 40G, instances = 8, chunk size = 128M 2018-07-29T12:06:58.609454Z 0 [Note] InnoDB: Completed initialization of buffer pool 2018-07-29T12:06:59.032412Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2018-07-29T12:06:59.049133Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2018-07-29T12:06:59.326492Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 10490107919709 2018-07-29T12:07:00.823548Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490113162240 2018-07-29T12:07:01.518904Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490118405120 2018-07-29T12:07:02.067879Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490123648000 2018-07-29T12:07:02.705089Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490128890880 2018-07-29T12:07:03.239903Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490134133760 2018-07-29T12:07:03.578357Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490139376640 2018-07-29T12:07:03.943284Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490144619520 2018-07-29T12:07:04.282333Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490149862400 2018-07-29T12:07:04.588314Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490155105280 2018-07-29T12:07:04.951946Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490160348160 2018-07-29T12:07:05.092327Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 10490161966131 2018-07-29T12:07:05.096425Z 0 [Note] InnoDB: Database was not shutdown normally! 2018-07-29T12:07:05.096461Z 0 [Note] InnoDB: Starting crash recovery. 2018-07-29T12:07:05.762574Z 0 [Note] InnoDB: Transaction 40308459058 was in the XA prepared state. 2018-07-29T12:07:05.847542Z 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo 2018-07-29T12:07:05.847633Z 0 [Note] InnoDB: Trx id counter is 40308459520 2018-07-29T12:07:05.847705Z 0 [Note] InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 2018-07-29T12:07:28.532792Z 0 [Note] InnoDB: Apply batch completed 2018-07-29T12:07:28.532896Z 0 [Note] InnoDB: Last MySQL binlog file position 0 103872326, file name db31b-bin.036723 2018-07-29T12:08:00.213805Z 0 [Note] InnoDB: Starting in background the rollback of uncommitted transactions 2018-07-29T12:08:00.213919Z 0 [Note] InnoDB: Rollback of non-prepared transactions completed 2018-07-29T12:08:00.535175Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1" 2018-07-29T12:08:00.535225Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2018-07-29T12:08:00.535535Z 0 [Note] InnoDB: Setting file '/var/lib/mysql/innodb-data/ibtmp1' size to 12 MB. Physically writing the file full; Please wait ... 2018-07-29T12:08:00.645871Z 0 [Note] InnoDB: File '/var/lib/mysql/innodb-data/ibtmp1' size is now 12 MB. 2018-07-29T12:08:00.650884Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. 2018-07-29T12:08:00.650969Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2018-07-29T12:08:00.661956Z 0 [Note] InnoDB: Waiting for purge to start 2018-07-29T12:08:00.712087Z 0 [Note] InnoDB: 5.7.21 started; log sequence number 10490161966131 2018-07-29T12:08:00.712176Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 61680ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.) 2018-07-29T12:08:00.713557Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/innodb-data/ib_buffer_pool 2018-07-29T12:08:00.714028Z 0 [Note] Plugin 'FEDERATED' is disabled. 2018-07-29T12:08:00.772547Z 0 [Note] Recovering after a crash using /var/lib/mysql/bin-logs/db31b-bin 2018-07-29T12:08:01.769336Z 0 [Note] Starting crash recovery... 2018-07-29T12:08:01.769430Z 0 [Note] InnoDB: Starting recovery for XA transactions... 2018-07-29T12:08:01.769443Z 0 [Note] InnoDB: Transaction 40308459058 in prepared state after recovery 2018-07-29T12:08:01.769451Z 0 [Note] InnoDB: Transaction contains changes to 1 rows 2018-07-29T12:08:01.769458Z 0 [Note] InnoDB: 1 transactions in prepared state after recovery 2018-07-29T12:08:01.769464Z 0 [Note] Found 1 prepared transaction(s) in InnoDB 2018-07-29T12:08:01.771322Z 0 [Note] Crash recovery finished. 2018-07-29T12:08:02.275856Z 0 [Warning] Failed to set up SSL because of the following SSL library error: SSL_CTX_set_default_verify_paths failed 2018-07-29T12:08:02.275942Z 0 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 2018-07-29T12:08:02.276079Z 0 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 2018-07-29T12:08:02.276856Z 0 [Note] Server socket created on IP: '0.0.0.0'. 2018-07-29T12:08:02.346087Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/user' is marked as crashed and should be repaired 2018-07-29T12:08:02.346759Z 0 [Warning] Checking table: './mysql/user' 2018-07-29T12:08:02.346827Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:02.361559Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/db' is marked as crashed and should be repaired 2018-07-29T12:08:02.362072Z 0 [Warning] Checking table: './mysql/db' 2018-07-29T12:08:02.362113Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:02.855879Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/tables_priv' is marked as crashed and should be repaired 2018-07-29T12:08:02.856406Z 0 [Warning] Checking table: './mysql/tables_priv' 2018-07-29T12:08:02.856463Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:03.025724Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/columns_priv' is marked as crashed and should be repaired 2018-07-29T12:08:03.026246Z 0 [Warning] Checking table: './mysql/columns_priv' 2018-07-29T12:08:03.026293Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:03.030327Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/procs_priv' is marked as crashed and should be repaired 2018-07-29T12:08:03.030823Z 0 [Warning] Checking table: './mysql/procs_priv' 2018-07-29T12:08:03.030865Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:03.365560Z 0 [ERROR] /usr/sbin/mysqld: Table './mysql/event' is marked as crashed and should be repaired 2018-07-29T12:08:03.366160Z 0 [Warning] Checking table: './mysql/event' 2018-07-29T12:08:03.366222Z 0 [ERROR] 1 client is using or hasn't closed the table properly 2018-07-29T12:08:03.372253Z 0 [Note] Event Scheduler: Loaded 2 events 2018-07-29T12:08:03.373209Z 1 [Note] Event Scheduler: scheduler thread started with id 1 2018-07-29T12:08:03.373566Z 0 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.7.21-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Server (GPL) 2018-07-29T12:08:03.409521Z 16 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:03.409591Z 18 [Warning] IP address '10.3.6.160' could not be resolved: Name or service not known 2018-07-29T12:08:03.409713Z 19 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:03.409728Z 22 [Warning] IP address '10.3.6.160' could not be resolved: Name or service not known 2018-07-29T12:08:03.410438Z 4 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:03.410452Z 3 [Warning] IP address '10.3.6.160' could not be resolved: Name or service not known 2018-07-29T12:08:03.410483Z 12 [Warning] IP address '10.3.6.160' could not be resolved: Name or service not known 2018-07-29T12:08:03.410877Z 13 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:03.410925Z 21 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:03.411001Z 20 [Warning] IP address '10.3.6.160' could not be resolved: Name or service not known 2018-07-29T12:08:03.410938Z 15 [Warning] IP address '10.3.6.159' could not be resolved: Name or service not known 2018-07-29T12:08:04.094394Z 21 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.094728Z 20 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.095081Z 19 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.095436Z 3 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.095817Z 24 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.096240Z 15 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.096573Z 18 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.096915Z 39 [ERROR] /usr/sbin/mysqld: Table './mysql/proc' is marked as crashed and should be repaired 2018-07-29T12:08:04.097628Z 21 [Warning] Checking table: './mysql/proc' 2018-07-29T12:08:44.255263Z 0 [Note] InnoDB: Buffer pool(s) load completed at 180729 12:08:44