| Bug #73322 | SEGFAULT while selecting from table with limit 1000 | ||
|---|---|---|---|
| Submitted: | 18 Jul 2014 8:31 | Modified: | 27 Sep 2016 13:15 |
| Reporter: | Shahriyar Rzayev | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: InnoDB storage engine | Severity: | S1 (Critical) |
| Version: | 5.6.19 | OS: | Linux (CentOS 6.5) |
| Assigned to: | CPU Architecture: | Any | |
[18 Jul 2014 8:32]
Shahriyar Rzayev
Same issue also with mysqldbexport from mysql-utilities:
[root@fabricsrv ~]# mysqldbexport --server=root@localhost --export=DATA --bulk-insert testing
# Source on localhost: ... connected.
Traceback (most recent call last):
File "/usr/bin/mysqldbexport", line 354, in <module>
export_databases(server_values, db_list, output_file, options)
File "/usr/lib/python2.6/site-packages/mysql/utilities/command/dbexport.py", line 1048, in export_databases
_export_data(source, server_values, db_list, output_file, options)
File "/usr/lib/python2.6/site-packages/mysql/utilities/command/dbexport.py", line 469, in _export_data
_export_table_data(source, table, output_file, options)
File "/usr/lib/python2.6/site-packages/mysql/utilities/command/dbexport.py", line 597, in _export_table_data
for data_rows in cur_table.retrieve_rows(retrieval_mode):
File "/usr/lib/python2.6/site-packages/mysql/utilities/common/table.py", line 999, in retrieve_rows
rows = cur.fetchall()
File "/usr/lib/python2.6/site-packages/mysql/connector/cursor.py", line 916, in fetchall
(rows, eof) = self._connection.get_rows()
File "/usr/lib/python2.6/site-packages/mysql/connector/connection.py", line 632, in get_rows
rows = self._protocol.read_text_result(self._socket, count)
File "/usr/lib/python2.6/site-packages/mysql/connector/protocol.py", line 301, in read_text_result
packet = sock.recv()
File "/usr/lib/python2.6/site-packages/mysql/connector/network.py", line 206, in recv_plain
raise errors.InterfaceError(errno=2013)
mysql.connector.errors.InterfaceError: 2013: Lost connection to MySQL server during query
From error log:
/lib64/libc.so.6(clone+0x6d)[0x35deee8b5d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f09ac02c140): SELECT * FROM `testing`.`wd_companies`
Connection ID (thread ID): 2
Status: NOT_KILLED
[18 Jul 2014 9:07]
Shahriyar Rzayev
bin/mysqld_safe: line 166: 27247 Segmentation fault (core dumped) nohup /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/var/lib/mysql --plugin-dir=/usr/local/mysql/lib/plugin --log-error=/var/lib/mysql/fabricsrv.err --open-files-limit=65535 --pid-file=/var/lib/mysql/fabricsrv.pid --socket=/var/lib/mysql/mysql.sock < /dev/null >> /var/lib/mysql/fabricsrv.err 2>&1 >> /var/lib/mysql/fabricsrv.err 2>&1 >> /var/lib/mysql/fabricsrv.err 2>&1 >> /var/lib/mysql/fabricsrv.err 2>&1 >> /var/lib/mysql/fabricsrv.err 2>&1 140718 13:33:05 mysqld_safe Number of processes running now: 0 140718 13:33:05 mysqld_safe mysqld restarted
[18 Jul 2014 10:19]
Shahriyar Rzayev
I dropped table wd_companies and there was another error for table wd_users:
From new core dump:
(gdb) bt
#0 0x00000035df20c8ac in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000ab6f2f in my_write_core (sig=11) at /root/mysql-5.6.19/mysys/stacktrace.c:422
#2 0x0000000000737107 in handle_fatal_signal (sig=11) at /root/mysql-5.6.19/sql/signal_handler.cc:248
#3 <signal handler called>
#4 0x0000000000c3c571 in row_sel_store_mysql_rec (mysql_rec=0x7fa1bc1fe1e0 "\371\322\024", prebuilt=0x7fa1bc1ff998,
rec=0x7fa1ebd80088 "\200", rec_clust=0, index=0x7fa1bc208528, offsets=0x7fa1e438e600)
at /root/mysql-5.6.19/storage/innobase/row/row0sel.cc:2941
#5 0x0000000000c40343 in row_search_for_mysql (buf=0x7fa1bc1fe1e0 "\371\322\024", mode=1, prebuilt=0x7fa1bc1ff998, match_mode=0, direction=1)
at /root/mysql-5.6.19/storage/innobase/row/row0sel.cc:4911
#6 0x0000000000b20193 in ha_innobase::general_fetch (this=0x7fa1bc1fc6c0, buf=0x7fa1bc1fe1e0 "\371\322\024", direction=1, match_mode=0)
at /root/mysql-5.6.19/storage/innobase/handler/ha_innodb.cc:7763
#7 0x0000000000b20705 in ha_innobase::rnd_next (this=0x7fa1bc1fc6c0, buf=0x7fa1bc1fe1e0 "\371\322\024")
at /root/mysql-5.6.19/storage/innobase/handler/ha_innodb.cc:7979
#8 0x0000000000647e99 in handler::ha_rnd_next (this=0x7fa1bc1fc6c0, buf=0x7fa1bc1fe1e0 "\371\322\024")
at /root/mysql-5.6.19/sql/handler.cc:2687
#9 0x00000000009965be in rr_sequential (info=0x7fa1bc08a0e0) at /root/mysql-5.6.19/sql/records.cc:478
#10 0x00000000007b6237 in sub_select (join=0x7fa1bc02c9c8, join_tab=0x7fa1bc08a050, end_of_records=false)
at /root/mysql-5.6.19/sql/sql_executor.cc:1259
#11 0x00000000007b5c1a in do_select (join=0x7fa1bc02c9c8) at /root/mysql-5.6.19/sql/sql_executor.cc:933
#12 0x00000000007b3acb in JOIN::exec (this=0x7fa1bc02c9c8) at /root/mysql-5.6.19/sql/sql_executor.cc:194
#13 0x000000000081599a in mysql_execute_select (thd=0x3ee6790, select_lex=0x3ee90d0, free_join=true)
at /root/mysql-5.6.19/sql/sql_select.cc:1100
#14 0x0000000000815cb1 in mysql_select (thd=0x3ee6790, tables=0x7fa1bc02c388, wild_num=1, fields=..., conds=0x0, order=0x3ee9298,
group=0x3ee91d0, having=0x0, select_options=2148829952, result=0x7fa1bc02c9a0, unit=0x3ee8a88, select_lex=0x3ee90d0)
at /root/mysql-5.6.19/sql/sql_select.cc:1221
#15 0x0000000000813cd7 in handle_select (thd=0x3ee6790, result=0x7fa1bc02c9a0, setup_tables_done_option=0)
at /root/mysql-5.6.19/sql/sql_select.cc:110
#16 0x00000000007ed9f1 in execute_sqlcom_select (thd=0x3ee6790, all_tables=0x7fa1bc02c388) at /root/mysql-5.6.19/sql/sql_parse.cc:5105
#17 0x00000000007e62c0 in mysql_execute_command (thd=0x3ee6790) at /root/mysql-5.6.19/sql/sql_parse.cc:2651
#18 0x00000000007f013a in mysql_parse (thd=0x3ee6790, rawbuf=0x7fa1bc02c140 "SELECT /*!40001 SQL_NO_CACHE */ * FROM `wd_users`", length=49,
parser_state=0x7fa1e4390680) at /root/mysql-5.6.19/sql/sql_parse.cc:6247
#19 0x00000000007e31f4 in dispatch_command (command=COM_QUERY, thd=0x3ee6790,
packet=0x3ff07e1 "илов!Фахраддин Халилов\373\n1995-06-01\001\060\001\060", packet_length=49)
at /root/mysql-5.6.19/sql/sql_parse.cc:1334
#20 0x00000000007e231e in do_command (thd=0x3ee6790) at /root/mysql-5.6.19/sql/sql_parse.cc:1036
#21 0x00000000007a9281 in do_handle_one_connection (thd_arg=0x3ee6790) at /root/mysql-5.6.19/sql/sql_connect.cc:982
#22 0x00000000007a8d89 in handle_one_connection (arg=0x3ee6790) at /root/mysql-5.6.19/sql/sql_connect.cc:898
---Type <return> to continue, or q <return> to quit---
#23 0x0000000000afd879 in pfs_spawn_thread (arg=0x3f5bc50) at /root/mysql-5.6.19/storage/perfschema/pfs.cc:1860
#24 0x00000035df2079d1 in start_thread () from /lib64/libpthread.so.0
#25 0x00000035deee8b5d in clone () from /lib64/libc.so.6
From error log:
stack_bottom = 7fa1e4390de0 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xab6ead]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x404)[0x736ee0]
/lib64/libpthread.so.0[0x35df20f710]
/usr/local/mysql/bin/mysqld[0xc3c571]
/usr/local/mysql/bin/mysqld[0xc40343]
/usr/local/mysql/bin/mysqld[0xb20193]
/usr/local/mysql/bin/mysqld[0xb20705]
/usr/local/mysql/bin/mysqld(_ZN7handler11ha_rnd_nextEPh+0x137)[0x647e99]
/usr/local/mysql/bin/mysqld(_Z13rr_sequentialP11READ_RECORD+0x6d)[0x9965be]
/usr/local/mysql/bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x1cf)[0x7b6237]
/usr/local/mysql/bin/mysqld[0x7b5c1a]
/usr/local/mysql/bin/mysqld(_ZN4JOIN4execEv+0x6a7)[0x7b3acb]
/usr/local/mysql/bin/mysqld[0x81599a]
/usr/local/mysql/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex_
unitP13st_select_lex+0x25f)[0x815cb1]
/usr/local/mysql/bin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x1c5)[0x813cd7]
/usr/local/mysql/bin/mysqld[0x7ed9f1]
/usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0xd14)[0x7e62c0]
/usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x428)[0x7f013a]
/usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xc94)[0x7e31f4]
/usr/local/mysql/bin/mysqld(_Z10do_commandP3THD+0x340)[0x7e231e]
/usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1be)[0x7a9281]
/usr/local/mysql/bin/mysqld(handle_one_connection+0x33)[0x7a8d89]
/usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x159)[0xafd879]
/lib64/libpthread.so.0[0x35df2079d1]
/lib64/libc.so.6(clone+0x6d)[0x35deee8b5d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fa1bc02c140): SELECT /*!40001 SQL_NO_CACHE */ * FROM `wd_users`
Connection ID (thread ID): 1
Status: NOT_KILLED
[18 Jul 2014 15:33]
Peter Laursen
Maybe it could be because max_allowed_packet is set too low on the new server (lower than or very close to the largest string stored in the table)? -- Peter -- not a MySQL/Oracle person?
[19 Jul 2014 6:29]
Shahriyar Rzayev
Beside this fact that at first time i could :
"Another fact that i could export tables from MySQL Workbench as "Insert Statements". And also i could query with select * to this table from mysql client."
Now i could not even query to these 2 table:
mysql> select * from wd_users;
ERROR 2013 (HY000): Lost connection to MySQL server during query
bash-4.1$ bin/mysqld_safe: line 166: 2162 Segmentation fault (core dumped) nohup /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/var/lib/mysql --plugin-dir=/usr/local/mysql/lib/plugin --log-error=/var/lib/mysql/fabricsrv.err --open-files-limit=65535 --pid-file=/var/lib/mysql/fabricsrv.pid --socket=/var/lib/mysql/mysql.sock < /dev/null >> /var/lib/mysql/fabricsrv.err 2>&1
140719 11:25:26 mysqld_safe Number of processes running now: 0
140719 11:25:26 mysqld_safe mysqld restarted
From new core dump:
Core was generated by `/usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/var/lib/mysql'.
Program terminated with signal 11, Segmentation fault.
#0 0x00000035df20c8ac in pthread_kill () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6_5.2.x86_64 libgcc-4.4.7-4.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64 nss-softokn-freebl-3.14.3-10.el6_5.x86_64
(gdb) bt
#0 0x00000035df20c8ac in pthread_kill () from /lib64/libpthread.so.0
#1 0x0000000000ab6f2f in my_write_core (sig=11) at /root/mysql-5.6.19/mysys/stacktrace.c:422
#2 0x0000000000737107 in handle_fatal_signal (sig=11) at /root/mysql-5.6.19/sql/signal_handler.cc:248
#3 <signal handler called>
#4 0x0000000000d27e2e in dict_lru_validate () at /root/mysql-5.6.19/storage/innobase/dict/dict0dict.cc:6342
#5 0x0000000000d1cc5e in dict_make_room_in_cache (max_tables=4000, pct_check=100)
at /root/mysql-5.6.19/storage/innobase/dict/dict0dict.cc:1318
#6 0x0000000000c5d5f6 in srv_master_evict_from_table_cache (pct_check=100) at /root/mysql-5.6.19/storage/innobase/srv/srv0srv.cc:2003
#7 0x0000000000c5dca7 in srv_master_do_idle_tasks () at /root/mysql-5.6.19/storage/innobase/srv/srv0srv.cc:2198
#8 0x0000000000c5e152 in srv_master_thread (arg=0x0) at /root/mysql-5.6.19/storage/innobase/srv/srv0srv.cc:2346
#9 0x00000035df2079d1 in start_thread () from /lib64/libpthread.so.0
#10 0x00000035deee8b5d in clone () from /lib64/libc.so.6
From error log:
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xab6ead]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x404)[0x736ee0]
/lib64/libpthread.so.0[0x35df20f710]
/usr/local/mysql/bin/mysqld[0xd27e2e]
/usr/local/mysql/bin/mysqld[0xd1cc5e]
/usr/local/mysql/bin/mysqld[0xc5d5f6]
/usr/local/mysql/bin/mysqld[0xc5dca7]
/usr/local/mysql/bin/mysqld[0xc5e152]
/lib64/libpthread.so.0[0x35df2079d1]
/lib64/libc.so.6(clone+0x6d)[0x35deee8b5d]
[21 Jul 2014 11:29]
Sveta Smirnova
Thank you for the report. Your table seems to be corrupted. Please try to fix it using instructions from http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html Don't forget to make binary backup first.
[21 Jul 2014 12:28]
Shahriyar Rzayev
There was a little variance that i want to inform you.. Before i couldn't even query for wd_users table but now i can mysql> select * from wd_users; . . 14484 rows in set (1,21 sec) but again there is a crash: 014-07-21 17:12:01 7f7e7637a700 InnoDB: Assertion failure in thread 140181125965568 in file dict0dict.cc line 6342 InnoDB: Failing assertion: table->can_be_evicted InnoDB: We intentionally generate a memory trap. . . . /usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xab6ead] /usr/local/mysql/bin/mysqld(handle_fatal_signal+0x404)[0x736ee0] /lib64/libpthread.so.0[0x35df20f710] /lib64/libc.so.6(gsignal+0x35)[0x35dee32925] /lib64/libc.so.6(abort+0x175)[0x35dee34105] /usr/local/mysql/bin/mysqld[0xd27e5f] /usr/local/mysql/bin/mysqld[0xd1cc5e] /usr/local/mysql/bin/mysqld[0xc5d5f6] /usr/local/mysql/bin/mysqld[0xc5dca7] /usr/local/mysql/bin/mysqld[0xc5e152] /lib64/libpthread.so.0[0x35df2079d1] /lib64/libc.so.6(clone+0x6d)[0x35deee8b5d] Now i will check with innodb_force_recovery beginning from 1 and will provide all necessary information. Thank You.
[21 Jul 2014 13:00]
Shahriyar Rzayev
Dear All, with innodb_force_recovery 1 to 3 there was no difference. But with 4 there is some information in error log for all tables in testing database: 2014-07-21 17:51:13 7035 [ERROR] InnoDB: Failed to find tablespace for table '"testing"."wd_users"' in the cache. Attempting to load the tablespace with space id 2073901. 2014-07-21 17:51:13 7035 [ERROR] InnoDB: Failed to find tablespace for table '"testing"."wd_companies"' in the cache. Attempting to load the tab lespace with space id 2073902. Also there was: 2014-07-21 17:51:39 9487 [ERROR] InnoDB: Failed to find tablespace for table '"mysql"."slave_master_info"' in the cache. Attempting to load the tablespace with space id 4. 2014-07-21 17:51:39 9487 [Warning] InnoDB: Allocated tablespace 4, old maximum was 0 2014-07-21 17:51:39 9487 [ERROR] InnoDB: Failed to find tablespace for table '"mysql"."slave_worker_info"' in the cache. Attempting to load the tablespace with space id 5. 2014-07-21 17:51:39 9487 [ERROR] InnoDB: Failed to find tablespace for table '"mysql"."slave_relay_log_info"' in the cache. Attempting to load the tablespace with space id 3. And Again the same stack-trace before i provide.
[21 Jul 2014 17:28]
Sveta Smirnova
Thank you for the feedback. You certainly have table corrupted. You need to continue trying innodb_force_recovery up to 6. If it fails with 6 please send us full error log file.
[22 Jul 2014 5:32]
Shahriyar Rzayev
I tried up to 6 and there was no difference. Will provide error log.
[22 Jul 2014 5:33]
Shahriyar Rzayev
Error Log:
Attachment: fabricsrv.err (application/octet-stream, text), 148.82 KiB.
[23 Jul 2014 13:54]
Shahriyar Rzayev
Dear all, I have also figured out that: *) mysql> select * from wd_companies limit 10; . . 10 rows in set (0,53 sec) *) mysql> select * from wd_companies limit 100; . . 100 rows in set (0,01 sec) *) mysql> select * from wd_companies limit 1000; ERROR 2013 (HY000): Lost connection to MySQL server during query And Again same crash.
[29 Jul 2014 19:05]
Sveta Smirnova
Thank you for the feedback. Is it possible to send us corrupted wd_companies files? Anyway, to partially fix this you need to select all rows, except one which causes crash, using LIMIT .. OFFSET, then re-create the table and load the dump. You will loose few rows unfortunately.
[29 Jul 2014 19:49]
Sveta Smirnova
Looking at the backtrace my last guess that this can be last row in the table (or few last rows). Please check.
[30 Aug 2014 1:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
[28 Oct 2015 6:36]
Shahriyar Rzayev
Uploaded a file mysql-bug-data-73322.tar.gz to sftp server. Please check.
[27 Sep 2016 13:15]
Shahriyar Rzayev
No more relevant
Description: Dear experts, I have encountered with interesting situation when using mysqldump to export database. Here is full situation, one of our site's server which is on shared hosting have been an disc failure according to official mail from hosting company. But they could take a physical files of database. MySQL was using with innodb_file_per_table. I want to recover it in my test VM which is CentOS 6.5 with Version: '5.6.19-debug-log'. So first off all i extract table creation statements using mysqlfrm from mysql-utilities. After i just run this statements to create exact empty tables. After all i discard empty table's tablespaces and imported those tablespaces which hosting company sent to us. Everything is ok. i can select from all of these tables. And for logical end i must take full backup off this database and send it to developers for examining if there was any data loss in file. Using mysqldump: root@fabricsrv mysql]# mysqldump -u root --single-transaction --set-gtid-purged=OFF --databases testing > /home/full_testing_dump.sqlmysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `wd_companies` at row: 118 In error log: Thread pointer: 0x275d7f0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f83fbffede0 thread_stack 0x40000 /usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0xab6ead] /usr/local/mysql/bin/mysqld(handle_fatal_signal+0x404)[0x736ee0] /lib64/libpthread.so.0[0x35df20f710] /usr/local/mysql/bin/mysqld(_ZNK5Field7is_nullEx+0x30)[0x677138] /usr/local/mysql/bin/mysqld(_ZN13Protocol_text5storeEP5Field+0x32)[0x72f872] /usr/local/mysql/bin/mysqld(_ZN10Item_field4sendEP8ProtocolP6String+0x3b)[0x66db97] /usr/local/mysql/bin/mysqld(_ZN8Protocol19send_result_set_rowEP4ListI4ItemE+0xb5)[0x72eaf7] /usr/local/mysql/bin/mysqld(_ZN11select_send9send_dataER4ListI4ItemE+0xc5)[0x79e4d3] /usr/local/mysql/bin/mysqld[0x7b99b2] /usr/local/mysql/bin/mysqld[0x7b6e07] /usr/local/mysql/bin/mysqld(_Z10sub_selectP4JOINP13st_join_tableb+0x2d5)[0x7b633d] /usr/local/mysql/bin/mysqld[0x7b5c1a] /usr/local/mysql/bin/mysqld(_ZN4JOIN4execEv+0x6a7)[0x7b3acb] /usr/local/mysql/bin/mysqld[0x81599a] /usr/local/mysql/bin/mysqld(_Z12mysql_selectP3THDP10TABLE_LISTjR4ListI4ItemEPS4_P10SQL_I_ListI8st_orderESB_S7_yP13select_resultP18st_select_lex$ /usr/local/mysql/bin/mysqld(_Z13handle_selectP3THDP13select_resultm+0x1c5)[0x813cd7] /usr/local/mysql/bin/mysqld[0x7ed9f1] /usr/local/mysql/bin/mysqld(_Z21mysql_execute_commandP3THD+0xd14)[0x7e62c0] /usr/local/mysql/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x428)[0x7f013a] /usr/local/mysql/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xc94)[0x7e31f4] /usr/local/mysql/bin/mysqld(_Z10do_commandP3THD+0x340)[0x7e231e] /usr/local/mysql/bin/mysqld(_Z24do_handle_one_connectionP3THD+0x1be)[0x7a9281] /usr/local/mysql/bin/mysqld(handle_one_connection+0x33)[0x7a8d89] /usr/local/mysql/bin/mysqld(pfs_spawn_thread+0x159)[0xafd879] /lib64/libpthread.so.0[0x35df2079d1] /lib64/libc.so.6(clone+0x6d)[0x35deee8b5d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7f83f402c140): SELECT /*!40001 SQL_NO_CACHE */ * FROM `wd_companies` Connection ID (thread ID): 1 Status: NOT_KILLED Here is gdb output: [root@fabricsrv mysql]# gdb /usr/local/mysql/bin/mysqld core.27042 Reading symbols from /usr/local/mysql/bin/mysqld...done. Core was generated by `/usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/var/lib/mysql'. Program terminated with signal 11, Segmentation fault. #0 0x00000035df20c8ac in pthread_kill () from /lib64/libpthread.so.0 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6_5.2.x86_64 libgcc-4.4.7-4.el6.x86_64 libstdc++-4.4.7-4.el6.x86_64 nss-softokn-freebl-3.14.3-10.el6_5.x86_64 (gdb) bt #0 0x00000035df20c8ac in pthread_kill () from /lib64/libpthread.so.0 #1 0x0000000000ab6f2f in my_write_core (sig=11) at /root/mysql-5.6.19/mysys/stacktrace.c:422 #2 0x0000000000737107 in handle_fatal_signal (sig=11) at /root/mysql-5.6.19/sql/signal_handler.cc:248 #3 <signal handler called> #4 0x0000000000677138 in Field::is_null (this=0x7f83f4150148, row_offset=0) at /root/mysql-5.6.19/sql/field.h:894 #5 0x000000000072f872 in Protocol_text::store (this=0x275dd10, field=0x7f83f4150148) at /root/mysql-5.6.19/sql/protocol.cc:1100 #6 0x000000000066db97 in Item_field::send (this=0x7f83f402ce08, protocol=0x275dd10, buffer=0x7f83fbffca20) at /root/mysql-5.6.19/sql/item.cc:6992 #7 0x000000000072eaf7 in Protocol::send_result_set_row (this=0x275dd10, row_items=0x2760250) at /root/mysql-5.6.19/sql/protocol.cc:847 #8 0x000000000079e4d3 in select_send::send_data (this=0x7f83f402c9a0, items=...) at /root/mysql-5.6.19/sql/sql_class.cc:2492 #9 0x00000000007b99b2 in end_send (join=0x7f83f402c9c8, join_tab=0x7f83f40755a0, end_of_records=false) at /root/mysql-5.6.19/sql/sql_executor.cc:2776 #10 0x00000000007b6e07 in evaluate_join_record (join=0x7f83f402c9c8, join_tab=0x7f83f40752a0) at /root/mysql-5.6.19/sql/sql_executor.cc:1601 #11 0x00000000007b633d in sub_select (join=0x7f83f402c9c8, join_tab=0x7f83f40752a0, end_of_records=false) at /root/mysql-5.6.19/sql/sql_executor.cc:1276 #12 0x00000000007b5c1a in do_select (join=0x7f83f402c9c8) at /root/mysql-5.6.19/sql/sql_executor.cc:933 #13 0x00000000007b3acb in JOIN::exec (this=0x7f83f402c9c8) at /root/mysql-5.6.19/sql/sql_executor.cc:194 #14 0x000000000081599a in mysql_execute_select (thd=0x275d7f0, select_lex=0x2760130, free_join=true) at /root/mysql-5.6.19/sql/sql_select.cc:1100 #15 0x0000000000815cb1 in mysql_select (thd=0x275d7f0, tables=0x7f83f402c388, wild_num=1, fields=..., conds=0x0, order=0x27602f8, group=0x2760230, having=0x0, select_options=2148829952, result=0x7f83f402c9a0, unit=0x275fae8, select_lex=0x2760130) at /root/mysql-5.6.19/sql/sql_select.cc:1221 #16 0x0000000000813cd7 in handle_select (thd=0x275d7f0, result=0x7f83f402c9a0, setup_tables_done_option=0) at /root/mysql-5.6.19/sql/sql_select.cc:110 #17 0x00000000007ed9f1 in execute_sqlcom_select (thd=0x275d7f0, all_tables=0x7f83f402c388) at /root/mysql-5.6.19/sql/sql_parse.cc:5105 #18 0x00000000007e62c0 in mysql_execute_command (thd=0x275d7f0) at /root/mysql-5.6.19/sql/sql_parse.cc:2651 #19 0x00000000007f013a in mysql_parse (thd=0x275d7f0, rawbuf=0x7f83f402c140 "SELECT /*!40001 SQL_NO_CACHE */ * FROM `wd_companies`", length=53, parser_state=0x7f83fbffe680) at /root/mysql-5.6.19/sql/sql_parse.cc:6247 #20 0x00000000007e31f4 in dispatch_command (command=COM_QUERY, thd=0x275d7f0, packet=0x285fd31 "", packet_length=53) at /root/mysql-5.6.19/sql/sql_parse.cc:1334 #21 0x00000000007e231e in do_command (thd=0x275d7f0) at /root/mysql-5.6.19/sql/sql_parse.cc:1036 #22 0x00000000007a9281 in do_handle_one_connection (thd_arg=0x275d7f0) at /root/mysql-5.6.19/sql/sql_connect.cc:982 #23 0x00000000007a8d89 in handle_one_connection (arg=0x275d7f0) at /root/mysql-5.6.19/sql/sql_connect.cc:898 #24 0x0000000000afd879 in pfs_spawn_thread (arg=0x269b620) at /root/mysql-5.6.19/storage/perfschema/pfs.cc:1860 #25 0x00000035df2079d1 in start_thread () from /lib64/libpthread.so.0 #26 0x00000035deee8b5d in clone () from /lib64/libc.so.6 Another fact that i could export tables from MySQL Workbench as "Insert Statements". And also i could query with select * to this table from mysql client. How to repeat: Surely i can not provide all necessary *.ibd files but i can upload error logs, core dump or something else which will be usefull for you investigating this issue. Suggested fix: No Idea.