| Bug #79475 | Insert a token of 84 4-bytes chars into fts index causes server crash | ||
|---|---|---|---|
| Submitted: | 1 Dec 2015 9:27 | Modified: | 3 Dec 2015 14:01 |
| Reporter: | Shaohua Wang | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: InnoDB storage engine | Severity: | S2 (Serious) |
| Version: | 5.6+ | OS: | Any |
| Assigned to: | CPU Architecture: | Any | |
[1 Dec 2015 9:30]
Shaohua Wang
Posted by developer:
MyISAM:
#0 0x00007ffff67b3267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007ffff67b4eca in __GI_abort () at abort.c:89
#2 0x00007ffff67f6c53 in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7ffff690cdad "*** %s ***: %s terminated\n") at ../sysdeps/posix/libc_fatal.c:175
#3 0x00007ffff6896e8c in __GI___fortify_fail (msg=<optimized out>, msg@entry=0x7ffff690cd95 "stack smashing detected") at fortify_fail.c:38
#4 0x00007ffff6896e30 in __stack_chk_fail () at stack_chk_fail.c:28
#5 0x0000000001cb7f5a in _ft_make_key (info=0x7fff9801ef60, keynr=0, keybuf=0x7fff9801fda6 "\376", <incomplete sequence \360\237\230>..., wptr=0x7fff98922910, filepos=0)
at /home/wonder/work/mysql/storage/myisam/ft_update.c:298
#6 0x9ff081989ff08198 in ?? ()
#7 0x9ff081989ff08198 in ?? ()
#8 0x9ff081989ff08198 in ?? ()
#9 0x9ff081989ff08198 in ?? ()
#10 0x000081989ff08198 in ?? ()
#11 0x0000000000000000 in ?? ()
InnoDB:
#0 __pthread_kill (threadid=<optimized out>, signo=6) at ../sysdeps/unix/sysv/linux/pthread_kill.c:62
#1 0x000000000184c230 in my_write_core (sig=6) at /home/wonder/work/mysql/mysys/stacktrace.c:247
#2 0x0000000000e577ad in handle_fatal_signal (sig=6) at /home/wonder/work/mysql/sql/signal_handler.cc:220
#3 <signal handler called>
#4 0x00007f8a3ce0e267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#5 0x00007f8a3ce0feca in __GI_abort () at abort.c:89
#6 0x0000000001ac5775 in ut_dbg_assertion_failed (expr=0x219a8f8 "len <= col->len || ((col->mtype) == 5 || (col->mtype) == 16 || (col->mtype) == 14) || (((col->mtype) == 15 || (col->mtype) == 16) && len == 2 * 2 * sizeof(double)) || (col->len == 0 && col->mtype == 1"..., file=0x219a628 "/home/wonder/work/mysql/storage/innobase/rem/rem0rec.cc", line=867) at /home/wonder/work/mysql/storage/innobase/ut/ut0dbg.cc:67
#7 0x000000000199ca81 in rec_get_converted_size_comp_prefix_low (index=0x7f89e0d56d40, fields=0x7f89e0e342e8, n_fields=7, v_entry=0x0, extra=0x0, temp=false) at /home/wonder/work/mysql/storage/innobase/rem/rem0rec.cc:864
#8 0x000000000199d107 in rec_get_converted_size_comp (index=0x7f89e0d56d40, status=0, fields=0x7f89e0e342e8, n_fields=7, extra=0x0) at /home/wonder/work/mysql/storage/innobase/rem/rem0rec.cc:1020
#9 0x0000000001aed359 in rec_get_converted_size (index=0x7f89e0d56d40, dtuple=0x7f89e0e342a0, n_ext=0) at /home/wonder/work/mysql/storage/innobase/include/rem0rec.ic:1653
#10 0x0000000001af82c5 in btr_cur_optimistic_insert (flags=0, cursor=0x7f8a3e4bdf90, offsets=0x7f8a3e4bdf40, heap=0x7f8a3e4bdf38, entry=0x7f89e0e342a0, rec=0x7f8a3e4bdf48, big_rec=0x7f8a3e4bdf30, n_ext=0, thr=0x7f89e0d821d8, mtr=0x7f8a3e4be3b0) at /home/wonder/work/mysql/storage/innobase/btr/btr0cur.cc:3095
#11 0x00000000019c2b06 in row_ins_clust_index_entry_low (flags=0, mode=2, index=0x7f89e0d56d40, n_uniq=2, entry=0x7f89e0e342a0, n_ext=0, thr=0x7f89e0d821d8, dup_chk_only=false) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:2443
#12 0x00000000019c4dcb in row_ins_clust_index_entry (index=0x7f89e0d56d40, entry=0x7f89e0e342a0, thr=0x7f89e0d821d8, n_ext=0, dup_chk_only=false) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:3158
#13 0x00000000019c526a in row_ins_index_entry (index=0x7f89e0d56d40, entry=0x7f89e0e342a0, thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:3288
#14 0x00000000019c57bc in row_ins_index_entry_step (node=0x7f89e0d81f28, thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:3438
#15 0x00000000019c5b5a in row_ins (node=0x7f89e0d81f28, thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:3580
#16 0x00000000019c6195 in row_ins_step (thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/row/row0ins.cc:3765
#17 0x000000000198f81c in que_thr_step (thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/que/que0que.cc:1031
#18 0x000000000198fb17 in que_run_threads_low (thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/que/que0que.cc:1113
#19 0x000000000198fcdc in que_run_threads (thr=0x7f89e0d821d8) at /home/wonder/work/mysql/storage/innobase/que/que0que.cc:1153
#20 0x0000000001c0983e in fts_eval_sql (trx=0x7f8a37783150, graph=0x7f89e0d82100) at /home/wonder/work/mysql/storage/innobase/fts/fts0sql.cc:245
#21 0x0000000001bebf98 in fts_write_node (trx=0x7f8a37783150, graph=0x7f89e0f5ae00, fts_table=0x7f8a3e4bef10, word=0x7f89e0d4b808, node=0x7f89e011ac78) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:3958
#22 0x0000000001bec4a3 in fts_sync_write_words (trx=0x7f8a37783150, index_cache=0x7f89e0f5ad78) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:4099
#23 0x0000000001bec99e in fts_sync_index (sync=0x7f89e0f5ad00, index_cache=0x7f89e0f5ad78) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:4414
#24 0x0000000001bece5f in fts_sync (sync=0x7f89e0f5ad00) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:4534
#25 0x0000000001beb5de in fts_add_doc_by_id (ftt=0x7f89e0e32b90, doc_id=2, fts_indexes=0x0) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:3637
#26 0x0000000001be9e37 in fts_add (ftt=0x7f89e0e32b90, row=0x7f89e033a0f8) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:2989
#27 0x0000000001bea750 in fts_commit_table (ftt=0x7f89e0e32b90) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:3205
#28 0x0000000001bea857 in fts_commit (trx=0x7f8a377828c0) at /home/wonder/work/mysql/storage/innobase/fts/fts0fts.cc:3254
#29 0x0000000001ab2fb3 in trx_commit_low (trx=0x7f8a377828c0, mtr=0x7f8a3e4bfc70) at /home/wonder/work/mysql/storage/innobase/trx/trx0trx.cc:2113
#30 0x0000000001ab318c in trx_commit (trx=0x7f8a377828c0) at /home/wonder/work/mysql/storage/innobase/trx/trx0trx.cc:2205
#31 0x0000000001ab38a5 in trx_commit_for_mysql (trx=0x7f8a377828c0) at /home/wonder/work/mysql/storage/innobase/trx/trx0trx.cc:2424
#32 0x0000000001888551 in innobase_commit_low (trx=0x7f8a377828c0) at /home/wonder/work/mysql/storage/innobase/handler/ha_innodb.cc:3881
#33 0x0000000001888a30 in innobase_commit (hton=0x38d27d0, thd=0x7f89e0115b50, commit_trx=false) at /home/wonder/work/mysql/storage/innobase/handler/ha_innodb.cc:4040
#34 0x0000000000ed1998 in ha_commit_low (thd=0x7f89e0115b50, all=false, run_after_commit=true) at /home/wonder/work/mysql/sql/handler.cc:1794
#35 0x0000000001620bea in TC_LOG_DUMMY::commit (this=0x2cfd008 <tc_log_dummy>, thd=0x7f89e0115b50, all=false) at /home/wonder/work/mysql/sql/tc_log.cc:28
#36 0x0000000000ed166e in ha_commit_trans (thd=0x7f89e0115b50, all=false, ignore_global_read_lock=false) at /home/wonder/work/mysql/sql/handler.cc:1689
#37 0x00000000016239db in trans_commit_stmt (thd=0x7f89e0115b50) at /home/wonder/work/mysql/sql/transaction.cc:458
#38 0x000000000151ef6c in mysql_execute_command (thd=0x7f89e0115b50, first_level=true) at /home/wonder/work/mysql/sql/sql_parse.cc:4946
#39 0x00000000015206a1 in mysql_parse (thd=0x7f89e0115b50, parser_state=0x7f8a3e4c1dd0) at /home/wonder/work/mysql/sql/sql_parse.cc:5508
#40 0x000000000151519a in dispatch_command (thd=0x7f89e0115b50, com_data=0x7f8a3e4c2e10, command=COM_QUERY) at /home/wonder/work/mysql/sql/sql_parse.cc:1425
#41 0x0000000001514064 in do_command (thd=0x7f89e0115b50) at /home/wonder/work/mysql/sql/sql_parse.cc:993
#42 0x000000000164f9da in handle_connection (arg=0x3d76060) at /home/wonder/work/mysql/sql/conn_handler/connection_handler_per_thread.cc:301
#43 0x0000000001cd3c58 in pfs_spawn_thread (arg=0x40292e0) at /home/wonder/work/mysql/storage/perfschema/pfs.cc:2192
#44 0x00007f8a3d9d76aa in start_thread (arg=0x7f8a3e4c3700) at pthread_create.c:333
#45 0x00007f8a3cedfeed in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
[3 Dec 2015 14:01]
Daniel Price
Posted by developer: Fixed as of the upcoming 5.6.29, 5.7.11, 5.8.0 release, and here's the changelog entry: Inserting a token of 84 4-byte characters into a full-text index raised an assertion. The maximum token length was 84 characters up to a maximum of 252 bytes, which did not account for 4-byte characters. The maximum byte length is now 336 bytes.
Description: we allow max token size up to 84 in both MyISAM and InnoDB, but when insert a token with 84 4-bytes chars into a fts index, the server crashes. The root cause is that we suppose max multiple-bytes char length is 3 bytes, which is not true. We support 4 bytes chars, e.g. in utf8mb4. See include/ft_global.h #define HA_FT_MAXBYTELEN 254 define HA_FT_MAXCHARLEN (HA_FT_MAXBYTELEN/3) How to repeat: --source include/have_simple_parser.inc set names utf8mb4; --replace_regex /\.dll/.so/ eval INSTALL PLUGIN simple_parser SONAME '$SIMPLE_PARSER'; create table t1( id int, title text, fulltext(title) with parser simple_parser ) engine = innodb default charset UTF8MB4; --server crashes with myisam too. SET SESSION debug="+d,fts_instrument_sync"; insert into t1 values(1, '😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁'); #insert into t1 values(1, 'mysql is good'); SET SESSION debug="-d,fts_instrument_sync"; select * from t1 where match(title) against('😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁😁'); drop table t1; UNINSTALL PLUGIN simple_parser; Suggested fix: #define HA_FT_MAXBYTELEN 336 define HA_FT_MAXCHARLEN (HA_FT_MAXBYTELEN/4)