Description:
MySQL crashes pretty often when using memcached plugin under severe load (about 20 concurrent connections constantly writing and reading entries with payload around 65k size), we use MEDIUMTEXT column type for payload, if it matters
here is the stack trace:
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/opt/mysql/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x2c)[0xe7324c]
/opt/mysql/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x451)[0x77e1c1]
/lib64/libpthread.so.0(+0xf100)[0x7fe3583ee100]
/lib64/libc.so.6(gsignal+0x37)[0x7fe3571f25f7]
/lib64/libc.so.6(abort+0x148)[0x7fe3571f3ce8]
/opt/mysql/usr/local/mysql/bin/mysqld[0x74efc8]
/opt/mysql/usr/local/mysql/bin/mysqld(_Z20rec_get_offsets_funcPKhPK12dict_index_tPmmPP16mem_block_info_t+0x54)[0xfa76b4]
/opt/mysql/usr/local/mysql/bin/mysqld[0x1080a01]
/opt/mysql/usr/local/mysql/bin/mysqld(_Z18ib_cursor_read_rowP11ib_cursor_tP10ib_tuple_tPPvPm+0x33e)[0x108655e]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(innodb_api_search+0x274)[0x7fe0042fbf84]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(innodb_api_store+0x75)[0x7fe0042fdc95]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(+0xb4eb)[0x7fe0042f94eb]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(conn_nread+0x1355)[0x7fe00451e875]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(+0x9de9)[0x7fe004512de9]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(event_base_loop+0x592)[0x7fe004525892]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(+0x18982)[0x7fe004521982]
/lib64/libpthread.so.0(+0x7dc5)[0x7fe3583e6dc5]
/lib64/libc.so.6(clone+0x6d)[0x7fe3572b3bdd]
the same thing happens with version 5.6.28, stack trace is slightly different:
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x30000
/opt/mysql/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0x8eb515]
/opt/mysql/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x41b)[0x653ffb]
/lib64/libpthread.so.0(+0xf100)[0x7fb871e35100]
/lib64/libc.so.6(gsignal+0x37)[0x7fb870c3a5f7]
/lib64/libc.so.6(abort+0x148)[0x7fb870c3bce8]
/opt/mysql/usr/local/mysql/bin/mysqld[0x9b61cd]
/opt/mysql/usr/local/mysql/bin/mysqld[0xa254d1]
/opt/mysql/usr/local/mysql/bin/mysqld[0xa25927]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(innodb_api_search+0x18e)[0x7fb1b5ba053e]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(innodb_api_store+0x70)[0x7fb1b5ba0b60]
/opt/mysql/usr/local/mysql/lib/plugin//innodb_engine.so(+0xa09f)[0x7fb1b5b9d09f]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(+0x13cba)[0x7fb852e29cba]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(conn_nread+0x211)[0x7fb852e2a6b1]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(+0x95bc)[0x7fb852e1f5bc]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(event_base_loop+0x286)[0x7fb852e31aa6]
/opt/mysql/usr/local/mysql/lib/plugin/libmemcached.so(+0x19582)[0x7fb852e2f582]
/lib64/libpthread.so.0(+0x7dc5)[0x7fb871e2ddc5]
/lib64/libc.so.6(clone+0x6d)[0x7fb870cfbbdd]
How to repeat:
create table with mediumtext column as a payload, map it to memcached plugin, connect with about 20 memcached clients and start performing reads and writes
