Bug #75200 MySQL crashed because of append operation
Submitted: 13 Dec 2014 11:27 Modified: 28 Jan 2015 12:55
Reporter: zhai weixiang (OCA) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Memcached Severity:S3 (Non-critical)
Version:5.7.5 OS:Any
Assigned to: CPU Architecture:Any
Tags: memcached

[13 Dec 2014 11:27] zhai weixiang
Description:
root@innodb_memcache 07:25:17>select * from containers where name = 'tt1';
+------+-----------+----------+-------------+---------------+-------+------------+--------------------+------------------------+
| name | db_schema | db_table | key_columns | value_columns | flags | cas_column | expire_time_column | unique_idx_name_on_key |
+------+-----------+----------+-------------+---------------+-------+------------+--------------------+------------------------+
| tt1  | test      | t1       | pk          | val1|val2     | c3    | c4         | c5                 | PRIMARY                |
+------+-----------+----------+-------------+---------------+-------+------------+--------------------+------------------------+
1 row in set (0.00 sec)

root@innodb_memcache 07:25:33>show create table test.t1\G
*************************** 1. row ***************************
       Table: t1
Create Table: CREATE TABLE `t1` (
  `pk` varchar(20) NOT NULL,
  `val1` int(11) DEFAULT NULL,
  `val2` int(11) DEFAULT NULL,
  `c3` bigint(20) DEFAULT NULL,
  `c4` bigint(20) DEFAULT NULL,
  `c5` bigint(20) DEFAULT NULL,
  PRIMARY KEY (`pk`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.00 sec)

root@innodb_memcache 07:25:40>select * from test.t1;
+-----+------+------+------+------+------+
| pk  | val1 | val2 | c3   | c4   | c5   |
+-----+------+------+------+------+------+
| pk1 |    8 |    9 |    0 |    7 |    0 |
| pk2 |    2 |    3 | NULL | NULL | NULL |
| pk3 |    1 |  111 |    0 |   13 |    0 |
| pk5 |   10 |   11 |    0 |   10 |    0 |
+-----+------+------+------+------+------+
4 rows in set (0.00 sec)

Then execute append from telnet:

$telnet 127.0.0.1  13407
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
get @@tt1
VALUE @@tt1 0 7
test/t1
END
get pk3
VALUE pk3 0 5
1|111
END
append pk3 0 0 6
 abcde

Segmentation fault happened and crashed the server.

backtrace:

#0  0x000000372c688ae6 in memcpy () from /lib64/libc.so.6
#1  0x00002adbb400d205 in innodb_api_link (engine=0x2adbac1c1640, cursor_data=0x2adc3c0008c0, key=0x2adc3c000ed8 "pk3 abcde", len=3, val_len=6, exp=0, cas=0x2adc34000b10, input_cas=0, flags=0, op=OPERATION_APPEND)
    at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_api.c:1414
#2  innodb_api_store (engine=0x2adbac1c1640, cursor_data=0x2adc3c0008c0, key=0x2adc3c000ed8 "pk3 abcde", len=3, val_len=6, exp=0, cas=0x2adc34000b10, input_cas=0, flags=0, op=OPERATION_APPEND)
    at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_api.c:1711
#3  0x00002adbb400931f in innodb_store (handle=0x2adbac1c1640, cookie=0x2adc340008c0, item=<value optimized out>, cas=0x2adc34000b10, op=OPERATION_APPEND, vbucket=<value optimized out>)
    at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_engine.c:1897
#4  0x00002ac6b5306a7f in complete_update_ascii (c=0x2adc340008c0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:1110
#5  complete_nread_ascii (c=0x2adc340008c0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:3506
#6  complete_nread (c=0x2adc340008c0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:3516
#7  0x00002ac6b5307623 in conn_nread (c=0x2adc340008c0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:5386
#8  0x00002ac6b52fd094 in event_handler (fd=<value optimized out>, which=<value optimized out>, arg=0x2adc340008c0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:5692
#9  0x00002ac6b530d7aa in event_process_active (base=0x2adbac1f11e0, flags=<value optimized out>) at /u01/project/mysql-lab/mysql-5.7.5-m15/libevent/event.c:392
#10 event_base_loop (base=0x2adbac1f11e0, flags=<value optimized out>) at /u01/project/mysql-lab/mysql-5.7.5-m15/libevent/event.c:544
#11 0x00002ac6b530b5b2 in worker_libevent (arg=0x2adbac1eafe0) at /u01/project/mysql-lab/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/thread.c:306
#12 0x000000372ca07851 in start_thread () from /lib64/libpthread.so.0
#13 0x000000372c6e767d in clone () from /lib64/libc.so.6

How to repeat:
Described above

Suggested fix:
I don't know.
[14 Dec 2014 7:36] Umesh Shastry
Hello Zhai,

Thank you for the report and test case.

Thanks,
Umesh
[14 Dec 2014 7:36] Umesh Shastry
// Build

[root@cluster-repo server]# md5sum mysql-5.7.5-m15-linux-glibc2.5-x86_64.tar.gz
b4c61a681b8a2d85527e7e2d2a595c84  mysql-5.7.5-m15-linux-glibc2.5-x86_64.tar.gz

//

[root@cluster-repo server]# telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
..

get @@tt1
VALUE @@tt1 0 7
test/t1
END
get pk1
VALUE pk1 2 3
1|2
END
append pk1 0 0 6
 abcde
Connection closed by foreign host.

//

(gdb) bt
#0  __pthread_kill (threadid=<value optimized out>, signo=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:63
#1  0x000000000061f0c4 in handle_fatal_signal (sig=11) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/sql/signal_handler.cc:219
#2  <signal handler called>
#3  memcpy () at ../sysdeps/x86_64/memcpy.S:102
#4  0x00007f76997f2eb3 in innodb_api_link (engine=<value optimized out>, cursor_data=0x7f7641503190, key=0x7f76415037a8 "pk1 abcde", len=3, val_len=<value optimized out>, exp=0, cas=0x2f243e0,
    input_cas=0, flags=0, op=OPERATION_APPEND) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_api.c:1414
#5  innodb_api_store (engine=<value optimized out>, cursor_data=0x7f7641503190, key=0x7f76415037a8 "pk1 abcde", len=3, val_len=<value optimized out>, exp=0, cas=0x2f243e0, input_cas=0,
    flags=0, op=OPERATION_APPEND) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_api.c:1711
#6  0x00007f76997ee7af in innodb_store (handle=0x7f763cd5af30, cookie=<value optimized out>, item=<value optimized out>, cas=<value optimized out>, op=<value optimized out>,
    vbucket=<value optimized out>) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/innodb_memcache/src/innodb_engine.c:1897
#7  0x00007f76982c8a9e in complete_update_ascii (c=0x2f24190)
    at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:1110
#8  complete_nread_ascii (c=0x2f24190) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:3506
#9  complete_nread (c=0x2f24190) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:3516
#10 conn_nread (c=0x2f24190) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:5386
#11 0x00007f76982bdafc in event_handler (fd=<value optimized out>, which=<value optimized out>, arg=0x2f24190)
    at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/memcached.c:5692
#12 0x00007f76982cfa66 in event_process_active (base=0x7f763cd721b0, flags=<value optimized out>) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/libevent/event.c:392
#13 event_base_loop (base=0x7f763cd721b0, flags=<value optimized out>) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/libevent/event.c:544
#14 0x00007f76982cd282 in worker_libevent (arg=0x7f763cd713f8) at /export/home/pb2/build/sb_0-13248356-1411047660.02/mysql-5.7.5-m15/plugin/innodb_memcached/daemon_memcached/daemon/thread.c:306
#15 0x0000003deb0079d1 in start_thread (arg=0x7f767cff9700) at pthread_create.c:301
#16 0x0000003deace89dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
[28 Jan 2015 3:17] Allen Lai
Posted by developer:
 
This bug is caused by trying to do "append" on a integer column. We should block this operation.
[28 Jan 2015 12:55] Daniel Price
Posted by developer:
 
Fixed as of the upcoming 5.6.24, 5.7.6 release, and here's the changelog entry:

A "memcached" "append" operation on an "INT" column caused a segmentation
fault. "append" operations on "INT" columns are not supported and are now
blocked.

Thank you for the bug report.
[27 Apr 2015 8:44] Laurynas Biveinis
commit cb7af34e0753c509e65fe45f8bc2d66538f7f20d
Author: Allen.Lai <zheng.lai@oracle.com>
Date:   Tue Jan 27 16:45:04 2015 +0800

    Bug#20209756 MYSQL CRASHED BECAUSE OF APPEND OPERATION
    
    This bug is caused by trying to do "append" on a integer column. We should
    block this operation.
    
    Reviewed-by: Jimmy Yang<jimmy.yang@oracle.com>
    RB: 7804