Bug #70224 Memcached allocates memory beyond configured limit
Submitted: 3 Sep 2013 14:19
Reporter: Miljenko Brkic Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Server: Memcached Severity:S1 (Critical)
Version:5.6.13 OS:Linux
Assigned to: CPU Architecture:Any
Tags: innodb, memcached

[3 Sep 2013 14:19] Miljenko Brkic 
Description:
Memcached allocates memory beyond configured limit, mysql process allocates all swap space.

Statistics (stats, stats slabs):

STAT engine_maxbytes 67108864
STAT bytes 89345503
STAT total_malloced 469760824 

How to repeat:
Use "cache_only" cache policy:

mysql> select * from cache_policies;
+--------------+------------+------------+---------------+--------------+
| policy_name  | get_policy | set_policy | delete_policy | flush_policy |
+--------------+------------+------------+---------------+--------------+
| cache_policy | cache_only | cache_only | cache_only    | cache_only   |
+--------------+------------+------------+---------------+--------------+
1 row in set (0.00 sec)

Memcached memory is limited to 64MB. Test program is setting new data in the cache in loop. Only Memcached is used (i.e. no any other operations on mysql).

Here is memory usage with time:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND                                                                                                                                    
 2159 mysql     20   0  806m 429m 1416 S    1 10.8   0:08.76 377m mysqld
 2159 mysql     20   0  810m 421m 1448 S    6 10.6   8:10.06 389m mysqld
 2159 mysql     20   0  810m 412m 1448 S   10 10.3  13:02.58 398m mysqld
 2159 mysql     20   0  810m 402m 1448 S   10 10.1  24:48.38 408m mysqld
 2159 mysql     20   0  810m 394m 1428 S    1  9.9  34:24.98 416m mysqld
...

Suggested fix:
Standalone Memcached does not have this problem:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND                                                                                                                                    
 1595 memcache  20   0 47344  896  676 S    0  0.0   0:00.52  45m memcached
 1595 memcache  20   0 51440 5012  776 S    6  0.1   8:55.15  45m memcached
 1595 memcache  20   0 51440 5012  776 S   10  0.1  16:13.69  45m memcached
 1595 memcache  20   0 51440 4924  744 S   13  0.1  26:57.56  45m memcached
...