Description:
Original bug description: https://bugs.launchpad.net/percona-server/+bug/1655403
includes quick fix.
setup innodb memcache, create a table with 65k varchar field.
Create single connection to memcache port.
set/get in a loop without closing connection
at some moment oom kills mysqld
#!/usr/bin/env ruby
require 'socket'
def bin_to_hex(s)
s.each_byte.map { |b| b.to_s(16) }.join
end
class MemcacheClient
def initialize
@s = TCPSocket.new 'localhost', 11211
end
def get(key)
@s.write("get #{key}\r\n")
info = @s.gets
if info.start_with?('VALUE')
count = info.split[-1].to_i
value = @s.read(count)
value += @s.gets # \r\n
value += @s.gets # END
end
info + (value || "")
end
def set(key, value)
value = value.to_s
@s.write("set #{key} 0 0 #{value.bytesize}\r\n#{value}\r\n")
@s.gets
end
end
def test_write_and_read(size=64000)
memcache_client = MemcacheClient.new
random = Random.new
1.upto(1_000_000) do |n|
k = bin_to_hex(random.bytes(20))
v = 'a' * size
puts n
$stdout.flush
memcache_client.set(k, v)
ret = memcache_client.get(k)
if !ret.include?(v)
raise "Error: mismatch: #{ret.inspect} != #{v.inspect}"
end
end
end
test_write_and_read
How to repeat:
docker run -it --rm --name m57 -e MYSQL_ALLOW_EMPTY_PASSWORD=1 mysql/mysql-server:5.7
inside container:
yum install ruby
cat /usr/share/mysql/innodb_memcached_config.sql | sed -e 's/c2 VARCHAR(1024)/c2 VARCHAR(65000)/' > innodb_memcached_config_changed.sql
mysql < innodb_memcached_config_changed.sql
mysql -e 'INSTALL PLUGIN daemon_memcached soname "libmemcached.so";'
ruby test_memcache.rb
Monitor memory consumption top -p $( pgrep -xn mysqld )
when script prints 75k 5GB RAM consumed.
If script killed memory consumption returns back to 300M
