Bug #102774 A scenario that result in memory leak

• Submitted: 2 Mar 2021 0:37
• Modified: 6 Mar 2021 7:13
• Reporter: Haixing Weng (OCA)
• Status: Verified
• Category: MySQL Server: DML
• Severity: S3 (Non-critical)
• Version: 5.7.31,8.0.22, 5.7.33
• OS: Any
• CPU Architecture: Any
• Tags: memory leak;

[2 Mar 2021 0:37] Haixing Weng

Description:
We usually gets memory overflow warning in our online system every several months.So we'd done some load testing in some scenario, and we found that even all tables of mysql are empty, the memory of mysql process will be growing at a slow rate (grow 25MB in 10 hours). 

How to repeat:
The cnf file had been attach below, use it to create a mysql process, and use sysbench to run a test loading oltp_read_only.lua.

1. create database sbtest;

2. prepare 6000 empty tables using sysbench

bin/sysbench --mysql-host=172.21.30.23 --mysql-port=61240 --mysql-user=hxdb --mysql-password=123456  --time=3600  --table_size=0 --report-interval=1 ./share/sysbench/oltp_read_only.lua --tables=6000 --table-size=0--threads=256 --mysql-ignore-errors=all --db-ps-mode=disable --luajit-cmd=off --thread-init-timeout=3600  prepare

3. run sysbench loading test

bin/sysbench --mysql-host=172.21.30.23 --mysql-port=61240 --mysql-user=hxdb --mysql-password=123456  --time=3600  --table_size=0 --report-interval=1 ./share/sysbench/oltp_read_only.lua --tables=6000 --table-size=0--threads=256 --mysql-ignore-errors=all --db-ps-mode=disable --luajit-cmd=off --thread-init-timeout=3600  run

4. using scripts or demo program to monitor $mysqld proc.RSS

5. we found that memory grows will nerver stop

6. I post our test result , with data & sampling graph

[2 Mar 2021 0:39] Haixing Weng

memory sampling graph

Attachment: sampling.png (image/png, text), 36.09 KiB.

[2 Mar 2021 0:39] Haixing Weng

memory sampling data

Attachment: 31818.pid.Rss (application/rss+xml, text), 532.00 KiB.

[2 Mar 2021 11:33] Haixing Weng

Suggest to running loading test in a long time, by using --time=36000(10hours) or 360000(5days), We observed 200MB memory grows, running in serveral hours up to now, and it seems to nerver stop

[3 Mar 2021 4:08] Haixing Weng

We still observe memory grows even we had started mysqld with jemalloc lib

[6 Mar 2021 7:13] MySQL Verification Team

Hello Haixing Weng,

Thank you for the report and feedback.
Observed the issue with 5.7.33 build with provided configuration and sysbench load.

regards,
Umesh