| Bug #104725 | Memory leak when set host_cache_size to 0 without using --skip-host-cache | ||
|---|---|---|---|
| Submitted: | 25 Aug 2021 15:28 | Modified: | 26 Aug 2021 23:58 |
| Reporter: | qian zhang (OCA) | Email Updates: | |
| Status: | Verified | Impact on me: | |
| Category: | MySQL Server: Connection Handling | Severity: | S2 (Serious) |
| Version: | 5.7.34 | OS: | Ubuntu (20.04.2 LTS) |
| Assigned to: | CPU Architecture: | x86 | |
| Tags: | host_cache_size, memory leak | ||
[26 Aug 2021 11:53]
MySQL Verification Team
Hi Mr. zhang, Thank you for your bug report. Before we proceed, we have to get some more info from you ...... First of all, it seems that you have killed server process and you still expect that MySQL server frees all of its memory. Are you aware of the fact that when a daemon is killed by the kernel on any Unix, including Linux, then daemon itself is responsible for releasing all memory that was allocated to server processes ??? We have designed our server so that some memory is not released until the end, when it is left up to the operating system to do what it does without any single error. Not a bug.
[26 Aug 2021 15:12]
qian zhang
The default signal of the kill command is SIGTERM(15), which just informs the process to perform a "clean exit", the program receives the signal safely, and usually performs some "preparatory work" before exiting, such as releasing resources, cleaning up temporary files, etc. If the preparatory work is finished, then proceed to the termination of the program. So, you look at it again?
[26 Aug 2021 15:27]
MySQL Verification Team
Sinisa this should be verified exactly as described.
You can mimik connecting from various hosts by adding entries in
/etc/hosts for 127.0.0.1, 127.0.0.2, 127.0.0.03 etc and connect with those bind addresses of mysql client.
=================================================================
==1624658==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 34272 byte(s) in 102 object(s) allocated from:
#0 0x7f8ce272193f in __interceptor_malloc (/lib64/libasan.so.6+0xae93f)
#1 0xcdbbb1 in add_hostname_impl /home/anon/bld/git/mysql-git/sql/hostname.cc:209
#2 0xcdbe47 in add_hostname /home/anon/bld/git/mysql-git/sql/hostname.cc:290
#3 0xcdcd04 in ip_to_hostname(sockaddr_storage*, char const*, char**, unsigned int*) /home/anon/bld/git/mysql-git/sql/hostname.cc:1017
#4 0x16c1c31 in check_connection /home/anon/bld/git/mysql-git/sql/sql_connect.cc:612
#5 0x16c23ac in login_connection /home/anon/bld/git/mysql-git/sql/sql_connect.cc:758
#6 0x16c251a in thd_prepare_connection(THD*) /home/anon/bld/git/mysql-git/sql/sql_connect.cc:910
#7 0x1946722 in handle_connection /home/anon/bld/git/mysql-git/sql/conn_handler/connection_handler_per_thread.cc:307
#8 0x22d6a3d in pfs_spawn_thread /home/anon/bld/git/mysql-git/storage/perfschema/pfs.cc:2197
#9 0x7f8ce265b298 in start_thread (/lib64/libpthread.so.0+0x9298)
SUMMARY: AddressSanitizer: 34272 byte(s) leaked in 102 allocation(s).
[26 Aug 2021 15:32]
MySQL Verification Team
8.0.26 was not affected.
[26 Aug 2021 23:58]
MySQL Verification Team
verified on 5.7 but 8.0 was not leaking.
[30 Aug 2021 16:01]
qian zhang
0001-Bug-104725-Memory-leak-when-set-host_cache_size-to-0.patch (*) I confirm the code being submitted is offered under the terms of the OCA, and that I am authorized to contribute it.
Contribution: 0001-Bug-104725-Memory-leak-when-set-host_cache_size-to-0.patch (application/octet-stream, text), 1.16 KiB.
[31 Aug 2021 12:16]
MySQL Verification Team
Thank you, Mr. zhang !!!!
Description: I use the following mysqld configuration file, When cross-host database access is performed, memory leaks will occur. [mysqld] basedir=/usr/local/mysql datadir=/mysql-files host_cache_size=0 # valgrind --tool=memcheck --leak-check=full /usr/local/mysql/bin/mysqld-debug --basedir=/usr/local/mysql --datadir=/mysql-files --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=acc-vm.err --pid-file=acc-vm.pid ==5824== Memcheck, a memory error detector ==5824== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==5824== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==5824== Command: /usr/local/mysql/bin/mysqld-debug --basedir=/usr/local/mysql --datadir=/mysql-files --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=acc-vm.err --pid-file=acc-vm.pid ==5824== ==5824== ==5824== HEAP SUMMARY: ==5824== in use at exit: 2,484 bytes in 7 blocks ==5824== total heap usage: 59,389 allocs, 59,382 frees, 197,369,733 bytes allocated ==5824== ==5824== 672 bytes in 2 blocks are definitely lost in loss record 5 of 6 ==5824== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==5824== by 0xF814D4: add_hostname_impl(char const*, char const*, bool, Host_errors*, unsigned long long) (hostname.cc:209) ==5824== by 0xF817E2: add_hostname(char const*, char const*, bool, Host_errors*) (hostname.cc:290) ==5824== by 0xF82FDF: ip_to_hostname(sockaddr_storage*, char const*, char**, unsigned int*) (hostname.cc:1017) ==5824== by 0x15B0A03: check_connection(THD*) (sql_connect.cc:612) ==5824== by 0x15B0FB2: login_connection(THD*) (sql_connect.cc:747) ==5824== by 0x15B1767: thd_prepare_connection(THD*) (sql_connect.cc:899) ==5824== by 0x174E279: handle_connection (connection_handler_per_thread.cc:307) ==5824== by 0x1E5AAF4: pfs_spawn_thread (pfs.cc:2197) ==5824== by 0x4C1E608: start_thread (pthread_create.c:477) ==5824== by 0x50A6292: clone (clone.S:95) ==5824== ==5824== LEAK SUMMARY: ==5824== definitely lost: 672 bytes in 2 blocks ==5824== indirectly lost: 0 bytes in 0 blocks ==5824== possibly lost: 0 bytes in 0 blocks ==5824== still reachable: 1,812 bytes in 5 blocks ==5824== suppressed: 0 bytes in 0 blocks # /usr/local/mysql/bin/mysqld-debug --version /usr/local/mysql/bin/mysqld-debug Ver 5.7.34-debug for Linux on x86_64 (MySQL Community Server (GPL)) How to repeat: 1. use the following mysqld configuration file: [mysqld] basedir=/usr/local/mysql datadir=/mysql-files host_cache_size=0 2. use valgrind to run mysqld: #valgrind --tool=memcheck --leak-check=full /usr/local/mysql/bin/mysqld-debug --basedir=/usr/local/mysql --datadir=/mysql-files --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=acc-vm.err --pid-file=acc-vm.pid 3.Access the database across hosts: 4.kill `pidof mysqld` 5.In the end, valgrind will report a memory leak. Suggested fix: sql/hostname.cc: static void add_hostname_impl(const char *ip_key, const char *hostname, bool validated, Host_errors *errors, ulonglong now) { Host_entry *entry; bool need_add= false; entry= hostname_cache_search(ip_key); //Because the host_cache_size parameter is set to 0, entry must be NULL if (likely(entry == NULL)) { entry= (Host_entry *) malloc(sizeof (Host_entry)); if (entry == NULL) return; need_add= true; ...... } ...... if (need_add) hostname_cache->add(entry); // hostname_cache.m_size is 0, so the add operation did nothing, so entry is definitely lost return; } At the beginning of the add_hostname_impl function, call hostname_cache_size to get the size of hostname_cache. If it is 0, do nothing and return directly.