| Bug #34513 | increasing memory consumption with creating/dropping tmp tables on debug binary | ||
|---|---|---|---|
| Submitted: | 13 Feb 2008 6:32 | Modified: | 25 Jan 2022 7:07 |
| Reporter: | Shane Bester (Platinum Quality Contributor) | Email Updates: | |
| Status: | Verified | Impact on me: | |
| Category: | MySQL Server: General | Severity: | S3 (Non-critical) |
| Version: | 5.1.23-debug, 8.0.30-debug | OS: | Windows |
| Assigned to: | CPU Architecture: | Any | |
| Tags: | memory leak | ||
[13 Feb 2008 6:32]
Shane Bester
[18 Feb 2008 20:59]
MySQL Verification Team
Thank you for the bug report.
[25 Jan 2022 7:07]
MySQL Verification Team
I left this running overnight on 8.0.30-debug and it seems that a leak really does still exist. It's a very very slow leak though. Dumping heap profile to mybin.hprof.0219.heap (483 MB currently in use) Dumping heap profile to mybin.hprof.0220.heap (484 MB currently in use) Dumping heap profile to mybin.hprof.0221.heap (485 MB currently in use) Dumping heap profile to mybin.hprof.0222.heap (486 MB currently in use) Dumping heap profile to mybin.hprof.0223.heap (518 MB currently in use) Dumping heap profile to mybin.hprof.0226.heap (519 MB currently in use) Dumping heap profile to mybin.hprof.0227.heap (520 MB currently in use) Dumping heap profile to mybin.hprof.0228.heap (521 MB currently in use) Dumping heap profile to mybin.hprof.0229.heap (522 MB currently in use) Dumping heap profile to mybin.hprof.0230.heap (523 MB currently in use) Dumping heap profile to mybin.hprof.0231.heap (524 MB currently in use) Dumping heap profile to mybin.hprof.0232.heap (525 MB currently in use) Dumping heap profile to mybin.hprof.0233.heap (526 MB currently in use) Dumping heap profile to mybin.hprof.0234.heap (527 MB currently in use) Dumping heap profile to mybin.hprof.0235.heap (528 MB currently in use) Dumping heap profile to mybin.hprof.0236.heap (529 MB currently in use) +---------+ | i | +---------+ | 1386000 | +---------+ 1 row in set (14 hours 30 min 10.94 sec) I'll leave it running another 24 hours and check how much memory is used and upload the HEAP profile here...
[25 Jan 2022 12:04]
MySQL Verification Team
Left 8.0.30-debug longer running. Current status: +---------+ | i | +---------+ | 1864000 | +---------+ 1 row in set (19 hours 27 min 1.02 sec) Dumping heap profile to mybin.hprof.0272.heap (563 MB currently in use) Dumping heap profile to mybin.hprof.0273.heap (564 MB currently in use) Dumping heap profile to mybin.hprof.0274.heap (565 MB currently in use)
[26 Jan 2022 4:22]
MySQL Verification Team
By now I'm on: +---------+ | i | +---------+ | 3491000 | +---------+ 1 row in set (1 day 11 hours 45 min 49.59 sec) Dumping heap profile to mybin.hprof.0371.heap (717 MB currently in use) Dumping heap profile to mybin.hprof.0372.heap (718 MB currently in use) Dumping heap profile to mybin.hprof.0373.heap (719 MB currently in use) Dumping heap profile to mybin.hprof.0374.heap (720 MB currently in use) Dumping heap profile to mybin.hprof.0375.heap (721 MB currently in use) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 336731 anon 20 0 2114696 1.1g 67048 S 100.0 3.6 2156:04 mysqld Will upload heap profile shortly
[26 Jan 2022 4:43]
MySQL Verification Team
memory profile
Attachment: mybin.hprof.0375.heap.pdf (application/pdf, text), 19.30 KiB.
[26 Jan 2022 4:44]
MySQL Verification Team
The leak is here:
67108896 (00000000048fbb29) /home/build/git/o_mysql-git/mysys/my_malloc.cc:382:my_raw_malloc
(00000000048fb71b) /home/build/git/o_mysql-git/mysys/my_malloc.cc:258:my_malloc
(000000000335e6b8) /home/build/git/o_mysql-git/include/prealloced_array.h:288:Prealloced_array::reserve
(000000000335dc4b) /home/build/git/o_mysql-git/include/prealloced_array.h:336:Prealloced_array::emplace_back
(000000000335d284) /home/build/git/o_mysql-git/include/prealloced_array.h:317:Prealloced_array::push_back
(0000000003352760) /home/build/git/o_mysql-git/sql/sql_plugin.cc:962:intern_plugin_lock
(0000000003352923) /home/build/git/o_mysql-git/sql/sql_plugin.cc:985:plugin_lock_by_name
(00000000036b0088) /home/build/git/o_mysql-git/sql/handler.cc:387:ha_resolve_by_name_raw
(00000000036558f3) /home/build/git/o_mysql-git/sql/dd_table_share.cc:581:fill_share_from_dd
(000000000365d371) /home/build/git/o_mysql-git/sql/dd_table_share.cc:2290:open_table_def
(000000000322be96) /home/build/git/o_mysql-git/sql/sql_base.cc:7195:open_table_uncached
(00000000033dc15d) /home/build/git/o_mysql-git/sql/sql_table.cc:1003:rea_create_tmp_table
(00000000033f1934) /home/build/git/o_mysql-git/sql/sql_table.cc:8925:create_table_impl
(00000000033f2829) /home/build/git/o_mysql-git/sql/sql_table.cc:9173:mysql_create_table_no_lock
(00000000033f6075) /home/build/git/o_mysql-git/sql/sql_table.cc:10075:mysql_create_table
[26 Jan 2022 4:44]
MySQL Verification Team
Appears to be this extra malloc leaking on debug build!!
static plugin_ref intern_plugin_lock(LEX *lex, plugin_ref rc) {
st_plugin_int *pi = plugin_ref_to_int(rc);
DBUG_TRACE;
mysql_mutex_assert_owner(&LOCK_plugin);
if (pi->state & (PLUGIN_IS_READY | PLUGIN_IS_UNINITIALIZED)) {
plugin_ref plugin;
#ifdef NDEBUG
/* built-in plugins don't need ref counting */
if (!pi->plugin_dl) return pi;
plugin = pi;
#else
/*
For debugging, we do an additional malloc which allows the
memory manager and/or valgrind to track locked references and
double unlocks to aid resolving reference counting problems.
*/
if (!(plugin = (plugin_ref)my_malloc(key_memory_plugin_ref, sizeof(pi),
MYF(MY_WME))))
return nullptr;
*plugin = pi;
#endif
pi->ref_count++;
DBUG_PRINT("info", ("thd: %p, plugin: \"%s\", ref_count: %d", current_thd,
pi->name.str, pi->ref_count));
if (lex) lex->plugins.push_back(plugin);
return plugin;
}
return nullptr;
}
[31 Aug 2023 10:07]
Sanket Sawant
Facing similar issue after upgrading to MySQL 8.0.30
1) we are seeing multiple queries like below in process list with status column as Opening Table and Closing Table.
Query:
SELECT SPECIFIC_SCHEMA AS PROCEDURE_CAT, NULL AS `PROCEDURE_SCHEM`, SPECIFIC_NAME AS `PROCEDURE_NAME`, IFNULL(PARAMETER_NAME, '') AS `COLUMN_NAME`, CASE WHEN PARAMETER_MODE = 'IN' THEN 1 WHEN PARAMETER_MODE = 'OUT' THEN 4 WHEN PARAMETER_MODE = 'INOUT' THEN 2 WHEN ORDINAL_POSITION = 0 THEN 5 ELSE 0 END AS `COLUMN_TYPE`, CASE WHEN UPPER(DATA_TYPE)='DECIMAL' THEN 3 WHEN UPPER(DATA_TYPE)='DECIMAL UNSIGNED' THEN 3 WHEN UPPER(DATA_TYPE)='TINYINT' THEN CASE WHEN LOCATE('(1)', DTD_IDENTIFIER) != 0 THEN -7 ELSE -6 END WHEN UPPER(DATA_TYPE)='TINYINT UNSIGNED' THEN CASE WHEN LOCATE('(1)', DTD_IDENTIFIER) != 0 THEN -7 ELSE -6 END WHEN UPPER(DATA_TYPE)='BOOLEAN' THEN 16 WHEN UPPER(DATA_TYPE)='SMALLINT' THEN 5 WHEN UPPER(DATA_TYPE)='SMALLINT UNSIGNED' THEN 5 WHEN UPPER(DATA_TYPE)='INT' THEN 4 WHEN UPPER(DATA_TYPE)='INT UNSIGNED' THEN 4 WHEN UPPER(DATA_TYPE)='FLOAT' THEN 7 WHEN UPPER(DATA_TYPE)='FLOAT UNSIGNED' THEN 7 WHEN UPPER(DATA_TYPE)='DOUBLE' THEN 8 WHEN UPPER(DATA_TYPE)='DOUBLE UNSIGNED' THEN 8 WHEN UPPER(DATA_TYPE)='NULL' THEN 0 WHEN UPPER(DATA_TYPE)='TIMESTAMP' THEN 93 WHEN UPPER(DATA_TYPE)='BIGINT' THEN -5 WHEN UPPER(DATA_TYPE)='BIGINT UNSIGNED' THEN -5 WHEN UPPER(DATA_TYPE)='MEDIUMINT' THEN 4 WHEN UPPER(DATA_TYPE)='MEDIUMINT UNSIGNED' THEN 4 WHEN UPPER(DATA_TYPE)='DATE' THEN 91 WHEN UPPER(DATA_TYPE)='TIME' THEN 92 WHEN UPPER(DATA_TYPE)='DATETIME' THEN 93 WHEN UPPER(DATA_TYPE)='YEAR' THEN 91 WHEN UPPER(DATA_TYPE)='VARCHAR' THEN 12 WHEN UPPER(DATA_TYPE)='VARBINARY' THEN -3 WHEN UPPER(DATA_TYPE)='BIT' THEN -7 WHEN UPPER(DATA_TYPE)='JSON' THEN -1 WHEN UPPER(DATA_TYPE)='ENUM' THEN 1 WHEN UPPER(DATA_TYPE)='SET' THEN 1 WHEN UPPER(DATA_TYPE)='TINYBLOB' THEN -3 WHEN UPPER(DATA_TYPE)='TINYTEXT' THEN 12 WHEN UPPER(DATA_TYPE)='MEDIUMBLOB' THEN -4 WHEN UPPER(DATA_TYPE)='MEDIUMTEXT' THEN -1 WHEN UPPER(DATA_TYPE)='LONGBLOB' THEN -4 WHEN UPPER(DATA_TYPE)='LONGTEXT' THEN -1 WHEN UPPER(DATA_TYPE)='BLOB' THEN -4 WHEN UPPER(DATA_TYPE)='TEXT' THEN -1 WHEN UPPER(DATA_TYPE)='CHAR' THEN 1 WHEN UPPER(DATA_TYPE)='BINARY' THEN -2 WHEN UPPER(DATA_TYPE)='GEOMETRY' THEN -2 WHEN UPPER(DATA_TYPE)='UNKNOWN' THEN 1111 WHEN UPPER(DATA_TYPE)='POINT' THEN -2 WHEN UPPER(DATA_TYPE)='LINESTRING' THEN -2 WHEN UPPER(DATA_TYPE)='POLYGON' THEN -2 WHEN UPPER(DATA_TYPE)='MULTIPOINT' THEN -2 WHEN UPPER(DATA_TYPE)='MULTILINESTRING' THEN -2 WHEN UPPER(DATA_TYPE)='MULTIPOLYGON' THEN -2 WHEN UPPER(DATA_TYPE)='GEOMETRYCOLLECTION' THEN -2 WHEN UPPER(DATA_TYPE)='GEOMCOLLECTION' THEN -2 ELSE 1111 END AS `DATA_TYPE`, UPPER(CASE WHEN LOCATE('UNSIGNED', UPPER(DATA_TYPE)) != 0 AND LOCATE('UNSIGNED', UPPER(DATA_TYPE)) = 0 THEN CONCAT(DATA_TYPE, ' UNSIGNED') ELSE DATA_TYPE END) AS `TYPE_NAME`, NUMERIC_PRECISION AS `PRECISION`, CASE WHEN LCASE(DATA_TYPE)='date' THEN 10 WHEN LCASE(DATA_TYPE)='time' THEN 8 WHEN LCASE(DATA_TYPE)='datetime' THEN 19 WHEN LCASE(DATA_TYPE)='timestamp' THEN 19 WHEN CHARACTER_MAXIMUM_LENGTH IS NULL THEN NUMERIC_PRECISION WHEN CHARACTER_MAXIMUM_LENGTH > 2147483647 THEN 2147483647 ELSE CHARACTER_MAXIMUM_LENGTH END AS LENGTH,NUMERIC_SCALE AS `SCALE`, 10 AS RADIX,1 AS `NULLABLE`, NULL AS `REMARKS`, NULL AS `COLUMN_DEF`, NULL AS `SQL_DATA_TYPE`, NULL AS `SQL_DATETIME_SUB`, CHARACTER_OCTET_LENGTH AS `CHAR_OCTET_LENGTH`, ORDINAL_POSITION, 'YES' AS `IS_NULLABLE`, SPECIFIC_NAME FROM INFORMATION_SCHEMA.PARAMETERS WHERE SPECIFIC_SCHEMA LIKE 'dbname' AND SPECIFIC_NAME LIKE 'SP_Name' AND (PARAMETER_NAME LIKE '%' OR PARAMETER_NAME IS NULL) ORDER BY SPECIFIC_SCHEMA, SPECIFIC_NAME, ROUTINE_TYPE, ORDINAL_POSITION
Above queries are not long running but frequently we are seeing this query its like whenever API calls Sp, this query is fired to search in parameter table.
[31 Aug 2023 14:09]
MySQL Verification Team
Sanket Sawant, your issue isn't related to this report. This is a bug report specific to debug builds.
