MySQL Bugs: #9673: valgrind shows not freed memory in InnoDB

Bug #9673	valgrind shows not freed memory in InnoDB
Submitted:	6 Apr 2005 11:26	Modified:	25 May 2005 10:39
Reporter:	Michael Widenius	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: InnoDB storage engine	Severity:	S3 (Non-critical)
Version:	5.0.3	OS:	Linux (Linux)
Assigned to:	Marko Mäkelä	CPU Architecture:	Any

Description:
mysql-test-run --skip-ndb --force --valgrind --valgrind-all t/auto_increment

Produces a list of not freed memory in InnoDB
(This is with valgrind 2.4)

Here is the errors (these are generated for a lot of tests)

=22171== 1056 bytes in 3 blocks are still reachable in loss record 4 of 7
==22171==    at 0x1B905BAC: malloc (in /usr/lib/valgrind/vgpreload_memcheck.so)
==22171==    by 0x1BB558E4: _IO_fdopen@@GLIBC_2.1 (in /lib/tls/libc.so.6)
==22171==    by 0x8459CE4: os_file_create_tmpfile (os0file.c:556)
==22171==    by 0x83AC8DC: dict_init (dict0dict.c:714)
==22171==    by 0x83A82A1: dict_boot (dict0boot.c:231)
==22171==    by 0x83A9323: dict_create (dict0boot.c:420)
==22171==    by 0x83A735E: innobase_start_or_create_for_mysql (srv0start.c:1381)
==22171==    by 0x829846D: innobase_init() (ha_innodb.cc:1262)
==22171==    by 0x8284893: ha_init() (handler.cc:385)
==22171==    by 0x81CC4E3: init_server_components() (mysqld.cc:2790)
==22171==    by 0x81CCA26: main (mysqld.cc:3095)

==22171== 19963702 bytes in 40106 blocks are still reachable in loss record 7 of
 7
==22171==    at 0x1B905BAC: malloc (in /usr/lib/valgrind/vgpreload_memcheck.so)
==22171==    by 0x8456AEF: ut_malloc_low (ut0mem.c:81)
==22171==    by 0x8456C71: ut_malloc (ut0mem.c:177)
==22171==    by 0x83E0607: yy_flex_alloc (lex.yy.c:2559)
==22171==    by 0x83E045A: yy_create_buffer (lex.yy.c:2234)
==22171==    by 0x83DF6DA: yylex (lex.yy.c:849)
==22171==    by 0x83DE1E0: yyparse (bison.simple:443)
==22171==    by 0x83DD9B5: pars_sql (pars0pars.c:1760)
==22171==    by 0x83AAC9F: dict_create_or_check_foreign_constraint_tables (dict0
crea.c:1228)
==22171==    by 0x83A74B8: innobase_start_or_create_for_mysql (srv0start.c:1541)
==22171==    by 0x829846D: innobase_init() (ha_innodb.cc:1262)
==22171==    by 0x8284893: ha_init() (handler.cc:385)
==22171==    by 0x81CC4E3: init_server_components() (mysqld.cc:2790)
==22171==    by 0x81CCA26: main (mysqld.cc:3095)

How to repeat:
mysql-test-run --skip-ndb --force --valgrind --valgrind-all t/auto_increment

Marko,

please look at this.

Heikki

Monty,
Did you by any chance use innodb_fast_shutdown? In that case, innobase_shutdown_for_mysql() will return before closing dict_foreign_err_file and calling ut_free_all_mem() in order to free all memory allocated by ut_malloc(). In other words, that would be the expected behaviour.

I couldn't repeat these problems with innodb_fast_shutdown=0, but I got some weird traces:

==19715== Syscall param write(buf) points to uninitialised byte(s)
==19715==    at 0x83C0024: write (in /home/msmakela/innobase/mysql-5.0.4/sql/mysqld)
==19715==    by 0x833A42F: os_thread_create (os0thread.c:158)
==19715==    by 0x8209361: innobase_start_or_create_for_mysql (srv0start.c:1555)
==19715==    by 0x81A020D: innobase_init() (ha_innodb.cc:1282)
==19715==    by 0x819512B: ha_init() (handler.cc:381)
==19715==    by 0x80E3E64: init_server_components() (mysqld.cc:2782)
==19715==    by 0x80E7B24: main (mysqld.cc:3087)
==19715==  Address 0x52BFA4AC is on thread 1's stack

Judging from the disassembly, the write() call is in __pthread_create_2_1, whatever that is. I was using a recent mysql 5.0.4-bk on GNU/Linux with kernel 2.6 and GNU libc 2.3.2, compiled with gcc 3.4.4 20050314 (prerelease) and linked statically.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Monty,
you should pass --skip-innodb-fast-shutdown to mysqld when testing for memory leaks. Otherwise InnoDB will skip most of its shutdown procedures, including the step to free all memory pools. (The memory pools will be freed by the operating system anyway.)

Sorry, it is indeed a bug. Someone changed 5.0 so that innobase_shutdown_for_mysql() will essentially crash InnoDB if innodb_fast_shutdown=1 (the default). The shutdown procedure should only be skipped if innodb_fast_shutdown=2.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/25257

Thank you for your bug report. This issue has been committed to our
source repository of that product and will be incorporated into the
next release.

If necessary, you can access the source repository and build the latest
available version, including the bugfix, yourself. More information 
about accessing the source trees is available at
    http://www.mysql.com/doc/en/Installing_source_tree.html

Additional info:

When the option innodb_fast_shutdown=2 was introduced, the InnoDB shutdown procedure was modified so that it would essentially crash InnoDB if innodb_fast_shutdown differs from 0. With innodb_fast_shutdown=1, InnoDB should wait for all threads to exit and release all allocated memory, as it does in MySQL 4.1.