MySQL Bugs: #25244: Consider compiling MySQL Server against the Google TCMalloc library

Bug #25244	Consider compiling MySQL Server against the Google TCMalloc library
Submitted:	22 Dec 2006 11:14	Modified:	5 Oct 2010 10:24
Reporter:	Lenz Grimmer	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Compiling	Severity:	S4 (Feature request)
Version:		OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
The Google Performance tools at http://goog-perftools.sourceforge.net/ contain an improved malloc() implementation that is supposed to boost performance significantly in comparison to the glibc version of malloc()

http://goog-perftools.sourceforge.net/doc/tcmalloc.html

"TCMalloc also reduces lock contention for multi-threaded programs. For small objects, there is virtually zero contention. For large objects, TCMalloc tries to use fine grained and efficient spinlocks."

How to repeat:
Compile MySQL against TCMalloc and compare the performance in benchmarks against glibc's malloc.

Suggested fix:
TCMalloc is BSD licensed, so it should be possible to include it in the MySQL server sources. The autoconf and Makefiles need to be updated to support linking against this library instead against glibc.

Should not even need to recompile, you can use LD_PRELOAD (if you are brave enough) to force it to hit that library in preference to the glibc malloc().

Good point, Samuel - this should be helpful to evaluate the general usefulness of using this library for MySQL by benchmarking the difference to glibc.

Bug #27063 marked as a duplicate of this one.

Should this not be noted as an S5 (Performance) rather than S4 (Feature)?
This change seriously improves scalability (multi-threading concurrency), affecting users/customers.

See also http://bugs.mysql.com/27063 for additional comments/insights.

Thank you for a reasonable feature request.

-ltcmalloc_minimal works much better than -ltcmalloc for me. The minimal version doesn't sample the stacktrace for allocation profiling which means it doesn't depend on libunwind.

More requests:
* provide the equivalent of SHOW TCMALLOC STATUS by calling MallocExtension::instance()->GetStats(...)

* provide RELEASE MEMORY -- by calling MallocExtension::instance()->ReleaseFreeMemory()

* provide a my.cnf variable to set the default number of bytes a thread should 
cache. This might be an issue for MySQL deployments with 1000+ threads. It can be set by a call to MallocExtension::SetNumericProperty(
          "tcmalloc.max_total_thread_cache_bytes",
          (size_t) opt_tcmalloc_max_thread_cache_size))

A forthcoming Google patch contains all of this.

It seems this subject was brought up in several separate bug reports which lack(ed) coordination :(

Work was done in the context of bug#47549, whose results took effect in MySQL 5.4.3: From that version, the start script would support options to make the server use tcmalloc, and such binaries were included in the MySQL 5.4.3 binary packages (Linux on x86 and x86_64, no others).

The start script change to use tcmalloc remained in the sources,
but the build tool changes to include tcmalloc in the packages fell victim to switching the release build process from autotools to cmake :(

This has been brought up again in bug#56267 which is now fixed, I expect that from version 5.5.7 onward the binary MySQL packages will again contain "libtcmalloc_minimal.so".

So I consider this feature request handled by bug#47549 and bug#56267 and will close it. "Duplicate" is not really appropriate, because this one is the oldest/first.
If Mark's other comments of Dec 1, 2008, are still a concern, please file a new request, so that it is easier to keep track of what is still needed.