Bug #25244 Consider compiling MySQL Server against the Google TCMalloc library
Submitted: 22 Dec 2006 12:14 Modified: 13 Nov 2007 18:12
Reporter: Lenz Grimmer
Status: Verified
Category:Server: Compiling Severity:S4 (Feature request)
Version: OS:Any
Assigned to: Target Version:
Triage: Triaged: D5 (Feature request)

[22 Dec 2006 12:14] Lenz Grimmer 
Description:
The Google Performance tools at http://goog-perftools.sourceforge.net/ contain an
improved malloc() implementation that is supposed to boost performance significantly in
comparison to the glibc version of malloc()

http://goog-perftools.sourceforge.net/doc/tcmalloc.html

"TCMalloc also reduces lock contention for multi-threaded programs. For small objects,
there is virtually zero contention. For large objects, TCMalloc tries to use fine grained
and efficient spinlocks."

How to repeat:
Compile MySQL against TCMalloc and compare the performance in benchmarks against glibc's
malloc.

Suggested fix:
TCMalloc is BSD licensed, so it should be possible to include it in the MySQL server
sources. The autoconf and Makefiles need to be updated to support linking against this
library instead against glibc.
[29 Dec 2006 12:40] Chris Samuel 
Should not even need to recompile, you can use LD_PRELOAD (if you are brave enough) to
force it to hit that library in preference to the glibc malloc().
[2 Jan 2007 15:11] Lenz Grimmer 
Good point, Samuel - this should be helpful to evaluate the general usefulness of using
this library for MySQL by benchmarking the difference to glibc.
[13 Mar 2007 8:56] Valeriy Kravchuk 
Bug #27063 marked as a duplicate of this one.
[14 Mar 2007 4:05] Arjen Lentz 
Should this not be noted as an S5 (Performance) rather than S4 (Feature)?
This change seriously improves scalability (multi-threading concurrency), affecting
users/customers.

See also http://bugs.mysql.com/27063 for additional comments/insights.
[13 Nov 2007 18:12] Valeriy Kravchuk 
Thank you for a reasonable feature request.
[1 Dec 2008 18:36] Mark Callaghan 
-ltcmalloc_minimal works much better than -ltcmalloc for me. The minimal version doesn't
sample the stacktrace for allocation profiling which means it doesn't depend on
libunwind.
[1 Dec 2008 19:07] Mark Callaghan 
More requests:
* provide the equivalent of SHOW TCMALLOC STATUS by calling
MallocExtension::instance()->GetStats(...)

* provide RELEASE MEMORY -- by calling MallocExtension::instance()->ReleaseFreeMemory()

* provide a my.cnf variable to set the default number of bytes a thread should 
cache. This might be an issue for MySQL deployments with 1000+ threads. It can be set by
a call to MallocExtension::SetNumericProperty(
          "tcmalloc.max_total_thread_cache_bytes",
          (size_t) opt_tcmalloc_max_thread_cache_size))

A forthcoming Google patch contains all of this.