Bug #47549 mysqld_safe: add support for preloading tcmalloc
Submitted: 23 Sep 2009 11:44 Modified: 28 Oct 2009 21:26
Reporter: Lars Heill Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: General Severity:S4 (Feature request)
Version:5.4 OS:Linux
Assigned to: Timothy Smith CPU Architecture:Any

[23 Sep 2009 11:44] Lars Heill
Description:
Add support for preloading tcmalloc in mysqld_safe.

How to repeat:
N/A, feature request

Suggested fix:
Add command line option instructing use of either co-packaged, system or user specified tcmalloc lib.
[1 Oct 2009 23:15] Paul Dubois
Noted in 5.4.3 changelog.

On Linux (and perhaps other systems), the performance of MySQL Server
can be improved by using a different malloc() implementation,
developed by Google and called tcmalloc. The gain is noticeable with
a higher number of simultaneous users. MySQL 5.4.1 was linked against
this library, which improved performance but imposed some
restrictions. As a result of these restrictions, MySQL 5.4.3 uses a
different approach:

* The server is linked against the default malloc() provided by the
  respective platform.

* Binary distributions for Linux include libtcmalloc_minimal.so (the
  shared library that can be linked against at runtime) in pkglibdir
  (that is, the same directory within the package where server plugins
  and similar object files are located). The version of tcmalloc
  included with MySQL 5.4.3 comes from google-perftools 1.4.

  If you want to try tcmalloc but are using a binary distribution for a
  non-Linux platform or a source distribution, you can install Google's
  tcmalloc_minimal. Some distributions provide it in a google-perftools
  package or with a similar name, or you can download it from Google at
  http://code.google.com/p/google-perftools/ and compile it yourself.

* mysqld_safe now supports a --malloc-lib option that enables
  administrators to specify that mysqld should use tcmalloc.

The --malloc-lib option works by modifying the LD_PRELOAD environment
value to affect dynamic linking so that the loader will find tcmalloc
when mysqld runs:

* If the option is not given, or is given without a value
  (--malloc-lib=), LD_PRELOAD is not modified and no attempt is made to
  use tcmalloc.

* If the option is given as --malloc-lib=tcmalloc, mysqld_safe looks
  for a tcmalloc library in /usr/lib and then in the MySQL pkglibdir
  location (for example, /usr/local/mysql/lib or whatever is
  appropriate). If tmalloc is found, its path name is added to the
  beginning of the LD_PRELOAD value for mysqld. If tcmalloc is not
  found, mysqld_safe aborts with an error.

* If the option is given as --malloc-lib=/path/to/some/library, that
  full path is added to the beginning of the LD_PRELOAD value. If the
  full path points to a nonexistent or unreadable file, mysqld_safe
  aborts with an error.

* For cases where mysqld_safe adds a path name to LD_PRELOAD, it adds
  the path to the beginning of any existing value the variable already
  has.  

As a result of the preceding changes, Linux users can use the
libtcmalloc_minimal.so now included in binary packages by adding
these lines to their my.cnf file:

[mysqld_safe]
malloc-lib=tcmalloc

Those lines also suffice for users on any platform who have installed
a tcmalloc package in /usr/lib. To use a specific tcmalloc library,
specify its full path name. Example:

[mysqld_safe]
malloc-lib=/opt/lib/libtcmalloc_minimal.so
[1 Oct 2009 23:18] Paul Dubois
Noted in 5.4.3 changelog.

On Linux (and perhaps other systems), the performance of MySQL Server
can be improved by using a different malloc() implementation,
developed by Google and called tcmalloc. The gain is noticeable with
a higher number of simultaneous users. MySQL 5.4.1 was linked against
this library, which improved performance but imposed some
restrictions. As a result of these restrictions, MySQL 5.4.3 uses a
different approach:

* The server is linked against the default malloc() provided by the
  respective platform.

* Binary distributions for Linux include libtcmalloc_minimal.so (the
  shared library that can be linked against at runtime) in pkglibdir
  (that is, the same directory within the package where server plugins
  and similar object files are located). The version of tcmalloc
  included with MySQL 5.4.3 comes from google-perftools 1.4.

  If you want to try tcmalloc but are using a binary distribution for a
  non-Linux platform or a source distribution, you can install Google's
  tcmalloc_minimal. Some distributions provide it in a google-perftools
  package or with a similar name, or you can download it from Google at
  http://code.google.com/p/google-perftools/ and compile it yourself.

* mysqld_safe now supports a --malloc-lib option that enables
  administrators to specify that mysqld should use tcmalloc.

The --malloc-lib option works by modifying the LD_PRELOAD environment
value to affect dynamic linking so that the loader will find tcmalloc
when mysqld runs:

* If the option is not given, or is given without a value
  (--malloc-lib=), LD_PRELOAD is not modified and no attempt is made to
  use tcmalloc.

* If the option is given as --malloc-lib=tcmalloc, mysqld_safe looks
  for a tcmalloc library in /usr/lib and then in the MySQL pkglibdir
  location (for example, /usr/local/mysql/lib or whatever is
  appropriate). If tmalloc is found, its path name is added to the
  beginning of the LD_PRELOAD value for mysqld. If tcmalloc is not
  found, mysqld_safe aborts with an error.

* If the option is given as --malloc-lib=/path/to/some/library, that
  full path is added to the beginning of the LD_PRELOAD value. If the
  full path points to a nonexistent or unreadable file, mysqld_safe
  aborts with an error.

* For cases where mysqld_safe adds a path name to LD_PRELOAD, it adds
  the path to the beginning of any existing value the variable already
  has.  

As a result of the preceding changes, Linux users can use the
libtcmalloc_minimal.so now included in binary packages by adding
these lines to their my.cnf file:

[mysqld_safe]
malloc-lib=tcmalloc

Those lines also suffice for users on any platform who have installed
a tcmalloc package in /usr/lib. To use a specific tcmalloc library,
specify its full path name. Example:

[mysqld_safe]
malloc-lib=/opt/lib/libtcmalloc_minimal.so
[1 Oct 2009 23:21] Paul Dubois
Noted in 5.4.3 changelog.

On Linux (and perhaps other systems), the performance of MySQL Server
can be improved by using a different malloc() implementation,
developed by Google and called tcmalloc. The gain is noticeable with
a higher number of simultaneous users. MySQL 5.4.1 was linked against
this library, which improved performance but imposed some
restrictions. As a result of these restrictions, MySQL 5.4.3 uses a
different approach:

* The server is linked against the default malloc() provided by the
  respective platform.

* Binary distributions for Linux include libtcmalloc_minimal.so (the
  shared library that can be linked against at runtime) in pkglibdir
  (that is, the same directory within the package where server plugins
  and similar object files are located). The version of tcmalloc
  included with MySQL 5.4.3 comes from google-perftools 1.4.

  If you want to try tcmalloc but are using a binary distribution for a
  non-Linux platform or a source distribution, you can install Google's
  tcmalloc_minimal. Some distributions provide it in a google-perftools
  package or with a similar name, or you can download it from Google at
  http://code.google.com/p/google-perftools/ and compile it yourself.

* mysqld_safe now supports a --malloc-lib option that enables
  administrators to specify that mysqld should use tcmalloc.

The --malloc-lib option works by modifying the LD_PRELOAD environment
value to affect dynamic linking so that the loader will find tcmalloc
when mysqld runs:

* If the option is not given, or is given without a value
  (--malloc-lib=), LD_PRELOAD is not modified and no attempt is made to
  use tcmalloc.

* If the option is given as --malloc-lib=tcmalloc, mysqld_safe looks
  for a tcmalloc library in /usr/lib and then in the MySQL pkglibdir
  location (for example, /usr/local/mysql/lib or whatever is
  appropriate). If tmalloc is found, its path name is added to the
  beginning of the LD_PRELOAD value for mysqld. If tcmalloc is not
  found, mysqld_safe aborts with an error.

* If the option is given as --malloc-lib=/path/to/some/library, that
  full path is added to the beginning of the LD_PRELOAD value. If the
  full path points to a nonexistent or unreadable file, mysqld_safe
  aborts with an error.

* For cases where mysqld_safe adds a path name to LD_PRELOAD, it adds
  the path to the beginning of any existing value the variable already
  has.  

As a result of the preceding changes, Linux users can use the
libtcmalloc_minimal.so now included in binary packages by adding
these lines to their my.cnf file:

[mysqld_safe]
malloc-lib=tcmalloc

Those lines also suffice for users on any platform who have installed
a tcmalloc package in /usr/lib. To use a specific tcmalloc library,
specify its full path name. Example:

[mysqld_safe]
malloc-lib=/opt/lib/libtcmalloc_minimal.so
[28 Oct 2009 21:26] Timothy Smith
This was committed and included in mysql-5.4.3, and is present in the mysql-trunk branch.