Bug #82920 Unable to start mysqld due to changes in mysqld_safe
Submitted: 9 Sep 2016 6:42 Modified: 3 Feb 2017 14:45
Reporter: Simon Mudd (OCA) Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: Command-line Clients Severity:S3 (Non-critical)
Version:5.5.52, 5.6.33, 5.7.15 OS:CentOS (6.X)
Assigned to: CPU Architecture:Any
Tags: mysqld_safe, regression, startup

[9 Sep 2016 6:42] Simon Mudd
On startup on 5.7.15 on CentOS 6 mysqld_safe prevents the init script starting the server with my configuration.

This used to work on MySQL 5.5 / 5.6 and 5.7 and now breaks on 5.7.15. I guess CentOS 7 uses systemd so should be ok (but not verified).

This seems to be caused by your "bugfix" as described in the release notes:

https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-15.html says:

"For mysqld_safe, the argument to --malloc-lib now must be one of the directories /usr/lib, /usr/lib64, /usr/lib/i386-linux-gnu, or /usr/lib/x86_64-linux-gnu. In addition, the --mysqld and --mysqld-version options can be used only on the command line and not in an option file. (Bug #24464380)"

So you're attempting to control something without understanding why it is used.

How to repeat:
Starting a server shows:

# /etc/init.d/mysqld  start
 mysqld_safe --mysqld option can only be used as command line option, found in config file
MySQL Daemon failed to start.
Starting mysqld:                                           [FAILED]

The reason is my /etc/my.cnf file has the following:

log-error = /var/log/mysqld.log
pid_file = /var/run/mysqld/mysqld.pid
core-file-size = unlimited
ledir  = /path/to/some/wrapper
mysqld = mysqld_wrapper

The wrapper script was due to reasons:

# /etc/my.cnf and /etc/init.d/mysql do not provide ways to adjust the
# behaviour of the mysqld that is being started, apart from changing certain
# mysqld parameters. However, sometimes the "environment" that the mysqld
# process is running in needs to be adjusted.
# Two current reasons for this are:
# 1. memory allocation on a NUMA architecture machine may cause swapping
#    when the box seems to have plenty of free memory. numactl helps to
#    control this.
# 2. ulimit settings: CentOS 6 sets a default NPROC (number of running processes
#    or threads limit) for all users, which is below the expected value
#    that might be needed by a mysqld server (max_connections + a few extra threads).
#    Calling ulimit prior to starting mysqld resolves this and only for MySQL
#    thus reducing the scope of the change.

# flush the linux filesystem buffer cache prior to starting MySQL
# and also prior to doing the NUMA stuff. Again this is to try to
# prevent the linux kernel using swap unnecessarily.
echo 3 > /proc/sys/vm/drop_caches

mysqld=/usr/sbin/mysqld         # Fix this if we need to later

# Ensure that we have a max user processes setting to a value that
# is significantly higher than _any_ possible max_connections setting
# we may want to configure on the server.
# Note: this setting is for the max number of concurrent threads.
# Given the highest setting we use is usually around 3000 max_connections
# this setting is 10 x that amount.  Make this limit a soft limit, so
# potentially we can increase it later...
ulimit -Su 30000

# use exec to avoid having an extra shell around.
exec $numactl --interleave all $mysqld "$@"

This has been running on MySQL 5.6 and 5.7 for several years (since 2012) and has prevented several issues.

Suggested fix:
Clearly you're trying to fix some sort of bug but are not aware how this option has been used. 

Please revert this as the logic you are trying to impose makes no sense at all and prevents me from modifying the runtime environment on startup which has been essential.
[9 Sep 2016 6:55] Simon Mudd
Change the Synopsis to make the "problem" clearer. The description in the ticket shows why this happens.
[9 Sep 2016 7:20] Shane Bester
Verified, it fails with message:
mysqld_safe --mysqld option can only be used as command line option, found in config file
[3 Feb 2017 14:45] Paul Dubois
Posted by developer:
Noted in 5.5.55, 5.6.36, 5.7.18, 8.0.1 changelogs.

Changes made to mysqld_safe in recent MySQL releases require the
--ledir, --mysqld, --mysqld-version options to be specified on the
command line; they can no longer be specified in option files. This
could cause failure of init scripts that invoke mysqld_safe. Such
scripts now pass MYSQLD_OPTS as the first command-line argument to
mysqld_safe, with the value set to such command line-only mysqld_safe
option values as may be required. On platforms that use systemd, the
MYSQLD_OPTS value can be set in /etc/sysconfig/mysqld with a line
such as this:

MYSQLD_OPTS=" --ledir=/mysqld_ledir --mysqld=my_wrapper "

The value of MYSQLD_OPTS can also include mysqld options for
mysqld_safe to pass to mysqld.