MySQL Bugs: #74648: MySQL /etc/init.d/mysqld init file changes cause breakages (yum packaging)

Bug #74648	MySQL /etc/init.d/mysqld init file changes cause breakages (yum packaging)
Submitted:	31 Oct 2014 11:32	Modified:	14 Apr 2015 10:17
Reporter:	Simon Mudd (OCA)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Packaging	Severity:	S2 (Serious)
Version:	5.7.5,5.6	OS:	Linux (CentOS 6)
Assigned to:		CPU Architecture:	Any
Tags:	init scripts, rpm, yum

Description:
When moving from the MySQL community 5.6 rpms to the MySQL 5.7 "yum" rpms various things have changed in rpm packaging. One of those changes is how the init script works.

One feature that I use heavily is the ability when using start or restart to add extra options which are passed on to mysqld but which are not part of the normal config. This is for example during initial server startup, upgrades, or reconfiguration. There are several reasons.

So previously I was able to do things like:

# /etc/init.d/mysql start --skip-slave-start --read-only --bind-address=127.0.0.1
# /etc/init.d/mysql start --init-file=/path/to/temporary/init.sql
# /etc/init.d/mysql start --relay-log-recovery=1
# /etc/init.d/mysql start --skip-grant-tables
# /etc/init.d/mysql start --skip-slave-start
# /etc/init.d/mysql start --skip-slave-start --skip-grant-tables --skip-networking

How to repeat:
The new /etc/init.d/mysqld script in MySQL 5.7 removes this functionality.

One might argue that we can edit the config file /etc/my.cnf but in reality if I make several changes as part of a build process then it really makes no sense to have to edit a config file for each stage of processing when the original mechanism worked very well and was quite useful and flexible.

Other adhoc config changes also fall into this category. E.g. If I want to start up a server but not start replication this fails (silently):

[root@myhost57 ~]# mysql -e"SHOW SLAVE STATUS\G" | grep Running
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
[root@myhost57 ~]# /etc/init.d/mysqld restart --skip-slave-start
Stopping mysqld:                                           [  OK  ]
Starting mysqld:                                           [  OK  ]
[root@myhost57 ~]# mysql -e"SHOW SLAVE STATUS\G" | grep Running
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
[root@myhost57 ~]# rpm -qa | grep y-server
mysql-community-server-5.7.5-0.6.m15.el6.x86_64

(replication still running)

vs the original 5.6 MySQL-server behaviour:

[root@myhost56 ~]#  mysql -e"SHOW SLAVE STATUS\G" | grep Running
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
[root@myhost56 ~]#  /etc/init.d/mysql restart --skip-slave-start
Shutting down MySQL........                                [  OK  ]
Starting MySQL................                             [  OK  ]
[root@myhost56 ~]#  mysql -e"SHOW SLAVE STATUS\G" | grep Running
             Slave_IO_Running: No
            Slave_SQL_Running: No
      Slave_SQL_Running_State: 
[root@myhost56 ~]# rpm -qa | grep MySQL-server
MySQL-server-5.6.14-1.el6.x86_64
[root@myhost56 ~]#

(replication stopped as requested)

This change in behaviour is very intrusive for tooling used to setup servers. Editing /etc/my.cnf is not really an option.

Suggested fix:
While many people on simpler setups may not see the use of this type of functionality I use it quite heavily and the change in rpm packaging seems to have removed this functionality without notice. Perhaps it was not intended or understood but the original MySQL-server rpm packaging has had this functionality since MySQL 5.0.

Please therefore re-enable this useful functionality.

Note: related to http://bugs.mysql.com/bug.php?id=72230

Actually this is not MySQL 5.7 specific but "the yum rpm build system specific", so going to tag as a 5.6 issue.  I don't use the yum community 5.6 rpms so haven't been affected by this before.

Adjust title to be clearer.

Hi Simon,

thanks for your report.

Sorry, for not documenting this change.

Adding the feature back is possible, however it's not a generic solution
as with introduction of systemd and service files there are no initscript
at all.

The need for e.g. --skip-grant-tables clearly exists I agree.

An other solution might be to read /etc/sysconfig/mysql 

Sample contents:

OPTS="--skip-grant-tables --skip-networking"

What do you think, other ways we can modify mysqld start up without too much trouble?

The move to systemd is something which I've not had to look at yet (luckily). It makes a large number of changes everywhere and indeed for many things it's very good.

However the need I'm talking about is for a DBA to do "tasks" when setting up or building a server. I gave examples but not the use cases:

(1) initial installation
(2) upgrades from an older version where the current grant system is now not understood by MySQL.
(3) configuration of a slave in a stopped state
(4) running special processing on startup
(5) starting up a server in read-only mode to prevent applications changing data
(6) starting up a server either bound to 127.0.0.1 or with networking disabled so that applications can not access the server while it's configuration is being adjusted.
(7) starting up a slave/server after a crash in a way which does / does not clean up old relay log files [2 different use cases here]

and these are the ones that come to mind without thinking too hard.
These are real use cases which I have on a day to day basis.

It's also true that the normal setup is stop or start MySQL and thats it. So if talking about systemd that's fine.

You need to make sure the DBA can do these other tasks somehow in a clean and clear fashion. The previously used method which has worked for some time has worked well and given the flexibility to do this.

Reading /etc/sysconfig/mysql does not work as we're not talking about static configuration settings.

If your suggestion of OPTS="--skip-grant-tables --skip-networking" means doing something like

# OPTS="--skip-grant-tables --skip-networking" /etc/init.d/mysqld start

Then that at least does follow a more traditional UNIX approach of providing an environment variable to the command which affects it's behaviour. That would work for this sort of case but I'd suggest a more descriptive variable name like MYSQLD_OPTS or similar.

I'm not familiar enough with systemd to know how this might work on a server using systemd, so feedback is welcome.

I'm not yet using MySQL CentOS 7 and one of the reasons is because of this huge change and the need to adapt current usage to work with systemd. So my current comments are more clearly focused really on CentOS / RHEL 6 (and 5 to some extent) environments where these changes are currently causing problems.

If you're familiar enough with systemd and the usage I'm talking about then suggestions on how these sorts of tasks can be done (remember they're part of a large build process) then that sort of feedback would be most welcome.

Hi Simon,

for platforms using initscript we are likely to add the feature back.

For new systemd it seems some research are required, current status
is not very user friendly.

Thank you for the response.
I think that's good news.

My thought for the case of systems running systemd might be to simply provide a command line tool (rename the script and put it in /usr/bin/ ?) which behaves the same way as the current /etc/init.d/mysqld and that would allow us to use that tool instead for this type of situations. (mysqld_init ?) I guess the only possible concern would be interference with the existing systemd behaviour, but I'd expect that perhaps that script can interact with systemd in such a way to not make that an issue.

I understand this requires some thought, but certainly fixing the issue for OS variants that do not use systemd yet, such as CentOS6/RHEL6/OEL6 etc would be a good start.

The feature has been added back to the init script.

Yes, thanks.
Works fine now, so thanks for resolving this.

Note: fixed in 5.7.6, as that's not shown here.

Hi Simon,

commit fixing this is here:

 https://github.com/mysql/mysql-server/commit/8be836297e77b783f5b7b01f5371272907f70004

Note the comment about systemd included in commit message.

Also have a look at the MySQL systemd documentation:

 https://dev.mysql.com/doc/refman/5.7/en/server-management-using-systemd.html