MySQL Bugs: #79442: MySQL 5.7 Server start failing after installation

Bug #79442	MySQL 5.7 Server start failing after installation
Submitted:	28 Nov 2015 9:59	Modified:	18 Dec 2015 18:39
Reporter:	Pradeep Agrawal	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Packaging	Severity:	S1 (Critical)
Version:	5.7	OS:	CentOS (CentOS release 6.6 (Final))
Assigned to:	Balasubramanian Kandasamy	CPU Architecture:	Any

Description:
After instllation MySQL start is failing, please find the details. 
# wget http://repo.mysql.com//mysql57-community-release-el6-7.noarch.rpm
# yum localinstall mysql57-community-release-el6-7.noarch.rpm
# yum install mysql-community-server
[root@pagrawal-oracle12-mysql57-1 ~]# service mysqld start
Initializing MySQL database:  2015-11-28T08:47:47.620414Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-11-28T08:47:47.623820Z 0 [ERROR] Can't change data directory owner to mysql
2015-11-28T08:47:47.623862Z 0 [ERROR] Aborting

                                                           [FAILED]
[root@pagrawal-oracle12-mysql57-1 ~]# service mysqld start
Initializing MySQL database:  2015-11-28T08:48:08.216504Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-11-28T08:48:08.219238Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
2015-11-28T08:48:08.219273Z 0 [ERROR] Aborting

                                                           [FAILED]
[root@pagrawal-oracle12-mysql57-1 ~]# service mysqld start
Initializing MySQL database:  2015-11-28T08:48:13.080433Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-11-28T08:48:13.083365Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
2015-11-28T08:48:13.083404Z 0 [ERROR] Aborting

                                                           [FAILED]
[root@pagrawal-oracle12-mysql57-1 ~]# service mysqld start
Initializing MySQL database:  2015-11-28T08:48:16.650635Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-11-28T08:48:16.653302Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
2015-11-28T08:48:16.653338Z 0 [ERROR] Aborting

                                                           [FAILED]

How to repeat:
Try above repo in cent os 6.6

If you are doing and fresh install (without older data) just remove the data directory and restart the server. Thanks.

Hi!

This seems similar to https://bugs.mysql.com/bug.php?id=78591

Do you have SELinux enabled?

You can test by doing:

$ selinuxenabled; echo $?

I have the same issue with CentOS 6.6 and MySQL 5.7.9 community server installed via yum.

I have SELinux enabled, and have used chcon, restorecon, and semanage to try to set the security context to mysqld_db_t for the nonstandard data directory. No joy.

I believe I installed this same version of MysQL installed successfully into a CentOS 6.7 VM using the same yum installation steps. However, the 6.7 image I started with had an older MySQL (5.1) preinstalled.

If I ensure that the data directory does not exist when starting mysqld for the first time, I think it gets created with the right security context, but is unable to change ownership, even though the directory is owned by mysql.

[root@localhost ~]# service mysqld start
MySQL-Datenbank initialisieren: 2015-11-30T19:53:33.967929Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2015-11-30T19:53:33.970712Z 0 [ERROR] Can't change data directory owner to mysql
2015-11-30T19:53:33.970956Z 0 [ERROR] Aborting

[FEHLGESCHLAGEN]

(also... not sure why it's decided to be in a German locale, but this is the least of my problems.)

root@localhost ~]# ls -Z -l /data/XXXXX
total 4
drwxr-x--x. 2 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 4096 Nov 30 20:53 mysql

The init script has created the certificate files before the server stops:

ls -Z -l /data/XXXXX/mysql/
total 32
-rw-------. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1679 Nov 30 20:53 ca-key.pem
-rw-r--r--. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1070 Nov 30 20:53 ca.pem
-rw-r--r--. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1078 Nov 30 20:53 client-cert.pem
-rw-------. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1675 Nov 30 20:53 client-key.pem
-rw-------. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1679 Nov 30 20:53 private_key.pem
-rw-r--r--. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 451 Nov 30 20:53 public_key.pem
-rw-r--r--. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1078 Nov 30 20:53 server-cert.pem
-rw-------. 1 unconfined_u:object_r:mysqld_db_t:s0 mysql mysql 1679 Nov 30 20:53 server-key.pem

Hi guys,

thanks for reporting and feedback.

There were in fact two bugs here, next release will fix the first issue.
We also have work in progress to fix the second issue, then service scripts
should work when SELinux is enabled.

Only workaround at the moment is to disable SELinux.

One more note: 

It's only for fresh installs this is a issue, systems with existing
data is not affected. 

The problem is in initialization process only.

Hi again guys,

can you please try the following workaround:

a) update to MySQL 5.7.10, it should be available in repos now, a yum update should fix that.

b) adjust SELinux file context on directory /var/lib/mysql-files by doing:

$ semanage fcontext -a -e /var/lib/mysql /var/lib/mysql-files
$ restorecon -R /var/lib/mysql-files

c) Make sure /var/lib/mysql is empty

d) Try to start server:

$ service mysqld start

Does this work for you?

I can give it a try when I get home; I'm traveling right now and the disk holding my CentOS VM is not with me.  I've had to step back to CentOS 6.6 (from 6.7) to better match the target environment for our software deployment--would this make a difference?

Eric

It should work for all CentOS 6.x releases.

Thanks for help so far!

Posted by developer:
 
Thanks for the bug report. We have pushed the fix to mysql-5.7 and trunk.
 
Due to SELinux policy mysqld can't read init-file located elsewhere than
/var/lib/mysql. --initialize wants clean datadir.
    
Move installing of validate password plugin to after initialize
is done and use /var/lib/mysql as directory for init-file option.
    
Use of other directory than /var/lib/mysql caused SELinux to deny
access to file used as argument to --init-file. This broke initscript for
new installations.

Noted in 5.7.11 changelog.

On SELinux, mysqld --initialize with an --init-file option could fail
to initialize the data directory.

Worth noting: although fixed in 5.7.11, the 5.7.10 Fedora 23 and EL6 RPM's were also updated with the fix; as a *-5.7.10-<literal>2</literal>-*.rpm release to the Yum repository.