Bug #85759 Upgrade from Ubuntu 14.04 to 16.04 has broken MySQL
Submitted: 2 Apr 2017 13:06 Modified: 19 May 2018 9:53
Reporter: Purvez Desai Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Server Severity:S2 (Serious)
Version:5.5 to 5.7 OS:Ubuntu (14.04 to 16.04)
Assigned to: CPU Architecture:Any

[2 Apr 2017 13:06] Purvez Desai
Description:
I upgraded my Ubuntu server from ver 14.04 LTS to 16.04 LTS.  The MySQL server on ver 14.04 was 5.5.  As part of the upgrade to 16.04 the Ubuntu upgrade process attempted to move to 5.7....which didn't work.  So I instead researched the web and found this : 

http://askubuntu.com/questions/760724/16-04-upgrade-broke-mysql-server

I then tried to apply the 'accepted answer' for the above.  When I tried the apt install  part I got the following output:

============
nexargi@server-02:/$ sudo apt install mysql-server
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  aptitude-common g++-4.8 gcc-4.9-base libboost-iostreams1.54.0 libboost-iostreams1.58.0 libc-ares2 libck-connector0 libcwidget3 libdbd-mysql-perl libdbi-perl libept1.4.12 libgcrypt11-dev libgcrypt20-dev
  libgnutlsxx27 libgpg-error-dev libisl10 libprocps3 libsigc++-2.0-0c2a libstdc++-4.8-dev libterm-readkey-perl libv8-3.14.5 libxapian22v5 linux-image-3.11.0-15-generic linux-image-extra-3.11.0-15-generic
  python-chardet python-cheetah python-configobj python-json-pointer python-jsonpatch python-oauth python-pkg-resources python-prettytable python-requests python-serial python-six python-urllib3 python-yaml
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  mysql-server-5.7 mysql-server-core-5.7
Suggested packages:
  mailx tinyca
Recommended packages:
  libhtml-template-perl
The following NEW packages will be installed:
  mysql-server mysql-server-5.7 mysql-server-core-5.7
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/10.6 MB of archives.
After this operation, 94.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
N: Ignoring file '50unattended-upgrades.ucf-old' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension
Preconfiguring packages ...
Selecting previously unselected package mysql-server-core-5.7.
(Reading database ... 98446 files and directories currently installed.)
Preparing to unpack .../mysql-server-core-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb ...
Unpacking mysql-server-core-5.7 (5.7.17-0ubuntu0.16.04.1) ...
Selecting previously unselected package mysql-server-5.7.
Preparing to unpack .../mysql-server-5.7_5.7.17-0ubuntu0.16.04.1_amd64.deb ...
Unpacking mysql-server-5.7 (5.7.17-0ubuntu0.16.04.1) ...
Selecting previously unselected package mysql-server.
Preparing to unpack .../mysql-server_5.7.17-0ubuntu0.16.04.1_all.deb ...
Unpacking mysql-server (5.7.17-0ubuntu0.16.04.1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up mysql-server-core-5.7 (5.7.17-0ubuntu0.16.04.1) ...
Setting up mysql-server-5.7 (5.7.17-0ubuntu0.16.04.1) ...
update-alternatives: using /etc/mysql/mysql.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Renaming removed key_buffer and myisam-recover options (if present)
ERROR: Unable to start MySQL server:
mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2 - No such file or directory)
mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
Please take a look at https://wiki.debian.org/Teams/MySQL/FAQ for tips on fixing common upgrade issues.
Once the problem is resolved, run apt-get --fix-broken install to retry.
dpkg: error processing package mysql-server-5.7 (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-5.7; however:
  Package mysql-server-5.7 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
Processing triggers for systemd (229-4ubuntu16) ...
Processing triggers for ureadahead (0.100.0-19) ...
Errors were encountered while processing:
 mysql-server-5.7
 mysql-server
N: Ignoring file '50unattended-upgrades.ucf-old' in directory '/etc/apt/apt.conf.d/' as it has an invalid filename extension
E: Sub-process /usr/bin/dpkg returned an error code (1)

================

Here is the list of /etc/mysql directory

nexargi@server-02:/$ cd /etc/mysql
nexargi@server-02:/etc/mysql$ ls -l
total 12
-rwxr-xr-x 1 root root  120 Jan 18 21:46 debian-start
lrwxrwxrwx 1 root root   24 Apr  2 12:29 my.cnf -> /etc/alternatives/my.cnf
-rw-r--r-- 1 root root  682 Jul 11  2016 mysql.cnf
drwxr-xr-x 2 root root 4096 Apr  2 12:29 mysql.conf.d

===========

Clearly the /etc/mysql/conf.d directory doesn't exist.  I would have expected the install script to deal with that.  

Will wait to hear on what to do next.

How to repeat:
Update Ubuntu from 14.04 to 16.04.

Purge all of mysql

then run apt install mysql-server
[2 Apr 2017 15:24] Purvez Desai
I thought I would highlight where I believe the problem lies.  From the previous output the following is of importance in my opinion:

ERROR: Unable to start MySQL server:
mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2 - No such file or directory)
mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
[3 Apr 2017 5:38] Lars Tangvald
A note first, that this is about the native packages, and in general bugs for them should be reported at https://bugs.launchpad.net/ubuntu/+source/mysql-5.7

The problem here is as you say that /etc/mysql/conf.d was missing. The 5.5 packages in Ubuntu 14 will create that directory on installation (you can test this by doing a fresh install of MySQL on a Ubuntu 14 box), so at some point it was probably removed.
This creates the problem because from Debian/Ubuntu's (really apt's) point of view any changes to the contents of /etc/mysql counts as a configuration customization, and will not be reverted when upgrading the packages. 
We could add logic to always create that directory if it's missing, but this quickly becomes a slippery slope of trying to "fix" any changes users make to their config that the server can't handle by default.
[3 Apr 2017 10:48] Purvez Desai
Hi Lars

Thanks very much for your response.  I agree that the conf.d directory did exist in 14.04.  All I did was run the do-release-upgrade on the server so clearly somewhere along the line the ubuntu release upgrader must have removed it.

I have reported the issue as a bug on launchpad as well but have had no response there.

In the interim I tried to install on top of the failed conversion from your apt repositories.  That failed too.  I'm attaching the error.log in /var/log/mysql.

I also did a dpkg -l | grep mysql and the output is as follows:

nexargi@server-02:/etc/mysql$ dpkg -l | grep mysql
ii  libdbd-mysql-perl                    4.033-1ubuntu0.1                           amd64        Perl5 database interface to the MySQL database
ii  libmysqlclient-dev                   5.7.17-0ubuntu0.16.04.1                    amd64        MySQL database development files
ii  libmysqlclient18:amd64               5.5.54-0ubuntu0.14.04.1                    amd64        MySQL database client library
ii  libmysqlclient20:amd64               5.7.17-0ubuntu0.16.04.1                    amd64        MySQL database client library
ii  mysql-apt-config                     0.8.3-1                                    all          Auto configuration for MySQL APT Repo.
ii  mysql-client                         5.7.17-1ubuntu16.04                        amd64        MySQL Client meta package depending on latest version
ii  mysql-common                         5.7.17-1ubuntu16.04                        amd64        MySQL Common
ii  mysql-community-client               5.7.17-1ubuntu16.04                        amd64        MySQL Client
ii  mysql-community-server               5.7.17-1ubuntu16.04                        amd64        MySQL Server
ii  mysql-server                         5.7.17-1ubuntu16.04                        amd64        MySQL Server meta package depending on latest version
rc  mysql-server-5.5                     5.5.54-0ubuntu0.14.04.1                    amd64        MySQL database server binaries and system database setup
rc  mysql-server-5.7                     5.7.17-0ubuntu0.16.04.1                    amd64        MySQL database server binaries and system database setup
ii  php5-mysql                           5.5.9+dfsg-1ubuntu4.21                     amd64        MySQL module for php5

==============

At this point I would be happy to get mysql5.7 working on the upgraded 16.04 ubuntu in whatever way possible.  Any assistance/guidance you can give would be very much appreciated.

Thanks
[3 Apr 2017 10:51] Purvez Desai
error log on trying to update from mysql repository.  See comments

Attachment: error.log (text/x-log), 3.00 KiB.

[3 Apr 2017 10:52] Purvez Desai
see comments section

Attachment: error.log (text/x-log), 3.00 KiB.

[4 Sep 2017 11:29] Marko Novakovic
Hello - - have the same problem here: - no idea what to try . . 

grega@proe:~$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  libegl1-mesa libgl1-mesa-dri libinput10 libmirclient9 libwayland-egl1-mesa libxatracker2 linux-headers-generic linux-headers-virtual
  linux-image-virtual linux-virtual
0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded.
2 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up mysql-server-5.7 (5.7.19-0ubuntu0.16.04.1) ...
ERROR: Unable to start MySQL server:
2017-09-04T11:22:41.505686Z 0 [ERROR] unknown variable 'validate_password_check_user_name=1'
2017-09-04T11:22:41.509433Z 0 [ERROR] Aborting
Please take a look at https://wiki.debian.org/Teams/MySQL/FAQ for tips on fixing common upgrade issues.
Once the problem is resolved, run apt-get --fix-broken install to retry.
dpkg: error processing package mysql-server-5.7 (--configure):
 subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of mysql-server:
 mysql-server depends on mysql-server-5.7; however:
  Package mysql-server-5.7 is not configured yet.

dpkg: error processing package mysql-server (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 mysql-server-5.7
 mysql-server
E: Sub-process /usr/bin/dpkg returned an error code (1)

------------------------------------------------
grega@proe:~$ dpkg -l | grep mysql
ii  libmysqlclient-dev                   5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database development files
ii  libmysqlclient20:amd64               5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database client library
ii  mysql-client                         5.7.19-0ubuntu0.16.04.1                    all          MySQL database client (metapackage depending on the latest version)
ii  mysql-client-5.7                     5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database client binaries
ii  mysql-client-core-5.7                5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database core client binaries
ii  mysql-common                         5.7.19-0ubuntu0.16.04.1                    all          MySQL database common files, e.g. /etc/mysql/my.cnf
iU  mysql-server                         5.7.19-0ubuntu0.16.04.1                    all          MySQL database server (metapackage depending on the latest version)
iF  mysql-server-5.7                     5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database server binaries and system database setup
ii  mysql-server-core-5.7                5.7.19-0ubuntu0.16.04.1                    amd64        MySQL database server binaries
ii  php5.6-mysql                         5.6.26-1+deb.sury.org~xenial+1             amd64        MySQL module for PHP
ii  python-mysql.connector               2.0.4-1                                    all          pure Python implementation of MySQL Client/Server protocol
[19 Apr 2018 9:53] MySQL Verification Team
Please report this bug to Ubusntu team how Lars already commented. Thanks.
[20 May 2018 1:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".