MySQL Bugs: #76149: mysql-apt-config configuration never finishes

Bug #76149	mysql-apt-config configuration never finishes
Submitted:	4 Mar 2015 19:20	Modified:	12 Oct 2015 15:31
Reporter:	Don Viszneki	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Package Repos	Severity:	S2 (Serious)
Version:	Unknown	OS:	Linux (Ubuntu 14.04)
Assigned to:		CPU Architecture:	Any
Tags:	apt, dpkg, sysadmin, systemsadministration, ubuntu

Description:
Upgrading, installing, or configuring this package, may result in indefinite spinlock.

As part of upgrade/install, a package will be "configured." You can see what it does by looking in /var/lib/dpkg/info/mysql-apt-config.config for example. A little familiarity with dpkg package configuration will be required to get an idea of what is happening in that file.

But so far, I cannot figure out what causes frontend (spawned by dpkg) and mysql-apt-config.config to spinlock indefinitely.

(Note by spinlock I only mean that between frontend and mysql-apt-confi processes, they are using up an entire CPU core, seemingly indefinitely.)

I could not determine with lsof what else they might be doing, but I'm guessing it's a true spinlock (wasting cpu waiting to acquire a lock) because A) I cannot see anything else that looks like it would actually use a lot of CPU like this, and B) it really seems to go on for a very long time (indefinitely, I assume.)

This will quite an annoyance for systems administrators who are not familiar enough with apt/dpkg to look any deeper into the problem. They may never be able to complete Software Updater until the problem is fixed!

How to repeat:
Go to http://dev.mysql.com/downloads/repo/apt/

Download the .deb package for Ubuntu 14.04. Here is mine with md5:

f893b0b1495c6c0458c40320c8f556b0  mysql-apt-config_0.3.2-1ubuntu14.04_all.deb

Install the package. (dpkg -i)

Update your package lists. (apt-get update)

Upgrade software. (apt-get upgrade)

OR

If you have already tried to upgrade, perhaps with "Software Updates" desktop GUI application from Canonical or similar program, and had to kill the program because it never finished working, first make sure you have cleaned up all the apt-related locks, and then you can trigger the problem this way:

dpkg --configure mysql-apt-config

Suggested fix:
Work-arounds:

On one affected system, I uninstalled mysql-apt-config (which afaict only removes the apt source from /etc/apt/sources.list.d/mysql-whatever) and then reinstalled the original package I downloaded (again package filename is mysql-apt-config_0.3.2-1ubuntu14.04_all.deb md5 "f893b0b1495c6c0458c40320c8f556b0." Now I will simply refuse to update this package until I learn that the problem has been addressed, or I find a better fix or workaround.

On a second affected system, even installing this original package caused the same spinlock, however I was probably less diligent about looking around for locked files, so there was no work-around for this problem.

I did not experience this problem with 0.3.2; however today I was running an update on my system and the mysql-apt-config package was to be updated from 0.3.2 to 0.3.4.  I experienced the console stuck at "Setting up mysql-apt-config (0.3.4-1ubutnu14.10) ...".  I had to kill the update process, kill the PIDs associated with the locked files, and reinstall the original 0.3.2 deb file.

Hello Don Viszneki,

Thank you for the report.
Observed similar behavior on Ubuntu 14.10.

Thanks,
Umesh

##  Downloaded .deb package from http://dev.mysql.com/downloads/repo/apt/
ushastry@ushastry:~/Downloads$ ls -l
..
-rw-rw-r--  1 ushastry ushastry     18266 Mar  5 10:14 mysql-apt-config_0.3.2-1ubuntu14.10_all.deb

// OS
ushastry@ushastry:~/Downloads$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.10
DISTRIB_CODENAME=utopic
DISTRIB_DESCRIPTION="Ubuntu 14.10"
NAME="Ubuntu"
VERSION="14.10 (Utopic Unicorn)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.10"
VERSION_ID="14.10"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

##
ushastry@ushastry:~/Downloads$ sudo dpkg -i mysql-apt-config_0.3.2-1ubuntu14.10_all.deb 
Selecting previously unselected package mysql-apt-config.
(Reading database ... 173043 files and directories currently installed.)
Preparing to unpack mysql-apt-config_0.3.2-1ubuntu14.10_all.deb ...
Unpacking mysql-apt-config (0.3.2-1ubuntu14.10) ...
Setting up mysql-apt-config (0.3.2-1ubuntu14.10) ...

^^ hanged here for long(waited >5min before closing)

##

ushastry@ushastry:~/Downloads$ ps aux|grep mysql

root      3491  0.0  0.0  65056  4000 pts/0    S+   10:15   0:00 sudo dpkg -i mysql-apt-config_0.3.2-1ubuntu14.10_all.deb
root      3492  0.1  0.9  64628 38388 pts/0    S+   10:15   0:00 dpkg -i mysql-apt-config_0.3.2-1ubuntu14.10_all.deb
root      3502 55.4  0.4  67868 16232 pts/0    R+   10:15   3:54 /usr/bin/perl -w /usr/share/debconf/frontend /var/lib/dpkg/info/mysql-apt-config.postinst configure 0.3.2-1ubuntu14.04
root      3508 36.7  0.0  16772  2656 pts/0    S+   10:15   2:35 /bin/bash /var/lib/dpkg/info/mysql-apt-config.config configure 0.3.2-1ubuntu14.04
ushastry  3586  0.0  0.0  13664  2184 pts/10   S+   10:22   0:00 grep --color=auto mysql

Also, see Bug #76161

Bug #76170 marked as duplicate of this

I have the same issue on Ubuntu 14.04.2. The problem is in /var/lib/dpkg/info/mysql-apt-config.config on line 90. The while loop looks for SELECT_PRODUCT != "Apply" but no matter what is pressed on the dialog SELECT_PRODUCT is always "Server". 

When I use the command prompt to sudo dpkg-reconfigure mysql-apt-config then it starts from the beginning (asking about server, then version). Then choosing apply at the first screen works. 

A very confusing dialog and the server/apply part is missing on upgrade.

In reply to Jacob Steelsmith:

Very good diagnosis :)

Please tell me, when to use dpkg-reconfigure in order to get the interactive stage of configuring this package. We went through this dialog when we first installed the package. Is the solution then to apt-get upgrade, kill the process when it gets stuck on mysql-apt-config, then dpkg-reconfigure?

Thanks Jacob!

Unfortunately I can't be sure. I did alter the script a bit and may have reset the debconf database for the package. To be sure I'll need to remove the mysql-apt-config package, install the older version and configure then upgrade, get into the loop, kill the process and try reconfigure. 

I don't believe it will work as the same script handles both configure and reconfigure. There may be code that checks for and skips the first screen where you choose server and sends the user to the version. 

The long term solution in my opinion would be to add an apply to the second screen (confusing) or use another logic branch or script to handle the server version on subsequent upgrades. 

In short the user is never presented with the server/client screen and sent to the server version screen with no apply, and the script is looking for that apply to be sent.

http://bugs.mysql.com/bug.php?id=76174 marked as duplicate of this one.

Hi Don/ Philip/ Jacob,

Thanks for this bug report on the mysql-apt-config package. This was quite an
unpleasant experience where a bug such severity passed through undetected to
our production repos.

We have finally fixed this issue affecting upgrades and delivered new
packages with version 0.3.3-2 for Debian7, Ubuntu12.04 and Ubuntu14.04 and
version 0.3.4-2 for Ubuntu 14.10. Same set of packages are also available at
http://dev.mysql.com/downloads/repo/apt/ in case you plan to download them
manually.

We would really appreciate if you can try upgrading your current installation
from 0.3.2-1 to directly to 0.3.3-2 or 0.3.4-2 depending on your platform.
The offending set of packages have been already removed from our repos. So
using 'apt-get upgrade' this time, you will be directly upgraded to the
correct version of mysql-apt-config.

I understand the unnecessary effort that was needed to get rid of the
misbehaving upgrade and revert package to the older version 0.3.2 of the
package.

Now that the problem has been resolved we would like to you keep enjoying our
awesome delivery of upgrades through MySQL software repositories. We would
like to hear if the new set of mysql-apt-config packages sufficiently resolve
the problem you guys have faced over the last week.

Akhil

Folks, luckily there is a workaround to update APT Repository.

This bug seems to be related to the config files being not purged (while there was a difference in the count of the possible entries).

The workaround (after you do apt-get update):
dpkg -P Will remove/delete everything including configuration files, which is just what you need.

# dpkg -P mysql-apt-config
# dpkg -i /var/cache/apt/archives/mysql-apt-config_0.3.3-2ubuntu14.04_all.deb

Now it asked me for the settings interactively and the setup finished successfully.
If this doesn't worked correctly for you, you could also download it directly.

1. dpkg -P mysql-apt-config
2. wget http://dev.mysql.com/get/mysql-apt-config_0.3.3-2ubuntu14.04_all.deb
3. dpkg -i mysql-apt-config_0.3.3-1ubuntu14.04_all.deb

It looks like every time we need to update/upgrade to the latest version of the MySQL APT Repository that we need to purge, so that it removes any configuration files otherwise we hang in a infinite loop.

I think there is a simple solution for this, the script needs to check if there is already a version installed likely a check of the configuration and ask the user if he or she want to change the existing configuration or if the user wants to maintain the same configuration.

WOOPS, I was too fast, correct workaround/upgrade to the latest version:

1. dpkg -P mysql-apt-config
2. wget http://dev.mysql.com/get/mysql-apt-config_0.3.3-2ubuntu14.04_all.deb
3. dpkg -i mysql-apt-config_0.3.3-2ubuntu14.04_all.deb

And as always, thanks for your support guys!
Now it's working fine!

Hi Akhil!

Thanks for the good news!

Ubuntu Software Updater had no problems this time!

I also did not have any problems updating this time.  Thanks.

Hi,

Thanks everyone for kindly sharing your feedback on the fix. We hope that such gap in our product is avoided in future releases.

Regards,
Akhil

Hi Akhil,

Everything works like suspected, the upgrade was fast witout any errors or warnings. This is a stable release.

Cheers,

Remzi

When I run Ubuntu 12.04 Update Manager, I am getting a message very similar to what is reported above:

    Failed to download repository information
    W:Failed to fetch http://repo.mysql.com/apt/ubuntu/dists/precise/Release  Unable to find expected entry 'utilities-1.4/binary-amd64/Packages' in Release file (Wrong sources.list entry or malformed file)
    , E:Some index files failed to download. They have been ignored, or old ones used instead.

I assume that a fix similar to what has been given will work for me, but since I'm running 12.04 and the fix is for 14.04, I would greatly appreciate detailed steps to follow for 12.04.  Thank you.

Hey Roger House,

Sorry to inform you, but I'm quite sure that your problem is unrelated.

Please file a separate bug report to ensure your problem gets the attention it needs!

As for similar solution: it looks like that APT source has bumped the utilities to version 1.5, under the package name mysql-utilities-1.5. This is pure speculation, but try installing that package instead.

Add a message on your new bug report that Oracle should consider both retaining old versions of software in the APT repository for sysadmins who must postpone upgrading, and also maintaining a metapackage which can assist in automatically upgrading when a version bump warrants a new package name.

Good luck!

I was directed to this issue from bug 76174. I am getting this error again in Ubuntu 14.04 server today:

W: Failed to fetch http://repo.mysql.com/apt/ubuntu/dists/trusty/InRelease  Unable to find expected entry 'mysql-utilities-1.6/binary-amd64/Packages' in Release file (Wrong sources.list entry or malformed file)

I looked at the solution indicated here but it did not fix:

dpkg -P mysql-apt-config
wget http://dev.mysql.com/get/mysql-apt-config_0.3.7-1ubuntu14.04_all.deb
dpkg -i mysql-apt-config_0.3.7-1ubuntu14.04_all.deb

I went through the configuration screens, but when I "apt-get update" I still get the same error as above.

I found that going through the configuration screen and selecting "mysql-utilities-1.5" instead of "mysql-utilities-1.6" will fix this error. I'm thinking it's perhaps because mysql-utilities-1.6 is incompatible with or not available for Ubuntu 14.04 server?

Hi John D'Orazio,

I am not an Oracle expert, but your problem seems to be unrelated to this bug.