Bug #91769 mysql_upgrade return code is '0' even after errors
Submitted: 23 Jul 2018 19:54 Modified: 7 Aug 2018 14:50
Reporter: Chris Horton Email Updates:
Status: Verified Impact on me:
Category:MySQL Server: Command-line Clients Severity:S4 (Feature request)
Version:5.7.22-22 OS:Linux
Assigned to: CPU Architecture:Any

[23 Jul 2018 19:54] Chris Horton
When running mysql_upgrade, and mysqld goes away for unrelated reasons, mysql_upgrade returns a '0' return code even though table upgrades fail and it throws error messages.  This '0' return code makes scripts act as if the mysql_upgrade ran successfully.

How to repeat:
I initially ran into the error when the OOM killer killed mysqld in the middle of the mysql_upgrade run.  I have also recreated the issue with a mysqld restart while mysql_upgrade is running.

$ sudo mysql_upgrade --force
Redacted many lines
Repairing tables
Failed to ALTER TABLE `database`.`table1` FORCE
Error: Lost connection to MySQL server during query
Failed to ALTER TABLE `database`.`table2` FORCE
Error: MySQL server has gone away
Failed to ALTER TABLE `database`.`table3` FORCE
Error: MySQL server has gone away
Upgrade process completed successfully.
mysql_upgrade: [ERROR] 2006: MySQL server has gone away

$ echo $?

$ mysql_upgrade --version
mysql_upgrade  Ver 2.0 Distrib 5.7.22-22, for Linux (x86_64)

Suggested fix:
Return appropriate non-zero error code when mysql_upgrade encounters errors in a run.
[7 Aug 2018 13:30] Sinisa Milivojevic

Thank you for your bug report.

What I see from the code is that for each error, mysql_upgrade is returning the message to the shell. That is all that it can do. It can not exit totally in the middle of the job as the upgrade should be totally atomic. It should not be done at all or it must be done fully.

It is a duty of the DBA to make sure that upgrade will go smoothly. As our manual says, no queries should run when you do mysql_upgrade. Actually, only mysql_upgrade should run. Hence, you are not supposed to get any OOM JUST with mysql_upgrade nor are you supposed to kill your own server in the middle of the upgrade.

There are many other ways in which you can make damage.

Hence, I do not think that this is a bug, since upgrade should run without problems from beginning to the end and hence, mysql_upgrade can not exit due to the error.
[7 Aug 2018 14:33] Chris Horton
A well-behaved command line tool in unix or linux only returns a 0 exit code on success.  Returning a 0 exit code when there are known errors is definitely a bug.

In the real world, you can't make claims like 'you are not supposed to
get any OOM JUST with mysql_upgrade'.  The fact of the matter is that you don't know what will be going on with a server or network during a long mysql_upgrade run.  In today's cloud-based infrastructure, you can most definitely not assume that the process will be run on the DB server with no network connectivity involved.  Not returning the correct exit code to indicate that the process had errors is objectively incorrect and most definitely a bug.  Regardless of when the process exits, it is aware that it hit errors during the run  (as shown by the error messages included in my original report) and should be able to indicate as much when it exits.

Why would you want to make the mysql_upgrade process unreliable for scripting?  Expecting a script to pick through the output for some error text, rather than being able to rely on the exit code to know whether there was an error, would be absurd and unreliable.
[7 Aug 2018 14:50] Sinisa Milivojevic

I have to agree with you on this issue.

0 (zero) can not be returned when there are errors. However, we did not envisage running it in the script, but that is one useful feature.

Verified as a feature request.