Bug #110549 mysql shell ignoring some CLI arguments passed to it on upgrade check utility
Submitted: 29 Mar 2023 12:12 Modified: 9 May 2023 11:02
Reporter: Eduardo Ortega (OCA) Email Updates:
Status: Closed Impact on me:
Category:Shell Upgrade Checker Severity:S3 (Non-critical)
Version:8.0.32 OS:Any
Assigned to: CPU Architecture:Any

[29 Mar 2023 12:12] Eduardo Ortega
When trying to run the upgrade check utility via shell integration just like in the example at https://dev.mysql.com/doc/mysql-shell/8.0/en/command-line-integration-overview.html, it ignores the user and password passed to it and instead prompts for the password for user root:

root@some-host~# mysqlsh -- util check-for-server-upgrade --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
Please provide the password for 'root@/mnt%2Fvitess%2Fmysql%2Fdatadir%2Fmysql.sock':

Notice that this happens after having `sudo`ed. If run without sudo, it does work:

user@some-host:~% mysqlsh -- util check-for-server-upgrade --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
    "serverAddress": "localhost:3306",

However, we need to run with sudo because our my.cnf is not world readable. There seems to be nothing strange in the env of the root user after sudo that explains this:

root@somme-host:~# env|grep -v MYSQL_PWD
LESSCLOSE=/usr/bin/lesspipe %s %s
LESSOPEN=| /usr/bin/lesspipe %s

How to repeat:
See description for steps.

Suggested fix:
Respect arguments passed in the CLI
[30 Mar 2023 15:43] Alfredo Kojima
Connection options must be passed before the --. Only options for the command being executed are allowed after the --.

So in the case that was included, the correct command is:

 mysqlsh --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   -- util check-for-server-upgrade  --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
[31 Mar 2023 9:39] Eduardo Ortega
Hi, Alfredo:

Thanks for checking. In that case,  the documentation at https://dev.mysql.com/doc/mysql-shell/8.0/en/command-line-integration-overview.html#mysql-... would need to be updated, because it does show it after the `--`. Furthermore, it does not explain why it works when not sudo-ed and fails when sudo-ed.

In addition, using the command you provided actually results in an error:

some-user@some-host:~%  mysqlsh --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   -- util check-for-server-upgrade  --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
WARNING: Using a password on the command line interface can be insecure.
ERROR: Argument connectionData: Invalid connection options, expected either a URI or a Connection Options Dictionary

So something seems definitely odd with how to upgrade check utility gets and uses the user & password.
[31 Mar 2023 13:03] Eduardo Ortega
For extra context, I see the failure after using `sudo` in our Ubuntu 18.04 servers. I don't see that when using `sudo` on my local Mac laptop.
[31 Mar 2023 13:11] Eduardo Ortega
So, after a bit of extra investigation on the subject, there seem to be two issues here:

1. The one originally reported on this bug: under some circumstances, the check upgrade util ignores the user, host and password passed to it after the `--`. Notice that the documentation indicates that passing them after the `--` is valid, both in [1] and [2].
2. If the user, host and password are moved before the `--` and some additional parameters are passed to the upgrade check util (e.g.  output format JSON), the utility fails with error this ERROR: Argument connectionData: Invalid connection options, expected either a URI or a Connection Options Dictionary

[1]: https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-utilities-upgrade.html#mysql-shel...
[2]: https://dev.mysql.com/doc/mysql-shell/8.0/en/command-line-integration-overview.html
[1 Apr 2023 1:36] MySQL Verification Team
Hi Eduardo,

I'm verifying this as documentation for sure needs to be updated to better reflect this.

Thanks for the effort
[20 Apr 2023 15:48] Pawel Andruszkiewicz
Posted by developer:
Starting with 8.0.32, Shell reads standard MySQL option defaults files (i.e. my.cnf/my.ini) from standard locations. It's possible that when logged in as root, Shell finds such file and tries to create a global session using data from that file (hence the password prompt for an unrelated user).

This can be verified using --print-defaults command line option.
[1 May 2023 9:24] Eduardo Ortega

That does not seem to be what is happening here:

root@some-host:~# mysqlsh  -- util check-for-server-upgrade --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
Please provide the password for 'root@/mnt%2Fvitess%2Fmysql%2Fdatadir%2Fmysql.sock':
Ctrl+C Cancelled

root@some-host:~# mysqlsh --print-defaults  -- util check-for-server-upgrade --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf
mysqlsh would have been started with the following arguments:
--socket=/mnt/vitess/mysql/datadir/mysql.sock -- util check-for-server-upgrade --user=my_user --password=***** --host=localhost --port=3306 --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf

`--print-defaults` is not showing any additional user arguments being read from other option files.
[5 May 2023 0:38] Juan Rene Ramirez Monarrez
Posted by developer:
Hi Eduardo

If you look at the output with print defaults, it is actually showing extra arguments.

You original call used the following arguments:

    mysqlsh -- util check-for-server-upgrade --user=my_user --password=$MYSQL_PWD --host=localhost --port=3306   --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf

While the output with print defaults indicates it is using:

    mysqlsh --socket=/mnt/vitess/mysql/datadir/mysql.sock -- util check-for-server-upgrade --user=my_user --password=***** --host=localhost --port=3306 --target-version=8.0.27 --output-format=JSON --config-path=/etc/mysql/my.cnf

This indicates that there's some options file defining a value for the "socket" option

On a case like this 2 sessions would be established:

- A global session using: --socket=/mnt/vitess/mysql/datadir/mysql.sock
- The session to be used in the upgrade checker using: --user=my_user --password=***** --host=localhost --port=3306

Since the password is not specified for the first session, the Shell prompts for it: expected behavior

The reason why the problem appears with sudo and not without it, would be because the options file is available in one case and not on the other

You can verify this by executing the command using --print-defaults in both cases.

As for the problem described by Pawel about the usage of Null for missing optional parameters, that is a bug we are already addressing.

A way to prevent the behavior of the unexpected prompt, is by using the --no-defaults shell argument, so no option files are read and the utility works only with the explicit options you are providing.

As for this report, I agree we should improve the documentation to warn about the connection data defined in option files.
[9 May 2023 11:02] Edward Gilmore
Posted by developer:
Updated MySQL Shell 8.0.33 documentation with notes in the command syntax and utilities sections explaining the default behavior if option files are present and how to ignore them if necessary.
Thank you for the bug report.