Description:
The upgrade checker's `removedFunctions` check (and every check that relies on `mysqlshdk::utils::SQL_iterator::next_sql_function()`) fails to detect usage of a removed function when there is whitespace — a space, tab, or newline — between the function name and its opening parenthesis, for example `PASSWORD ('x')` or `PASSWORD\n('x')` instead of `PASSWORD('x')`.
`func (args)` with intervening whitespace is valid SQL that executes normally under the default `sql_mode` (IGNORE_SPACE is not required). So these are real, working objects that use a removed function.
The miss occurs only for object types whose definitions the server stores verbatim: stored procedures, functions, triggers, and events. Expression-based objects (views, generated columns, functional indexes, CHECK constraints, DEFAULT expressions) are unaffected, because the server rewrites those definitions into a canonical whitespace-free form (`password(...)`) before storing them in information_schema — so those remain correctly detected.
Root cause:
`next_sql_function()` treats a token as a function call only when the very next character after the name is `(`. It does not skip whitespace between the name and the parenthesis:
// mysqlshdk/libs/utils/utils_lexing.cc
std::string_view SQL_iterator::next_sql_function() {
std::string_view token;
while (!(token = next_token()).empty()) {
if (!std::isalpha(token.front())) continue;
if (valid() && get_char() == '(') break; // fails when a space is here
}
return token;
}
For the stored text `PASSWORD ('x')`, `next_token()` returns `"PASSWORD"` and leaves the cursor on the space; the `get_char() == '('` test is then false, so the name is discarded and the usage is never reported.
Impact:
an object such as a stored procedure/function/trigger/event that calls a removed function using the very common `FUNC (args)` formatting style passes the upgrade check, and then fails at runtime after the upgrade (the removed function does not resolve).
How to repeat:
On a MySQL 5.7 server (so the removed `PASSWORD()` function still exists):
CREATE DATABASE test;
USE test;
DELIMITER //
-- caught today (no space)
CREATE PROCEDURE p_nospace() BEGIN SELECT PASSWORD('x'); END//
-- MISSED (space before the paren) -- the bug
CREATE PROCEDURE p_space() BEGIN SELECT PASSWORD ('x'); END//
DELIMITER ;
-- control: same call in a view; the server normalizes the space away,
-- so this one is (correctly) caught
CREATE VIEW v_ws AS SELECT PASSWORD ('x') AS h;
Run the checker:
mysqlsh --user=... --password=... --socket=/tmp/mysql_sandbox5744.sock \
-- util check-for-server-upgrade --target-version=8.0.40 \
--include=removedFunctions
Observed output (only the view is reported; the space-form procedure is missed):
1) Usage of removed functions (removedFunctions)
Error: The following DB objects use functions that were removed in the
latest MySQL version. ...
test.v_ws - VIEW uses removed function PASSWORD
More information:
https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html#mysql-nutshell-removals
Errors: 1
Warnings: 0
Notices: 0
ERROR: 1 errors were found. ...
Expected: `test.p_nospace`, `test.p_space`, and `test.v_ws` should all be
reported (all three call the removed `PASSWORD()` function). `p_space` is
silently missed solely because of the space before `(`.
You can confirm the space is preserved verbatim in the routine definition:
SELECT routine_name, routine_definition
FROM information_schema.routines
WHERE routine_schema='test';
-- p_space stores: BEGIN SELECT PASSWORD ('x'); END (space kept)
-- p_nospace stores: BEGIN SELECT PASSWORD('x'); END
Version-independent reproduction at the lexer level (no server needed) — the
following returns an empty string instead of "PASSWORD":
mysqlshdk::utils::SQL_iterator it("PASSWORD ('x')");
it.next_sql_function(); // returns "" ; expected "PASSWORD"
Suggested fix:
Skip whitespace between the function-name token and the parenthesis check in
`SQL_iterator::next_sql_function()`:
std::string_view SQL_iterator::next_sql_function() {
std::string_view token;
while (!(token = next_token()).empty()) {
if (!std::isalpha(token.front())) continue;
while (valid() && std::isspace(get_char())) ++(*this); // skip whitespace
if (valid() && get_char() == '(') break;
}
return token;
}
Description: The upgrade checker's `removedFunctions` check (and every check that relies on `mysqlshdk::utils::SQL_iterator::next_sql_function()`) fails to detect usage of a removed function when there is whitespace — a space, tab, or newline — between the function name and its opening parenthesis, for example `PASSWORD ('x')` or `PASSWORD\n('x')` instead of `PASSWORD('x')`. `func (args)` with intervening whitespace is valid SQL that executes normally under the default `sql_mode` (IGNORE_SPACE is not required). So these are real, working objects that use a removed function. The miss occurs only for object types whose definitions the server stores verbatim: stored procedures, functions, triggers, and events. Expression-based objects (views, generated columns, functional indexes, CHECK constraints, DEFAULT expressions) are unaffected, because the server rewrites those definitions into a canonical whitespace-free form (`password(...)`) before storing them in information_schema — so those remain correctly detected. Root cause: `next_sql_function()` treats a token as a function call only when the very next character after the name is `(`. It does not skip whitespace between the name and the parenthesis: // mysqlshdk/libs/utils/utils_lexing.cc std::string_view SQL_iterator::next_sql_function() { std::string_view token; while (!(token = next_token()).empty()) { if (!std::isalpha(token.front())) continue; if (valid() && get_char() == '(') break; // fails when a space is here } return token; } For the stored text `PASSWORD ('x')`, `next_token()` returns `"PASSWORD"` and leaves the cursor on the space; the `get_char() == '('` test is then false, so the name is discarded and the usage is never reported. Impact: an object such as a stored procedure/function/trigger/event that calls a removed function using the very common `FUNC (args)` formatting style passes the upgrade check, and then fails at runtime after the upgrade (the removed function does not resolve). How to repeat: On a MySQL 5.7 server (so the removed `PASSWORD()` function still exists): CREATE DATABASE test; USE test; DELIMITER // -- caught today (no space) CREATE PROCEDURE p_nospace() BEGIN SELECT PASSWORD('x'); END// -- MISSED (space before the paren) -- the bug CREATE PROCEDURE p_space() BEGIN SELECT PASSWORD ('x'); END// DELIMITER ; -- control: same call in a view; the server normalizes the space away, -- so this one is (correctly) caught CREATE VIEW v_ws AS SELECT PASSWORD ('x') AS h; Run the checker: mysqlsh --user=... --password=... --socket=/tmp/mysql_sandbox5744.sock \ -- util check-for-server-upgrade --target-version=8.0.40 \ --include=removedFunctions Observed output (only the view is reported; the space-form procedure is missed): 1) Usage of removed functions (removedFunctions) Error: The following DB objects use functions that were removed in the latest MySQL version. ... test.v_ws - VIEW uses removed function PASSWORD More information: https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html#mysql-nutshell-removals Errors: 1 Warnings: 0 Notices: 0 ERROR: 1 errors were found. ... Expected: `test.p_nospace`, `test.p_space`, and `test.v_ws` should all be reported (all three call the removed `PASSWORD()` function). `p_space` is silently missed solely because of the space before `(`. You can confirm the space is preserved verbatim in the routine definition: SELECT routine_name, routine_definition FROM information_schema.routines WHERE routine_schema='test'; -- p_space stores: BEGIN SELECT PASSWORD ('x'); END (space kept) -- p_nospace stores: BEGIN SELECT PASSWORD('x'); END Version-independent reproduction at the lexer level (no server needed) — the following returns an empty string instead of "PASSWORD": mysqlshdk::utils::SQL_iterator it("PASSWORD ('x')"); it.next_sql_function(); // returns "" ; expected "PASSWORD" Suggested fix: Skip whitespace between the function-name token and the parenthesis check in `SQL_iterator::next_sql_function()`: std::string_view SQL_iterator::next_sql_function() { std::string_view token; while (!(token = next_token()).empty()) { if (!std::isalpha(token.front())) continue; while (valid() && std::isspace(get_char())) ++(*this); // skip whitespace if (valid() && get_char() == '(') break; } return token; }