Description:
STRING_TO_VECTOR() / TO_VECTOR() refuses any vector literal that has a non-zero component smaller in magnitude than FLT_MIN (about 1.1755e-38). These are valid IEEE-754 float32 values (denormals, down to 1.4e-45). The error is ERROR 6138 (HY000): Data cannot be converted to a valid vector. Values that underflow to zero (for example 1e-50) are refused the same way, while an explicit 0 is accepted.
This refusal doesn't keep denormals out of VECTOR columns. Binary input such as X'C2160100' (1e-40) bypasses the text conversion and has always been accepted. The result is that VECTOR_TO_STRING() output of such a stored vector can't be converted back with STRING_TO_VECTOR().
The rest of the server accepts underflow silently. FLOAT and DOUBLE columns, CAST(... AS DOUBLE) and JSON all store tiny values as denormals or 0 without a warning, even in strict mode, while still refusing overflow while STRING_TO_VECTOR() refuses it.
How to repeat:
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.2e-38,1]')); -- OK: [1.20000e-38,1.00000e+00]
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.1e-38,1]')); -- ERROR 6138
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-45,1]')); -- ERROR 6138 (smallest denormal)
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-50,1]')); -- ERROR 6138 (underflows to 0)
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[0,1]')); -- OK
-- For comparison, FLOAT accepts the same values without a warning
-- (strict mode):
CREATE TABLE f (x FLOAT);
INSERT INTO f VALUES ('1e-40'), ('1e-50'); -- OK: 9.99995e-41, 0
DROP TABLE f;
Suggested fix:
Stop checking errno and let the existing std::isinf() / std::isnan() checks handle out-of-range values. Parse failures are already detected by the input != end loop condition.
Initially reported on github https://github.com/mysql/mysql-server/issues/849 and the fix provided at https://github.com/mysql/mysql-server/pull/828 but it was not taken into consideration due to internal policies, hence the duplicate bug report.
Description: STRING_TO_VECTOR() / TO_VECTOR() refuses any vector literal that has a non-zero component smaller in magnitude than FLT_MIN (about 1.1755e-38). These are valid IEEE-754 float32 values (denormals, down to 1.4e-45). The error is ERROR 6138 (HY000): Data cannot be converted to a valid vector. Values that underflow to zero (for example 1e-50) are refused the same way, while an explicit 0 is accepted. This refusal doesn't keep denormals out of VECTOR columns. Binary input such as X'C2160100' (1e-40) bypasses the text conversion and has always been accepted. The result is that VECTOR_TO_STRING() output of such a stored vector can't be converted back with STRING_TO_VECTOR(). The rest of the server accepts underflow silently. FLOAT and DOUBLE columns, CAST(... AS DOUBLE) and JSON all store tiny values as denormals or 0 without a warning, even in strict mode, while still refusing overflow while STRING_TO_VECTOR() refuses it. How to repeat: SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.2e-38,1]')); -- OK: [1.20000e-38,1.00000e+00] SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.1e-38,1]')); -- ERROR 6138 SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-45,1]')); -- ERROR 6138 (smallest denormal) SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-50,1]')); -- ERROR 6138 (underflows to 0) SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[0,1]')); -- OK -- For comparison, FLOAT accepts the same values without a warning -- (strict mode): CREATE TABLE f (x FLOAT); INSERT INTO f VALUES ('1e-40'), ('1e-50'); -- OK: 9.99995e-41, 0 DROP TABLE f; Suggested fix: Stop checking errno and let the existing std::isinf() / std::isnan() checks handle out-of-range values. Parse failures are already detected by the input != end loop condition. Initially reported on github https://github.com/mysql/mysql-server/issues/849 and the fix provided at https://github.com/mysql/mysql-server/pull/828 but it was not taken into consideration due to internal policies, hence the duplicate bug report.