Bug #121559 STRING_TO_VECTOR() refuses valid float32 denormal components with ER_TO_VECTOR_CONVERSION
Submitted: 20 Oct 6:59
Reporter: Catalin Besleaga (OCA) Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Server: Data Types Severity:S3 (Non-critical)
Version:9.7, 2X OS:Any
Assigned to: CPU Architecture:Any
Tags: VECTOR_TO_STRING

[20 Oct 6:59] Catalin Besleaga
Description:
STRING_TO_VECTOR() / TO_VECTOR() refuses any vector literal that has a non-zero component smaller in magnitude than FLT_MIN (about 1.1755e-38). These are valid IEEE-754 float32 values (denormals, down to 1.4e-45). The error is ERROR 6138 (HY000): Data cannot be converted to a valid vector. Values that underflow to zero (for example 1e-50) are refused the same way, while an explicit 0 is accepted.

This refusal doesn't keep denormals out of VECTOR columns. Binary input such as X'C2160100' (1e-40) bypasses the text conversion and has always been accepted. The result is that VECTOR_TO_STRING() output of such a stored vector can't be converted back with STRING_TO_VECTOR().

The rest of the server accepts underflow silently. FLOAT and DOUBLE columns, CAST(... AS DOUBLE) and JSON all store tiny values as denormals or 0 without a warning, even in strict mode, while still refusing overflow while STRING_TO_VECTOR() refuses it.

How to repeat:
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.2e-38,1]'));  -- OK: [1.20000e-38,1.00000e+00]
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1.1e-38,1]'));  -- ERROR 6138
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-45,1]'));    -- ERROR 6138 (smallest denormal)
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[1e-50,1]'));    -- ERROR 6138 (underflows to 0)
SELECT VECTOR_TO_STRING(STRING_TO_VECTOR('[0,1]'));        -- OK

-- For comparison, FLOAT accepts the same values without a warning
-- (strict mode):
CREATE TABLE f (x FLOAT);
INSERT INTO f VALUES ('1e-40'), ('1e-50');  -- OK: 9.99995e-41, 0
DROP TABLE f;

Suggested fix:
Stop checking errno and let the existing std::isinf() / std::isnan() checks handle out-of-range values. Parse failures are already detected by the input != end loop condition.

Initially reported on github https://github.com/mysql/mysql-server/issues/849 and the fix provided at https://github.com/mysql/mysql-server/pull/828 but it was not taken into consideration due to internal policies, hence the duplicate bug report.