Bug #120989 Documentation: FLOAT(p) boundary on floating-point-types page contradicts numeric-type-syntax page and actual server beh

Submitted: 23 Jul 12:44
Reporter: Sebastian Deppisch Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Server: Documentation Severity:S3 (Non-critical)
Version:9.4.0, 8.3.0(behavior); manual: multiple OS:Any
Assigned to: CPU Architecture:Any

[23 Jul 12:44] Sebastian Deppisch
Description:
The manual page "Floating-Point Types (Approximate Value) - FLOAT, DOUBLE"
states in all current versions (8.0, 8.4, 9.x):

  "A precision from 0 to 23 results in a 4-byte single-precision FLOAT
   column. A precision from 24 to 53 results in an 8-byte double-precision
   DOUBLE column."

e.g. https://dev.mysql.com/doc/refman/8.4/en/floating-point-types.html

This contradicts both the actual server behavior and another page of the
same manual. The "Numeric Data Type Syntax" page states correctly:

  "If p is from 0 to 24, the data type becomes FLOAT with no M or D
   values. If p is from 25 to 53, the data type becomes DOUBLE with no
   M or D values."

https://dev.mysql.com/doc/refman/8.4/en/numeric-type-syntax.html

The server implements the latter: FLOAT(24) creates a 4-byte FLOAT
column; only FLOAT(25) and above create a DOUBLE column. This is
intentional and correct, since IEEE 754 single precision has 24
significand bits (23 stored + 1 implicit), so a precision of 24 bits
still fits into a 4-byte FLOAT. The implementation in
sql/create_field.cc (identical in 8.0 and 8.4) uses
PRECISION_FOR_FLOAT = 24:

  /* change FLOAT(precision) to FLOAT or DOUBLE */
  if (display_width_in_codepoints && !fld_decimals) {
    const size_t tmp_length = m_max_display_width_in_codepoints;
    if (tmp_length > PRECISION_FOR_DOUBLE) {
      my_error(ER_WRONG_FIELD_SPEC, MYF(0), fld_name);
      return true;
    } else if (tmp_length > PRECISION_FOR_FLOAT) {
      sql_type = MYSQL_TYPE_DOUBLE;
    ...

i.e. DOUBLE is chosen only for p > 24. The floating-point-types page is
therefore off by one at the lower boundary; it should read 0 to 24 /
25 to 53, matching the numeric-type-syntax page.

For reference, other systems document the same boundary that MySQL
actually implements: MariaDB (0-24 -> FLOAT, 25-53 -> DOUBLE),
PostgreSQL (float(1)-float(24) -> real, float(25)-float(53) -> double
precision), and SQL Server (n = 1-24 -> 4 bytes with REAL defined as
float(24), n = 25-53 -> 8 bytes).

How to repeat:
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 9.4.0     |
+-----------+

mysql> CREATE TABLE t (a FLOAT(23), b FLOAT(24), c FLOAT(25));
Query OK, 0 rows affected

mysql> DESCRIBE t;
+-------+--------+------+-----+---------+-------+
| Field | Type   | Null | Key | Default | Extra |
+-------+--------+------+-----+---------+-------+
| a     | float  | YES  |     | NULL    |       |
| b     | float  | YES  |     | NULL    |       |
| c     | double | YES  |     | NULL    |       |
+-------+--------+------+-----+---------+-------+

According to the floating-point-types page, column b (FLOAT(24)) should
be DOUBLE. It is FLOAT, as correctly described on the
numeric-type-syntax page. Verified on 9.4.0 and 8.3.0 (Docker official
images).

Suggested fix:
On https://dev.mysql.com/doc/refman/<version>/en/floating-point-types.html
(all maintained versions), change the sentence to:

  "A precision from 0 to 24 results in a 4-byte single-precision FLOAT
   column. A precision from 25 to 53 results in an 8-byte double-precision
   DOUBLE column."

This aligns it with the numeric-type-syntax page and the actual
server behavior.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.048 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.