Description:
In previous versions of MySQL it was possible to create a table:
CREATE TABLE t1 (
a VARCHAR(255) NOT NULL,
b VARCHAR(255) NOT NULL,
c VARCHAR(255) NOT NULL,
d VARCHAR(175) NOT NULL,
g GEOMETRY NOT NULL SRID 4326,
PRIMARY KEY (a, b, c, d),
SPATIAL KEY idx_g (g)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1;
and then insert some rows, for example:
INSERT INTO t1(a,b,c,d,g) VALUES ('a', 'b', 'c', 'd', ST_GeomFromText('POINT(37.7749 -122.4194)', 4326));
In 9.7, this table can't be created:
CREATE TABLE t1 (
a VARCHAR(255) NOT NULL,
b VARCHAR(255) NOT NULL,
c VARCHAR(255) NOT NULL,
d VARCHAR(175) NOT NULL,
g GEOMETRY NOT NULL SRID 4326,
PRIMARY KEY (a, b, c, d),
SPATIAL KEY idx_g (g)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1'
Returned error: ER_TOO_BIG_ROWSIZE(1118) (42000): Row size too large (> 8126). Changing some columns to TEXT or BLOB may help. In current row format, BLOB prefix of 0 bytes is stored inline.
This issue is still there after applying a related bug-fix:
https://github.com/mysql/mysql-server/pull/657#issuecomment-4410209000
The reason is:
1. In previous versions of MySQL there was a possible overflow during estimation of the maximum number of bytes occupied by indexed fields. That's because for a geometry field, ULINT_MAX was returned.
2. The overflow was dangerous when the estimation was executed to decide whether a node should be x-latched.
3. But it was not a big problem when the estimation was executed to check whether a given table might be created.
4. In 9.7, these two use cases started to share implementation (get_field_max_size), which perhaps was not a fortunate decision, because they have different requirements:
- for x-latch decision: it must never underestimate, should take into account only index fields, and should be efficient.
- for DDL decision: it should not break backward compatibility, it might underestimate, doesn't need to be efficient, and should take into account all fields.
5. Because of the shared implementation, the bug fix that was critical for the x-latch decision broke the backward compatibility for the DDL decision.
How to repeat:
Try:
CREATE TABLE t1 (
a VARCHAR(255) NOT NULL,
b VARCHAR(255) NOT NULL,
c VARCHAR(255) NOT NULL,
d VARCHAR(175) NOT NULL,
g GEOMETRY NOT NULL SRID 4326,
PRIMARY KEY (a, b, c, d),
SPATIAL KEY idx_g (g)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1;
INSERT INTO t1(a,b,c,d,g) VALUES ('a', 'b', 'c', 'd', ST_GeomFromText('POINT(37.7749 -122.4194)', 4326));
Suggested fix:
I would suggest a split of implementations for these two different use cases.
Alternatively, let's restore the overflow when the function is being used to check if a table might be created, which would most likely require a new argument.
The following fix shows how to patch the existing solution:
https://github.com/polchawa-percona/mysql-server/pull/1
Assumption:
https://github.com/mysql/mysql-server/pull/657#issuecomment-4410209000
has been merged first (it solves the most critical part of the bug).
Side note: splitting might also be beneficial for efficiency - smaller code used in regular workloads and less branching.
Description: In previous versions of MySQL it was possible to create a table: CREATE TABLE t1 ( a VARCHAR(255) NOT NULL, b VARCHAR(255) NOT NULL, c VARCHAR(255) NOT NULL, d VARCHAR(175) NOT NULL, g GEOMETRY NOT NULL SRID 4326, PRIMARY KEY (a, b, c, d), SPATIAL KEY idx_g (g) ) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1; and then insert some rows, for example: INSERT INTO t1(a,b,c,d,g) VALUES ('a', 'b', 'c', 'd', ST_GeomFromText('POINT(37.7749 -122.4194)', 4326)); In 9.7, this table can't be created: CREATE TABLE t1 ( a VARCHAR(255) NOT NULL, b VARCHAR(255) NOT NULL, c VARCHAR(255) NOT NULL, d VARCHAR(175) NOT NULL, g GEOMETRY NOT NULL SRID 4326, PRIMARY KEY (a, b, c, d), SPATIAL KEY idx_g (g) ) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1' Returned error: ER_TOO_BIG_ROWSIZE(1118) (42000): Row size too large (> 8126). Changing some columns to TEXT or BLOB may help. In current row format, BLOB prefix of 0 bytes is stored inline. This issue is still there after applying a related bug-fix: https://github.com/mysql/mysql-server/pull/657#issuecomment-4410209000 The reason is: 1. In previous versions of MySQL there was a possible overflow during estimation of the maximum number of bytes occupied by indexed fields. That's because for a geometry field, ULINT_MAX was returned. 2. The overflow was dangerous when the estimation was executed to decide whether a node should be x-latched. 3. But it was not a big problem when the estimation was executed to check whether a given table might be created. 4. In 9.7, these two use cases started to share implementation (get_field_max_size), which perhaps was not a fortunate decision, because they have different requirements: - for x-latch decision: it must never underestimate, should take into account only index fields, and should be efficient. - for DDL decision: it should not break backward compatibility, it might underestimate, doesn't need to be efficient, and should take into account all fields. 5. Because of the shared implementation, the bug fix that was critical for the x-latch decision broke the backward compatibility for the DDL decision. How to repeat: Try: CREATE TABLE t1 ( a VARCHAR(255) NOT NULL, b VARCHAR(255) NOT NULL, c VARCHAR(255) NOT NULL, d VARCHAR(175) NOT NULL, g GEOMETRY NOT NULL SRID 4326, PRIMARY KEY (a, b, c, d), SPATIAL KEY idx_g (g) ) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=2 CHARSET=latin1; INSERT INTO t1(a,b,c,d,g) VALUES ('a', 'b', 'c', 'd', ST_GeomFromText('POINT(37.7749 -122.4194)', 4326)); Suggested fix: I would suggest a split of implementations for these two different use cases. Alternatively, let's restore the overflow when the function is being used to check if a table might be created, which would most likely require a new argument. The following fix shows how to patch the existing solution: https://github.com/polchawa-percona/mysql-server/pull/1 Assumption: https://github.com/mysql/mysql-server/pull/657#issuecomment-4410209000 has been merged first (it solves the most critical part of the bug). Side note: splitting might also be beneficial for efficiency - smaller code used in regular workloads and less branching.