MySQL Bugs: #72361: After OPTIMIZE TABLE a SELECT on InnoDB spatial index crashes the server

Bug #72361	After OPTIMIZE TABLE a SELECT on InnoDB spatial index crashes the server
Submitted:	16 Apr 2014 22:22	Modified:	6 May 2014 13:47
Reporter:	Hartmut Holzgraefe	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: GIS	Severity:	S3 (Non-critical)
Version:	mysql-5.7.4-labs-april	OS:	Linux
Assigned to:		CPU Architecture:	Any

Description:
After OPTIMIZE table on an InnoDB table with spatial key any SELECT that would use this index will crash the server:

2014-04-17 00:08:15 0x7f548d035700  InnoDB: Assertion failure in thread 140001119786752 in file page0cur.cc line 262
InnoDB: Failing assertion: mode == 3 || mode == 4 || mode == 1 || mode == 2 || dict_index_is_spatial(index)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
22:08:15 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68149 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f5448000c50
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f548d034de0 thread_stack 0x40000
/usr/local/mysql-labs-april/bin/mysqld(my_print_stacktrace+0x38) [0xd30375]
/usr/local/mysql-labs-april/bin/mysqld(handle_fatal_signal+0x3aa) [0x7a7f7e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xfbb0) [0x7f5498603bb0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37) [0x7f5497a44f77]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7f5497a485e8]
/usr/local/mysql-labs-april/bin/mysqld() [0xfc1e9b]
/usr/local/mysql-labs-april/bin/mysqld(page_cur_search_with_match(buf_block_t const*, dict_index_t const*, dtuple_t const*, unsigned long, unsigned long*, unsigned long*, page_cur_t*, rtr_info*)+0x14e) [0xeb7f78]
/usr/local/mysql-labs-april/bin/mysqld(btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, unsigned long, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*)+0x1a48) [0xfeab51]
/usr/local/mysql-labs-april/bin/mysqld(btr_estimate_n_rows_in_range(dict_index_t*, dtuple_t const*, unsigned long, dtuple_t const*, unsigned long)+0x10e) [0xff41ab]
/usr/local/mysql-labs-april/bin/mysqld(ha_innobase::records_in_range(unsigned int, st_key_range*, st_key_range*)+0x523) [0xe367b5]
/usr/local/mysql-labs-april/bin/mysqld(handler::multi_range_read_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x2db) [0x8074e1]
/usr/local/mysql-labs-april/bin/mysqld(DsMrr_impl::dsmrr_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x64) [0x808c82]
/usr/local/mysql-labs-april/bin/mysqld(ha_innobase::multi_range_read_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x77) [0xe3ec6d]
/usr/local/mysql-labs-april/bin/mysqld() [0x9dbb51]
/usr/local/mysql-labs-april/bin/mysqld() [0x9d2e71]
/usr/local/mysql-labs-april/bin/mysqld(SQL_SELECT::test_quick_select(THD*, Bitmap<64u>, unsigned long long, unsigned long long, bool, st_order::enum_order)+0x103d) [0x9cc565]
/usr/local/mysql-labs-april/bin/mysqld() [0xa94936]
/usr/local/mysql-labs-april/bin/mysqld(JOIN::estimate_rowcount()+0x5d9) [0xa94045]
/usr/local/mysql-labs-april/bin/mysqld(JOIN::make_join_plan()+0x32f) [0xa92217]
/usr/local/mysql-labs-april/bin/mysqld(JOIN::optimize()+0xce2) [0xa88630]
/usr/local/mysql-labs-april/bin/mysqld(mysql_prepare_and_optimize_select(THD*, List<Item>&, unsigned long long, select_result*, st_select_lex*, bool*)+0x115) [0xaf8f1d]
/usr/local/mysql-labs-april/bin/mysqld(mysql_union_prepare_and_optimize(THD*, LEX*, select_result*, st_select_lex_unit*, unsigned long)+0x1f9) [0xb4d784]
/usr/local/mysql-labs-april/bin/mysqld(explain_query(THD*, st_select_lex_unit*, select_result*)+0x222) [0xc242bb]
/usr/local/mysql-labs-april/bin/mysqld() [0xab0ef2]
/usr/local/mysql-labs-april/bin/mysqld(mysql_execute_command(THD*)+0xd2f) [0xaa9e3f]
/usr/local/mysql-labs-april/bin/mysqld(mysql_parse(THD*, Parser_state*)+0x4a0) [0xab1de5]
/usr/local/mysql-labs-april/bin/mysqld(dispatch_command(enum_server_command, THD*, char*, unsigned long)+0xd6b) [0xaa711c]
/usr/local/mysql-labs-april/bin/mysqld(do_command(THD*)+0x321) [0xaa5edf]
/usr/local/mysql-labs-april/bin/mysqld(handle_connection+0x193) [0xbb5ca3]
/usr/local/mysql-labs-april/bin/mysqld(pfs_spawn_thread+0x14e) [0xd48fc1]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7f6e) [0x7f54985fbf6e]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5497b089cd]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f544800fb00): explain select count(*) from t1 where st_contains(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way)
Connection ID (thread ID): 1
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
Aborted (core dumped)

How to repeat:
CREATE TABLE `t1` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `way` geometry NOT NULL,
  PRIMARY KEY (`id`),
  SPATIAL KEY `way` (`way`)
) ENGINE=InnoDB;

INSERT INTO t1 SET way = GeomFromText('POINT(1 1)');                
INSERT INTO t1 SET way = GeomFromText('POINT(1 2)');               
INSERT INTO t1 SET way = GeomFromText('POINT(1 3)');               
INSERT INTO t1 SET way = GeomFromText('POINT(1 4)');                
INSERT INTO t1 SET way = GeomFromText('POINT(1 5)');                
INSERT INTO t1 SET way = GeomFromText('POINT(2 1)');                
INSERT INTO t1 SET way = GeomFromText('POINT(2 2)');                
INSERT INTO t1 SET way = GeomFromText('POINT(2 3)');                
INSERT INTO t1 SET way = GeomFromText('POINT(2 4)');                
INSERT INTO t1 SET way = GeomFromText('POINT(2 5)');                
INSERT INTO t1 SET way = GeomFromText('POINT(3 1)');                
INSERT INTO t1 SET way = GeomFromText('POINT(3 2)');                
INSERT INTO t1 SET way = GeomFromText('POINT(3 3)');                
INSERT INTO t1 SET way = GeomFromText('POINT(3 4)');                
INSERT INTO t1 SET way = GeomFromText('POINT(3 5)');                
INSERT INTO t1 SET way = GeomFromText('POINT(4 1)');                
INSERT INTO t1 SET way = GeomFromText('POINT(4 2)');                
INSERT INTO t1 SET way = GeomFromText('POINT(4 3)');                
INSERT INTO t1 SET way = GeomFromText('POINT(4 4)');                
INSERT INTO t1 SET way = GeomFromText('POINT(4 5)');                
INSERT INTO t1 SET way = GeomFromText('POINT(5 1)');                
INSERT INTO t1 SET way = GeomFromText('POINT(5 2)');                
INSERT INTO t1 SET way = GeomFromText('POINT(5 3)');                
INSERT INTO t1 SET way = GeomFromText('POINT(5 4)');                
INSERT INTO t1 SET way = GeomFromText('POINT(5 5)');                

-- this query works
SELECT COUNT(*) 
  FROM t1 
 WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);

OPTIMIZE TABLE t1;

-- this query works as it ignores the spatial index
SELECT COUNT(*) 
  FROM t1 IGNORE INDEX (way)
 WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);

-- this query crashes the server now 
SELECT COUNT(*) 
  FROM t1 
 WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);

Suggested fix:
Don't crash?

Hello Hartmut,

Thank you for the bug report.
Verified as described with the debug build.

Thanks,
Umesh

// Non debug build is not crashing

mysql> select version();
+----------------------+
| version()            |
+----------------------+
| 5.7.4-labs-april-log |
+----------------------+
1 row in set (0.00 sec)

mysql> CREATE TABLE `t1` (
    ->   `id` int(11) NOT NULL AUTO_INCREMENT,
    ->   `way` geometry NOT NULL,
    ->   PRIMARY KEY (`id`),
    ->   SPATIAL KEY `way` (`way`)
    -> ) ENGINE=InnoDB;

Query OK, 0 rows affected (0.03 sec)

mysql>
mysql> INSERT INTO t1 SET way = GeomFromText('POINT(1 1)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(1 2)');
INSERT INTO t1 SET way = GeomFromText('POINT(1 3)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(1 3)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(1 4)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(1 5)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(2 1)');
INSERT INTO t1 SET way = GeomFromText('POINT(2 2)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(2 2)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(2 3)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(2 4)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(2 5)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(3 1)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(3 2)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(3 3)');
Query OK, 1 row affected (0.01 sec)

mysql> INSEINSERT INTO t1 SET way = GeomFromText('POINT(3 4)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(3 5)');
INSERT INTO t1 SET way = GeomFromText('POINT(4 1)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(4 1)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(4 2)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(4 3)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(4 4)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(4 5)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(5 1)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(5 2)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(5 3)');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(5 4)');
Query OK, 1 row affected (0.01 sec)

mysql> INSERT INTO t1 SET way = GeomFromText('POINT(5 5)');
Query OK, 1 row affected (0.00 sec)

mysql> SELECT COUNT(*)
    ->   FROM t1
    ->  WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);
+----------+
| COUNT(*) |
+----------+
|        1 |
+----------+
1 row in set (0.00 sec)

mysql>
mysql> OPTIMIZE TABLE t1;
+---------+----------+----------+-------------------------------------------------------------------+
| Table   | Op       | Msg_type | Msg_text                                                          |
+---------+----------+----------+-------------------------------------------------------------------+
| test.t1 | optimize | note     | Table does not support optimize, doing recreate + analyze instead |
| test.t1 | optimize | status   | OK                                                                |
+---------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.08 sec)

mysql> SELECT COUNT(*)
    ->   FROM t1 IGNORE INDEX (way)
    ->  WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);
+----------+
| COUNT(*) |
+----------+
|        1 |
+----------+
1 row in set (0.01 sec)

mysql> SELECT COUNT(*)
    ->   FROM t1
    ->  WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);
+----------+
| COUNT(*) |
+----------+
|        1 |
+----------+
1 row in set (0.00 sec)

// With debug build - it crashing

mysql> SELECT COUNT(*)    FROM t1   WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way);
2014-04-20 06:28:14 0x7fb70c5a2700  InnoDB: Assertion failure in thread 140424162977536 in file page0cur.cc line 262
InnoDB: Failing assertion: mode == 3 || mode == 4 || mode == 1 || mode == 2 || dict_index_is_spatial(index)
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
00:58:14 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68149 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7fb6c8000ca0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fb70c5a1dc0 thread_stack 0x40000
bin/mysqld-debug(my_print_stacktrace+0x35) [0xd65041]
bin/mysqld-debug(handle_fatal_signal+0x412) [0x7c437e]
/lib64/libpthread.so.0() [0x39ed40f710]
/lib64/libc.so.6(gsignal+0x35) [0x39ed032925]
/lib64/libc.so.6(abort+0x175) [0x39ed034105]
bin/mysqld-debug() [0xffe43b]
bin/mysqld-debug(page_cur_search_with_match(buf_block_t const*, dict_index_t const*, dtuple_t const*, unsigned long, unsigned long*, unsigned long*, page_cur_t*, rtr_info*)+0x104) [0xeee8e3]
bin/mysqld-debug(btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, unsigned long, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*)+0x1af5) [0x1027fe8]
bin/mysqld-debug(btr_estimate_n_rows_in_range(dict_index_t*, dtuple_t const*, unsigned long, dtuple_t const*, unsigned long)+0xfe) [0x1031002]
bin/mysqld-debug(ha_innobase::records_in_range(unsigned int, st_key_range*, st_key_range*)+0x54a) [0xe68c2c]
bin/mysqld-debug(handler::multi_range_read_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x2dd) [0x8253a9]
bin/mysqld-debug(DsMrr_impl::dsmrr_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x6c) [0x826c04]
bin/mysqld-debug(ha_innobase::multi_range_read_info_const(unsigned int, st_range_seq_if*, void*, unsigned int, unsigned int*, unsigned int*, Cost_estimate*)+0x7d) [0xe71227]
bin/mysqld-debug() [0xa070ff]
bin/mysqld-debug() [0x9fe164]
bin/mysqld-debug(SQL_SELECT::test_quick_select(THD*, Bitmap<64u>, unsigned long long, unsigned long long, bool, st_order::enum_order)+0x10ab) [0x9f7dc1]
bin/mysqld-debug() [0xac26c8]
bin/mysqld-debug(JOIN::estimate_rowcount()+0x513) [0xac1de7]
bin/mysqld-debug(JOIN::make_join_plan()+0x344) [0xabfefc]
bin/mysqld-debug(JOIN::optimize()+0xcd7) [0xab631b]
bin/mysqld-debug(mysql_prepare_and_optimize_select(THD*, List<Item>&, unsigned long long, select_result*, st_select_lex*, bool*)+0x125) [0xb2675b]
bin/mysqld-debug(mysql_select(THD*, List<Item>&, unsigned long long, select_result*, st_select_lex*)+0x69) [0xb2681d]
bin/mysqld-debug(handle_select(THD*, select_result*, unsigned long)+0x1a2) [0xb24ed6]
bin/mysqld-debug() [0xadfa2c]
bin/mysqld-debug(mysql_execute_command(THD*)+0xdc0) [0xad83b0]
bin/mysqld-debug(mysql_parse(THD*, Parser_state*)+0x4d5) [0xae0823]
bin/mysqld-debug(dispatch_command(enum_server_command, THD*, char*, unsigned long)+0xde3) [0xad54f8]
bin/mysqld-debug(do_command(THD*)+0x352) [0xad4223]
bin/mysqld-debug(handle_connection+0x1a4) [0xbe5157]
bin/mysqld-debug(pfs_spawn_thread+0x14e) [0xdeef59]
/lib64/libpthread.so.0() [0x39ed4079d1]
/lib64/libc.so.6(clone+0x6d) [0x39ed0e8b6d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fb6c800fba0): SELECT COUNT(*)    FROM t1   WHERE ST_CONTAINS(GeomFromText('POLYGON((2 2,4 2, 4 4, 2 4, 2 2))'),way)
Connection ID (thread ID): 2
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
ERROR 2013 (HY000): Lost connection to MySQL server during query

Posted by developer:
 
Since Bug#13975225 fix, OPTIMIZE TABLE maps to online ALTER (WL#6255 inside InnoDB).

It seems to me that this check in ha_innobase::check_if_supported_inplace_alter() is insufficent:

	if ((ha_alter_info->handler_flags
	     & Alter_inplace_info::ADD_SPATIAL_INDEX)
	    || ((ha_alter_info->handler_flags
		 & Alter_inplace_info::ADD_PK_INDEX)
		&& innobase_spatial_exist(altered_table))) {
		DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED);
	}

Instead of only checking for ADD_PK_INDEX (ADD PRIMARY KEY), this should invoke innobase_need_rebuild().

Fixed as of 5.7.5, and here's the changelog entry:

After running "OPTIMIZE TABLE" on an "InnoDB" table with a spatial index,
running a "SELECT" statement that uses the spatial index could result in a
crash. 

Thank you for the bug report.