MySQL Bugs: #73776: GIS: crash in split_rtree_node when inserting into spatial index

Bug #73776	GIS: crash in split_rtree_node when inserting into spatial index
Submitted:	30 Aug 2014 16:10	Modified:	26 Sep 2014 17:26
Reporter:	Shane Bester (Platinum Quality Contributor)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: GIS	Severity:	S2 (Serious)
Version:	5.7.5	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
Affects trunk debug/release builds.

Version: '5.7.5-m15'  MySQL Community Server (GPL)
mysqld got exception 0xc0000005 ;

mysqld.exe!split_rtree_node()[gis0geo.cc:565]
mysqld.exe!rtr_page_split_and_insert()[gis0rtree.cc:1100]
mysqld.exe!btr_cur_pessimistic_insert()[btr0cur.cc:3382]
mysqld.exe!row_ins_sec_index_entry_low()[row0ins.cc:3053]
mysqld.exe!row_ins_sec_index_entry()[row0ins.cc:3277]
mysqld.exe!row_ins_index_entry_step()[row0ins.cc:3454]
mysqld.exe!row_ins()[row0ins.cc:3592]
mysqld.exe!row_ins_step()[row0ins.cc:3765]
mysqld.exe!row_insert_for_mysql_using_ins_graph()[row0mysql.cc:1584]
mysqld.exe!ha_innobase::write_row()[ha_innodb.cc:6758]
mysqld.exe!handler::ha_write_row()[handler.cc:7432]
mysqld.exe!write_record()[sql_insert.cc:1347]
mysqld.exe!mysql_insert()[sql_insert.cc:708]
mysqld.exe!mysql_execute_command()[sql_parse.cc:3367]
mysqld.exe!mysql_parse()[sql_parse.cc:5434]
mysqld.exe!dispatch_command()[sql_parse.cc:1253]
mysqld.exe!do_command()[sql_parse.cc:841]
mysqld.exe!handle_connection()[connection_handler_per_thread.cc:298]
mysqld.exe!pfs_spawn_thread()[pfs.cc:2140]
mysqld.exe!pthread_start()[my_winthread.c:74]
mysqld.exe!_callthreadstartex()[threadex.c:376]
mysqld.exe!_threadstartex()[threadex.c:354]

How to repeat:
table structure is simply:
create table t8(a geometrycollection not null,spatial key(a))engine=innodb;

Then import the attached .sql file.

Affects both windows/linux builds, although sometimes needing different testcases.

split_rtree_node (node=0x160bc9d0, n_entries=<optimized out>, all_size=16104, key_size=44, min_size=0, size1=46, size2=46, d_buffer=0x162acb00, n_dim=2, first_rec=0x0) at ./mysql-trunk-clean/storage/innobase/gis/gis0geo.cc:585
585        mbr_join(g1, next->coords, n_dim);
(gdb) list
580                     }
581
582                     pick_next(node, n_entries, g1, g2, &next, &next_node, n_dim);
583                     if (next_node == 1) {
584                             size1 += key_size;
585                             mbr_join(g1, next->coords, n_dim);
586                     } else {
587                             size2 += key_size;
588                             mbr_join(g2, next->coords, n_dim);
589                     }
(gdb) p next
$1 = (rtr_split_node_t *) 0x0

import with mysql client. run mysqld in valgrind if necessary.

Attachment: bug73776_testcase.sql (application/octet-stream, text), 1.17 MiB.

Posted by developer:
 
In function mbr_join_square, the square could be a infinity value caused this bug.
We need to detect this, and return DBL_MAX in this case.

Posted by developer:
 
Fixed as of the upcoming 5.7.6 release, and here's the changelog entry:

An "INSERT" operation on a spatial index resulted in a crash in
"split_rtree_node()". The "mbr_join_square" function failed to check for
infinity and NaN (not a number) values. 

Thank you for the bug report.