Bug #30286 spatial index cause corruption and server crash!
Submitted: 7 Aug 2007 21:32 Modified: 5 Nov 2007 2:52
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: MyISAM storage engine Severity:S1 (Critical)
Version:5.0.45, 5.0.48BK, 5.1.21 OS:Any
Assigned to: Alexey Botchkov CPU Architecture:Any
Tags: corruption, crash, myisam, rtree, spatial

[7 Aug 2007 21:32] Shane Bester
Description:
on 5.0.45 on windows:

mysql> insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
ERROR 126 (HY000): Incorrect key file for table '.\f\t1.MYI'; try to repair it
mysql> check table t1 extended;
+-------+-------+----------+----------------------------+
| Table | Op    | Msg_type | Msg_text                   |
+-------+-------+----------+----------------------------+
| f.t1  | check | warning  | Table is marked as crashed |
| f.t1  | check | status   | OK                         |
+-------+-------+----------+----------------------------+
2 rows in set (0.00 sec)

on 5.1.21-debug on linux, server crashes!

Version: '5.1.21-beta-debug'  socket: '/tmp/mysql.sock'  port: 3306  yes
070807 23:12:03 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=8388572
read_buffer_size=131072
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 337612 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x8db0b90
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x4286482c, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x81fd55d handle_segfault + 541
0x84856e8 _mi_kpos + 96
0x84ae411 rtree_insert_req + 408
0x84ae7fe rtree_insert_level + 457
0x84aeb50 rtree_insert + 140
0x8493dbc mi_write + 1020
0x847aead _ZN9ha_myisam9write_rowEPh + 99
0x82ed67b _ZN7handler12ha_write_rowEPh + 33
0x827d044 _Z12write_recordP3THDP8st_tableP12st_copy_info + 1968
0x827b8fa _Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb + 2248
0x820cc1c _Z21mysql_execute_commandP3THD + 10268
0x821372e _Z11mysql_parseP3THDPKcjPS2_ + 372
0x8208ec8 _Z16dispatch_command19enum_server_commandP3THDPcj + 2320
0x82085ac _Z10do_commandP3THD + 612
0x82070ef handle_one_connection + 253
0x40038aa7 _end + 931867479
0x4017ec2e _end + 933203166
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/refman/5.1/en/resolve-stack-dump.html
and follow instructions on how to resolve the stack trace.
Resolved stack trace is much more helpful in diagnosing the
problem, so please do resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8de08c0 = insert ignore into `t1`(`a`) values (GeomFromWKB(@v))
thd->thread_id=1

How to repeat:
drop table if exists `t1`;
create table `t1` (`a` geometry not null, spatial index(`a`))engine=myisam;
set @v=unhex(concat('01','01000000',lpad(conv(abs(-98),10,16),16,'0'),lpad(conv(abs(65),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-104),10,16),16,'0'),lpad(conv(abs(-123),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-2),10,16),16,'0'),lpad(conv(abs(-128),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-32),10,16),16,'0'),lpad(conv(abs(-53),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-95),10,16),16,'0'),lpad(conv(abs(87),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-113),10,16),16,'0'),lpad(conv(abs(-14),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-58),10,16),16,'0'),lpad(conv(abs(87),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-128),10,16),16,'0'),lpad(conv(abs(12),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-53),10,16),16,'0'),lpad(conv(abs(112),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(25),10,16),16,'0'),lpad(conv(abs(-118),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-63),10,16),16,'0'),lpad(conv(abs(-113),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-98),10,16),16,'0'),lpad(conv(abs(38),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-9),10,16),16,'0'),lpad(conv(abs(7),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-53),10,16),16,'0'),lpad(conv(abs(58),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(7),10,16),16,'0'),lpad(conv(abs(-82),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-91),10,16),16,'0'),lpad(conv(abs(-8),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(13),10,16),16,'0'),lpad(conv(abs(-68),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-13),10,16),16,'0'),lpad(conv(abs(117),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(8),10,16),16,'0'),lpad(conv(abs(37),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-128),10,16),16,'0'),lpad(conv(abs(56),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-98),10,16),16,'0'),lpad(conv(abs(-31),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-94),10,16),16,'0'),lpad(conv(abs(-8),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-8),10,16),16,'0'),lpad(conv(abs(-47),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-113),10,16),16,'0'),lpad(conv(abs(57),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(62),10,16),16,'0'),lpad(conv(abs(-53),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-53),10,16),16,'0'),lpad(conv(abs(-26),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(37),10,16),16,'0'),lpad(conv(abs(-104),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-2),10,16),16,'0'),lpad(conv(abs(31),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
set @v=unhex(concat('01','01000000',lpad(conv(abs(-124),10,16),16,'0'),lpad(conv(abs(27),10,16),16,'0')));
insert ignore into `t1`(`a`) values (GeomFromWKB(@v));
check table t1 extended;

Suggested fix:
.
[17 Aug 2007 11:33] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32691

ChangeSet@1.2679, 2007-08-17 15:29:47+05:00, holyfoot@mysql.com +3 -0
  Bug #30286 spatial index cause corruption and server crash!
  
  Sometimes the result of the DOUBLE claculations can be bigger
  than DBL_MAX constant we use in code. If that happens in
  rtree_pick_key it can fail as it cannot find suitable key.
  
  So we shouldn't use DBL_MAX constant as a biggest double we
  can ever get.
[30 Aug 2007 12:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/33433

ChangeSet@1.2679, 2007-08-30 16:40:21+05:00, holyfoot@mysql.com +4 -0
  Bug #30286 spatial index cause corruption and server crash!
  
  As the result of DOUBLE claculations can be bigger
  than DBL_MAX constant we use in code, we shouldn't use this constatn
  as a biggest possible value.
  Particularly the rtree_pick_key function set 'min_area= DBL_MAX' relying
  that any rtree_area_increase result will be less so we return valid
  key. Though in rtree_area_increase function we calculate the area
  of the rectangle, so the result can be 'inf' if the rectangle is
  huge enough, which is bigger than DBL_MAX.
  
  Code of the rtree_pick_key modified so we always return a valid key.
[5 Oct 2007 11:47] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/34960

ChangeSet@1.2679, 2007-10-05 15:40:32+05:00, holyfoot@mysql.com +4 -0
  Bug #30286 spatial index cause corruption and server crash!
  
  As the result of DOUBLE claculations can be bigger
  than DBL_MAX constant we use in code, we shouldn't use this constatn
  as a biggest possible value.
  Particularly the rtree_pick_key function set 'min_area= DBL_MAX' relying
  that any rtree_area_increase result will be less so we return valid
  key. Though in rtree_area_increase function we calculate the area
  of the rectangle, so the result can be 'inf' if the rectangle is
  huge enough, which is bigger than DBL_MAX.
  
  Code of the rtree_pick_key modified so we always return a valid key.
[29 Oct 2007 8:40] Bugs System
Pushed into 4.1.24
[29 Oct 2007 8:44] Bugs System
Pushed into 5.0.52
[29 Oct 2007 8:47] Bugs System
Pushed into 5.1.23-beta
[29 Oct 2007 8:51] Bugs System
Pushed into 6.0.4-alpha
[5 Nov 2007 2:52] Paul Dubois
Noted in 4.1.24, 5.0.52, 5.1.23, 6.0.4 changelogs.

Setting certain values on a table using a spatial index could cause the server to crash.