MySQL Bugs: #76748: the server crash when test st_intersects with st

Bug #76748	the server crash when test st_intersects with st_buffer
Submitted:	20 Apr 2015 6:01	Modified:	22 Apr 2015 13:12
Reporter:	zkong kong	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: GIS	Severity:	S2 (Serious)
Version:	5.7.7-rc	OS:	Linux (centos4.3)
Assigned to:		CPU Architecture:	Any

Description:
Build the mysql-5.7.7-rc on centos 4.3 with the gcc 4.8.2 (boost 1.5.7) and ran the follow case the server crashed.
 select t1.geo from t,t1 where st_intersects(t1.geo, st_buffer(t.geo,50));

How to repeat:
build the server using: cmake ../ -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS="-g -O3" -DCMAKE_CXX_FLAGS="-g -O3"  -DCMAKE_INSTALL_PREFIX=/home/mysql/mysql577/ -DWITH_BOOST=/home/mysql/gis/mysql-5.7.7-rc/source_downloads/ -DCMAKE_SKIP_BUILD_RPATH=ON

the test data

Attachment: test.sql (application/octet-stream, text), 72.01 KiB.

using gdb to debug the server found that in Gis_polygon::Gis_polygon inner_container_type v(wkb, nbytes, get_flags(), srid); 
the get_flags return value is not assigned to v third argument correctlly.
Modify the Flags_t copy constructor then the test can pass. But I don't know why.

901,903c901,908
<       compile_time_assert(sizeof(*this) == sizeof(uint64));
<       *(reinterpret_cast<uint64 *>(this))=
<         *(reinterpret_cast<const uint64 *>(&o));
---
>       this->bo= o.bo;
>       this->dim= o.dim;
>       this->nomem= o.nomem;
>       this->geotype= o.geotype;
>       this->nbytes= o.nbytes;
>       this->props= o.props;
>       this->zm= o.zm;
>       this->unused= o.unused;

Version: '5.8.0-m17-asan'  MySQL Community Server (GPL)
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd0bbd700 (LWP 6443)]
in size (this=...) at spatial.h:2270
2270        return m_geo_vect ? m_geo_vect->size() : 0;
(gdb) bt
#0  in size (this=...) at spatial.h:2270
#1  in Gis_polygon_ring::set_ring_order  at spatial.cc:1905
#2  in Gis_polygon::set_polygon_ring_order  at spatial.cc:2044
#3  in Geometry::normalize_ring_order  at spatial.cc:4353
#4  in bg_buffer  at item_geofunc_buffer.cc:376
#5  in Item_func_buffer::val_str at item_geofunc_buffer.cc:598
#6  in Item_func_spatial_rel::val_int  at item_geofunc_relchecks.cc:196
#7  in skip_record  at sql_executor.h:455
#8  in JOIN_CACHE::check_match  at sql_join_buffer.cc:2085
#9  in JOIN_CACHE::generate_full_extensions at sql_join_buffer.cc:2034
#10 in JOIN_CACHE_BNL::join_matching_records  at sql_join_buffer.cc:1924
#11 in JOIN_CACHE::join_records  at sql_join_buffer.cc:1729
#12 in sub_select_op  at sql_executor.cc:1052
#13 in do_select  at sql_executor.cc:935
#14 in JOIN::exec  at sql_executor.cc:198
#15 in handle_query  at sql_select.cc:185
#16 in execute_sqlcom_select  at sql_parse.cc:4690
#17 in mysql_execute_command  at sql_parse.cc:2418
#18 in mysql_parse  at sql_parse.cc:5110
#19 in dispatch_command  at sql_parse.cc:1213
#20 in do_command  at sql_parse.cc:797
#21 in handle_connection  at connection_handler_per_thread.cc:299
#22 in pfs_spawn_thread at pfs.cc:2147
#23 in start_thread () from /usr/lib64/libpthread.so.0
#24 in clone () from /usr/lib64/libc.so.6

Thanks for the report!   Affects 5.7 and 5.8.  Might have to use ASAN build to repeat the crashes.

Noted in 5.7.8, 5.8.0 changelogs.

Unaligned memory access could cause spatial operations to fail.