Bug #94560 record comparison in spatial index non-leaf rtree node seems incorrect

Submitted: 5 Mar 2019 11:43 Modified: 7 Mar 2019 4:29
Reporter: Jie Zhou Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: GIS Severity:S2 (Serious)
Version:5.7, 8.0 OS:Any
Assigned to: CPU Architecture:Any
Tags: gis, spatial index

[5 Mar 2019 11:43] Jie Zhou
Description:
According to the function cmp_rec_rec_with_match:
---------
	for (; cur_field < rec1_n_fields && cur_field < rec2_n_fields;
	     cur_field++) {

		ulint	mtype;
		ulint	prtype;

		/* If this is node-ptr records then avoid comparing node-ptr
		field. Only key field needs to be compared. */
		if (cur_field == dict_index_get_n_unique_in_tree(index)) {
			break;
		}

		...
			const dict_col_t*	col;

			col	= dict_index_get_nth_col(index, cur_field);

			mtype = col->mtype;
			prtype = col->prtype;

			/* If the index is spatial index, we mark the
			prtype of the first field as MBR field. */
			if (cur_field == 0 && dict_index_is_spatial(index)) {
				ut_ad(DATA_GEOMETRY_MTYPE(mtype));
				prtype |= DATA_GIS_MBR;
			}
		}
                 ...
		ret = cmp_data(mtype, prtype,
			       rec1_b_ptr, rec1_f_len,
			       rec2_b_ptr, rec2_f_len);
		if (ret) {
			goto order_resolved;
		}
	}
---------
If the record is located in a non-leaf node of spatial index, it consists of two fields, mbr and node-ptr.

"if (cur_field == 0 && dict_index_is_spatial(index)) prtype |= DATA_GIS_MBR;" 
It only deals with the first col.

For the second field, mtype will be set to col->mtype.
It equals to index->fields[1]->col->mtype, which is the primary key's mtype.
So that we compare node-ptr field by pk's mtype.
It is ok in most cases since pk is always set to a int type in real world.
But if pk is varchar type or others, some unexpected problem may happen.

How to repeat:
Currently, I just read the code direcly and find the problem.

Suggested fix:
if (dict_index_is_spatial(index)
    && (!page_is_leaf(page_align(rec1)) || !page_is_leaf(page_align(rec2)))) {
    cmp_spatial_index_nonleaf = true;
}
if (cur_field == 1 && cmp_spatial_index_nonleaf) {
    /* mark as node-ptr */
    mtype = DATA_SYS_CHILD;
    prtype = DATA_NOT_NULL;
}
[7 Mar 2019 4:29] MySQL Verification Team
Hi,

Thanks for your report and suggested fix.

all best
Bogdan

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.040 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.