Bug #42759 | optimizer choosing different plans/index for the same query | ||
---|---|---|---|
Submitted: | 11 Feb 2009 12:35 | Modified: | 14 Dec 2009 16:33 |
Reporter: | walid bakkar | Email Updates: | |
Status: | No Feedback | Impact on me: | |
Category: | MySQL Server: Optimizer | Severity: | S5 (Performance) |
Version: | 5.0.51, 5.1.30 | OS: | Linux |
Assigned to: | CPU Architecture: | Any |
[11 Feb 2009 12:35]
walid bakkar
[11 Feb 2009 13:10]
Valeriy Kravchuk
Thank you for the problem report. What exact version of 5.1, 5.1.x, you had used? Please, send also the results of: select count(*) from field join0 where join0.field_type=2; select count(*) from field join0 where join0.field_numeric > 1 AND join0.field_numeric < 3; select count(*) from field join0 where join0.field_numeric > 887 AND join0.field_numeric < 889; select count(*) from field join0 where join0.field_type=2 AND join0.field_numeric > 1 AND join0.field_numeric < 3; select count(*) from field join0 where join0.field_type=2 AND join0.field_numeric > 887 AND join0.field_numeric < 889;
[11 Feb 2009 13:13]
walid bakkar
this is the type of data distribution in the tables (index/cardinality): mysql> show index from DOCUMENT; +----------+------------+-----------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | +----------+------------+-----------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+ | DOCUMENT | 0 | PRIMARY | 1 | docid | A | 527757 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | rootkey_index | 1 | rootkey | A | 527757 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | latestkey_index | 1 | latest | A | 12 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | form_index | 1 | form | A | 18 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | creator_index | 1 | creator | A | 16 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | creation_date_index | 1 | creation_date | A | 175919 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | workflow_state_index | 1 | workflow_state | A | 18 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | last_modified_by_index | 1 | last_modified_by | A | 18 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | last_modified_date_index | 1 | last_modified_date | A | 75393 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | timer_date_index | 1 | timer_date | A | 16 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | timer_processed_index | 1 | timer_processed | A | 16 | NULL | NULL | YES | BTREE | | | DOCUMENT | 1 | dockey_versionnumber_unique | 1 | rootkey | A | 527757 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | dockey_versionnumber_unique | 2 | version_number | A | 527757 | NULL | NULL | | BTREE | | | DOCUMENT | 1 | dockey_versionnumber_unique | 3 | language | A | 527757 | NULL | NULL | | BTREE | | +----------+------------+-----------------------------+--------------+--------------------+-----------+-------------+----------+--------+------+------------+---------+ 14 rows in set (0.51 sec) mysql> show index from FIELD; +-------+------------+---------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+ | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | +-------+------------+---------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+ | FIELD | 0 | PRIMARY | 1 | docid | A | 2719642 | NULL | NULL | | BTREE | | | FIELD | 0 | PRIMARY | 2 | field_name | A | 57112489 | NULL | NULL | | BTREE | | | FIELD | 0 | PRIMARY | 3 | field_position | A | 57112489 | NULL | NULL | | BTREE | | | FIELD | 1 | docid_index | 1 | docid | A | 865340 | NULL | NULL | | BTREE | | | FIELD | 1 | field_name_index | 1 | field_name | A | 11 | NULL | NULL | | BTREE | | | FIELD | 1 | field_type_index | 1 | field_type | A | 11 | NULL | NULL | | BTREE | | | FIELD | 1 | field_string_index | 1 | field_string | A | 19037496 | NULL | NULL | | BTREE | | | FIELD | 1 | field_numeric_index | 1 | field_numeric | A | 11 | NULL | NULL | YES | BTREE | | | FIELD | 1 | field_date_index | 1 | field_date | A | 11 | NULL | NULL | YES | BTREE | | +-------+------------+---------------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+ 9 rows in set (0.48 sec)
[11 Feb 2009 13:41]
walid bakkar
select version(); +-----------+ | version() | +-----------+ | 5.1.30 | +-----------+ 1 row in set (0.04 sec) mysql> explain select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1000) AND (join0.field_numeric < 3000)) ids order by d_docid ASC; +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ | 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 13 | Using filesort | | 2 | DERIVED | join0 | range | PRIMARY,docid_index,field_name_index,field_type_index,field_numeric_index | field_numeric_index | 9 | NULL | 156 | Using where; Using temporary | | 2 | DERIVED | d | eq_ref | PRIMARY,form_index | PRIMARY | 4 | pbm.join0.docid | 1 | Using where | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ 3 rows in set (0.16 sec) mysql> explain select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1) AND (join0.field_numeric < 2)) ids order by d_docid ASC; +----+-------------+------------+------+---------------------------------------------------------------------------+------------+---------+-------------+-------+-------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+------+---------------------------------------------------------------------------+------------+---------+-------------+-------+-------------------------------------------+ | 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 3799 | Using filesort | | 2 | DERIVED | d | ref | PRIMARY,form_index | form_index | 386 | | 51200 | Using where; Using index; Using temporary | | 2 | DERIVED | join0 | ref | PRIMARY,docid_index,field_name_index,field_type_index,field_numeric_index | PRIMARY | 390 | pbm.d.docid | 1 | Using where; Distinct | +----+-------------+------------+------+---------------------------------------------------------------------------+------------+---------+-------------+-------+-------------------------------------------+ 3 rows in set (11.49 sec) mysql> select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1000) AND (join0.field_numeric < 3000)) ids order by d_docid ASC; 13 rows in set (0.03 sec) mysql> select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1) AND (join0.field_numeric < 2)) ids order by d_docid ASC; 3799 rows in set (12.31 sec) mysql> select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 force index (field_numeric_index) on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1) AND (join0.field_numeric < 2)) ids order by d_docid ASC; 3799 rows in set (4.28 sec) mysql> select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 force index (field_numeric_index) on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 887) AND (join0.field_numeric < 889)) ids order by d_docid ASC; 15248 rows in set (0.23 sec) select d_docid from (select distinct d.docid as d_docid from (DOCUMENT d left outer join FIELD join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 887) AND (join0.field_numeric < 889)) ids order by d_docid ASC; 15248 rows in set (11.91 sec) mysql> select count(*) from FIELD join0 where join0.field_type=2; +----------+ | count(*) | +----------+ | 1480999 | +----------+ 1 row in set (0.73 sec) mysql> select count(*) from FIELD join0 where join0.field_numeric > 1 AND join0.field_numeric < 2; +----------+ | count(*) | +----------+ | 44720 | +----------+ 1 row in set (0.05 sec) mysql> select count(*) from FIELD join0 where join0.field_numeric > 887 AND join0.field_numeric < 889; +----------+ | count(*) | +----------+ | 15248 | +----------+ 1 row in set (0.02 sec) mysql> select count(*) from FIELD join0 where join0.field_numeric > 1000 AND join0.field_numeric < 3000; +----------+ | count(*) | +----------+ | 210 | +----------+ 1 row in set (0.00 sec) mysql> select count(*) from FIELD join0 where join0.field_type=2 AND join0.field_numeric > 1 AND join0.field_numeric < 2; +----------+ | count(*) | +----------+ | 44720 | +----------+ 1 row in set (3.68 sec) mysql> select count(*) from FIELD join0 where join0.field_type=2 AND join0.field_numeric > 887 AND join0.field_numeric < 889; +----------+ | count(*) | +----------+ | 15248 | +----------+ 1 row in set (0.32 sec) mysql> select count(*) from FIELD join0 where join0.field_type=2 AND join0.field_numeric > 1000 AND join0.field_numeric < 3000; +----------+ | count(*) | +----------+ | 210 | +----------+ 1 row in set (0.02 sec)
[11 Feb 2009 16:16]
Valeriy Kravchuk
I had missed one more column. So, please, send also the results of: select count(*) from field join0 where join0.field_name=N'sectionA.number'; This looks like a bug, maybe similar to bug #32254, but for single-column key that allows range access.
[11 Feb 2009 17:46]
walid bakkar
mysql> select count(*) from FIELD join0 where join0.field_name=N'sectionA.number'; +----------+ | count(*) | +----------+ | 528144 | +----------+ 1 row in set (1.14 sec)
[11 Feb 2009 17:57]
walid bakkar
http://bugs.mysql.com/bug.php?id=32254 is about index merge being used unecessarily. this is not the only problem happening in this case, since in the previous exemple i have three plans: one with index_merge, one with a form_index and one using the correct field_numeric_index.
[11 Feb 2009 18:33]
Valeriy Kravchuk
Please, check if you will still get 3 different plans after converting the field table to MyISAM. Inform about the results. I need to know if the bug is engine-dependent.
[12 Feb 2009 13:13]
walid bakkar
those are the results after converting tables to myisam. we only get two plans in this case, but please note the time the explain took for each of these three queries. (same as before, same queries, different ranges of values) MyISAM ------ mysql> explain select d_docid from (select distinct d.docid as d_docid from (doc d left outer join fie join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1000) AND (join0.field_numeric < 3000)) ids order by d_docid ASC; +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ | 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 13 | Using filesort | | 2 | DERIVED | join0 | range | PRIMARY,docid_index,field_name_index,field_type_index,field_numeric_index | field_numeric_index | 9 | NULL | 1782 | Using where; Using temporary | | 2 | DERIVED | d | eq_ref | PRIMARY,form_index | PRIMARY | 4 | pbm.join0.docid | 1 | Using where | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+------+------------------------------+ 3 rows in set (0.01 sec) mysql> explain select d_docid from (select distinct d.docid as d_docid from (doc d left outer join fie join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 1) AND (join0.field_numeric < 2)) ids order by d_docid ASC; +----+-------------+------------+-------------+---------------------------------------------------------------------------+-----------------------------------+---------+-----------------+-------+-----------------------------------------------------------------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+-------------+---------------------------------------------------------------------------+-----------------------------------+---------+-----------------+-------+-----------------------------------------------------------------------------------------------+ | 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 3799 | Using filesort | | 2 | DERIVED | join0 | index_merge | PRIMARY,docid_index,field_name_index,field_type_index,field_numeric_index | field_type_index,field_name_index | 4,386 | NULL | 23406 | Using intersect(field_type_index,field_name_index); Using where; Using index; Using temporary | | 2 | DERIVED | d | eq_ref | PRIMARY,form_index | PRIMARY | 4 | pbm.join0.docid | 1 | Using where | +----+-------------+------------+-------------+---------------------------------------------------------------------------+-----------------------------------+---------+-----------------+-------+-----------------------------------------------------------------------------------------------+ 3 rows in set (2 min 16.26 sec) mysql> explain select d_docid from (select distinct d.docid as d_docid from (doc d left outer join fie join0 on d.docid=join0.docid and join0.field_name=N'sectionA.number' and join0.field_type=2) where (d.form = N'performanceBenchMarking/PBMComplexForm4') AND (join0.field_numeric > 887) AND (join0.field_numeric < 889)) ids order by d_docid ASC; +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+-------+------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+-------+------------------------------+ | 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 15248 | Using filesort | | 2 | DERIVED | join0 | range | PRIMARY,docid_index,field_name_index,field_type_index,field_numeric_index | field_numeric_index | 9 | NULL | 33381 | Using where; Using temporary | | 2 | DERIVED | d | eq_ref | PRIMARY,form_index | PRIMARY | 4 | pbm.join0.docid | 1 | Using where | +----+-------------+------------+--------+---------------------------------------------------------------------------+---------------------+---------+-----------------+-------+------------------------------+ 3 rows in set (5.65 sec)
[14 Nov 2009 16:33]
Valeriy Kravchuk
Please, try to repeat with a newer version, 5.1.40, and inform about the results.
[15 Dec 2009 0:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".