Bug #79681 Queries running much slower in version 5.7 versus 5.6
Submitted: 16 Dec 2015 21:53 Modified: 17 Dec 2015 12:12
Reporter: Mike Savoie Email Updates:
Status: Verified Impact on me:
Category:MySQL Server: Optimizer Severity:S5 (Performance)
Version:5.7, 5.7.10 OS:Red Hat (Redhat 6 update 5)
Assigned to: CPU Architecture:Any

[16 Dec 2015 21:53] Mike Savoie
The same queries run against MySQL 5.7.9 server is taking 15 times longer than when it is run against MySQL 5.6 server.

How to repeat:
Run query below against MySQL 5.6 and MySQL 5.7. The elapsed time(seconds) was 9.641 against MySQL 5.6. The elapsed time(seconds) against MySQL 5.7 was 151.757

select `coguda00`.`PRODUCT_BRAND_KEY` as "C0" , min(`coguda01`.`PRODUCT_BRAND_EN`) as "C1" , sum(`SLS_SALES_FACT`.`SALE_TOTAL`) as "C2"  from `gosalesdw_1021`.`SLS_PRODUCT_DIM` `coguda00`, `gosalesdw_1021`.`SLS_PRODUCT_BRAND_LOOKUP` `coguda01`, `gosalesdw_1021`.`SLS_PRODUCT_LINE_LOOKUP` `coguda02`, `gosalesdw_1021`.`SLS_PRODUCT_LOOKUP` `coguda03`, `gosalesdw_1021`.`SLS_PRODUCT_SIZE_LOOKUP` `coguda04`, `gosalesdw_1021`.`SLS_PRODUCT_TYPE_LOOKUP` `coguda05`, `gosalesdw_1021`.`SLS_PRODUCT_COLOR_LOOKUP` `coguda06`, `gosalesdw_1021`.`SLS_SALES_FACT` `SLS_SALES_FACT`, (select `SLS_RETAILER_DIM`.`RETAILER_TYPE_CODE` as "Retailer_type_code" , `SLS_RETAILER_DIM`.`RETAILER_KEY` as "Retailer_key"  from `gosalesdw_1021`.`SLS_RTL_DIM` `SLS_RETAILER_DIM` where `SLS_RETAILER_DIM`.`RETAILER_TYPE_CODE` = 7 group by `SLS_RETAILER_DIM`.`RETAILER_TYPE_CODE`, `SLS_RETAILER_DIM`.`RETAILER_KEY`) `Retailer`, `gosalesdw_1021`.`GO_REGION_DIM` `coguda30`, `gosalesdw_1021`.`SLS_RTL_DIM` `coguda31`, `gosalesdw_1021`.`SLS_PRODUCT_DIM` `coguda40`, `gosalesdw_1021`.`SLS_PRODUCT_LINE_LOOKUP` `coguda41`, `gosalesdw_1021`.`SLS_PRODUCT_TYPE_LOOKUP` `coguda42`, `gosalesdw_1021`.`SLS_PRODUCT_LOOKUP` `coguda43`, `gosalesdw_1021`.`SLS_PRODUCT_COLOR_LOOKUP` `coguda44`, `gosalesdw_1021`.`SLS_PRODUCT_SIZE_LOOKUP` `coguda45`, `gosalesdw_1021`.`SLS_PRODUCT_BRAND_LOOKUP` `coguda46`, `gosalesdw_1021`.`GO_TIME_DIM` `GO_TIME_DIM`, `gosalesdw_1021`.`EMP_EMPLOYEE_DIM` `EMP_EMPLOYEE_DIM`, `gosalesdw_1021`.`GO_BRANCH_DIM` `GO_BRANCH_DIM` where `coguda00`.`PRODUCT_KEY` = `SLS_SALES_FACT`.`PRODUCT_KEY` and `coguda31`.`RETAILER_SITE_KEY` = `SLS_SALES_FACT`.`RETAILER_SITE_KEY` and `Retailer`.`Retailer_key` = `coguda31`.`RETAILER_KEY` and `coguda40`.`PRODUCT_KEY` = `SLS_SALES_FACT`.`PRODUCT_KEY` and `SLS_SALES_FACT`.`ORDER_DAY_KEY` = `GO_TIME_DIM`.`DAY_KEY` and `SLS_SALES_FACT`.`EMPLOYEE_KEY` = `EMP_EMPLOYEE_DIM`.`EMPLOYEE_KEY` and `EMP_EMPLOYEE_DIM`.`BRANCH_CODE` = `GO_BRANCH_DIM`.`BRANCH_CODE` and `GO_BRANCH_DIM`.`BRANCH_KEY` = 9929 and cast(`GO_TIME_DIM`.`CURRENT_YEAR` as char( 4 )) = '2011' and `coguda40`.`PRODUCT_LINE_CODE` = 993 and `coguda40`.`PRODUCT_TYPE_KEY` = 961 and `coguda40`.`PRODUCT_LINE_CODE` = `coguda41`.`PRODUCT_LINE_CODE` and `coguda40`.`PRODUCT_NUMBER` = `coguda43`.`PRODUCT_NUMBER` and `coguda40`.`PRODUCT_SIZE_CODE` = `coguda45`.`PRODUCT_SIZE_CODE` and `coguda40`.`PRODUCT_TYPE_CODE` = `coguda42`.`PRODUCT_TYPE_CODE` and `coguda40`.`PRODUCT_COLOR_CODE` = `coguda44`.`PRODUCT_COLOR_CODE` and `coguda46`.`PRODUCT_BRAND_CODE` = `coguda40`.`PRODUCT_BRAND_CODE` and `coguda43`.`PRODUCT_LANGUAGE` = 'EN' and `coguda30`.`REGION_CODE` = 740 and `coguda30`.`COUNTRY_CODE` = `coguda31`.`RTL_COUNTRY_CODE` and `coguda00`.`PRODUCT_LINE_CODE` = `coguda02`.`PRODUCT_LINE_CODE` and `coguda00`.`PRODUCT_NUMBER` = `coguda03`.`PRODUCT_NUMBER` and `coguda00`.`PRODUCT_SIZE_CODE` = `coguda04`.`PRODUCT_SIZE_CODE` and `coguda00`.`PRODUCT_TYPE_CODE` = `coguda05`.`PRODUCT_TYPE_CODE` and `coguda00`.`PRODUCT_COLOR_CODE` = `coguda06`.`PRODUCT_COLOR_CODE` and `coguda01`.`PRODUCT_BRAND_CODE` = `coguda00`.`PRODUCT_BRAND_CODE` and `coguda03`.`PRODUCT_LANGUAGE` = 'EN' group by `coguda00`.`PRODUCT_BRAND_KEY` order by `C1` asc;
[16 Dec 2015 22:56] Mike Savoie
I have added the file mysql-bug-data-79681.rar (WinRAR tool was used to zip). It contains complete dump of the gosalesdw_1021 mysql schema and a file which contains 5 SQL statements which are much slower in 5.7.9 than it was in 5.6
[17 Dec 2015 12:12] MySQL Verification Team
Hello Mike,

Thank you for the report and test case.
Observed the issue(for 2 queries it was 9-10 times slow) with 5.7.10 when compared to 5.6.28.

[12 Jan 2016 15:28] Øystein Grøvlen
The reason for the regression is that MySQL 5.7 tries to estimate the filtering effect of conditions that are not used by indexes.  For columns that are not indexed, these estimates are not very accurate, and in this particular case the filtering effect is over-estimated.

  set optimizer_switch='condition_fanout_filter=off';

Another alternative is to add some indexes.  I was able to improve the performance of Q1 in 5.7 to the level of 5.6 by adding the following two indexes:
  sls_product_dim(product_line_code, product_type_key);

One comment to the queries:  I do not understand why there is two instances of sls_product_dim in the join queries (coduga00 and coduga20/coduga40).  Through their join conditions with sls_sales_fact, matching rows from the two instances will have the same product_key.  Since product_key are PRIMARY KEY in sls_product_dim, rows are joined with themselves.  This self-join does not add anything to the result except execution overhead.
[13 Jan 2016 10:41] Øystein Grøvlen
Looking at the queries, I think a lot can be gained from eliminating tables that does not affect the query result.  AFAICT, the following query should be equivalent to Q1:

select `coguda00`.`product_brand_key` as "c0" ,
       min(`coguda01`.`product_brand_en`) as "c1" ,
       sum(`sls_sales_fact`.`sale_total`) as "c2" 
  `gosalesdw_1021`.`sls_product_dim` `coguda00`,
  `gosalesdw_1021`.`sls_product_brand_lookup` `coguda01`,
  `gosalesdw_1021`.`sls_product_lookup` `coguda03`,
  `gosalesdw_1021`.`sls_sales_fact` `sls_sales_fact`,
  `gosalesdw_1021`.`go_region_dim` `coguda30`,
  `gosalesdw_1021`.`sls_rtl_dim` `coguda31`,
  `gosalesdw_1021`.`go_time_dim` `go_time_dim`,
  `gosalesdw_1021`.`emp_employee_dim` `emp_employee_dim`,
  `gosalesdw_1021`.`go_branch_dim` `go_branch_dim` 
  `coguda00`.`product_key` = `sls_sales_fact`.`product_key` and
  `coguda31`.`retailer_site_key` = `sls_sales_fact`.`retailer_site_key` and
  `coguda31`.`retailer_key` in 
    (select retailer_key from `gosalesdw_1021`.`sls_rtl_dim` `sls_retailer_dim` 
     where `sls_retailer_dim`.`retailer_type_code` = 7) and
  `sls_sales_fact`.`order_day_key` = `go_time_dim`.`day_key` and
  `sls_sales_fact`.`employee_key` = `emp_employee_dim`.`employee_key` and
  `emp_employee_dim`.`branch_code` = `go_branch_dim`.`branch_code` and
  `go_branch_dim`.`branch_key` = 9929 and
  `go_time_dim`.`current_year` = 2011 and 
  `coguda00`.`product_line_code` = 993 and 
  `coguda00`.`product_type_key` = 961 and
  `coguda30`.`region_code` = 740 and 
  `coguda30`.`country_code` = `coguda31`.`rtl_country_code` and 
  `coguda00`.`product_number` = `coguda03`.`product_number` and 
  `coguda01`.`product_brand_code` = `coguda00`.`product_brand_code` and 
  `coguda03`.`product_language` = 'en'
group by `coguda00`.`product_brand_key` order by `c1` asc;

On MySQL 5.7.9, the above query execute in 0.5% of the time used by the original query!