Bug #29921 MYSQLparse uses 3X more CPU in MySQL 5.0 than yyparse in MySQL 4.0
Submitted: 20 Jul 2007 3:39 Modified: 4 Mar 2008 22:29
Reporter: Mark Callaghan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Parser Severity:S5 (Performance)
Version:5.0.37 OS:Any
Assigned to: Marc ALFF CPU Architecture:Any
Tags: bfsm_2007_08_02, bfsm_2007_10_18, Contribution, parser, performance, regression

[20 Jul 2007 3:39] Mark Callaghan
Description:
I have a benchmark that consists of replaying mysqldump output against a database. When one database session is used to reload the database (no concurrent inserts), MySQL 5.0.37 takes ~50% more time than MySQL 4.0.26 (~17000 vs ~11000 seconds).

Some of the slowdown for MySQL 5 is due to support for row_format=compact in InnoDB. However, oprofile output shows MYSQLparse with ~20% of the time for MySQL 5 and yyparse with ~9% of the time for MySQL 4. Given that the MySQL 5 test takes 50% more time, MYSQLparse uses about 3x more CPU in MySQL 5 than yyparse in MySQL 4 for this benchmark.

One of the problems is that MySQL 5.0 uses utf8 internally. My clients use latin1 and the default charset for the db is latin1. Some time is spent converting and copying strings from latin1 to utf8. Some time is wasted in my_charset_same (there has to be a better way to check for charset equality than to compare strings). More time is probably used because the YACC grammar has grown. But this is a big regression.

How to repeat:
Dump and reload a database. I will rerun my tests using MyISAM tables to determine how much of the slowdown is from InnoDB. But the oprofile output I have shows that parsing is a big part of the problem.

Suggested fix:
Reduce the YACC grammar? Write a parser by hand? I don't have a good suggestion at this time, but I really want my performance to improve.

Updates with oprofile results to follow.
[20 Jul 2007 3:41] Mark Callaghan
This is oprofile output for the benchmark using InnoDB tables and MySQL 4.0.26. I have listed the top 30 functions:
samples  %        app name                 symbol name
20790098  9.6387  mysqld                   yyparse()
13978509  6.4807  mysqld                   yylex(void*)
12371259  5.7355  libc-2.3.5.so            memcpy
9134302   4.2348  mysqld                   cmp_dtuple_rec_with_match
6339029   2.9389  mysqld                   buf_calc_page_new_checksum
5467357   2.5348  mysqld                   log_group_write_buf
5280038   2.4479  mysqld                   btr_search_guess_on_hash
5228149   2.4239  mysqld                   mysql_parse(THD*, char*, unsigned int)
4697479   2.1778  mysqld                   page_cur_insert_rec_write_log
4529755   2.1001  mysqld                   Item_int::save_in_field(Field*, bool)
3836652   1.7787  libpthread-2.3.5.so      __pthread_mutex_unlock_usercnt
3082379   1.4290  mysqld                   setup_fields(THD*, st_table_list*, List<Item>&, bool, List<Item>*, bool)
2903332   1.3460  mysqld                   btr_cur_optimistic_insert
2814160   1.3047  libpthread-2.3.5.so      pthread_getspecific
2799271   1.2978  mysqld                   page_cur_search_with_match
2710013   1.2564  mysqld                   Item_string::save_in_field(Field*, bool)
2593889   1.2026  mysqld                   row_insert_for_mysql
2593100   1.2022  mysqld                   Item_string::~Item_string()
2545890   1.1803  mysqld                   page_cur_insert_rec_low
2434105   1.1285  mysqld                   page_rec_get_next
2414043   1.1192  mysqld                   Item_int::~Item_int()
2340320   1.0850  libc-2.3.5.so            __GI_____strtoll_l_internal
2238759   1.0379  libpthread-2.3.5.so      pthread_mutex_trylock
2137703   0.9911  mysqld                   btr_cur_search_to_nth_level
2092474   0.9701  mysqld                   rec_get_nth_field
2048178   0.9496  mysqld                   mtr_commit
2002929   0.9286  mysqld                   alloc_root
[20 Jul 2007 3:43] Mark Callaghan
These are the top 30 from MySQL 5.0.37 using InnoDB tables:
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        app name                 symbol name
67093324 19.4305  mysqld                   MYSQLparse(void*)
14889263  4.3120  mysqld                   rec_get_offsets_func
12838530  3.7181  libc-2.3.5.so            memcpy
12018824  3.4807  mysqld                   log_group_write_buf
9654171   2.7959  mysqld                   MYSQLlex(void*, void*)
7601190   2.2013  mysqld                   cmp_dtuple_rec_with_match
7159070   2.0733  libc-2.3.5.so            strstr
6870511   1.9897  libpthread-2.3.5.so      __pthread_mutex_unlock_usercnt
5872749   1.7008  mysqld                   buf_calc_page_new_checksum
5670346   1.6422  mysqld                   get_text(st_lex*)
5587272   1.6181  mysqld                   btr_search_guess_on_hash
5515860   1.5974  mysqld                   alloc_root
4594264   1.3305  mysqld                   Query_arena::free_items()
4551895   1.3182  mysqld                   Item_int::save_in_field(Field*, bool)
4479174   1.2972  mysqld                   copy_and_convert(char*, unsigned int, charset_info_st*, char const*, unsigned int, charset_info_st*, unsigned int*)
4186962   1.2126  mysqld                   page_cur_insert_rec_low
4151865   1.2024  mysqld                   Item_string::save_in_field(Field*, bool)
3901579   1.1299  libpthread-2.3.5.so      pthread_mutex_trylock
3822935   1.1071  mysqld                   Item::used_tables() const
3753735   1.0871  mysqld                   setup_fields(THD*, Item**, List<Item>&, bool, List<Item>*, bool)
3544822   1.0266  mysqld                   page_cur_search_with_match
3405702   0.9863  mysqld                   mem_area_free
3388987   0.9815  mysqld                   Item::check_cols(unsigned int)
3334453   0.9657  libpthread-2.3.5.so      pthread_mutex_unlock
3290290   0.9529  mysqld                   my_uni_utf8
3199040   0.9265  mysqld                   fill_record(THD*, Field**, List<Item>&, bool)
3169733   0.9180  mysqld                   buf_page_get_gen
[20 Jul 2007 3:44] Mark Callaghan
These are the top 30 from MySQL 5.0.37 with system_charset_info changed to use latin1 rather than utf8:
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000
samples  %        app name                 symbol name
40761850 19.4885  mysqld                   MYSQLparse(void*)
9352251   4.4714  mysqld                   rec_get_offsets_func
8789276   4.2022  libc-2.3.5.so            memcpy
8090572   3.8682  mysqld                   log_group_write_buf
5784592   2.7656  mysqld                   MYSQLlex(void*, void*)
4662155   2.2290  mysqld                   cmp_dtuple_rec_with_match
4392068   2.0999  libc-2.3.5.so            strstr
4300587   2.0561  mysqld                   get_text(st_lex*)
4191729   2.0041  libpthread-2.3.5.so      __pthread_mutex_unlock_usercnt
3746288   1.7911  mysqld                   buf_calc_page_new_checksum
3337162   1.5955  mysqld                   alloc_root
3009014   1.4386  mysqld                   btr_search_guess_on_hash
2867279   1.3709  mysqld                   Query_arena::free_items()
2716966   1.2990  libpthread-2.3.5.so      pthread_mutex_trylock
2614492   1.2500  mysqld                   page_cur_search_with_match
2557889   1.2229  mysqld                   Item_string::save_in_field(Field*, bool)
2511082   1.2006  mysqld                   page_cur_insert_rec_low
2503175   1.1968  mysqld                   Item_int::save_in_field(Field*, bool)
2385286   1.1404  mysqld                   Item::used_tables() const
2231738   1.0670  mysqld                   setup_fields(THD*, Item**, List<Item>&, bool, List<Item>*, bool)
2155671   1.0306  mysqld                   mem_area_free
2104924   1.0064  mysqld                   buf_page_get_gen
1927640   0.9216  mysqld                   Item::check_cols(unsigned int)
1890692   0.9040  mysqld                   fill_record(THD*, Field**, List<Item>&, bool)
1725761   0.8251  libpthread-2.3.5.so      pthread_getspecific
1715146   0.8200  mysqld                   page_cur_insert_rec_write_log
1709014   0.8171  mysqld                   btr_cur_search_to_nth_level
[20 Jul 2007 14:09] Mark Callaghan
The test was 4% faster after I changed system_charset_info to use latin1 (the default database charset) rather than utf8. That is only a small part of the problem.
[20 Jul 2007 14:50] MySQL Verification Team
See bug: http://bugs.mysql.com/bug.php?id=11605
[20 Jul 2007 23:23] Mark Callaghan
I reran the database reload tests using MyISAM tables. The slowdown is bad. The table below lists the time to finish loading data that uses ~100GB for InnoDB tables. Tests were run for 1, 2, 4, 8 and 16 concurrent sessions inserting into ~400 tables. Sessions did not insert into the same tables and all table types were MyISAM. The server had 4 CPU cores and 12 disks with SW RAID 0. Times are in seconds. Clients and server are on the same host.

               number of concurrent insert sessions
version   1      2       4        8     16
4.0.26   8831   4631    3007    3082    3174
5.0.37  14008   7373    4957    5582    5408

This regression is worse than 50%.
[21 Jul 2007 4:36] Mark Callaghan
This is a simple test case. MySQL 5.0.37 does this in 48 seconds. MySQL 4.0.26 does this in 31 seconds. This is on a 2.4GHz Opteron.

drop table if exists t1;
create table t1 (i int) type = myisam;
insert into t1 values (1);

update t1 set i = 2;
<repeat previous line 1M times>

------------

Another simple test takes 30 seconds for MySQL 4 and 44 seconds for MySQL 5
drop table if exists i;
create table i (i int) type = myisam;

insert into i values (1);
<repeat 1M times>
[21 Jul 2007 18:15] Mark Callaghan
I tried removing many of the new features from the parser for 5.0. This includes: triggers, stored procedures and views. The parser was not any faster with support for those features removed.

I then used rdtsc to determine what part of MYSQLparse/yyparse was slower in 5.0. The biggest slowdown is parsing the VALUES list for the insert statment. For each entry in the VALUES list in the 'insert into Foo values (....)' statement, the expr token uses ~5000 cycles in MySQL 4 and ~7500 cycles in MySQL 5. This is the primary factor in the 50% slowdown in MySQL 5

The expr nonterminal in MySQL 4 is simple:
expr:   expr_expr       {$$ = $1; }
        | simple_expr   {$$ = $1; };

The expr nonterminal in MySQL 5 is complex. Notice all of the memory alloc/dealloc operations and other work that includes in the common case: new List, push_front call, pop call, delete list.

expr:
          bool_term { Select->expr_list.push_front(new List<Item>); }
          bool_or_expr
          {
            List<Item> *list= Select->expr_list.pop();
            if (list->elements)
            {
              list->push_front($1);
              $$= new Item_cond_or(*list);
              /* optimize construction of logical OR to reduce
                 amount of objects for complex expressions */
            }
            else
              $$= $1;
            delete list;
          }
        ;
[21 Jul 2007 19:52] Mark Callaghan
I changed the values terminal in sql_yacc.yy to use simple_expr and avoid the overhead of expr mentioned previously. This was done to determine whether the overhead of expr is what makes MySQL 5 slower. The new definition of values is:

values:
  values ','  simple_expr
  {
    if (Lex->insert_list->push_back($3))
      YYABORT;
  }
  | simple_expr
    {
      if (Lex->insert_list->push_back($1))
        YYABORT;
    }
  ;

I then ran this SQL to compare MySQL 4 with MySQL 5 modifed as described above:
drop table if exists i;
create table i (i int) type = myisam;

insert into i values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10),
   (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
<repeat insert statement 1M times>

The time to run this was:
 * MySQL 4 -> 32 seconds
 * MySQL 5 original -> 39 seconds
 * MySQL 5 modified -> 32 seconds

So expr in sql_yacc.yy for MySQL 5 is a big part of the regression.
[21 Jul 2007 20:01] Mark Callaghan
The bool_term nonterminal has the same problem as the expr terminal. Both allocate a List, push it onto another data structure, pop it, and then deallocate the list. Unfortunately, all of this must be done to parse a literal. This is very expensive.
[21 Jul 2007 20:32] MySQL Verification Team
Thank you for the bug report.
[21 Jul 2007 23:59] Mark Callaghan
In MySQL 5.0.37, I changed the definition of the values nonterminal in sql_yacc.yy to use simple_expr rather than expr_or_default. This is incorrect, but allows me to avoid the overhead in expr and bool_term. I reran the database reload tests. The change makes performance for MySQL 5.0 much better for the tests with low concurrency (1 or 2 sessions). Performance improvement for 4 or more concurrent sessions does not improve as much. I think this is because I have saturated the CPU on the server -- it has 4 CPU cores.

'5.0.37 hack' has the change to sql_yacc.yy. This lists the number of seconds to reload about 100GB of data (when measured in InnoDB datafiles) into MyISAM tables.
               number of concurrent insert sessions
version        1      2       4        8     16
4.0.26        8831   4631    3007    3082    3174
5.0.37 orig  14008   7373    4957    5582    5408
5.0.37 hack  10760   5917    4699    5513    5234
[22 Jul 2007 8:24] Konstantin Osipov
Mark, this is an excellent analysis.
We've had observed the regression internally but never got a change to analyze the cause.
[22 Jul 2007 16:57] Mark Callaghan
Thanks. My hope for an easy fix might not come true. I changed the grammar to not optimize for expressions with many AND and OR terms and to use the 2 arg constructor for Item_cond_or and Item_cond_and. This change removed the extra new/delete calls. But the performance after this change was not that much better. The grammar for expressions is very different in MySQL 5 than in MySQL 4. There is much more nesting to reach simple_expr. Perhaps changing the grammar to be more similar to MySQL 4 will get back the performance.
[3 Aug 2007 23:29] Marc ALFF
Hi Mark

Many thanks for your report and analysis.

I suspect that many different factors have an effect on the performance
degradation, so to address this in general, I would like to propose to:
- keep bug#29921 open, and use it to collect observations / comments
- fix each issue found independently, with a dedicated bug report.
This would make the fix/review/merge process much easier

Below are some notes of various aspects identified so far, that I plan
to work on.

Regards,
-- Marc

ANALYSIS / things to investigate

Grammar complexity

The number of rules / terminals / non terminals, etc has increased between
4.1 and 5.0. See related comments in bug#11605.

A direct effect of this is that more reduces are needed to parse the same
construct, causing bison to execute more loops internally.

Impact: to assess, I assume it's very low.
Possible mitigation: Flatten the rules, to avoid unnecessary R1->R2->R3
reduces, where R1->R3 can be done instead.
Risk: Make sure operators priority are not affected when flattening the
rules.

Actions complexity

The issue with the lists used in the AND and OR boolean expressions falls
into this category. This specific problem will be fixed with Bug#30237,
but other similar issues could also exist.
Action: code review of actions in general
Impact: localized changes expected, low

Memory allocation

There is a difference between :
- new Item()
- new (THD->mem_root) Item()
that could have an impact here,
since the later does not need to get to thread local storage again and
again.
Action: code cleanup, mostly.
Performance impact to assess

Impact of new features

In general, constructs like VIEW, TRIGGER, PROCEDURE, FUNCTION etc
do affect a lot the grammar with new statements, but should not affect
existing statements that do not use the feature.
Stored procedures have an effect of changing THD::mem_root during parsing,
but since this code is not involved when parsing regular statements,
there should be no impact at all.
To investigate: the only known side effect of having new statements
supported but unused in the parser is the total size of the parser state
automaton.
There is a concern that at some point increasing this size could
cause caching faults, but it's not verified: it's only an hypothesis.

Impact of grammar ambiguities

Same as above, focused at the following rules:
- keyword
- keyword_sp
Early experiment showed that the parser size is greatly reduced (divided by
a factor 3) by removing these rules. The idea is that not having ambiguities
with keywords like this should help the parser to find statements faster.
Action: assess the real effect on execution time, instead of code size.

Parser stack

Some rules do use a stack maintained by the mysql code, independently
of the stack maintained by bison.
See for example the rules:
- udf_expr_list
- expr_list
- ident_list
which uses Select->expr_list.
Action: investigate, assess performance impact compared to using $$

Expressions

Because expressions are used in many places, optimizing parsing of
expressions (compared to say DDL statements like CREATE PROCEDURE) should
have the most positive impact on overall performances.
This area will be investigated first.

MYSQLlex

Investigate if the lexer itself is involved in the regression.
Action: implement the suggestion about comparing charsets by number,
not by name
[9 Aug 2007 14:48] Marc ALFF
See related Bug#30237, Bug#30333
[13 Aug 2007 12:37] Marko Mäkelä
See also Bug #11604 and Bug #11605. (Sorry for the previous entry, which contained wrong bug numbers.)
[13 Aug 2007 19:04] Mark Callaghan
I will attach changes to sql_yacc.yy that removed about half of the performance regression from MySQL 4.0. The numbers below are from a benchmark that reports the number of seconds to reload all tables of a large (100GB as InnoDB) database.

Values for the 'binary' column:
4_64 - 64-bit build of MySQL 4.0.26
5_orig - 64-bit build of MySQL 5.0.37 with parser performance problem
5_fix - 64-bit build of MySQL 5.0.37 with faster grammar

DOP=x lists the number of concurrent sessions. The regression is worst for DOP=1.

First, numbers for MyISAM tables. 
For DOP=1, MySQL5 is 58% slower without the fix and 29% slower with it.

binary  DOP=1   DOP=2   DOP=4   DOP=8   DOP=16
4_64    10061    5729    3436   3543    3557
5_fix   12995    6995    5369   6378    6493
5_orig  15808    8232    5622   6608    6849

Numbers for InnoDB tables.
For DOP=1, MySQL5 is 52% slower without the fix and 30% slower with it. 

binary  DOP=1   DOP=2   DOP=4   DOP=8   DOP=16
4_64    12741    8489    7160    7825    8018
5_fix   16532   10515    8508    9726    9282
5_orig  19315   11744    9368    9859    9401
[13 Aug 2007 19:05] Mark Callaghan
Patch for changes to sql_yacc.yy in MySQL 5.0.37

Attachment: yacc.c (text/x-csrc), 16.88 KiB.

[17 Aug 2007 18:03] Marc ALFF
Analysis, grammar complexity

Release 4.1:

Query        | States | Reduces |
Select 1     |     43 |      33 |
Select 1,2   |     55 |      43 |
Select 1,2,3 |     67 |      53 |
Increment    |    +12 |     +10 |

'State' counts the number of state transitions executed while parsing the
query.
'Reduces' counts the number of rule reductions executed.
'Increment' is the additional cost of having 1 expression

Reduces for an expression:
NUM_literal: NUM
literal: NUM_literal
simple_expr: literal
expr: simple_expr
--> 4 reduces

Release 5.0 (5.0-runtime)

Query        | States | Reduces |
Select 1     |     60 |      50 |
Select 1,2   |     87 |      75 |
Select 1,2,3 |    114 |     100 |
Increment    |    +27 |     +25 |

Reduces for an expression:
NUM_literal: NUM
literal: NUM_literal
simple_expr: literal
factor: simple_expr
term: factor
value_expr: term
bit_factor: value_expr
bit_term: bit_factor
bit_expr: bit_term
predicate: bit_expr
bool_pri: predicate
bool_test: bool_pri
bool_factor: bool_test
(reduce action @71): { Select->expr_list.push_front(new List<Item>); }
bool_and_expr: /* empty */
bool_term: bool_factor bool_and_expr
(reduce action @70): { Select->expr_list.push_front(new List<Item>); }
bool_or_expr: /* empty */
expr: bool_term bool_or_expr
--> 19 reduces

Release 5.0 + bug#30237 (5.0-perf)

Query        | States | Reduces |
Select 1     |     57 |      47 |
Select 1,2   |     81 |      69 |
Select 1,2,3 |    105 |      91 |
Increment    |    +24 |     +22 |

Reduces for an expression:
NUM_literal: NUM
literal: NUM_literal
simple_expr: literal
factor: simple_expr
term: factor
value_expr: term
bit_factor: value_expr
bit_term: bit_factor
bit_expr: bit_term
predicate: bit_expr
bool_pri: predicate
bool_test: bool_pri
bool_factor: bool_test
bool_term: bool_factor
bool_xterm: bool_term
expr: bool_xterm
--> 16 reduces

Release 5.1 (5.1-runtime)

Query        | States | Reduces |
Select 1     |     60 |      50 |
Select 1,2   |     87 |      75 |
Select 1,2,3 |    114 |     100 |
Increment    |    +27 |     +25 |

Reduces for an expression:
Same as 5.0

----------

The regression in general is caused in part by the number of reduces needed
to parse an expression, which increased from 5 to 19 between 4.1 and 5.0.
This cause the bison generated code to loop 19 times instead of 4 in this
case, and cause the differences in MYSQLparse() seen with oprofile/gprof
[24 Aug 2007 15:33] Marc ALFF
See related bug#30625
[25 Aug 2007 1:13] Marc ALFF
Performance update

Test case:
1,000,000 inserts of 100 values into a blackhole table.

This test forces the evaluation of 100 Milions expressions in the parser,
and is designed to expose the parser cost, by removing the cost of
the runtime execution of the query (blackhole).

All the tests below have been executed on the same hardware,
a single CPU amd64 box running Linux.

MySQL 4.1.24, gprof

Total time: 6m6.915s (367s)
time in yyparse(): 48.48s

MySQL 5.0.48, gprof

Total time: 10m6.592s (607s)
time in MYSQLparse(): 125.13s

Regression in total time: 240s
Regression in MYSQLparse(): 76.65s

MySQL 5.0.48 + Bug#30237, gprof

Total time: 8m12.277s (492s)
time in MYSQLparse(): 97.10s

Regression in total time: 125s
Regression in MYSQLparse(): 48.62s

Based on the total time,
the fix for Bug#30237 reclaimed 115 seconds from 240, or around 50 percent.

Please keep in mind that:
a) while this number is encouraging, it needs to be validated against
 the original use case, as a verification.
b) there is other work in progress (bug#30625) that also improve performances,
 so there is more to gain.

As a preview, prototyping of bug#30625 gives the following results:

MySQL 5.0.48 + Bug#30237 + Bug#30625, gprof

Total time: 7m31.371 (451s)
time in MYSQLparse(): 64.15s

This represent a regression of 84 seconds instead of 240,
or a reclaim of 65 percents (to confirm).

The patch for Bug#30625 is not available yet, it's to be posted soon.
The patch for Bug#30237 is already available.

Regards,
-- Marc
[28 Aug 2007 16:24] Marc ALFF
See related bug#30625, for which a performance improvement patch is available.
[11 Sep 2007 0:07] Marc ALFF
GPROF profiling for 4.1

Attachment: perf-4.1.txt.gz (application/x-gzip, text), 127.24 KiB.

[11 Sep 2007 0:08] Marc ALFF
GPROF profiling for 5.0.45

Attachment: perf-5.0.45.txt.gz (application/x-gzip, text), 144.20 KiB.

[11 Sep 2007 0:08] Marc ALFF
GPROF profiling for 5.0.50

Attachment: perf-5.0.txt.gz (application/x-gzip, text), 144.70 KiB.

[11 Sep 2007 0:10] Marc ALFF
time with gprof, 4.1

Attachment: time-4.1-gprof.log (text/x-log), 1.23 KiB.

[11 Sep 2007 0:11] Marc ALFF
time with gprof, 5.0.45

Attachment: time-5.0.45-gprof.log (text/x-log), 1.38 KiB.

[11 Sep 2007 0:12] Marc ALFF
time with gprof, 5.0.50

Attachment: time-5.0-gprof.log (text/x-log), 1.35 KiB.

[11 Sep 2007 0:14] Marc ALFF
time results, 4.1

Attachment: time-4.1.log (text/x-log), 1.28 KiB.

[11 Sep 2007 0:14] Marc ALFF
time results, 5.0.45

Attachment: time-5.0.45.log (text/x-log), 1.43 KiB.

[11 Sep 2007 0:15] Marc ALFF
time results, 5.0.50

Attachment: time-5.0.log (text/x-log), 1.43 KiB.

[11 Sep 2007 0:26] Marc ALFF
test

1,000,000 inserts of 100 values in a blackhole table,
all performed on the same hardware (dual core pentium64)

Timing under gprof

4.1.24:  6m46 (406 s)
5.0.45: 10m23 (623 s)
5.0.50:  8m50 (530 s)

Time spent in MYSQLparse() / yyparse()

4.1.24: 25.21 s
5.0.45: 68.82 s
5.0.50: 45.01 s

Timing under a regular build (no profiling)

4.1.24: 5m58 (358 s)
5.0.45: 7m25 (445 s)
5.0.50: 5m48 (348 s)
[11 Sep 2007 0:47] Marc ALFF
Hi Mark

Please see the new performance numbers (previous comments in the bug report),
obtained in a test environment (focused on stressing the parser).

Could you try MySQL 5.0.50 (from the BK repository) or later,
with the load data benchmark that was used when reporting the bug,
and inform us of the changes in performances ?

Thanks a lot for your feedback.

Regards,
-- Marc
[17 Sep 2007 19:05] Mark Callaghan
I will do this soon. I was distracted by http://bugs.mysql.com/bug.php?id=30738
[27 Sep 2007 17:35] Mark Callaghan
Can you provide a patch file with only your changes? I need to push this into 5.0.37 for testing and the diff of sql_yacc.yy between 5.0.37 and 5.0.50 (from bk) has too many changes.
[5 Oct 2007 22:04] Marc ALFF
See related bug#17229, which is another symptom of the same issue.
[4 Mar 2008 21:52] Marc ALFF
See related blog:

http://mysqlha.blogspot.com/2008/02/is-mysql-getting-faster.html
[4 Mar 2008 22:29] Marc ALFF
Several performance related issues have been fixed in the parser:
- Bug#30237 (Performance regression in boolean expressions)
- Bug#30333 (Performance, expressions lists in the parser)
- Bug#30625 (Performance, reduce depth for expressions)

The performance degradation reported has been addressed,
so this bug is now closed.

Feel free to re-open this bug report, with new performance measures,
if further improvement is needed.