Bug #56528 Increased server footprint with MTR
Submitted: 3 Sep 2010 7:31 Modified: 4 Nov 2010 1:12
Reporter: Evgeny Potemkin Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Tests Severity:S7 (Test Cases)
Version:recent 5.5, trunk, next-mr OS:Any
Assigned to: Marc ALFF CPU Architecture:Any

[3 Sep 2010 7:31] Evgeny Potemkin
Description:
After fixing bug#55576 mysql server memory footprint has increased from ~70M to ~325M in the latest 5.5-bugfixing tree. It's more than 4 times.
Before the fix my hardware was able to run mtr tests in 20 threads. Now even 10 threads are causing excessive swap because now 10 threads occupy more that 3G of memory instead of 700M.
One could imagine the impact on hosting/VDS/VPS providers.

How to repeat:
Compile recent 5.5-bugfixing tree and 5.5-bugfixing tree before revid:marc.alff@oracle.com-20100825002143-nci9ruqp4xq9yeno.
Run mtr tests, run `top` in parallel, check RSS/RES column.

Suggested fix:
Design a better fix for the bug#55576 to avoid 4+ times increase of memory footprint.
[3 Sep 2010 16:47] Marc ALFF
To clarify,

the default server memory footprint has _not_ changed,
because the performance schema is disabled by default,
so this will not affect people in production that takes a binary
with all the default values for the performance schema configuration parameters.

What did change, is that MTR, because the performance schema is enabled in the
MTR test suite .cnf files, now need more memory for the server.

Changing category as tests.

The fix to limit impact on the test suite itself, where the server is run
with typically low settings, is to also provide explicitly lower settings
for the number of mutexes / rwlock instances to use.
[8 Sep 2010 15:58] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/117778

3204 Evgeny Potemkin	2010-09-08
      Bug#56528 Increased server footprint with PFS.
      Fix for the bug 55576 increased number of rwlocks and mutexes instances
      allocated by default to suit needs of performance schema instrumentation for
      some tests with InnoDB. This led to increased memory footprint by more than 4
      times and most of those instances were unused in most cases.
      
      This fix reverts the increased number of default instances and sets appropriate
      number of instances only for tests that need it.
     @ mysql-test/suite/perfschema/r/start_server_no_cond_class.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_cond_inst.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_file_class.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_file_inst.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_mutex_class.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_mutex_inst.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_rwlock_class.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_rwlock_inst.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_thread_class.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_no_thread_inst.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/r/start_server_off.result
        Adjusted test case result after fixing bug#56528.
     @ mysql-test/suite/perfschema/t/server_init-master.opt
        Bug#56528 Increased server footprint with PFS.
        Allocate more rwlocks and mutexes instances.
     @ mysql-test/suite/perfschema/t/start_server_on-master.opt
        Bug#56528 Increased server footprint with PFS.
        Allocate more rwlocks and mutexes instances.
     @ storage/perfschema/pfs_server.h
        Bug#56528 Increased server footprint with PFS.
        Reverted to old default number of rwlocks and mutexes instances.
[8 Sep 2010 18:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/117786

3203 Marc Alff	2010-09-08
      Bug#56528 Increased server footprint with MTR
      
      With recent changes in the performance schema default sizing parameters,
      the memory used by a mysqld binary increased accordingly.
      
      This negatively affects the MTR test suite,
      because running several tests in parallel now consumes more ressources.
      
      The fix is to leave the default production values unchanged,
      and to configure the MTR environment to limit memory
      used when running tests in the test suite, which is ok
      because only a few objects are typically used within a test script.
      
      This fix:
      - changed the default configuration in MTR to use less memory
      - adjusted the performance schema tests accordingly
      
      Note that 1,000 mutex instances was too short and caused test failures
      in the past in team trees, so the default used is now 10,000 in MTR.
      
      The amount of memory used by the performance schema itself
      can be observed with the statement SHOW ENGINE PERFORMANCE_SCHEMA STATUS
[9 Sep 2010 16:27] Marc ALFF
Pushed into:
- mysql-5.5-bugfixing
- mysql-trunk-bugfixing
- mysql-next-mr-bugfixing
[2 Oct 2010 18:13] Bugs System
Pushed into mysql-trunk 5.6.1-m4 (revid:alexander.nozdrin@oracle.com-20101002180948-852x1cuv7c6i85ea) (version source revid:alexander.nozdrin@oracle.com-20101002180857-an32jpuwzemsp4f2) (merge vers: 5.6.1-m4) (pib:21)
[2 Oct 2010 18:14] Bugs System
Pushed into mysql-next-mr (revid:alexander.nozdrin@oracle.com-20101002181053-6iotvl26uurcoryp) (version source revid:alexander.nozdrin@oracle.com-20101002180917-h0n62akupm3z20nt) (pib:21)
[2 Oct 2010 18:16] Bugs System
Pushed into mysql-5.5 5.5.7-rc (revid:alexander.nozdrin@oracle.com-20101002180831-590ka2tuit9qoxbb) (version source revid:alexander.nozdrin@oracle.com-20101002180831-590ka2tuit9qoxbb) (merge vers: 5.5.7-rc) (pib:21)
[4 Nov 2010 1:12] Paul DuBois
Added to
http://dev.mysql.com/doc/refman/5.5/en/performance-schema-runtime-configuration.html:

With Performance Schema enabled, the number of Performance Schema
instances affects the server memory footprint, perhaps to a large
extent.  It may be necessary to tune the values of Performance
Schema system variables to find the number of instances that balances
insufficient instrumentation against excessive memory consumption.