Bug #47243 Allow MTR to read max auto setting for --parallel from env. variable
Submitted: 10 Sep 2009 10:22 Modified: 18 Dec 2009 23:51
Reporter: Bjørn Munch Email Updates:
Status: Closed Impact on me:
None 
Category:Tools: MTR / mysql-test-run Severity:S3 (Non-critical)
Version:5.1,5.4 OS:Any
Assigned to: Bjørn Munch CPU Architecture:Any

[10 Sep 2009 10:22] Bjørn Munch 
Description:
MTR's method to determine parallel level when using --parallel=auto is far from perfect, in particular it may select a too high number, possibly causing tests to fail due to timeouts or resource problems.

This could be helped if it was possible to "hardcode" the auto level per host, thus overriding MTR's algorithm. This could be done after some manual testing to find a proper number.

How to repeat:
We see in PB2 that tests on Solaris, on hosts where parallel is set to 8, that tests often fail with timout or out of memory. At least the latter is clearly related to the number of parallel test threads.

Suggested fix:
Let MTR check the existence of some /etc file, and if it exists, use the number there and skip the code that calculates this from system info.

There is no need to implement something for Windows (currently), because auto is set to 1 on that platform anyway.
[1 Oct 2009 20:55] Bjørn Munch 
It's easier to have this set with an envorinment variable, new patch coming.
[1 Oct 2009 20:58] Bjørn Munch 
It's easier to have this set with an envorinment variable, new patch coming.
[1 Oct 2009 21:15] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85443

2837 Bjorn Munch	2009-10-01
      Bug #47243 Allow MTR to read max auto setting for --parallel from env. variable
      In stead of hard-coded 8, read from $MTR_MAX_PARALLEL if set
[2 Oct 2009 11:14] Jonathan Perkin 
Looks good.
[5 Oct 2009 11:08] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85732

2838 Bjorn Munch	2009-10-05
      Bug #47243 Allow MTR to read max auto setting for --parallel from env. variable
      In stead of hard-coded 8, read from $MTR_MAX_PARALLEL if set
[5 Oct 2009 14:35] Bjørn Munch 
Pushed to 5.1-mtr, trunk-mtr, next-mr-mtr and 6.0-codebase-mtr
[22 Oct 2009 20:17] Bugs System 
Pushed into 6.0.14-alpha (revid:alik@sun.com-20091022201524-0efl2fzebfuuf0vk) (version source revid:bjorn.munch@sun.com-20091005111718-rd5rr1ufarj4cr3w) (merge vers: 6.0.14-alpha) (pib:13)
[22 Oct 2009 20:18] Bugs System 
Pushed into 5.5.0-beta (revid:alik@sun.com-20091022201318-jfvtrzd6lb07cwp5) (version source revid:bjorn.munch@sun.com-20091005111204-j1k0y4hznyyd3775) (merge vers: 5.4.5-beta) (pib:13)
[22 Oct 2009 23:13] Paul DuBois 
It seems the --parallel option is not documented in the mysqltest manual. mtr --help says:

  parallel=N            Run tests in N parallel threads (default=1)
                        Use parallel=auto for auto-setting of N

Is there anything else that should be said about it? What considerations go into "auto-setting" the value?
[23 Oct 2009 7:33] Bugs System 
Pushed into 5.1.41 (revid:bjorn.munch@sun.com-20091021073307-ummbh6668hvfxqjv) (version source revid:bjorn.munch@sun.com-20091021073307-ummbh6668hvfxqjv) (merge vers: 5.1.41) (pib:13)
[23 Oct 2009 15:26] Paul DuBois 
Added description of --parallel option to mysqltest manual.
[18 Dec 2009 10:37] Bugs System 
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:52] Bugs System 
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:07] Bugs System 
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:21] Bugs System 
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
[15 Aug 2016 16:30] Paul DuBois 
Noted in 5.1.41 changelog.
[15 Aug 2016 16:30] Paul DuBois 
mysql-test-run.pl now checks the MTR_MAX_PARALLEL environment
variable. If set, this variable specifies the maximum number of
parallel workers that can be spawned when the --parallel=auto option
is specified. If --parallel=auto is not specified, MTR_MAX_PARALLEL
variable has no effect.