MySQL Bugs: #27753: enable mysql-test-run.pl to ignore tests based on wildcard

Bug #27753	enable mysql-test-run.pl to ignore tests based on wildcard
Submitted:	11 Apr 2007 10:55	Modified:	11 Feb 2008 18:16
Reporter:	Hakan Küçükyılmaz	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Tests	Severity:	S4 (Feature request)
Version:		OS:	Any
Assigned to:	Magnus Blåudd	CPU Architecture:	Any

Description:
Currently mysql-test-runl.pl --skip-test=falcon will skip all tests starting with falcon. The test index_merge_falcon will not be skipped.

The same applies for --do-test=falcon. It will run all tests starting with falcon but not index_merge_falcon.

How to repeat:
Run mysql-test-run.pl --do-test=falcon in 5.1-falcon tree.

Suggested fix:
The options --skip-test and --do-test should work on wildcard basis.

Best regards,

Hakan

Thank you for a reasonable feature request.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/35003

ChangeSet@1.2534, 2007-10-05 19:23:44+02:00, msvensson@shellback.(none) +2 -0
  Bug#27753 enable mysql-test-run.pl to ignore tests based on wildcard

Pushed into 5.1.23-beta

Pushed into 5.0.52

Both --do-test=<pattern> and --skip-test=<pattern> uses perl regex syntax now, and  supports:

1. The old way, if there is no regex characters in the <pattern> it will match beginning of the testname. In perl regex that is ^pattern

Example:
 ./mtr --do-test=al # Matches all test whose name starts with "al"
 ./mtr --skip-test=al # Skips all test whose name starts with "al"

2. The new way, if there is a regex character in the <pattern> it will be evaluated and use against the testname of each test found.

Example:
 ./mtr --do-test=a.s # Matches all tests whose names start with an "a" and third char is "s"
 ./mtr --do-test=a.*s # Matches all tests whose names start with an "a" and followed by an "s" somewhere in the name
./mtr --do-test=a.*s$ # Matches all tests whose names start with an "a" and ends with an "s"

I hope I won't have to explain all possibilities :)

In theory, everything described on http://perldoc.perl.org/perlre.html should work.

Question regarding this:
"[24 Oct 10:07] Magnus Svensson
[...]
2. The new way, if there is a regex character in the <pattern> it will be evaluated and
use against the testname of each test found.

Example:
 ./mtr --do-test=a.s # Matches all tests whose names start with an "a" and third char is
"s"
 ./mtr --do-test=a.*s # Matches all tests whose names start with an "a" and followed by an
"s" somewhere in the name
./mtr --do-test=a.*s$ # Matches all tests whose names start with an "a" and ends with an
"s"

I hope I won't have to explain all possibilities :)"

No, you don't need to explain all the possibilities. :-)

But I do wonder about the examples shown above. The pattern explanations should say "Matches all tests whose names *contain* an 'a'" (not "*start* with an 'a'"), unless the patterns are always assumed to begin with an explicit ^ character.

So my question: Does --do-test=a.* match test names that begin with an "a" or that contain an "a".
That is, is --do-test=a.* equivalent to --do-test=^a.*, or is the implied ^ used only if the --do-test argument contains no pattern characters?

There seems to be a bug. The intention is that in order to be backward compatible a ^will be prepended to the pattern if it does not contain any special regex characters.

My check for that was as simple as 
>>
if ( $from =~ /[a-z0-9]/ ) {
 # No regex in pattern
  $from= "^$from";
}
<<

But that does not work and hence my faulty explanation. Will fix so it works like you say.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/36835

ChangeSet@1.2549, 2007-11-01 11:02:28+01:00, msvensson@pilot.mysql.com +1 -0
  Bug#27753 enable mysql-test-run.pl to ignore tests based on wildcard
   - Fix problem in first implementation

This should do the trick. Philip please review.

sub init_pattern {
  my ($from, $what)= @_;
  return undef unless defined $from;
  if ( $from =~ /^[a-z0-9\.]*$/ ) {
    # Does not contain any regex (except . that we allow as
    # separator betwen suite and testname), make the pattern match
    # beginning of string
    $from= "^$from";
    mtr_verbose("$what='$from'");
  }
  # Check that pattern is a valid regex
  eval { "" =~/$from/; 1 } or
    mtr_error("Invalid regex '$from' passed to $what\nPerl says: $@");
  return $from;
}

The patch from the last comment appears to do the trick. The following work as expected:

perl mysql-test-run.pl --do-test=.*state - skips tests ending in state

perl mysql-test-run.pl --skip-test=.*state log_state - skips this test

perl mysql-test-run.pl main.log_state - runs just this test

perl mysql-test-run.pl --do-test=myisam - runs all tests starting with myisam

and regular expression errors are properly reported.

However, Paul's example:

Example:
 ./mtr --do-test=a.s # Matches all tests whose names start with an "a" and third char is
"s"

will not work because a stand-alone dot is taken as a suite/test separator. In order for the script to match "a" in any position, the following syntax is required:

 ./mtr --do-test=.*a

That is, there must be a * in order for the string to qualify as a pattern, and then it will be matched as "^.*a" which covers all strings containing "a".

Pushed into 6.0.5-alpha

Pushed into 5.1.23-rc

Pushed into 5.0.54

Noted in 5.0.54, 5.1.23, 6.0.5 changelogs.

The argument for the mysql-test-run.pl --do-test and --skip-test options is
now interpreted as a Perl regular expression if there is a pattern 
metacharacter in the argument value. This allows more flexible 
specification of which tests to perform or skip.