Bug #20010 mysql-test-run.pl: --record and --require conflict (test fails)
Submitted: 22 May 2006 20:31 Modified: 6 Nov 2006 14:47
Reporter: Guilhem Bichot Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Tests Severity:S3 (Non-critical)
Version:5.1-bk OS:Linux (linux)
Assigned to: Magnus Blåudd CPU Architecture:Any

[22 May 2006 20:31] Guilhem Bichot
Description:
have not tested with 5.0.
Happened when creating a new test like this:
-- source include/master-slave.inc
Then to create the result for this test I wanted to do as usual:
./mysql-test-run.pl --record mytest.test
and it failed like this:
mysqltest: In included file "./include/master-slave.inc": At line 12: Could not open ./r/slave-stopped.result: errno = 13
This is because master-slave.inc does a
--require r/slave-stopped.result 
and mysqltest tries to WRITE the result of the test to r/slave-stopped.result!
The bug is indeed that mysqltest uses a wrong logic:

  if (command->record_file[0])
  {

    /* A result file was specified for _this_ query  */
    if (record)
    {
      /*
	 Recording in progress
         Dump the output from _this_ query to the specified record_file
      */
      str_to_file(command->record_file, ds->str, ds->length);

    } else {

      /*
	The output from _this_ query should be checked against an already
	existing file which has been specified using --require or --result
      */
      check_result(ds, command->record_file, command->require_file);
    }
  }
It says it all: it considers --record and the presence of the "require" command as mutually exclusive, which it is not.

How to repeat:
see description
[24 May 2006 9:18] Magnus Blåudd
Can't repeat, when I do:
./mysql-test-run.pl --record mytest.test 

It will say it can't connect to the slave, and that is normal as the test must start with "rpl" for the slave to be started.

If I rename it to rpl_mytest.test it works fine. Even when I remove the r/slave-stopped.result file. It will be recreated.
[24 May 2006 9:36] Guilhem Bichot
ok, I can reproduce the problem on two machines with a tree of May 12th, and cannot reproduce on a third machine with a tree of May 18th. I guess this bug has been fixed somewhere between the two moments, though I see nothing in the "bk changes".
I propose to close this as "can't repeat", and when I pull a fresh tree I'll retest and re-open if needed.
[20 Sep 2006 9:37] Guilhem Bichot
Here is a testcase from today, 5.1-maint:
create t/cursor.test containing this:
-- source include/have_innodb.inc
and don't create the result file.
./mysql-test-run.pl  --record cursor

Logging: ./mysql-test-run.pl --record cursor
Using binlog format 'stmt'
Using ndbcluster when necessary, mysqld supports it
Setting mysqld to support SSL connections
Binaries are debug compiled
Using MTR_BUILD_THREAD      = 0
Using MASTER_MYPORT         = 9306
Using MASTER_MYPORT1        = 9307
Using SLAVE_MYPORT          = 9308
Using SLAVE_MYPORT1         = 9309
Using SLAVE_MYPORT2         = 9310
Using NDBCLUSTER_PORT       = 9310
Using NDBCLUSTER_PORT_SLAVE = 9311
Using IM_PORT               = 9312
Using IM_MYSQLD1_PORT       = 9313
Using IM_MYSQLD2_PORT       = 9314
Killing Possible Leftover Processes
Removing Stale Files
Installing Master Database
Installing Master Database
Stopping All Servers
Shutting-down Instance Manager
=======================================================
Starting Tests in the 'main' suite

TEST                           RESULT         TIME (ms)
-------------------------------------------------------

cursor                         [ fail ]

Errors are (from /home/mysql_src/mysql-5.1-non-mater-cursors/mysql-test/var/log/mysqltest-time) :
/home/mysql_src/mysql-5.1-non-mater-cursors/client/.libs/lt-mysqltest: File './r/have_innodb.require' not found (Errcode: 13)
mysqltest: In included file "./include/have_innodb.inc": At line 3: Could not open ./r/have_innodb.require: errno = 13
[20 Sep 2006 10:24] Magnus Blåudd
Guilhem, I can reproduce if I do a "chmod u-w r/innodb.require". I noticed you also had changed the Bitkeeper config file to check out in "get" mode.

That means the problem is that --record are trying to rewrite the .require file. That should probably not be the case?

I would almost like to say that we should remove the --record option as I feel it seems to be so easy to introduce problems.  But of course I can see the usefulness in some way. ;)

I suggest that 
 * --record should not rewrite require files.
 * If there is no require file --record should fail and dump the suggested require file to .require_reject
 *  When recording a testcase the requires should be checked. I.e the result of the queries should be checked against the content of the file. If the check fails, the record should fail since it should not be possible to record a .result file when not all conditions are met.
 * If a require fails during record the query should be dumped to .require-reject
[2 Oct 2006 9:14] Guilhem Bichot
Hi Magnus,
yes, your proposed solution is fine with me.
[5 Oct 2006 19:38] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13190

ChangeSet@1.2587, 2006-10-05 18:00:20+02:00, msvensson@neptunus.(none) +1 -0
  Bug #20010  	mysql-test-run.pl: --record and --require conflict (test fails)
  - When --record is passed to mysqltest the whole testcase should be executed
  as it normally is while the output form the test is acumulating
  in ds_res. When test has finished ds_res should simply be written to the
  specified result file(if any) instead of comapring it against the result file.
  Simplify handling of --require and also the cecking of result files by splitting
  check_result function into one  function 'check_require' that is specialised in
  checking require's and leave 'check_result' to do just that.
  - "mysqltest --record" has been considered unsafe, but with this really simple
  logic, it should be safe to use.
[6 Nov 2006 14:47] Paul Dubois
Test suite change. No changelog entry needed.