MySQL Bugs: #20010: mysql-test-run.pl: --record and --require conflict (test fails)

Bug #20010	mysql-test-run.pl: --record and --require conflict (test fails)
Submitted:	22 May 2006 20:31	Modified:	6 Nov 2006 14:47
Reporter:	Guilhem Bichot	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Tests	Severity:	S3 (Non-critical)
Version:	5.1-bk	OS:	Linux (linux)
Assigned to:	Magnus Blåudd	CPU Architecture:	Any

Description:
have not tested with 5.0.
Happened when creating a new test like this:
-- source include/master-slave.inc
Then to create the result for this test I wanted to do as usual:
./mysql-test-run.pl --record mytest.test
and it failed like this:
mysqltest: In included file "./include/master-slave.inc": At line 12: Could not open ./r/slave-stopped.result: errno = 13
This is because master-slave.inc does a
--require r/slave-stopped.result 
and mysqltest tries to WRITE the result of the test to r/slave-stopped.result!
The bug is indeed that mysqltest uses a wrong logic:

  if (command->record_file[0])
  {

    /* A result file was specified for _this_ query  */
    if (record)
    {
      /*
	 Recording in progress
         Dump the output from _this_ query to the specified record_file
      */
      str_to_file(command->record_file, ds->str, ds->length);

    } else {

      /*
	The output from _this_ query should be checked against an already
	existing file which has been specified using --require or --result
      */
      check_result(ds, command->record_file, command->require_file);
    }
  }
It says it all: it considers --record and the presence of the "require" command as mutually exclusive, which it is not.

How to repeat:
see description

Can't repeat, when I do:
./mysql-test-run.pl --record mytest.test 

It will say it can't connect to the slave, and that is normal as the test must start with "rpl" for the slave to be started.

If I rename it to rpl_mytest.test it works fine. Even when I remove the r/slave-stopped.result file. It will be recreated.

ok, I can reproduce the problem on two machines with a tree of May 12th, and cannot reproduce on a third machine with a tree of May 18th. I guess this bug has been fixed somewhere between the two moments, though I see nothing in the "bk changes".
I propose to close this as "can't repeat", and when I pull a fresh tree I'll retest and re-open if needed.

Here is a testcase from today, 5.1-maint:
create t/cursor.test containing this:
-- source include/have_innodb.inc
and don't create the result file.
./mysql-test-run.pl  --record cursor

Logging: ./mysql-test-run.pl --record cursor
Using binlog format 'stmt'
Using ndbcluster when necessary, mysqld supports it
Setting mysqld to support SSL connections
Binaries are debug compiled
Using MTR_BUILD_THREAD      = 0
Using MASTER_MYPORT         = 9306
Using MASTER_MYPORT1        = 9307
Using SLAVE_MYPORT          = 9308
Using SLAVE_MYPORT1         = 9309
Using SLAVE_MYPORT2         = 9310
Using NDBCLUSTER_PORT       = 9310
Using NDBCLUSTER_PORT_SLAVE = 9311
Using IM_PORT               = 9312
Using IM_MYSQLD1_PORT       = 9313
Using IM_MYSQLD2_PORT       = 9314
Killing Possible Leftover Processes
Removing Stale Files
Installing Master Database
Installing Master Database
Stopping All Servers
Shutting-down Instance Manager
=======================================================
Starting Tests in the 'main' suite

TEST                           RESULT         TIME (ms)
-------------------------------------------------------

cursor                         [ fail ]

Errors are (from /home/mysql_src/mysql-5.1-non-mater-cursors/mysql-test/var/log/mysqltest-time) :
/home/mysql_src/mysql-5.1-non-mater-cursors/client/.libs/lt-mysqltest: File './r/have_innodb.require' not found (Errcode: 13)
mysqltest: In included file "./include/have_innodb.inc": At line 3: Could not open ./r/have_innodb.require: errno = 13

Guilhem, I can reproduce if I do a "chmod u-w r/innodb.require". I noticed you also had changed the Bitkeeper config file to check out in "get" mode.

That means the problem is that --record are trying to rewrite the .require file. That should probably not be the case?

I would almost like to say that we should remove the --record option as I feel it seems to be so easy to introduce problems.  But of course I can see the usefulness in some way. ;)

I suggest that 
 * --record should not rewrite require files.
 * If there is no require file --record should fail and dump the suggested require file to .require_reject
 *  When recording a testcase the requires should be checked. I.e the result of the queries should be checked against the content of the file. If the check fails, the record should fail since it should not be possible to record a .result file when not all conditions are met.
 * If a require fails during record the query should be dumped to .require-reject

Hi Magnus,
yes, your proposed solution is fine with me.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/13190

ChangeSet@1.2587, 2006-10-05 18:00:20+02:00, msvensson@neptunus.(none) +1 -0
  Bug #20010  	mysql-test-run.pl: --record and --require conflict (test fails)
  - When --record is passed to mysqltest the whole testcase should be executed
  as it normally is while the output form the test is acumulating
  in ds_res. When test has finished ds_res should simply be written to the
  specified result file(if any) instead of comapring it against the result file.
  Simplify handling of --require and also the cecking of result files by splitting
  check_result function into one  function 'check_require' that is specialised in
  checking require's and leave 'check_result' to do just that.
  - "mysqltest --record" has been considered unsafe, but with this really simple
  logic, it should be safe to use.

Test suite change. No changelog entry needed.