Bug #52464 mysqltest does not invalidate 'source' cache
Submitted: 30 Mar 2010 10:08 Modified: 21 Jun 2017 17:26
Reporter: Sven Sandberg Email Updates:
Status: Verified Impact on me:
None 
Category:Tools: MTR / mysql-test-run Severity:S3 (Non-critical)
Version:5.1+ OS:Any
Assigned to: CPU Architecture:Any

[30 Mar 2010 10:08] Sven Sandberg
Description:
When using a $variable in a 'source' command in a test script, and the same 'source' command is executed several times, then only the value that the $variable had the first time the file was sourced is used, even if the $variable has a different value later.

So it seems that mtr caches the contents of the file and never invalidates the cache.

How to repeat:
$ cat mysql-test/t/tmp.test
--let $f= tmp_a.inc
--let $i= 2
while ($i) {
  --echo sourcing $f
  --source $f
  --dec $i
  --let $f= tmp_b.inc
}

$ cat mysql-test/t/tmp_a.inc        
--echo a

$ cat mysql-test/t/tmp_b.inc        
--echo b

$ ./mtr tmp
[...]
sourcing tmp_a.inc
a
sourcing tmp_b.inc
a
[...]
[4 May 2010 8:16] Jørgen Austvik
Another test that demonstrates the same:

---------------8<-------------------------8<-------------------------8<-------
--let $cnt= 9
while ($cnt) {
  --echo Cnt outside: $cnt
  --perl
use strict;
use IO::File;
my $random = int(rand(10));
print "Perl random: $random\n";
my $vardir = $ENV{MYSQLTEST_VARDIR} or die "Need MYSQLTEST_VARDIR";
my $file_name = "$vardir/tmp/random.inc";
my $F = IO::File->new($file_name, 'w') or die "Could not open '$file_name' for writing";
print $F "--let \$MTR_RANDOM= $random\n";
print $F "--echo Random from include: \$MTR_RANDOM\n";
$F->close();
EOF
  --source $MYSQLTEST_VARDIR/tmp/random.inc
  --echo Random from loop: $MTR_RANDOM
  --dec $cnt
}
---------------8<-------------------------8<-------------------------8<-------

example output:
---------------8<-------------------------8<-------------------------8<-------
worker[1] Using MTR_BUILD_THREAD 666, with reserved ports 16660..16669
Cnt outside: 9
Perl random: 7
Random from include: 7
Random from loop: 7
Cnt outside: 8
Perl random: 6
Random from include: 7
Random from loop: 7
Cnt outside: 7
Perl random: 1
Random from include: 7
Random from loop: 7
Cnt outside: 6
Perl random: 4
Random from include: 7
Random from loop: 7
...
Cnt outside: 1
Perl random: 1
Random from include: 7
Random from loop: 7
main.while-source                        [ pass ]    169
---------------8<-------------------------8<-------------------------8<-------

The file has off course been updated correctly:
---------------8<-------------------------8<-------------------------8<-------
[ja155679@khepri02:mysql-test] cat var/tmp/random.inc 
--let $MTR_RANDOM= 1
--echo Random from include: $MTR_RANDOM
---------------8<-------------------------8<-------------------------8<-------

I have a test where I want to populate a table until X% of the memory is used. To figure out how much memory is used, I have to call external programs and parse and accumulate their output. I do the parsing and accumulation in perl, and then save the output from the perl command to a include file so that it can be used from MTR.

For me this problem is a showstopper, and I would very much welcome a --don't-cache-source or similar setting as a short-term fix for this.
[21 Jun 2017 17:26] MySQL Verification Team
Unfortunately still here with 5.7.18

kind regards
bogdan kecman

[root@localhost mysql-test]# ./mysql-test-run tmp
Logging: ./mysql-test-run  tmp
MySQL Version 5.7.18
Checking supported features...
 - SSL connections supported
Collecting tests...
Checking leftover processes...
 - found old pid 46644 in 'mysqld.1.pid', killing it...
   process did not exist!
 - found old pid 46646 in 'mysqld.2.pid', killing it...
   process did not exist!
Removing old var directory...
Creating var directory '/root/BUILD/mysql-5.7.18/mysql-test/var'...
Installing system database...
Using parallel: 1

==============================================================================

TEST                                      RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
worker[1] mysql-test-run: WARNING: running this script as _root_ will cause some tests to be skipped
sourcing tmp_a.inc
a
sourcing tmp_b.inc
a
main.tmp                                 [ pass ]      9
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 0.009 of 9 seconds executing testcases

Completed: All 1 tests were successful.