Bug #56601 Test uses Unix path for temporary file, fails, and writes misleading message
Submitted: 6 Sep 2010 19:06 Modified: 16 Nov 2010 2:02
Reporter: Joerg Bruehe Email Updates:
Status: Closed Impact on me:
None 
Category:Tests: Engine Severity:S3 (Non-critical)
Version:5.5.6-m3 OS:Windows
Assigned to: Mattias Jonsson CPU Architecture:Any

[6 Sep 2010 19:06] Joerg Bruehe
Description:
Found in the release build of 5.5.6-m3 on Windows:

=====
parts.partition_special_myisam           [ fail ]
        Test ended at YYYY-MM-DD HH:MM:SS

CURRENT_TEST: parts.partition_special_myisam
Could not open file for writing /test/load.in at G:/pb2/test/sb_1-2235634-1283491829.74/mysql-5.5.6-rc-win32/mysql-test/var\tmp34A6.tmp line 1.
mysqltest: At line NNN: command "perl" failed with error 2

The result from queries just before the failure was:
< snip >
...
#
# Bug#53770: Server crash at handler.cc:2076 on LOAD DATA
#            after timed out COALESCE PARTITION
CREATE TABLE t1 ( i INT NOT NULL AUTO_INCREMENT PRIMARY KEY, f INT )
ENGINE = MyISAM PARTITION BY HASH(i) PARTITIONS 3;
# Connection 1 starts transaction and gets lock
START TRANSACTION;
SELECT * FROM t1;
i   f
SET lock_wait_timeout = 2;
# Connection 2 tries to coalesce partitions (timeout):
ALTER TABLE t1 COALESCE PARTITION 2;
ERROR HY000: Lock wait timeout exceeded; try restarting transaction

More results from queries before failure can be found in /PATH/mysql-test/var/log/partition_special_myisam.log
=====

This happens when an embedded Perl snippet cannot write a temporary file.

Note that the path name reported is "/test/load.in" which should not be used and is very likely to fail everywhere, so why does it work on most platforms?

Well, in reality it isn't used - here is the code snippet from
"mysql-test/suite/parts/t/partition_special_myisam.test":

=====
--connect (con3,localhost,root,,)

perl;
open( LD, ">" . "/tmp/load.in" ) || die "Could not open file for writing " . $ENV{'MYSQLTEST_DATADIR'} . "/test/load.in";
print LD "1\n2\n3\n";
close( LD );
EOF
=====

So the real path name is "/tmp/load.in" which will succeed on all Unix platforms (that is why the test passes there), but will still fail on Windows (as experienced in the build).

How to repeat:
Run the suite on Windows.

Suggested fix:
The reason is non-portable test code in the fix for bug#53770.

Proposed fixes:
1) Search the test files for other Perl snippets opening a temporary file.
They typically use path names built from $ENV{MYSQL_TMP_DIR} or $ENV{'MYSQLTEST_VARDIR'}

2) By all means, don't write error messages which write different path names from those used in the code!
[15 Sep 2010 8:24] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/118277

3207 Mattias Jonsson	2010-09-15
      Bug#56659: Mismatch of CAPITAL vs small letters in "unified filelist" partitioning output
      Bug#56657: Test still uses "--exec rm -f ..." which is non-portable
      Bug#56601: Test uses Unix path for temporary file, fails, and writes misleading message
      
      Several tests that was written in a non portable way (failed on windows)
      
      Fixed by
      1) backporting the fix for replace_result to also apply to list_files
      (mysqltest from mysql-trunk)
      2) replacing all #p#/#sp#/#tmp# to #P#/#SP#/#TMP#/
      (innodb always converts filenames to lower case in windows).
      3) replacing '--exec rm -f' with '--remove_files_wildcard'
      4) replacing a perl snippet with '--write_file'
     @ client/mysqltest.cc
        backport from mysql-trunk to allow replace_result to apply
        also on list_files
     @ mysql-test/suite/parts/inc/partition_check_drop.inc
        Compensate for differences between innodb on windows vs unix.
        Using mysqltest command, instead of unix command to remove files.
     @ mysql-test/suite/parts/inc/partition_crash.inc
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/inc/partition_fail.inc
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/inc/partition_layout.inc
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/inc/partition_layout_check1.inc
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/inc/partition_layout_check2.inc
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/r/partition_recover_myisam.result
        updated result
     @ mysql-test/suite/parts/r/partition_special_myisam.result
        updated result
     @ mysql-test/suite/parts/t/part_supported_sql_func_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter1_1_2_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter1_2_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter2_1_1_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter2_1_2_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter2_2_2_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_alter4_innodb.test
        Test takes very long time, require --big flag
     @ mysql-test/suite/parts/t/partition_debug_sync_innodb.test
        compensate for differences between innodb on windows vs unix
     @ mysql-test/suite/parts/t/partition_recover_myisam.test
        more generic suppression (failed in windows)
     @ mysql-test/suite/parts/t/partition_special_myisam.test
        Using portable mysqltest command 'write_file' instead of perl snippet.
[21 Sep 2010 10:33] Joerg Bruehe
I checked the attached patch.

The fixes for bug#56601, bug#56657, and bug#56659 are approved.
The backport in "client/mysqltest.cc" is approved.

The generic suppression in test "partition_recover_myisam" is approved.

I have mixed feelings about classifying "partition_alter*" and "part_supported_sql_func_innodb" as "big" tests:
- Yes, these tests may take quite long, so they may qualify as "big".
  Depending on the platform, their combined run time may reach 1 hour.
- OTOH, I fear we have a tendency to disable tests rather than fix them.
  This may cause us to lose test coverage.

I have no firm opinion whether we should do this change.
Please check whether we have continuous PB2 coverage for "big" tests on all platforms in at least the main trees, so that it doesn't get lost.
If we do *not* have this, get a second review for this change.
[22 Sep 2010 14:22] John Embretsen
Patch conditionally approved. See review comment for Bug#56659 (changeset fixes multiple bugs).
[1 Oct 2010 19:50] Mattias Jonsson
Pushed into mysql-5.5-bugteam and mysql-trunk-merge
[3 Oct 2010 1:18] Paul DuBois
Changes to test suite. No changelog entry needed.
[9 Nov 2010 19:46] Bugs System
Pushed into mysql-5.5 5.5.7-rc (revid:sunanda.menon@sun.com-20101109182959-otkxq8vo2dcd13la) (version source revid:sunanda.menon@sun.com-20101109182959-otkxq8vo2dcd13la) (merge vers: 5.5.7-rc) (pib:21)
[13 Nov 2010 16:19] Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:alexander.nozdrin@oracle.com-20101113152450-2zzcm50e7i4j35v7) (merge vers: 5.6.1-m4) (pib:21)
[13 Nov 2010 16:36] Bugs System
Pushed into mysql-next-mr (revid:alexander.nozdrin@oracle.com-20101113160336-atmtmfb3mzm4pz4i) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (pib:21)