Bug #20037 mysqltest requires cygwin on windows(part 1, new mysqltest commands)
Submitted: 24 May 2006 8:54 Modified: 7 Aug 2007 19:43
Reporter: Magnus Blåudd Email Updates:
Status: Closed Impact on me:
Category:MySQL Server: Tests Severity:S3 (Non-critical)
Version: OS:Microsoft Windows (windows)
Assigned to: Magnus Blåudd CPU Architecture:Any

[24 May 2006 8:54] Magnus Blåudd
When running mysqltest on Windows it requires Cygwin. It should be possible to run without it.

A plan for how ho to do this is as follows:

>below is my suggestion on how to get rid of our dependcy on cygwin. 
>> Comments are appreicated. If I don't here anything from you 
>> within this week, I will proceed with the below suggestion.
>> Best regards
>> Magnus
>> ** Current commands in mysqltest:
>> --exec <command>
>> Currently used to execute some system dependent commands. 
>> Typical example
>> would be to 'echo' something to mysql('echo' does not work on windows
>> the same way as echo on linux) or call 'rm', 'mv'.
>> There is also one test where exec is used to execute "grep" in order
>> to remove some parts in a .TRG file.
>> It should be changed  used _only_ to execute our own binaries
>> like mysql, mysqltest, mysqldump, mysql etc. Maybe we can
>> enforce this by checking that the program being executed is in
>> the list of allowed programs to execute. Will append the output
>> from these commands to the result file.
>> In order to avoid rewiring the testcases too much, we might want to
>> allow  "exec" of commands on the
>> form "--exec echo "something" | <MYSQL_PROGRAM>, and handle it
>> internally in mysql, but it's probably easier to rewrite the testcase
>> to use an intermediate file.
>> It's common to exec something and pipe stderr to stdout like 
>> "2>&1", if
>> we can't get that to work on windows we should rewrite it on the fly
>> before executing.
>> --system <command>
>> Currently used to execute some system commands. Not used that 
>> frequently.
>> When used it will call 'rm', 'rm -f', 'rmdir', 'grep', 'mv', 'echo',
>> 'chmod', 'cp', 'cat', 'test -s'.
>> To be removed?
>> ** New commands to be added to mysqltest:
>> --rm <filename/directoryname>
>> Removes a file or directory. Replaces 'rm', 'rm -f' and 'rmdir'
>> --cp <from> <to>
>> Copy a file. Replaces 'cp'.
>> --mv <from> <to>
>> Move a file. Replaces 'mv'. Actually just a "cp" followed by a "rm".
>> --file_exists <filename>
>> Check for a file existence. Replace 'test -s'
>> Can be combined with an expected "--error 1" to detect that a 
>> file does not
>> exist
>> --perl <scriptname> <arg1> .. <argn>
>> Call a perl script and include it's output in the result log
>> (unless --disable_result_log has been used)
>> --write <filename> <what to write>
>> Writes the <what to write> to the file <filename>
>> --chmod <mod> <filename>
>> I am not sure if this one should be added.
>> Change the file permissions of the file. Should support only the mode
>> flags "-r", "+r", "-w" and "+w" as these are the only ones
>> that exist on windows. The "-r" will return the same error as 
>> if the file
>> doesn't exist at all on windows.
>> * Not to be implemented
>> cat
>> Only used in one place "cat /dev/null > file" and that could 
>> be replace
>> by a small perl script. Or maybe just "write <filename> 0" would do?
>> grep
>> Only used in two places. Need to call perl script. There can 
>> be standard
>> script grep.pl, but I think the "search and replace" thing 
>> that is done 
>> is really just a "one liner" in perl.

How to repeat:
I created this mainly as a place holder to keep track of it.

Suggested fix:
See above.
[14 Jun 2006 15:31] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

[31 Jul 2006 14:24] Magnus Blåudd
This is the first patch for this bug, which adds new commands to mysqltest.

Next step is to remove the "cygwin hacks" from mysqltest and get the things that don't work after that running with the help of the new commands.
[6 Sep 2006 13:55] Chad MILLER
Even better, use the shell here-document convention for specifying the end.  Three letters "EOF" shouldn't be special.

--perl <<XYZ
print "asdf asdf \
EOF is legal here.\n";

$ man 1 bash  #search for "Here Documents"
[2 Aug 2007 7:43] Magnus Blåudd
The new commands has been added to mysqltest and now needs to be documented. Please see http://lists.mysql.com/commits/9820 for an explanation (and examples) of what has been added.

Follow up bug reports have been written for remaining issues running mysql-test-run.pl on Windows without cygwin.
[2 Aug 2007 7:48] Magnus Blåudd
As an effect of these new commands all uses of "--exec rm <file>" should be banned from .test scripts. Instead we should use remove_file etc.

Please update/remove the section of http://dev.mysql.com/doc/mysqltest/en/mysqltest-commands.html that says:
"On Cygwin, the command is executed from cmd.exe, so commands such as rm cannot be executed with exec. Use system  instead.

--exec $MYSQL_DUMP --xml --skip-create test
--exec rm $MYSQLTEST_VARDIR/tmp/t1
exec $MYSQL_SHOW test -v -v;" 
to indicate this.
[2 Aug 2007 8:06] Magnus Blåudd
Tried to find out what version this has gone into. I know it was done in 5.1 and then backported to 4.1 and 5.0 to get the same version of mysqltest in all version . This was done in August 2006, can't see exactly what version that we had at that time.
[7 Aug 2007 7:28] Magnus Blåudd
> New commands:
> 1) copy_file from_file to_file
> Fails if to_file exists.


> 2) write_file file_name
>    file-content
>    EOF

It's "write_file file_name [DELIMITER]", i.e you can optionally choose what your delimiter should be.

> Overwrites any existing file.
> Why the inconsistency between treatment of existing files for
> copy_file and write_file?

Yes, changed "write_file" to throw an error if the file exist. Makes sense.

2b) There is also "append_file <file_name> [<delimiter>]"
2c) There is also "cat_file <file_name>"

> 3) file_exists file_name
> 4) remove_file file_name
> 5) perl
>   code to execute
>   EOF
> Are the filename arguments to these commands subject to variable
> substitution?

yes, the arguments are substituted, but not the content between the command and EOF

> Can the EOF terminator for write_file and perl be changed?

yes, there is an optional argument two write_file, append_file and perl that takes the delimiter to use.

6) There is also "diff_files file_name1 file_name2"

7) There is also chmod_file <octal> <filename>

8) Both "skip" and "exit" is in the manual but not "die <message>"

9) As part of bug#29579 the command "send_quit" was added, it will send COM_QUIT to the server. Please leave that out of the manual for a little while - maybe we should change that to a more generic command that allows the sending of any such command to the server?

10) The syntax for "let" has been extended to allow the retrieval of individual values in a larger result set(this is especially suited for reading values from a SHOW query)

let $variable= query_get_value(<query to run>,<column name>,<row no>);

CREATE TABLE t1(a int, b varchar(255), c datetime);
+Field	Type	Null	Key	Default	Extra
+a	int(11)	YES		NULL	
+b	varchar(255)	YES		NULL	
+c	datetime	YES		NULL

let $value= query_get_value(SHOW COLUMNS FROM t1, Type, 1);
echo $value;

11) eval_result

I suggest deprecating this command and removing it from the manual, it's not used anywhere.
[7 Aug 2007 9:40] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:


ChangeSet@1.2506, 2007-08-07 11:40:03+02:00, msvensson@pilot.(none) +7 -0
  Bug#20037 mysqltest requires cygwin on windows(part 1, new mysqltest commands)
   - Update comments
   - Make "write_file" fail if file already exist
   - Remove temporary files created by test cases
[7 Aug 2007 19:43] Paul Dubois
The mysqltest manual now describes these commands:


and extended syntax for the let command.
[4 Sep 2007 17:12] Bugs System
Pushed into 5.1.23-beta
[4 Sep 2007 17:13] Bugs System
Pushed into 5.0.50