MySQL Bugs: #43532: mtr should not rely on diff to report test results

Bug #43532	mtr should not rely on diff to report test results
Submitted:	10 Mar 2009 11:39	Modified:	17 Jun 2009 9:39
Reporter:	Georgi Kodinov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Tools: MTR / mysql-test-run	Severity:	S4 (Feature request)
Version:	5.0, 5.1, 6.0	OS:	Any
Assigned to:	Bjørn Munch	CPU Architecture:	Any
Tags:	pb2

Description:
When the diff utility is not installed and a test fails the mysql-test-run.pl will return the full content of the two files (the actual and expected result) to the user.
This is not good because diff is a GNU tool and we can't safely redistribute it.
 

How to repeat:
Run the test on windows and don't install the diff binary

Suggested fix:
Implement a better looking comparison in mtr itself and don't use the diff external command

Solution from Bjørn that makes diff functionality available to MTR but not to tests, hence addressing both the need from MTR/developers and the need to keep unixisms unavailable for tests:

Install diff but under a different name, say 'mtrdiff' and then modify the mysqltest binary to check for this if it can't find diff on Windows.  This should not be very difficult; there's already a separate function used on Windows only to check whether we have diff.

That way MTR can use diff but tests that try to will fail, since they
shouldn't know about mtrdiff.

Installing it under a different name is easier to handle than fiddling
with the path, and safer.

Until this fix is in place, we temporarily make diff available on Windows test servers (in PB2).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/68863

2767 Bjorn Munch	2009-03-11
      Bug #43532 mtr should not rely on diff to report test results
      mtr on Windows does not give decent diff due to missing diff install
      Modified to look for 'mtrdiff' if diff not available.
      modified:
        client/mysqltest.cc

Just one comment, please use snprintf instead of sprintf:
+    snprintf(cmd, sizeof(cmd), "%s -v", diff_name);

Otherwise ok.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/68892

2767 Bjorn Munch	2009-03-11
      Bug #43532 mtr should not rely on diff to report test results
      mtr on Windows does not give decent diff due to missing diff install
      Modified to look for 'mtrdiff' if diff not available.
      modified:
        client/mysqltest.cc

Ok, but I think you need to use 'my_snprintf' - sorry for not mentioning before.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/68899

2767 Bjorn Munch	2009-03-11
      Bug #43532 mtr should not rely on diff to report test results
      mtr on Windows does not give decent diff due to missing diff install
      Modified to look for 'mtrdiff' if diff not available.
      modified:
        client/mysqltest.cc

Pushed to 6.0-mtr and 5.1-mtr

Pushed into 5.1.34 (revid:alik@sun.com-20090319121953-frpl3uvd05v07sho) (version source revid:bjorn.munch@sun.com-20090311135250-c1eu1n9ias4ce9x4) (merge vers: 5.1.33) (pib:6)

Pushed into 6.0.11-alpha (revid:alik@sun.com-20090319122149-fpfkb3vxi7gk3guh) (version source revid:bjorn.munch@sun.com-20090311140700-qnu2uk70le7w8smc) (merge vers: 6.0.11-alpha) (pib:6)

Test suite change. No changelog entry needed.

Pushed into 5.1.34-ndb-6.2.18 (revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (version source revid:jonas@mysql.com-20090508185236-p9b3as7qyauybefl) (merge vers: 5.1.34-ndb-6.2.18) (pib:6)

Pushed into 5.1.34-ndb-6.3.25 (revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (version source revid:jonas@mysql.com-20090509063138-1u3q3v09wnn2txyt) (merge vers: 5.1.34-ndb-6.3.25) (pib:6)

Pushed into 5.1.34-ndb-7.0.6 (revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (version source revid:jonas@mysql.com-20090509154927-im9a7g846c6u1hzc) (merge vers: 5.1.34-ndb-7.0.6) (pib:6)

Can we please port this to 5.0 ?

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/75713

2767 Georgi Kodinov	2009-06-05
      Bug #43532 : backport of the 5.1 code to 5.0 mysqltest

Pushed into 5.0.84 (revid:gkodinov@mysql.com-20090616082753-kwe0l8uoictxhojf) (version source revid:azundris@mysql.com-20090605212937-9kcqi5zjat2f2awm) (merge vers: 5.0.83) (pib:6)

Pushed into 5.1.36 (revid:joro@sun.com-20090616102155-3zhezogudt4uxdyn) (version source revid:azundris@mysql.com-20090606160500-zm37psz7u1m96y5l) (merge vers: 5.1.36) (pib:6)

Test suite change. No changelog entry needed.

Pushed into 5.4.4-alpha (revid:alik@sun.com-20090616183122-chjzbaa30qopdra9) (version source revid:azundris@mysql.com-20090606160423-aym8s5rn51nmg387) (merge vers: 6.0.12-alpha) (pib:11)

Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)

Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)

Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)

Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)