Description:
MySQL Cluster run some automatic tests called autotest with results currently visible at http://ndbmaster.se.oracle.com/~autotest. It is based on the executable atrt.
Atrt takes an testcase file as argument describing what tests to run.
When new test are added or some old removed from a testcase file errors are introduced.
Often it is due to a missing empty line or an extra empty line. The syntax is quite strict and each testcase description should be terminated with exactly one empty line, no other empty lines are allowed. Any comment line starting with # are ignored and not counted as empty lines.
When this occur the only visible sign is that the number of tests reported in autotest gui have decreased, which in itself is no error.
For example only 418 tests are reported on 2015-06-27 in daily-basic for mysql-5.6-cluster-7.4 while there are 420 tests defined in storage/ndb/test/run-test/daily-basic-tests.txt.
How to repeat:
Look on results for daily-basic on 7.4 at 2015-06-27 and compare with daily-basic-tests.txt.
Suggested fix:
* add checks of testcase file syntax in build time
* add checks of testcase file syntax in mtr
* change syntax to less error prune syntax
* let atrt report when there are syntax errors in testcase file
* make it visible in autotest webgui that testing was aborted
* document syntax of testcase file
