MySQL Bugs: #43607: mysqltest command disable_abort_on

Bug #43607	mysqltest command disable_abort_on_error invalidated by error command
Submitted:	12 Mar 2009 17:03	Modified:	17 Mar 2010 11:41
Reporter:	Ingo Strüwing	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Tools: MTR / mysql-test-run	Severity:	S3 (Non-critical)
Version:	6.0	OS:	Any
Assigned to:	Bjørn Munch	CPU Architecture:	Any

Description:
If an SQL statement is expected to return an error, but returns a different error or success, the test is aborted, even if 'disable_abort_on_error' is in effect.

'disable_abort_on_error' works so that the test case is not aborted if an SQL statement, which is expected to succeed, returns any error.

But if the statement is preceded by an 'error <some_error_number>' command, the test case is aborted anyway, if another error number, even success, is returned.

This can be quite inconvenient, but I am not sure if this is a bug at all. Perhaps it works as designed. Using 'error 0' can temporarily disable 'disable_abort_on_error' for the next statement. :-/

However, I hope it will be fixed.

How to repeat:
Select an arbitrary test file, e.g. backup_default.test
Insert the following sequence, e.g. at the end:

--disable_abort_on_error
#--error 1
#--error 0
#--error ER_PARSE_ERROR
ILLEGAL_SQL_STATEMENT;
#--error 1
SHOW PROCESSLIST;
--enable_abort_on_error

With all the '#' in place, the test shows how 'disable_abort_on_error' ignores the error from 'ILLEGAL_SQL_STATEMENT'.

Commenting in the first 'error 1' shows that the test case aborts with "wrong error".

Commenting in 'error 0' shows that 'disable_abort_on_error' is disabled by it. The test case aborts with "syntax error".

Commenting in the second 'error 1' shows that 'disable_abort_on_error' does not defeat a test abort due to a successful statement, which is expected to fail.

Commenting in 'error ER_PARSE_ERROR' shows the normal behavior. The SQL statement returns the expected error, so the test continues. [Even with 'disable_abort_on_error' in effect. :) So one case works at least ;).]

Suggested fix:
Let the test continue in all the above cases, when 'disable_abort_on_error' is in effect.

Suggest triage values:
Defect: medium. I'm unsure if this might be as designed. :(
Workaround: none.
Impact: minimal. Feature used during test case development [almost] only.

Behavior probably not intended

Hi,
I added the "--disable_abort_on_error" and the $mysql_errno
feature years ago and can confirm that the behaviour
related to SQL commands is intended.
Commands like for example "remove_file" or 
"if (`<some SQL command>`)" did not exist in 2004.
1. "--disable_abort_on_error" should apply to
   SQL commands.
2. In case we have a sequence of SQL statements
   between "--disable_abort_on_error" and
   "--enable_abort_on_error" than it should be easy
   for the test author to switch to strict checking
   for one particular statement of this sequence.
   If he writes a "--error <number>" before some
   statement than this should overrule any
   "*_abort_on_error" setting.
3. In case we run with "--disable_abort_on_error"
   than we might want to exploit the SQL code of
   the last statement for test flow control.
   Therefore the SQL code of the last statement
   is stored im $mysql_errno.

Based on my current experiences I think
- 2. is neither clear bad nor clear good
- "--disable_abort_on_error" + $mysql_errno
  would be also very useful for commands like
  connect/disconnect/connection/remove_file/exec/...
- long statements sequences between
  "--disable_abort_on_error" and "--enable_abort_on_error"
  are rare and in many cases weak test design
- an option which affects just the next statement like
  "--error undefined" + $mysql_errno would have most
  probably been a better solution

There is probably a significant number of tricky tests
which might fail in case the current behaviour gets changed.

Last comment is right, this is not a bug, in fact it is intentional and documented.  Sorry for not closing this before; it ended up behind a large stack of bugs and not seriously looked at.

Closing (from Not a Bug) to get this off my list