Bug #54166 ha_rnd_init() result code not checked

Description:
This is about 10-15 bugs.

If your storage engine has the nerve to return an error code from rnd_init() (as all documentation suggests you can, as well as the 'int' return type) a lot of places in the code don't check the return type, continue, and you can end up in a world of pain.

It's also the case for rnd_end and other handler functions, but this is the one that bit me first.

How to repeat:
This patch produces a nice list of where they are (gcc only of course, but easy to wrap in a macro).

--- ../mysql-5.1.46/sql/handler.h	2010-04-07 00:06:06.000000000 +1000
+++ sql/handler.h	2010-06-01 17:37:48.230727373 +1000
@@ -1188,7 +1188,7 @@
     inited=NONE;
     DBUG_RETURN(index_end());
   }
-  int ha_rnd_init(bool scan)
+  int ha_rnd_init(bool scan) __attribute__ ((warn_unused_result))
   {
     int result;
     DBUG_ENTER("ha_rnd_init");

Suggested fix:
-Werror and the above patch.

Thank you for the report.

Verified as described:

$cat bug54166.log  | grep ha_rnd_init
item_subselect.cc:2044: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:10854: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:13944: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:14072: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_update.cc:1976: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
log_event.cc:9063: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
log_event_old.cc:843: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
log_event_old.cc:2487: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:208: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:245: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:256: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
filesort.cc:538: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
handler.cc:2150: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
filesort.cc:538: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
handler.cc:2150: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
item_subselect.cc:2044: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:208: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:245: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
records.cc:256: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:10854: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:13944: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_select.cc:14072: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result
sql_update.cc:1976: warning: ignoring return value of ‘int handler::ha_rnd_init(bool)’, declared with attribute warn_unused_result

Are there plans to fix this?

MariaDB and Drizzle have implemented fixes (also for other handler calls).

index_init is also affected.

InnoDB has various workarounds (I've only audited parts of the codepath in Drizzle though, but it *should* be the same in MySQL 5.5 and thus okay) - see https://bugs.launchpad.net/drizzle/+bug/720552

MyISAM and NDB just set local vars, no problem.

Other engines may have issues if they can possibly return an error from index_init and then not preserve it for subsequent index calls.

Noted in 5.6.6 changelog. 

Code for the storage engine API did not check the return value from 
the ha_rnd_init() and ha_index_init() functions.