Bug #42345 server crashes if an engine fails to initalize
Submitted: 26 Jan 11:22 Modified: 29 Jan 18:07
Reporter: Sergey Vojtovich
Status: Duplicate
Category:Server: SE API Severity:S3 (Non-critical)
Version:5.1+ OS:Any
Assigned to: Ramil Kalimullin Target Version:5.1+
Triage: Triaged: D1 (Critical)

[26 Jan 11:22] Sergey Vojtovich
Description:
If an engine fails to initialize (during INSTALL PLUGIN or server startup), further
accesses to MyISAM tables (including grant tables) may result in server crash.

This is affecting all engines with init() function that may return non-zero status on
error. At least InnoDB, NDB and Falcon are pretty affected. Third party engines may be
affected as well.

Though it was discovered while fixing pure Falcon BUG#42275, it is also affecting
mysql-5.1.

Both risk and effort to fix are low.

The problem seem to be around handlerton to plugin mapping. Handlerton slot in
hton2plugin mapping is calculated after plugin init() function gets called. If init()
function fails, handlerton slot left untouched and holding 0 value (which is usually
MyISAM slot).

Later, when we call deinit() function, we're actually shutting down MyISAM and setting
it's slot to NULL.

How to repeat:
Trigger an engine init() function to return non-zero status. The easiest way to repeat
this is to modify some trivial storage engine to always return non-zero.

E.g. always return 1 from blackhole_init() in ha_blackhole.cc and start the server (if an
engine is compiled-in or installed) or issue INSTALL PLUGIN statement.

Suggested fix:
This probably may be fixed by the following patch:

=== modified file 'sql/handler.cc'
--- sql/handler.cc	2008-12-10 20:14:50 +0000
+++ sql/handler.cc	2009-01-26 10:22:51 +0000
@@ -433,6 +433,8 @@
   {
     if (plugin->plugin->init(hton))
     {
+      my_free((uchar*) hton, MYF(0));
+      plugin->data= NULL;
       sql_print_error("Plugin '%s' init function returned error.",
                       plugin->name.str);
       goto err;
[26 Jan 12:04] Miguel Solorzano
Thank you for the bug report.
[29 Jan 18:07] Ramil Kalimullin
Duplicate, see bug#40757: "Starting server on Windows with innodb_flush_method=wrong_value
causes crash"