Bug #41399 Maria: mutex deadlock detection warning LOCK_audit_mask LOCK_plugin

Description:
safe_mutex: Found wrong usage of mutex 'LOCK_audit_mask' and 'LOCK_plugin'
Mutex currently locked (in reverse order):
LOCK_plugin                       sql_plugin.cc  line 1778
LOCK_audit_mask                   sql_audit.cc  line 346

in 6.0-maria pushbuild (see URL in private post)

How to repeat:
pushbuild. Can't repeat on my computer.

Suggested fix:
find out where the bug is

This happened right after merging the mutex deadlock detector into 6.0-maria.
Warning must be removed for us to be able to merge into 6.0-main, we must:
- either disable detection for those mutexes
- or fix the problem

So, here's the scenario.

1) when NULL_AUDIT plugin is installed (like with 'INSTALL PLUGIN NULL_AUDIT SONAME 'adt_null.so', which I used in my machine, though in debx86-b it probably is automatically installed at startup, see the "configure" line for machine debx86-b: it says "checking whether to use NULL Audit Plug-in... yes"):

Breakpoint 3, initialize_audit_plugin (plugin=0x98c4550) at sql_audit.cc:297
297       pthread_mutex_lock(&LOCK_audit_mask);
(gdb) bt
#0  initialize_audit_plugin (plugin=0x98c4550) at sql_audit.cc:297
#1  0x084e8df0 in plugin_initialize (plugin=0x98c4550) at sql_plugin.cc:1006
#2  0x084e9135 in mysql_install_plugin (thd=0x97c0700, name=0x97c1ea4, 
    dl=0x97c1eac) at sql_plugin.cc:1660
#3  0x083154a4 in mysql_execute_command (thd=0x97c0700) at sql_parse.cc:4565
#4  0x08315c3d in mysql_parse (thd=0x97c0700, 
    inBuf=0x98b90d8 "INSTALL PLUGIN NULL_AUDIT SONAME 'adt_null.so'", 
    length=46, found_semicolon=0xaf943e80) at sql_parse.cc:5737
#5  0x08316bd2 in dispatch_command (command=COM_QUERY, thd=0x97c0700, 
    packet=0x98988c9 "INSTALL PLUGIN NULL_AUDIT SONAME 'adt_null.so'", 
    packet_length=46) at sql_parse.cc:1006

i.e. LOCK_plugin was first locked (see assertion in plugin_initialize()) and then LOCK_audit_mask is locked.

But then, when the plugin is uninstalled (plugin_deinitialize()) (on my machine "UNINSTALL PLUGIN NULL_AUDIT"; in debx86-b it's at shutdown):

Breakpoint 4, finalize_audit_plugin (plugin=0x98c4550) at sql_audit.cc:346
346       pthread_mutex_lock(&LOCK_audit_mask);
(gdb) n
347       plugin_foreach(current_thd, calc_class_mask, MYSQL_AUDIT_PLUGIN,

and plugin_foreach is going to lock LOCK_plugin; which is the reverse order.

However, I could not provoke the deadlock itself, I suspect (after looking at debug trace) that it's because mysql_install_plugin() and mysql_uninstall_plugin() are serialized by a WRITE lock on the mysql.plugin table. That is, this WRITE lock actually makes the wrong mutex order non-harmful it seems.
It's for Monty to decide what he wants to do with that (disable detection?).

Btw the AUDIT_NULL plugin is in mysql-6.0-maria/plugin/audit_null/.libs/adt_null.so .
As I start mysqld with "mtr --start-and-exit" it wants plugins in a certain directory, here's how I place it:
cp /m/bzrrepos/mysql-6.0-maria/plugin/audit_null/.libs/adt_null.so /home/mysql_src/bzrrepos/mysql-6.0-maria/mysql-test/lib/mysql/plugin/adt_null.so

Taking this bug, as time for merge into 6.0-main is coming soon.
I'm disabling detection for one lock call, as it's safe (table lock protects).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/61519

2777 Guilhem Bichot	2008-12-12
      Two small fixes: 1) when server is shutting down, one can get errno 2013 so expect it in tests (actually,
      I'm taking what's already in wait_until_connected_again.inc)
      2) fix for BUG#41399 "Maria: mutex deadlock detection warning LOCK_audit_mask LOCK_plugin": this is a safe case
      where table lock serializes problematic cases, so we silence the detector here.

revision-id: guilhem@mysql.com-20081212150535-zmw4un0wil38oeno
Was 6.0-maria specific, not in 5.1-maria.

Pushed into 6.0.9-alpha  (revid:guilhem@mysql.com-20081212150535-zmw4un0wil38oeno) (version source revid:guilhem@mysql.com-20081213204800-0nubni3t4ihn4hv9) (pib:5)

Internal only. No documentation needed.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85290

3125 Luis Soares	2009-10-01
      Partial backport for BUG#41399, more precisely, the changes to 
      wait_until_disconnected.inc.

Pushed into 6.0.14-alpha (revid:alik@sun.com-20091027094604-9p7kplu1vd2cvcju) (version source revid:zhenxing.he@sun.com-20091026140226-uhnqejkyqx1aeilc) (merge vers: 6.0.14-alpha) (pib:13)

Internal only; no further documentation needed.

Pushed into 5.5.0-beta (revid:alik@sun.com-20091110093229-0bh5hix780cyeicl) (version source revid:alik@sun.com-20091027095744-rf45u3x3q5d1f5y0) (merge vers: 5.5.0-beta) (pib:13)

Internal only; no documentation needed.