Bug #72215 When LOCK_plugin conflicts very much, one uninstall-audit-plugin operation crash
Submitted: 3 Apr 2014 3:13 Modified: 20 Apr 2014 6:46
Reporter: Hao Liu Email Updates:
Status: Verified Impact on me:
None 
Category:MySQL Server: User-defined functions ( UDF ) Severity:S3 (Non-critical)
Version:5.6.16 OS:Linux
Assigned to: CPU Architecture:Any

[3 Apr 2014 3:13] Hao Liu
Description:
When LOCK_plugin conflicts very much, one uninstall-audit-plugin operation may crash. In order to simulate Lock_plugin conflict,I modify the reap_plugins function(sql/sql_plugin.cc: reap_plugins) to usleep(10) after mysql_mutex_assert_owner(&LOCK_plugin). As below:

 sql/sql_plugin.cc
 982 static void reap_plugins(void)
 983 {
 984   uint count, idx;
 985   struct st_plugin_int *plugin, **reap, **list;
 986
 987   mysql_mutex_assert_owner(&LOCK_plugin);
 988
 989   usleep(10);
 990   if (!reap_needed)
 991     return;

Then I use 160 sysbench threads to select/update the server. Another session to install/uninstall audit_null plugin.The the mysqld crash when I uninstall audit_null plugin.

How to repeat:
1) modify the reap_plugins function as above;
2) compile the mysqld;
3) use sysbench to select/update the server
$ ./sysbench --num-threads=160 --test=tests/db/oltp.lua --oltp-reconnect-mode=random --mysql-table-engine=innodb --mysql-db=test --oltp-table-size=200000 --mysql-user=root --mysql-socket=/u01/my3928/run/mysql.sock --max-requests=100000000 run
4) use another session to install/uninstall the audit_null plugin.

Suggested fix:
When insert_dynamic to thd->audit_class_plugins after my_plugin_lock, judge whether the plugin variale is null
[3 Apr 2014 3:19] Hao Liu
the core backtrace:

#0  plugins_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:445
445       st_mysql_audit *data= plugin_data(plugin, struct st_mysql_audit *);
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
(gdb) bt
#0  plugins_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:445
#1  event_class_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:490
#2  general_class_handler(THD *, uint, typedef __va_list_tag __va_list_tag *) (thd=0x199ade0,
    event_subtype=<value optimized out>, ap=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:90
#3  0x0000000000679221 in mysql_audit_notify (thd=0x199ade0, event_class=0, event_subtype=2)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:215
#4  0x00000000006d46db in mysql_audit_general (command=<value optimized out>, thd=0x199ade0,
    packet=<value optimized out>, packet_length=1074791817) at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.h:177
#5  dispatch_command (command=<value optimized out>, thd=0x199ade0, packet=<value optimized out>,
    packet_length=1074791817) at /u01/xianlin.lh/mysql-5.6.16/sql/sql_parse.cc:2000
#6  0x000000000069cfcd in do_handle_one_connection (thd_arg=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_connect.cc:982
#7  0x000000000069d102 in handle_one_connection (arg=<value optimized out>)
    at /u01/xianlin.lh/mysql-5.6.16/sql/sql_connect.cc:898
#8  0x0000003c6e4077f1 in start_thread () from /lib64/libpthread.so.0
#9  0x0000003c6e0e570d in clone () from /lib64/libc.so.6
[5 Apr 2014 6:54] MySQL Verification Team
Hello Liu,

Thank you for the report.
Verified as described.

Thanks,
Umesh
[20 Apr 2014 6:46] Hao Liu
I thint it is very ugly to add a sleep function for reproducing the bug. So I add a test case using DEBUG_SYNC to reproduce it. The patch has also included the modification for fixing the bug.
[20 Apr 2014 6:47] Hao Liu
a patch for the bug, including the test case and the fix

Attachment: plugin_uninstall_audit_core.patch (application/octet-stream, text), 2.97 KiB.

[24 Aug 2015 11:41] Georgi Kodinov
Thank you very much for your analysis and proposed fix.
But before I can process it I'd need the OCA agreement handled. 
Can you please use the "contributions" tab on the bug db or use a git pull request ?
[29 Nov 2017 11:04] MySQL Verification Team
related:
https://bugs.mysql.com/bug.php?id=88693