| Bug #72215 | When LOCK_plugin conflicts very much, one uninstall-audit-plugin operation crash | ||
|---|---|---|---|
| Submitted: | 3 Apr 2014 3:13 | Modified: | 20 Apr 2014 6:46 |
| Reporter: | Hao Liu | Email Updates: | |
| Status: | Verified | Impact on me: | |
| Category: | MySQL Server: User-defined functions ( UDF ) | Severity: | S3 (Non-critical) |
| Version: | 5.6.16 | OS: | Linux |
| Assigned to: | CPU Architecture: | Any | |
[3 Apr 2014 3:19]
Hao Liu
the core backtrace:
#0 plugins_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:445
445 st_mysql_audit *data= plugin_data(plugin, struct st_mysql_audit *);
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64
(gdb) bt
#0 plugins_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:445
#1 event_class_dispatch (thd=0x199ade0, event_subtype=<value optimized out>, ap=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:490
#2 general_class_handler(THD *, uint, typedef __va_list_tag __va_list_tag *) (thd=0x199ade0,
event_subtype=<value optimized out>, ap=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:90
#3 0x0000000000679221 in mysql_audit_notify (thd=0x199ade0, event_class=0, event_subtype=2)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.cc:215
#4 0x00000000006d46db in mysql_audit_general (command=<value optimized out>, thd=0x199ade0,
packet=<value optimized out>, packet_length=1074791817) at /u01/xianlin.lh/mysql-5.6.16/sql/sql_audit.h:177
#5 dispatch_command (command=<value optimized out>, thd=0x199ade0, packet=<value optimized out>,
packet_length=1074791817) at /u01/xianlin.lh/mysql-5.6.16/sql/sql_parse.cc:2000
#6 0x000000000069cfcd in do_handle_one_connection (thd_arg=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_connect.cc:982
#7 0x000000000069d102 in handle_one_connection (arg=<value optimized out>)
at /u01/xianlin.lh/mysql-5.6.16/sql/sql_connect.cc:898
#8 0x0000003c6e4077f1 in start_thread () from /lib64/libpthread.so.0
#9 0x0000003c6e0e570d in clone () from /lib64/libc.so.6
[5 Apr 2014 6:54]
MySQL Verification Team
Hello Liu, Thank you for the report. Verified as described. Thanks, Umesh
[20 Apr 2014 6:46]
Hao Liu
I thint it is very ugly to add a sleep function for reproducing the bug. So I add a test case using DEBUG_SYNC to reproduce it. The patch has also included the modification for fixing the bug.
[20 Apr 2014 6:47]
Hao Liu
a patch for the bug, including the test case and the fix
Attachment: plugin_uninstall_audit_core.patch (application/octet-stream, text), 2.97 KiB.
[24 Aug 2015 11:41]
Georgi Kodinov
Thank you very much for your analysis and proposed fix. But before I can process it I'd need the OCA agreement handled. Can you please use the "contributions" tab on the bug db or use a git pull request ?
[29 Nov 2017 11:04]
MySQL Verification Team
related: https://bugs.mysql.com/bug.php?id=88693
[20 Nov 3:43]
Nelson Fahey
The LOCK_plugin conflict sounds tricky. Simulating the race condition with usleep(10) is a clever approach. Seems like the install/uninstall process, particularly of the audit_null plugin, combined with the heavy load from sysbench, triggers the crash. Almost like navigating a difficult level in Slope Game, where one wrong move leads to a quick reset. https://slopegamefree.com/
Description: When LOCK_plugin conflicts very much, one uninstall-audit-plugin operation may crash. In order to simulate Lock_plugin conflict，I modify the reap_plugins function(sql/sql_plugin.cc: reap_plugins) to usleep(10) after mysql_mutex_assert_owner(&LOCK_plugin). As below: sql/sql_plugin.cc 982 static void reap_plugins(void) 983 { 984 uint count, idx; 985 struct st_plugin_int *plugin, **reap, **list; 986 987 mysql_mutex_assert_owner(&LOCK_plugin); 988 989 usleep(10); 990 if (!reap_needed) 991 return; Then I use 160 sysbench threads to select/update the server. Another session to install/uninstall audit_null plugin.The the mysqld crash when I uninstall audit_null plugin. How to repeat: 1) modify the reap_plugins function as above; 2) compile the mysqld; 3) use sysbench to select/update the server $ ./sysbench --num-threads=160 --test=tests/db/oltp.lua --oltp-reconnect-mode=random --mysql-table-engine=innodb --mysql-db=test --oltp-table-size=200000 --mysql-user=root --mysql-socket=/u01/my3928/run/mysql.sock --max-requests=100000000 run 4) use another session to install/uninstall the audit_null plugin. Suggested fix: When insert_dynamic to thd->audit_class_plugins after my_plugin_lock， judge whether the plugin variale is null