MySQL Bugs: #46014: rpl_stm_reset_slave crashes the server sporadically in pb2

Bug #46014	rpl_stm_reset_slave crashes the server sporadically in pb2
Submitted:	7 Jul 2009 14:34	Modified:	16 Sep 2009 9:37
Reporter:	Georgi Kodinov	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Replication	Severity:	S3 (Non-critical)
Version:	5.1-bugteam	OS:	Windows (test-max-win_ws2008-x86)
Assigned to:	Davi Arnaut	CPU Architecture:	Any

Description:
rpl_stm_reset_slave causes crashes like the following :

rpl.rpl_stm_reset_slave                  [ fail ]
        Test ended at 2009-07-07 12:06:05

CURRENT_TEST: rpl.rpl_stm_reset_slave
mysqltest: In included file ".\extra\rpl_tests\rpl_reset_slave.test": At line 39: query 'stop slave' failed: 2013: Lost connection to MySQL server during query

How to repeat:
run the test suite enough times on windows

Suggested fix:
n/a

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

http://lists.mysql.com/commits/80781

2788 Davi Arnaut 2009-08-13
Bug#46013: rpl_extraColmaster_myisam fails on pb2
Bug#45243: crash on win in sql thread clear_tables_to_lock() -> free()
Bug#45242: crash on win in mysql_close() -> free()
Bug#45238: rpl_slave_skip, rpl_change_master failed (lost connection) for STOP SLAVE
Bug#46030: rpl_truncate_3innodb causes server crash on windows
Bug#46014: rpl_stm_reset_slave crashes the server sporadically in pb2

When killing a user session on the server, it's necessary to
interrupt (notify) the thread associated with the session that
the connection is being killed so that the thread is woken up
if waiting for I/O. On a few platforms (Mac, Windows and HP-UX)
where the SIGNAL_WITH_VIO_CLOSE flag is defined, this interruption
procedure is to asynchronously close the underlying socket of
the connection.

In order to enable this schema, each connection serving thread
registers its VIO (I/O interface) so that other threads can
access it and close the connection. But only the owner thread of
the VIO might delete it as to guarantee that other threads won't
see freed memory (the thread unregisters the VIO before deleting
it). A side note: closing the socket introduces a harmless race
that might cause a thread attempt to read from a closed socket,
but this is deemed acceptable.

The problem is that this infrastructure was meant to only be used
by server threads, but the slave I/O thread was registering the
VIO of a mysql handle (a client API structure that represents a
connection to another server instance) as a active connection of
the thread. But under some circumstances such as network failures,
the client API might destroy the VIO associated with a handle at
will, yet the VIO wouldn't be properly unregistered. This could
lead to accesses to freed data if a thread attempted to kill a
slave I/O thread whose connection was already broken.

There was a attempt to work around this by checking whether
the socket was being interrupted, but this hack didn't work as
intended due to the aforementioned race -- attempting to read
from the socket would yield a "bad file descriptor" error.

The solution is to add a hook to the client API that is called
from the client code before the VIO of a handle is deleted.
This hook allows the slave I/O thread to detach the active vio
so it does not point to freed memory.
@ server-tools/instance-manager/mysql_connection.cc
Add stub method required for linking.
@ sql-common/client.c
Invoke hook.
@ sql/client_settings.h
Export hook.
@ sql/slave.cc
Introduce hook that clears the active VIO before it is freed
by the client API.

Queued to 5.0-bugteam

The user visible effect is that a STOP SLAVE statement might lead to a crash on Windows or Mac.

See BUG#45243 for documentation info.

Pushed into 5.0.86 (revid:joro@sun.com-20090902102337-n5rw8227wwp5cpx8) (version source revid:davi.arnaut@sun.com-20090813200720-utqy73cj0orcy80z) (merge vers: 5.0.86) (pib:11)

Bugfix also noted in 5.0.86 changelog.

Set status to Patch Pending, waiting for 5.4 push.

Pushed into 5.1.39 (revid:joro@sun.com-20090902154533-8actmfcsjfqovgsb) (version source revid:ramil@mysql.com-20090814091316-07dvnrvaj0th0th2) (merge vers: 5.1.38) (pib:11)

Now documented in the following changelogs: 5.0.86, NDB-6.2.19, NDB-6.3.27, NDB-7.0.8, 5.1.39 (should have documented for Cluster releases, not 5.1.37-main).

Set status to NDI, waiting for push to 5.4.

Pushed into 5.4.4-alpha (revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (version source revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (merge vers: 5.4.4-alpha) (pib:11)

Bugfix also documented in the 5.4.4 changelog.

Closed.

Pushed into 5.1.39-ndb-6.3.28 (revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (version source revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (merge vers: 5.1.39-ndb-6.3.28) (pib:11)

Pushed into 5.1.39-ndb-7.0.9 (revid:jonas@mysql.com-20091001072547-kv17uu06hfjhgjay) (version source revid:jonas@mysql.com-20091001071652-irejtnumzbpsbgk2) (merge vers: 5.1.39-ndb-7.0.9) (pib:11)

Pushed into 5.1.39-ndb-7.1.0 (revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (version source revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (merge vers: 5.1.39-ndb-7.1.0) (pib:11)

Moved 5.4 changelog entry from 5.4.4 to 5.4.3.

Pushed into 5.1.39-ndb-6.2.19 (revid:jonas@mysql.com-20091005103850-dwij2dojwpvf5hi6) (version source revid:jonas@mysql.com-20090930185117-bhud4ek1y0hsj1nv) (merge vers: 5.1.39-ndb-6.2.19) (pib:11)