Bug #45243 crash on win in sql thread clear_tables_to_lock() -> free()
Submitted: 1 Jun 2009 12:28 Modified: 16 Sep 2009 9:35
Reporter: Andrei Elkin Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Row Based Replication ( RBR ) Severity:S3 (Non-critical)
Version:5.1-bugteam OS:Microsoft Windows
Assigned to: Davi Arnaut CPU Architecture:Any
Tags: rpl_row_basic_3innodb, test failure
Triage: Triaged: D1 (Critical)

[1 Jun 2009 12:28] Andrei Elkin
Description:
The crash is found on PB2.
In executing
  rpl.rpl_row_basic_3innodb

slave crashed at the end of processing one of two rows events (l.373 of the rpl_row_basic.test)

 --echo [expecting slave to replicate correctly]
  connection master;
  INSERT INTO t4 VALUES (1, "", 1);
  INSERT INTO t4 VALUES (2, repeat(_utf8'a', 128), 2);

Here is the stack:

kernel32!HeapFree
mysqld!free [f:\dd\vctools\crt_bld\self_x86\crt\src\free.c @ 110]
mysqld!Relay_log_info::clear_tables_to_lock [g:\pb2\build\sb_0-555450-1243608735.23\mysql-5.1.36-win-x86\sql\rpl_rli.cc @ 1207]
mysqld!Rows_log_event::do_apply_event [g:\pb2\build\sb_0-555450-1243608735.23\mysql-5.1.36-win-x86\sql\log_event.cc @ 7456]
mysqld!apply_event_and_update_pos [g:\pb2\build\sb_0-555450-1243608735.23\mysql-5.1.36-win-x86\sql\slave.cc @ 2047]
mysqld!exec_relay_log_event [g:\pb2\build\sb_0-555450-1243608735.23\mysql-5.1.36-win-x86\sql\slave.cc @ 2174]
mysqld!handle_slave_sql [g:\pb2\build\sb_0-555450-1243608735.23\mysql-5.1.36-win-x86\sql\slave.cc @ 2891]

How to repeat:
http://pb2.norway.sun.com/web.py?template=mysql_show_test_failure&test_failure_id=2183238
[29 Jun 2009 13:04] Zhenxing He
This may relate to bug#40796
[8 Jul 2009 7:33] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/78181

3013 Georgi Kodinov	2009-07-08
      Bug#38998, Bug#46029, Bug#45243, Bug#46030 making tests experimental
[8 Jul 2009 13:30] Bugs System
Pushed into 5.1.37 (revid:joro@sun.com-20090708131116-kyz8iotbum8w9yic) (version source revid:joro@sun.com-20090708073149-qxyfa5vkh1m0pvol) (merge vers: 5.1.37) (pib:11)
[9 Jul 2009 7:37] Bugs System
Pushed into 5.1.37 (revid:joro@sun.com-20090708131116-kyz8iotbum8w9yic) (version source revid:joro@sun.com-20090708073149-qxyfa5vkh1m0pvol) (merge vers: 5.1.37) (pib:11)
[10 Jul 2009 11:20] Bugs System
Pushed into 5.4.4-alpha (revid:anozdrin@bk-internal.mysql.com-20090710111017-bnh2cau84ug1hvei) (version source revid:joro@sun.com-20090708121727-rekm6n1iu4vmvcfa) (merge vers: 5.4.4-alpha) (pib:11)
[13 Aug 2009 20:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/80781

2788 Davi Arnaut	2009-08-13
      Bug#46013: rpl_extraColmaster_myisam fails on pb2
      Bug#45243: crash on win in sql thread clear_tables_to_lock() -> free()
      Bug#45242: crash on win in mysql_close() -> free()
      Bug#45238: rpl_slave_skip, rpl_change_master failed (lost connection) for STOP SLAVE
      Bug#46030: rpl_truncate_3innodb causes server crash on windows
      Bug#46014: rpl_stm_reset_slave crashes the server sporadically in pb2
      
      When killing a user session on the server, it's necessary to
      interrupt (notify) the thread associated with the session that
      the connection is being killed so that the thread is woken up
      if waiting for I/O. On a few platforms (Mac, Windows and HP-UX)
      where the SIGNAL_WITH_VIO_CLOSE flag is defined, this interruption
      procedure is to asynchronously close the underlying socket of
      the connection.
      
      In order to enable this schema, each connection serving thread
      registers its VIO (I/O interface) so that other threads can
      access it and close the connection. But only the owner thread of
      the VIO might delete it as to guarantee that other threads won't
      see freed memory (the thread unregisters the VIO before deleting
      it). A side note: closing the socket introduces a harmless race
      that might cause a thread attempt to read from a closed socket,
      but this is deemed acceptable.
      
      The problem is that this infrastructure was meant to only be used
      by server threads, but the slave I/O thread was registering the
      VIO of a mysql handle (a client API structure that represents a
      connection to another server instance) as a active connection of
      the thread. But under some circumstances such as network failures,
      the client API might destroy the VIO associated with a handle at
      will, yet the VIO wouldn't be properly unregistered. This could
      lead to accesses to freed data if a thread attempted to kill a
      slave I/O thread whose connection was already broken.
      
      There was a attempt to work around this by checking whether
      the socket was being interrupted, but this hack didn't work as
      intended due to the aforementioned race -- attempting to read
      from the socket would yield a "bad file descriptor" error.
      
      The solution is to add a hook to the client API that is called
      from the client code before the VIO of a handle is deleted.
      This hook allows the slave I/O thread to detach the active vio
      so it does not point to freed memory.
     @ server-tools/instance-manager/mysql_connection.cc
        Add stub method required for linking.
     @ sql-common/client.c
        Invoke hook.
     @ sql/client_settings.h
        Export hook.
     @ sql/slave.cc
        Introduce hook that clears the active VIO before it is freed
        by the client API.
[13 Aug 2009 21:07] Davi Arnaut
Queued to 5.0-bugteam
[13 Aug 2009 22:01] Davi Arnaut
The user visible effect is that a STOP SLAVE statement might lead to a crash on Windows or Mac.
[26 Aug 2009 13:45] Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[26 Aug 2009 13:46] Bugs System
Pushed into 5.1.37-ndb-6.3.27 (revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (version source revid:jonas@mysql.com-20090826105955-bkj027t47gfbamnc) (merge vers: 5.1.37-ndb-6.3.27) (pib:11)
[26 Aug 2009 13:48] Bugs System
Pushed into 5.1.37-ndb-6.2.19 (revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (version source revid:jonas@mysql.com-20090825194404-37rtosk049t9koc4) (merge vers: 5.1.37-ndb-6.2.19) (pib:11)
[27 Aug 2009 16:32] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090827163030-6o3kk6r2oua159hr) (version source revid:jonas@mysql.com-20090826132541-yablppc59e3yb54l) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[27 Aug 2009 16:46] Jon Stephens
Documented bugfix in the 5.1.37 changelog as follows:

        In some cases, a STOP SLAVE statement could cause the replication slave
        to crash. This issue was specific to MySQL on Windows or Macintosh
        platforms.

Set bug status to Patch Queued waiting for pushes to 5.0/5.4 trees.
[2 Sep 2009 10:24] Bugs System
Pushed into 5.0.86 (revid:joro@sun.com-20090902102337-n5rw8227wwp5cpx8) (version source revid:davi.arnaut@sun.com-20090813200720-utqy73cj0orcy80z) (merge vers: 5.0.86) (pib:11)
[2 Sep 2009 13:01] Jon Stephens
Bugfix also noted in 5.0.86 changelog.

Set status to Patch Pending, waiting for 5.4 push.
[2 Sep 2009 16:42] Bugs System
Pushed into 5.1.39 (revid:joro@sun.com-20090902154533-8actmfcsjfqovgsb) (version source revid:ramil@mysql.com-20090814091316-07dvnrvaj0th0th2) (merge vers: 5.1.38) (pib:11)
[3 Sep 2009 20:43] Jon Stephens
Now documented in the following changelogs: 5.0.86, NDB-6.2.19, NDB-6.3.27, NDB-7.0.8, 5.1.39 (should have documented for Cluster releases, not 5.1.37-main).

Set status to NDI, waiting for push to 5.4.
[14 Sep 2009 16:03] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (version source revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (merge vers: 5.4.4-alpha) (pib:11)
[16 Sep 2009 9:35] Jon Stephens
Also documented in the 5.4.4 changelog.

Closed.
[1 Oct 2009 5:58] Bugs System
Pushed into 5.1.39-ndb-6.3.28 (revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (version source revid:jonas@mysql.com-20091001055605-ap2kiaarr7p40mmv) (merge vers: 5.1.39-ndb-6.3.28) (pib:11)
[1 Oct 2009 7:25] Bugs System
Pushed into 5.1.39-ndb-7.0.9 (revid:jonas@mysql.com-20091001072547-kv17uu06hfjhgjay) (version source revid:jonas@mysql.com-20091001071652-irejtnumzbpsbgk2) (merge vers: 5.1.39-ndb-7.0.9) (pib:11)
[1 Oct 2009 13:25] Bugs System
Pushed into 5.1.39-ndb-7.1.0 (revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (version source revid:jonas@mysql.com-20091001123013-g9ob2tsyctpw6zs0) (merge vers: 5.1.39-ndb-7.1.0) (pib:11)
[2 Oct 2009 0:04] Paul Dubois
Moved 5.4 changelog entry from 5.4.4 to 5.4.3.
[2 Oct 2009 0:07] Paul Dubois
Moved 5.4 changelog entry from 5.4.4 to 5.4.3.
[5 Oct 2009 10:50] Bugs System
Pushed into 5.1.39-ndb-6.2.19 (revid:jonas@mysql.com-20091005103850-dwij2dojwpvf5hi6) (version source revid:jonas@mysql.com-20090930185117-bhud4ek1y0hsj1nv) (merge vers: 5.1.39-ndb-6.2.19) (pib:11)