Bug #38563 ndb_mgmd performs network I/O with client(s) while holding mutex
Submitted: 5 Aug 2008 11:50 Modified: 5 Oct 2008 18:29
Reporter: Magnus Blaudd
Status: Closed
Category:Server: Cluster Severity:S3 (Non-critical)
Version:6.2 OS:Any
Assigned to: Magnus Blaudd Target Version:6.2
Triage: Needs Triage: D2 (Serious)

[5 Aug 2008 11:50] Magnus Blaudd 
Description:
The ndb_mgmd will perform network I/O with the client while holding the MgmApiSesion mutex
and sometimes also the SocketServer::m_session_mutex. This in combination with not
checking the return value when printing to m_outputstream leads to long running threads
(that are writing to clients that no longer exist) holding mutexes in the ndb_mgmd.

How to repeat:
Run ./testMgm in a loop. Second loops will take really long time before the threads
disconnected(and thus hanging) in the first loop will timeout and relase the mutex so that
other threads may join.

Suggested fix:
1. Use BufferedOutputStream that will buffer the result and write it when the mutexes has
been released.
2. Avoid locking the m_session_mutex by taking a copy of the session pointers and release
the m_session_mutex while calling the callback function passed to
'SocketServer::foreachSession'
[5 Aug 2008 13:54] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50918

2637 Magnus Svensson	2008-08-05
      Bug#38563 ndb_mgmd performs network I/O with client(s) while holding mutex
[5 Aug 2008 13:59] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50919

2645 Magnus Svensson	2008-08-05 [merge]
      Merge bug#38563
[5 Aug 2008 14:34] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/50923

2697 Magnus Svensson	2008-08-05 [merge]
      Merge bug#38563
[11 Sep 2008 21:17] Jon Stephens 
Documented bugfix in the NDB 6.2.16 and 6.3,17 changelogs as follows:

        ndb_mgmd performed unnecessary network I/O with the client. This in
        combination with other factors led to long-running threads that were
        attempting to write to clients that no longer existed.
[5 Oct 2008 18:29] Jon Stephens 
Already documented for NDB 6.2/6.3; closed.
[13 Dec 2008 0:29] Bugs System 
Pushed into 6.0.6-alpha  (revid:msvensson@mysql.com-20080805115352-uyhcspxinmbgp1kb)
(version source revid:jonas@mysql.com-20080813092004-7zlf6eu87i4ziwm2) (pib:5)