Bug #39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

• Submitted: 2 Sep 2008 11:59
• Modified: 14 Nov 2008 15:13
• Reporter: Magnus Blaudd
• Status: Closed
• Category: Server: Cluster
• Severity: S3 (Non-critical)
• Version: 6.2
• OS: Any
• Assigned to: Magnus Blaudd
• Target Version: 6.2.17, 6.3.19, 6.4.0
• Triage: Triaged: D3 (Medium)

[2 Sep 2008 11:59] Magnus Blaudd

Description:
While  the patch for "Bug #38609 Segfault in Logger::Log causes ndbd to hang
indefinately" fixed the problem this time, it could potentially happen again.To avoid
that we need to fix two things:

1. Segfault in Logger::log causes the 'handler_error' function to generate tracefile and
print even more using Logger::log. Since the first call to Logger::log has locked the
mutex, it will deadlock indefinitley.
 - Don't use g_eventLogger in the crash pat, but instead instantiate a new eventlogger on
the stack and use that for the crash path.

2. Add ATRIBUTE_FORMAT directives to all sprintf like functions so that the compiler will
tell when the format specifier used is wrong.

msvensson@pilot:~/mysql/6.2-bug38609$ bzr diff
=== modified file 'BUILD/SETUP.sh'
--- BUILD/SETUP.sh	2008-08-08 19:59:32 +0000
+++ BUILD/SETUP.sh	2008-08-09 09:01:54 +0000
@@ -95,7 +95,7 @@ if [ "x$warning_mode" != "xpedantic" ]; 
   warnings="$warnings -Wwrite-strings -Wunused-function -Wunused-label -Wunused-value
-Wunused-variable"
 
   # Make "printf like format specifier warnings" into error
-  # warnings="$warnings -Werror=format"
+  warnings="$warnings -Werror=format"
 
 # For more warnings, uncomment the following line
 # warnings="$global_warnings -Wshadow"

=== modified file 'storage/ndb/src/kernel/vm/SimulatedBlock.hpp'
--- storage/ndb/src/kernel/vm/SimulatedBlock.hpp	2008-05-29 15:58:58 +0000
+++ storage/ndb/src/kernel/vm/SimulatedBlock.hpp	2008-08-09 09:04:15 +0000
@@ -393,8 +393,10 @@ protected:
   /**
    * General info event (sent to cluster log)
    */
-  void infoEvent(const char * msg, ...) const ;
-  void warningEvent(const char * msg, ...) const ;
+  void infoEvent(const char * msg, ...) const
+    ATTRIBUTE_FORMAT(printf, 2, 3);
+  void warningEvent(const char * msg, ...) const
+    ATTRIBUTE_FORMAT(printf, 2, 3);

How to repeat:
See bug#38609

Suggested fix:
See description

[23 Oct 2008 11:44] Magnus Blaudd

This is a follow up bug that serves as a placeholder so we remember to rewrite the ndbd so
it does not hang again if it should get a similar segfault.

[24 Oct 2008 13:01] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/56990

2673 Magnus Svensson	2008-10-24
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

[6 Nov 2008 18:26] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu)
(version source revid:[e-mail]-20081106163505-fydpp10qhh0c4rwa) (pib:5)

[6 Nov 2008 18:28] Bugs System

Pushed into 5.1.29-ndb-6.3.19  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu)
(version source revid:[e-mail]-20081106164627-iqgph2ulgkb1h25o) (pib:5)

[6 Nov 2008 18:43] Bugs System

Pushed into 5.1.29-ndb-6.4.0  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu)
(version source revid:[e-mail]-20081106165259-m4m9n6y2kq9023tr) (pib:5)

[7 Nov 2008 9:01] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58143

2721 Magnus Svensson	2008-11-07
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

[7 Nov 2008 9:03] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081107080133-v5irq5yguluvc68r)
(version source revid:[e-mail]-20081107080133-v5irq5yguluvc68r) (pib:5)

[7 Nov 2008 10:12] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58152

2722 Magnus Svensson	2008-11-07
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2
       - Fix one more warning for "too many format specifiers"

[7 Nov 2008 12:04] Bugs System

Pushed into 5.1.29-ndb-6.3.19  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu)
(version source revid:[e-mail]-20081107103331-07n7lk0xet5rq163) (pib:5)

[7 Nov 2008 12:05] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu)
(version source revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (pib:5)

[7 Nov 2008 12:07] Bugs System

Pushed into 5.1.29-ndb-6.4.0  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu)
(version source revid:[e-mail]-20081107110038-ua1v67kljsc7my1w) (pib:5)

[14 Nov 2008 13:28] Magnus Blaudd

Pushed to 6.2, 6.3 and 6.4

[14 Nov 2008 15:13] Jon Stephens

Documented bugfix in the NDB-6.2.17 and NDB 6.3.18 changelogs as follows:

        A segfault in Logger::Log caused ndbd to hang indefinitely. This fix
        improves on an earlier one for this issue, first made in MySQL Cluster
        NDB 6.2.16 and MySQL Cluster NDB 6.3.17. 

        See also Bug# 38609.

[13 Dec 2008 0:25] Bugs System

Pushed into 6.0.9-alpha  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu)
(version source revid:[e-mail]-20081209185954-9svcixh2p5hsfi6w) (pib:5)