Bug #39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

• Submitted: 2 Sep 2008 9:59
• Modified: 14 Nov 2008 14:13
• Reporter: Magnus Blåudd
• Status: Closed
• Category: MySQL Cluster: Cluster (NDB) storage engine
• Severity: S3 (Non-critical)
• Version: 6.2
• OS: Any
• Assigned to: Magnus Blåudd
• CPU Architecture: Any

[2 Sep 2008 9:59] Magnus Blåudd

Description:
While  the patch for "Bug #38609 Segfault in Logger::Log causes ndbd to hang indefinately" fixed the problem this time, it could potentially happen again.To avoid that we need to fix two things:

1. Segfault in Logger::log causes the 'handler_error' function to generate tracefile and print even more using Logger::log. Since the first call to Logger::log has locked the mutex, it will deadlock indefinitley.
 - Don't use g_eventLogger in the crash pat, but instead instantiate a new eventlogger on the stack and use that for the crash path.

2. Add ATRIBUTE_FORMAT directives to all sprintf like functions so that the compiler will tell when the format specifier used is wrong.

msvensson@pilot:~/mysql/6.2-bug38609$ bzr diff
=== modified file 'BUILD/SETUP.sh'
--- BUILD/SETUP.sh	2008-08-08 19:59:32 +0000
+++ BUILD/SETUP.sh	2008-08-09 09:01:54 +0000
@@ -95,7 +95,7 @@ if [ "x$warning_mode" != "xpedantic" ]; 
   warnings="$warnings -Wwrite-strings -Wunused-function -Wunused-label -Wunused-value -Wunused-variable"
 
   # Make "printf like format specifier warnings" into error
-  # warnings="$warnings -Werror=format"
+  warnings="$warnings -Werror=format"
 
 # For more warnings, uncomment the following line
 # warnings="$global_warnings -Wshadow"

=== modified file 'storage/ndb/src/kernel/vm/SimulatedBlock.hpp'
--- storage/ndb/src/kernel/vm/SimulatedBlock.hpp	2008-05-29 15:58:58 +0000
+++ storage/ndb/src/kernel/vm/SimulatedBlock.hpp	2008-08-09 09:04:15 +0000
@@ -393,8 +393,10 @@ protected:
   /**
    * General info event (sent to cluster log)
    */
-  void infoEvent(const char * msg, ...) const ;
-  void warningEvent(const char * msg, ...) const ;
+  void infoEvent(const char * msg, ...) const
+    ATTRIBUTE_FORMAT(printf, 2, 3);
+  void warningEvent(const char * msg, ...) const
+    ATTRIBUTE_FORMAT(printf, 2, 3);

How to repeat:
See bug#38609

Suggested fix:
See description

[23 Oct 2008 9:44] Magnus Blåudd

This is a follow up bug that serves as a placeholder so we remember to rewrite the ndbd so it does not hang again if it should get a similar segfault.

[24 Oct 2008 11:01] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/56990

2673 Magnus Svensson	2008-10-24
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

[6 Nov 2008 17:26] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu) (version source revid:[e-mail]-20081106163505-fydpp10qhh0c4rwa) (pib:5)

[6 Nov 2008 17:28] Bugs System

Pushed into 5.1.29-ndb-6.3.19  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu) (version source revid:[e-mail]-20081106164627-iqgph2ulgkb1h25o) (pib:5)

[6 Nov 2008 17:43] Bugs System

Pushed into 5.1.29-ndb-6.4.0  (revid:[e-mail]-20081024110037-l01cf88y6d1n3xuu) (version source revid:[e-mail]-20081106165259-m4m9n6y2kq9023tr) (pib:5)

[7 Nov 2008 8:01] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58143

2721 Magnus Svensson	2008-11-07
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2

[7 Nov 2008 8:03] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081107080133-v5irq5yguluvc68r) (version source revid:[e-mail]-20081107080133-v5irq5yguluvc68r) (pib:5)

[7 Nov 2008 9:12] Bugs System

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/58152

2722 Magnus Svensson	2008-11-07
      Bug#39180 Segfault in Logger::Log causes ndbd to hang indefinately - part2
       - Fix one more warning for "too many format specifiers"

[7 Nov 2008 11:04] Bugs System

Pushed into 5.1.29-ndb-6.3.19  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (version source revid:[e-mail]-20081107103331-07n7lk0xet5rq163) (pib:5)

[7 Nov 2008 11:05] Bugs System

Pushed into 5.1.29-ndb-6.2.17  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (version source revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (pib:5)

[7 Nov 2008 11:07] Bugs System

Pushed into 5.1.29-ndb-6.4.0  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (version source revid:[e-mail]-20081107110038-ua1v67kljsc7my1w) (pib:5)

[14 Nov 2008 12:28] Magnus Blåudd

Pushed to 6.2, 6.3 and 6.4

[14 Nov 2008 14:13] Jon Stephens

Documented bugfix in the NDB-6.2.17 and NDB 6.3.18 changelogs as follows:

        A segfault in Logger::Log caused ndbd to hang indefinitely. This fix
        improves on an earlier one for this issue, first made in MySQL Cluster
        NDB 6.2.16 and MySQL Cluster NDB 6.3.17. 

        See also Bug# 38609.

[12 Dec 2008 23:25] Bugs System

Pushed into 6.0.9-alpha  (revid:[e-mail]-20081107091143-ha8yihhvzbp1wwcu) (version source revid:[e-mail]-20081209185954-9svcixh2p5hsfi6w) (pib:5)