Bug #46113 ndbd fails to start when built with gcc-4.4.0 (e.g Fedora 11)
Submitted: 10 Jul 2009 15:20 Modified: 10 Oct 2009 17:19
Reporter: Andrew Morgan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-7.0 OS:Linux (Fedora 11)
Assigned to: Magnus Blåudd CPU Architecture:Any
Tags: 7.0.6, cluster, Fedora, Fedora 11, gcc, gcc 4.4.0, ndbd
Triage: Triaged: D3 (Medium) / R6 (Needs Assessment) / E2 (Low)

[10 Jul 2009 15:20] Andrew Morgan
Description:
I've built 7.0.6 from source and installed it with...

autoreconf --force --install
./configure --with-plugins=max --prefix=/usr/local/mysql make make install scripts/mysql_install_db --datadir=/usr/local/mysql/data

I then start the management node successfully but the ndbd fails to start correctly.

ndb_mgmd --initial -c conf/config.ini
ndbd --initial -c localhost:1186

I've tried this several times on 2 different machines (each running Fedora 11).

Config and log files located at:
www.clusterdb.com/Fedora11/config.ini 
www.clusterdb.com/Fedora11/ndb_1_cluster.log
www.clusterdb.com/Fedora11/ndb_1_out.log
www.clusterdb.com/Fedora11/ndb_2_error.log
www.clusterdb.com/Fedora11/ndb_2_out.log

How to repeat:
As described above (on a host using Fedora 11).

Suggested fix:
Suspect the problem is with the fact that Fedora 11 contains gcc 4.4.0
[10 Jul 2009 19:55] Sveta Smirnova
Thank you for the report.

Failure to start verified on RedHat 5, although logs in my case are partially same to logs provided.

To repeat: `./mtr --suite=ndb`
[16 Jul 2009 12:19] Magnus Blåudd
This problems occur with gcc 4.4.0 where behaviour has changed according to the changelog "G++ now properly implements value-initialization, so objects with     an initializer of () and an implicitly defined default constructor will be zero-initialized before the default constructor is called."

Crash occurs in DbdihInit, where we first put an object in a list and then run the default constructor which "properly" overwrites the member variables used for the list with zeroes. :(

The below patch 

pilot:~/mysql/7.0/storage/ndb/src/kernel/blocks$ bzr diff dbdih/
=== modified file 'storage/ndb/src/kernel/blocks/dbdih/DbdihInit.cpp'
--- storage/ndb/src/kernel/blocks/dbdih/DbdihInit.cpp	2009-05-27 15:21:45 +0000
+++ storage/ndb/src/kernel/blocks/dbdih/DbdihInit.cpp	2009-07-16 12:16:05 +0000
@@ -48,7 +48,7 @@ void Dbdih::initData() 
     Ptr<TakeOverRecord> ptr;
     while (c_activeTakeOverList.seize(ptr))
     {
-      new (ptr.p) TakeOverRecord();
+      new (ptr.p) TakeOverRecord;
     }
     while (c_activeTakeOverList.first(ptr))
     {

We should probably change "tactic"  and avoid code like this in the future.
[17 Jul 2009 7:40] Jonas Oreland
is it possible to add a warning-flag, so we can find-out if there are more occurrences like this
[17 Jul 2009 7:48] Magnus Blåudd
Couldn't find flag. But I agree, we should try to autodetect this somehow...
[8 Sep 2009 6:50] Sveta Smirnova
Bug #46113 was marked as duplicate of this one
[9 Sep 2009 8:23] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/82769
[9 Sep 2009 8:58] Bugs System
Pushed into 5.1.37-ndb-7.0.8 (revid:magnus.blaudd@sun.com-20090909084945-5huc5c74gyrzxa7p) (version source revid:magnus.blaudd@sun.com-20090909084945-5huc5c74gyrzxa7p) (merge vers: 5.1.37-ndb-7.0.8) (pib:11)
[9 Sep 2009 9:07] Bugs System
Pushed into 5.1.35-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20090909085908-tq7g6xkftkkqcjuv) (version source revid:magnus.blaudd@sun.com-20090909085908-tq7g6xkftkkqcjuv) (merge vers: 5.1.35-ndb-7.1.0) (pib:11)
[9 Sep 2009 9:25] Magnus Blåudd
Pushed to 6.3, 7.0 and 7.1
[9 Sep 2009 13:58] Jon Stephens
Documented bugfix in the NDB-6.3.27 and 7.0.8 changelogs as follows:

        ndbd was not built correctly when compiled using gcc 4.4.0. (The
        ndbd binary was built, but could not be started.)

Closed.
[11 Sep 2009 14:24] Magnus Blåudd
Just found more problems when release compiling.
[11 Sep 2009 14:37] Jon Stephens
Docs note: Removed (commented out) changelog entry until we have a new fix, per IRC discussion with Magnus.
[6 Oct 2009 20:55] Magnus Blåudd
Corruption in DbUtil::initResultSet causes failure to create event operation when keyinfo is overwritten.

Can be detected in any version when by checking return value from rs.next like below.

=== modified file 'storage/ndb/src/kernel/blocks/dbutil/DbUtil.cpp'
--- storage/ndb/src/kernel/blocks/dbutil/DbUtil.cpp	2009-08-05 11:05:34 +0000
+++ storage/ndb/src/kernel/blocks/dbutil/DbUtil.cpp	2009-10-06 20:49:37 +0000
@@ -2323,7 +2323,7 @@
 	   rsit.curr.i,
 	   ((AttributeHeader*)&rsit.data[0])->getDataSize() + 1);
 #endif
-    rs.next(rsit, ((AttributeHeader*)&rsit.data[0])->getDataSize() + 1);
+    ndbrequire(rs.next(rsit, ((AttributeHeader*)&rsit.data[0])->getDataSize() + 1));
   }
 }
[7 Oct 2009 14:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86020
[7 Oct 2009 14:19] Bugs System
Pushed into 5.1.39-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20091007141857-8l1yifui7nmvb8wf) (version source revid:magnus.blaudd@sun.com-20091007141857-8l1yifui7nmvb8wf) (merge vers: 5.1.39-ndb-7.1.0) (pib:11)
[8 Oct 2009 9:56] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86115
[8 Oct 2009 11:42] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86139
[8 Oct 2009 12:41] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86165
[8 Oct 2009 13:43] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86194
[9 Oct 2009 14:13] Bugs System
Pushed into 5.1.39-ndb-6.3.28 (revid:magnus.blaudd@sun.com-20091009130938-s7tv5vj7f842wo9g) (version source revid:magnus.blaudd@sun.com-20091009090956-4lkn0peii3linumx) (merge vers: 5.1.39-ndb-6.3.28) (pib:13)
[9 Oct 2009 14:20] Bugs System
Pushed into 5.1.39-ndb-7.0.9 (revid:magnus.blaudd@sun.com-20091009141821-rrrm9xm2j8f3hstt) (version source revid:magnus.blaudd@sun.com-20091009141821-rrrm9xm2j8f3hstt) (merge vers: 5.1.39-ndb-7.0.9) (pib:13)
[9 Oct 2009 14:27] Bugs System
Pushed into 5.1.39-ndb-7.1.0 (revid:magnus.blaudd@sun.com-20091009142206-4g25a32akb8t3o5u) (version source revid:magnus.blaudd@sun.com-20091009142206-4g25a32akb8t3o5u) (merge vers: 5.1.39-ndb-7.1.0) (pib:13)
[10 Oct 2009 17:19] Jon Stephens
Changelog entry re-enabled, added to NDB-6.3.28 and 7.0.9 changelogs. Closed.
[15 Oct 2009 15:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/87008

3018 Martin Skold	2009-10-15 [merge]
      Merge
      modified:
        config/ac-macros/ha_ndbcluster.m4
        mysql-test/suite/ndb/my.cnf
        mysql-test/suite/ndb/r/ndb_config.result
        mysql-test/suite/ndb_binlog/r/ndb_binlog_variants.result
        mysql-test/suite/ndb_binlog/t/ndb_binlog_variants.test
        mysql-test/suite/ndb_team/r/ndb_dd_backuprestore.result
        sql/ha_ndbcluster.cc
        sql/ha_ndbcluster.h
        storage/ndb/include/kernel/signaldata/ContinueFragmented.hpp
        storage/ndb/include/kernel/signaldata/DumpStateOrd.hpp
        storage/ndb/include/kernel/signaldata/NodeFailRep.hpp
        storage/ndb/include/ndb_global.h.in
        storage/ndb/include/ndbapi/NdbDictionary.hpp
        storage/ndb/include/ndbapi/NdbOperation.hpp
        storage/ndb/ndbapi-examples/ndbapi_scan/ndbapi_scan.cpp
        storage/ndb/src/kernel/blocks/backup/Backup.cpp
        storage/ndb/src/kernel/blocks/backup/Backup.hpp
        storage/ndb/src/kernel/blocks/cmvmi/Cmvmi.cpp
        storage/ndb/src/kernel/blocks/cmvmi/Cmvmi.hpp
        storage/ndb/src/kernel/blocks/dbdict/Dbdict.cpp
        storage/ndb/src/kernel/blocks/dbdict/Dbdict.hpp
        storage/ndb/src/kernel/blocks/dbdict/printSchemaFile.cpp
        storage/ndb/src/kernel/blocks/dbdih/printSysfile.cpp
        storage/ndb/src/kernel/blocks/dbdih/printSysfile/printSysfile.cpp
        storage/ndb/src/kernel/blocks/dblqh/DblqhMain.cpp
        storage/ndb/src/kernel/blocks/dblqh/redoLogReader/reader.cpp
        storage/ndb/src/kernel/blocks/dbtc/Dbtc.hpp
        storage/ndb/src/kernel/blocks/dbtc/DbtcMain.cpp
        storage/ndb/src/kernel/blocks/dbtup/Dbtup.hpp
        storage/ndb/src/kernel/blocks/dbtup/DbtupGen.cpp
        storage/ndb/src/kernel/blocks/dbtup/tuppage.hpp
        storage/ndb/src/kernel/blocks/dbtux/Dbtux.hpp
        storage/ndb/src/kernel/blocks/dbutil/DbUtil.cpp
        storage/ndb/src/kernel/blocks/dbutil/DbUtil.hpp
        storage/ndb/src/kernel/blocks/lgman.cpp
        storage/ndb/src/kernel/blocks/lgman.hpp
        storage/ndb/src/kernel/blocks/ndbcntr/NdbcntrMain.cpp
        storage/ndb/src/kernel/blocks/qmgr/QmgrMain.cpp
        storage/ndb/src/kernel/blocks/suma/Suma.cpp
        storage/ndb/src/kernel/blocks/suma/Suma.hpp
        storage/ndb/src/kernel/blocks/trix/Trix.hpp
        storage/ndb/src/kernel/blocks/tsman.cpp
        storage/ndb/src/kernel/blocks/tsman.hpp
        storage/ndb/src/kernel/vm/DLFifoList.hpp
        storage/ndb/src/kernel/vm/DLHashTable.hpp
        storage/ndb/src/kernel/vm/DLList.hpp
        storage/ndb/src/kernel/vm/DataBuffer.hpp
        storage/ndb/src/kernel/vm/SimulatedBlock.cpp
        storage/ndb/src/kernel/vm/SimulatedBlock.hpp
        storage/ndb/src/mgmapi/LocalConfig.cpp
        storage/ndb/src/mgmapi/Makefile.am
        storage/ndb/src/mgmsrv/ConfigInfo.cpp
        storage/ndb/src/ndbapi/NdbDictionary.cpp
        storage/ndb/src/ndbapi/NdbOperation.cpp
        storage/ndb/test/ndbapi/testNdbApi.cpp
        storage/ndb/test/run-test/daily-basic-tests.txt