| Bug #73030 | Cluster goes down with Error 2341 | ||
|---|---|---|---|
| Submitted: | 18 Jun 2014 8:23 | Modified: | 17 Nov 2014 8:23 |
| Reporter: | erkan yanar | Email Updates: | |
| Status: | Duplicate | Impact on me: | |
| Category: | MySQL Cluster: Cluster (NDB) storage engine | Severity: | S2 (Serious) |
| Version: | mysql-5.6.15 ndb-7.3.4 | OS: | Linux |
| Assigned to: | CPU Architecture: | Any | |
[18 Jun 2014 8:24]
erkan yanar
Errorlog of node3
Attachment: ndb_3_error.log (text/x-log), 3.97 KiB.
[18 Jun 2014 8:25]
erkan yanar
compressed log
Attachment: ndb_3_out.log.gz (application/gzip, text), 294.60 KiB.
[18 Jun 2014 8:26]
erkan yanar
Trace log
Attachment: ndb_3_trace.log.8 (application/octet-stream, text), 1015.03 KiB.
[18 Jun 2014 8:26]
erkan yanar
mgmd log
Attachment: ndb_1_cluster.log (text/x-log), 952.34 KiB.
[17 Nov 2014 8:20]
MySQL Verification Team
Thank you for the report. Colleague of mine confirmed that this is duplicate of Bug #56929 Thanks, Umesh
Description: We have a cluster failing. From the ndbd error log: Time: Monday 19 May 2014 - 06:04:46 Status: Temporary error, restart node Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug) Error: 2341 Error data: DbtupTrigger.cpp Error object: DBTUP (Line: 2184) 0x00000002 Program: ndbd Pid: 35713 Version: mysql-5.6.15 ndb-7.3.4 Trace: /usr/local/mysql/data/ndb_3_trace.log.8 [t1..t1] As the second ndbd fails with the same error Looking for the trace log I ended looking into DbtuxMaint.cpp 133 treeAdd(c_ctx, frag, treePos, ent); 134 frag.m_entryCount++; 135 frag.m_entryBytes += searchKey.get_data_len(); 136 frag.m_entryOps++; 137 break; 138 case TuxMaintReq::OpRemove: 139 jam(); 140 ok = searchToRemove(c_ctx, frag, searchKey, ent, treePos); 141 #ifdef VM_TRACE 142 if (debugFlags & DebugMaint) { 143 debugOut << treePos << (! ok ? " - error" : "") << endl; 144 } 145 #endif 146 if (! ok) { 147 jam(); 148 // there is no "Building" state so this will have to do 149 if (indexPtr.p->m_state == Index::Online) { 150 jam(); 151 req->errorCode = TuxMaintReq::SearchError; 152 } 153 break; 154 } So there is another state than online. I.e. Building, Defining, Not Defined etc. But this is not helping in any way. (at least for me) How to repeat: As we even don't know what triggers the shutdown, there is no way to find out how to reproduce it.