Bug #45794 GCOV testing crashed Data Node hitting ndbrequire(state & Page_entry::HOT);
Submitted: 26 Jun 2009 19:16 Modified: 7 Dec 2009 12:00
Reporter: Jonathan Miller Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:mysql-5.1-telco-7.0 OS:Linux
Assigned to: Pekka Nousiainen CPU Architecture:Any

[26 Jun 2009 19:16] Jonathan Miller
Description:
During the initail run of the GCOV build some of the test failed. After investigation it was found that they we having a resource issue:

Error 1296 Got error 625 Out of memory increase IndexMemory

Looking at the default configurations, it seemed that the cluster configuration in include/default_ndbd.cnf where very low.

DataMemory=                    20M
IndexMemory=                   1M

So I increased them to:

DataMemory=                    60M
IndexMemory=                   10M

And restarted testing to see if this would correct the resource shortages.

ndb.ndb_dd_sql_features hit ndbrequire(state & Page_entry::HOT) in pgman.cpp.

Time: Friday 26 June 2009 - 19:04:22
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug)
Error: 2341
Error data: pgman.cpp
Error object: PGMAN (Line: 606) 0x00000002
Program: /data0/ACRT_CLONES/builds/clone-mysql-5.1-telco-7.0-2009-06-19.7237/storage/ndb/src/kernel/ndbd
Pid: 683
Trace: /data0/ACRT_CLONES/builds/clone-mysql-5.1-telco-7.0-2009-06-19.7237/mysql-test/var/mysql_cl

Code from pgman.cpp

  D("lirs_stack_pop");
  Page_stack& pl_stack = m_page_stack;
  Page_queue& pl_queue = m_page_queue;

  Ptr<Page_entry> ptr;
  bool ok = pl_stack.first(ptr);
  ndbrequire(ok);
  Page_state state = ptr.p->m_state;

  D(ptr << ": pop from stack");

  ndbrequire(state & Page_entry::HOT); <-- Failed Require
  ndbrequire(state & Page_entry::ONSTACK);
  pl_stack.remove(ptr);
  state &= ~ Page_entry::HOT;
  state &= ~ Page_entry::ONSTACK;
  ndbrequire(! (state & Page_entry::ONQUEUE));

I will upload trace and logs shortly.

How to repeat:
1) Compile using AMD-GCOV suite.
2) Change ndbd default setting as show above
3) Disable ndb_config :Defaults changed or it will fail
4) ./mysql-test-run.pl --suite=ndb
   or
   ./mysql-test-run.pl --do-test=ndb_dd_sql_features
[26 Jun 2009 19:18] Jonathan Miller
Trace Files

Attachment: ndb_dd_sql_features.tgz (application/x-compressed-tar, text), 189.35 KiB.

[19 Nov 2009 15:15] Pekka Nousiainen
see also bug#48910