Bug #9389 NDB start/stop failure (also causing subsequent test failures)
Submitted: 24 Mar 2005 18:28 Modified: 13 Oct 2005 14:34
Reporter: Joerg Bruehe Email Updates:
Status: No Feedback Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:5.0.3 and up OS:Some Unix (shifting)
Assigned to: Assigned Account CPU Architecture:Any

[24 Mar 2005 18:28] Joerg Bruehe
Description:
Final release build based on ChangeSet
  1.1847.1.1 05/03/22 21:52:01 jimw@mysql.com +11 -0
  Hack to ensure that CHAR's created in 5.0 are not converted to VARCHAR even if
  the row type is dynamic (For 4.1 tables old 'VARCHAR' fields are converted to
  true VARCHAR in the next ALTER TABLE)

  This ensures that one can use MySQL 5.0 privilege tables with MySQL 4.1.

  (Fix imported from main 5.0 tree to 5.0.3 build tree.)

Problem occurs on 'hpux11' and on 'sol10' (x86), on both only in "default" test run whereas '--ps-protocol' passes (on both). No tree saved yet, as the build progresses.

Initial failure symptom:

ndb_autodiscover               [ fail ]

Errors are (from /var/tmp/mysqldev/hpux11/test/mysql-max-5.0.3-beta-hp-hpux11.11-hppa2.0w/mysql-test/var/log/mysqltest-time) :
waitNodeState(NOT_STARTED, -1) timeout after 121 attemps
/var/tmp/mysqldev/hpux11/test/mysql-max-5.0.3-beta-hp-hpux11.11-hppa2.0w/bin/mysqltest: command "$NDB_TOOLS_DIR/ndb_waiter --no-defaults --not-started > /dev/null" failed
(the last lines may be the most important ones)

Subsequent test failures which I attribute to the initial problem:

ndb_autodiscover2              [ fail ]

Errors are (from /export/home/mysqldev/sol10/test/mysql-max-5.0.3-beta-pc-solaris2.10-i386/mysql-test/var/log/mysqltest-time) :
/export/home/mysqldev/sol10/test/mysql-max-5.0.3-beta-pc-solaris2.10-i386/bin/mysqltest: At line 9: query 'select * from t9 order by a' failed: 1146: Table 'test.t9' doesn't exist
(the last lines may be the most important ones)

*****

-------------------------------------------------------
*** r/ps_1general.result        Tue Mar 22 22:46:46 2005
--- r/ps_1general.reject        Wed Mar 23 18:17:37 2005
***************
*** 261,266 ****
--- 261,267 ----
  information_schema
  mysql
  test
+ test_only_ndb_tables
  prepare stmt4 from ' show tables from test like ''t2%'' ';
  execute stmt4;
  Tables_in_test (t2%)
-------------------------------------------------------

*****

-------------------------------------------------------
*** r/rpl000009.result  Tue Mar 22 22:46:47 2005
--- r/rpl000009.reject  Wed Mar 23 18:21:13 2005
***************
*** 37,42 ****
--- 37,43 ----
  mysqltest
  mysqltest2
  test
+ test_only_ndb_tables
  create table mysqltest2.t1(n int, s char(20));
  create table mysqltest2.t2(n int, s text);
  insert into mysqltest2.t1 values (1, 'one'), (2, 'two'), (3, 'three');
-------------------------------------------------------

*****
-------------------------------------------------------
*** r/rpl_create_database.result        Tue Mar 22 22:46:47 2005
--- r/rpl_create_database.reject        Wed Mar 23 18:26:38 2005
***************
*** 28,33 ****
--- 28,34 ----
  mysqltest_prometheus
  mysqltest_sisyfos
  test
+ test_only_ndb_tables
  SHOW DATABASES;
  Database
  information_schema
***************
*** 65,70 ****
--- 66,72 ----
  mysqltest_prometheus
  mysqltest_sisyfos
  test
+ test_only_ndb_tables
  SHOW DATABASES;
  Database
  information_schema
-------------------------------------------------------

*****

-------------------------------------------------------
*** r/schema.result     Tue Mar 22 22:46:47 2005
--- r/schema.reject     Wed Mar 23 18:47:21 2005
***************
*** 8,11 ****
--- 8,12 ----
  foo
  mysql
  test
+ test_only_ndb_tables
  drop schema foo;
-------------------------------------------------------

*****

-------------------------------------------------------
*** r/show_check.result Tue Mar 22 22:46:47 2005
--- r/show_check.reject Wed Mar 23 18:48:57 2005
***************
*** 54,62 ****
--- 54,64 ----
  information_schema
  mysql
  test
+ test_only_ndb_tables
  show databases like "test%";
  Database (test%)
  test
+ test_only_ndb_tables
  create table t1 (f1 int not null, f2 int not null, f3 int not null, f4 int not null, primary key(f1,f2,f3,f4));
  insert into t1 values (1,1,1,0),(1,1,2,0),(1,1,3,0),(1,2,1,0),(1,2,2,0),(1,2,3,0),(1,3,1,0),(1,3,2,0),(1,3,3,0),(1,1,1,1),(1,1,2,1),(1,1,3,1),(1,2,1,1),(1,2,2,1),(1,2,3,1),(1,3,1,1),(1,3,2,1),(1,3,3,1);
  analyze table t1;
-------------------------------------------------------

How to repeat:
Build + test on these.
[14 Jun 2005 15:30] Martin Skold
This problem is not seen in last 5.0.7 build,
should I close it?
[14 Jul 2005 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[10 Aug 2005 12:40] Joerg Bruehe
Sorry, got out of sight for some time.
The problem still exists, but is shifting platforms:
In 5.0.11, it occurs just on "cane" (FreeBSD), both with native and with Linuxthreads:

ndb_autodiscover               [ fail ]

Errors are (from PATH/mysqltest-time) :
waitNodeState(NOT_STARTED, -1) timeout after 121 attemps
At line 489: command "$NDB_TOOLS_DIR/ndb_waiter --no-defaults --not-started > /dev/null" failed
(the last lines may be the most important ones)

Ending Tests

Occurred:
cane-5.0-cluster.log   normal
cane-linuxthreads-5.0-cluster.log   normal
cane-linuxthreads-5.0-cluster.log   ps-protocol
[16 Aug 2005 9:59] Tomas Ulin
Is this only on freebsd with linux threads, if so it is a know issue, see below

http://bugs.mysql.com/bug.php?id=8369
[16 Aug 2005 11:21] Tomas Ulin
problem limited to freeBSD and lacking support there, thus setting low prio
[18 Aug 2005 3:18] Stewart Smith
I was talking with someone who is more familiar with FreeBSD and they were saying it's quite possibly a bug in the FreeBSD implementation of LinuxThreads. The developers of this library could be interested to work with us - although it's probably not a priority.
[13 Oct 2005 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[14 Dec 2005 19:36] Joerg Bruehe
Sorry, got out of sight a bit.
The problem of NDB not starting was reported again in bug#15619,
and was traced back to a too aggressive compiler optimization there.

I will not yet close this, but hope I can do it after a successful build + test round.