MySQL Bugs: #19911: ndb does not work on sol10 x86

Bug #19911	ndb does not work on sol10 x86
Submitted:	18 May 2006 14:55	Modified:	28 Feb 2008 8:38
Reporter:	Joerg Bruehe	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S1 (Critical)
Version:	5.1.10-beta	OS:	Solaris (Solaris 10 (x86))
Assigned to:	Magnus Blåudd	CPU Architecture:	Any
Tags:	ndb, solaris, x86

Description:
Test build of 5.1.10-beta.

This effect was not found in previous logs, and it is specific to "sol10x86":

Several tests show a server crash (error 2013) on commands which contain a clause "engine=ndb" or "engine=ndbcluster".

Example:
=====
ndb_alter_table2               [ fail ]

Errors are (from /PATH/mysqltest-time) :
mysqltest: At line NNN: query 'CREATE TABLE t1 (
a INT NOT NULL PRIMARY KEY,
b INT NOT NULL
) ENGINE=ndbcluster' failed: 2013: Lost connection to MySQL server during query
(the last lines may be the most important ones)
Result from queries before failure can be found in r/ndb_alter_table2.log

Killing Possible Leftover Processes
=====

More examples (shortened):
=====
ndb_alter_table3               [ fail ]
mysqltest: At line NNN: query 'create table t1 ( a int primary key, b varchar(10), c varchar(10), index (b) )
engine=ndb' failed: 2013: Lost connection to MySQL server during query
=====
ndb_alter_table                [ fail ]
mysqltest: At line NNN: query 'CREATE TABLE t1 (
a INT NOT NULL,
b INT NOT NULL
) ENGINE=ndbcluster' failed: 2013: Lost connection to MySQL server during query
=====
ndb_bitfield                   [ fail ]
mysqltest: At line NNN: query 'create table t1 (
pk1 int not null primary key,
b bit(64)
) engine=ndbcluster' failed: 2013: Lost connection to MySQL server during query
=====
ndb_cache                      [ fail ]
mysqltest: At line NNN: query 'CREATE TABLE t1 ( pk int not null primary key,
a int, b int not null, c varchar(20)) ENGINE=ndbcluster' failed: 2013: Lost connection to MySQL server during query
=====
ndb_dd_backuprestore           [ fail ]
mysqltest: At line NNN: query 'CREATE LOGFILE GROUP log_group1
ADD UNDOFILE './log_group1/undofile.dat'
INITIAL_SIZE 16M
UNDO_BUFFER_SIZE = 1M
ENGINE=NDB' failed: 2013: Lost connection to MySQL server during query
=====

Further tests with similar effects:
ndb_basic
ndb_binlog_ignore_db
ndb_blob
ndb_blob_partition
ndb_cache2
ndb_cache_multi2
ndb_cache_multi
ndb_charset
ndb_condition_pushdown
ndb_dd_disk2memory
ndb_index
ndb_index_ordered
ndb_index_unique
ndb_insert
ndb_limit
ndb_multi_row
ndb_replace
ndb_restore
ndb_subquery
ndb_transaction
ndb_truncate
ndb_update
ndb_view

In between, some few tests passed:
ndb_binlog_basic2
ndb_config
ndb_config2
ndb_types
All other NDB tests were reported as "disabled" or "skipped".

All these results were identical for both runs using NDB:
1) perl ./mysql-test-run.pl --ps-protocol --comment="ps+rowrepl+NDB" --mysqld=--binlog-format=row ... --with-openssl ... --with-ndbcluster --ndbcluster_port=9350
2) perl ./mysql-test-run.pl --comment=NDB --with-ndbcluster-only ... --with-openssl ... --with-ndbcluster --ndbcluster_port=9350

How to repeat:
Found by the test suite.

Sorry, the list in the first entry is incomplete:
1) Also tests with "engine = ndb" ("engine = ndbcluster") are affected.
2) Also the "rpl_ndb_*" tests are affected (above, I had only listed "ndb_*").

Here is a complete list:
ndb_alter_table2          ndb_index_unique       rpl_ndb_blob2
ndb_alter_table3          ndb_insert             rpl_ndb_blob
ndb_alter_table           ndb_limit              rpl_ndb_dd_advance
ndb_basic                 ndb_minmax             rpl_ndb_dd_basic
ndb_binlog_ignore_db      ndb_multi_row          rpl_ndb_delete_nowhere
ndb_bitfield              ndb_partition_error    rpl_ndb_func003
ndb_blob                  ndb_partition_key      rpl_ndb_idempotent
ndb_blob_partition        ndb_partition_list     rpl_ndb_insert_ignore
ndb_cache2                ndb_partition_range    rpl_ndb_multi
ndb_cache                 ndb_read_multi_range   rpl_ndb_multi_update2
ndb_cache_multi2          ndb_replace            rpl_ndb_multi_update3
ndb_cache_multi           ndb_restore            rpl_ndb_row_001
ndb_charset               ndb_subquery           rpl_ndb_sp003
ndb_condition_pushdown    ndb_transaction        rpl_ndb_trig004
ndb_dd_backuprestore      ndb_truncate           rpl_ndb_UUID
ndb_dd_disk2memory        ndb_update             rpl_row_basic_7ndb
ndb_index                 ndb_view
ndb_index_ordered         rpl_ndb_basic

joerg, how did sol10 x86 go for 5.1.12

5.1.12-beta test on sol10x86: Same problem, no change detected.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

Seems I forgot to change the status after adding my last comment ...

5.1.15 and current (as of this writing) 5.1.16-beta from BK still affected, on a completely new Solaris 10/x86 setup.

The crashes all generate core dumps with very similar stack traces, example following:

./sol10-x86-b/test/mysql-5.1.16-beta-solaris10-i386/mysql-test/var/log/ndb_alter_table/core
libc.so.1`_lwp_kill+0x15(7, 6)
write_core+0x21(6)
handle_segfault+0x146(6, 0, d0a7d5a4)
libc.so.1`__sighndlr+0xf(6, 0, d0a7d5a4, 81feba8)
libc.so.1`call_user_handler+0x22b(6, 0, d0a7d5a4)
libc.so.1`sigacthandler+0xbb(6, 0, d0a7d5a4)
libc.so.1`_lwp_kill+0x15(7, 6)
libc.so.1`raise+0x1f(6)
libc.so.1`abort+0x64(8b11f16, 886e078, 65737341, 6f697472, 6166206e, 64656c69)
libc.so.1`_assert+0x6b(84e9405, 84e93ed, 39c)
__1cVndbcluster_get_schema6FpnTst_ndbcluster_share_pnOCluster_schema__v_+0x111(
8afd8b8, 886e078)
__1cbLndb_binlog_thread_handle_schema_event6FpnDTHD_pnDNdb_pnRNdbEventOperation_
pnEList4nOCluster_schema___7pnLst_mem_root__i_+0x387(8806068, 87b9818, 8b20878, 
d0a7df34, d0a7df40, d0a7df10)
ndb_binlog_thread_func+0xf1d(0)
libc.so.1`_thr_setup+0x4e(d0ab1000)
libc.so.1`_lwp_start(d0ab1000, 0, 0, d0a7dff8, d0c2d230, d0ab1000)

No change in 5.1.20, just the list of affected tests differs a bit.

mysqldev@sol10-x86-b:~/magnus/mysql-5.1.22-rc-pb589/mysql-test> CC=cc CFLAGS="-xO3 -mt -fsimple=1 -ftrap=%none -nofstore -xbuiltin=%all -xlibmil -xlibmopt -xtarget=generic" CXX=CC CXXFLAGS="-xO3 -mt -fsimple=1 -ftrap=%none -nofstore -xbuiltin=%all -features=no%except -xlibmil -xlibmopt -xtarget=generic"  ./configure --prefix=/usr/local/mysql --localstatedir=/usr/local/mysql/data --libexecdir=/usr/local/mysql/bin --with-comment="MySQL Community Server (GPL)" --with-server-suffix="" --enable-thread-safe-client --enable-local-infile --with-mysqld-libs=-lmtmalloc --with-named-curses=-lcurses --with-pic --with-client-ldflags="-static" --with-mysqld-ldflags="-static" --with-zlib-dir=bundled --with-big-tables --with-ssl --with-readline --with-embedded-server --with-archive-storage-engine --with-blackhole-storage-engine --with-csv-storage-engine --with-example-storage-engine --with-federated-storage-engine --with-partition --with-innodb --with-extra-charsets=complex --with-ndbcluster --with-debug

mysqldev@sol10-x86-b:~/magnus/mysql-5.1.22-rc-pb589/mysql-test> ./mtr ndb_basic
Logging: ./mtr ndb_basic
MySQL Version 5.1.22
<snip>
TEST                           RESULT         TIME (ms)
-------------------------------------------------------
ndb.ndb_basic                  [ pass ]         104998
-------------------------------------------------------

mysqldev@sol10-x86-b:~/magnus/mysql-5.1.22-rc-pb589/mysql-test> ./mtr --do-test=ndb
Logging: ./mtr --do-test=ndb
MySQL Version 5.1.22
TEST                           RESULT         TIME (ms)
-------------------------------------------------------
ndb.ndb_alter_table            [ pass ]          66498
ndb.ndb_alter_table2           [ pass ]           6148
ndb.ndb_alter_table3           [ pass ]           8177
ndb.ndb_backup_print           [ pass ]           5328
ndb.ndb_basic                  [ pass ]          95963
ndb.ndb_binlog_basic           [ pass ]          18217
ndb.ndb_binlog_ddl_multi       [ pass ]          64443
ndb.ndb_binlog_discover        [ pass ]          26964
ndb.ndb_binlog_format          [ pass ]           4175
mysql-test-run: WARNING: Forcing kill of process 6487
ndb.ndb_binlog_ignore_db       [ pass ]           4991
ndb.ndb_binlog_log_bin         [ pass ]          20149
ndb.ndb_binlog_multi           [ pass ]          11345
ndb.ndb_bitfield               [ pass ]          12753
ndb.ndb_blob                   [ pass ]          55433
ndb.ndb_blob_partition         [ pass ]           7076
ndb.ndb_bug26793               [ pass ]           3484
ndb.ndb_cache                  [ pass ]           2714
ndb.ndb_cache2                 [ pass ]          16743
ndb.ndb_cache_multi            [ pass ]           3203
ndb.ndb_cache_multi2           [ pass ]           3513
ndb.ndb_charset                [ pass ]          30393
ndb.ndb_condition_pushdown     [ pass ]          21113
ndb.ndb_config                 [ pass ]           1740
ndb.ndb_config2                [ pass ]            199
ndb.ndb_cursor                 [ pass ]           4826
ndb.ndb_database               [ pass ]           3490
ndb.ndb_dd_alter               [ pass ]         108054
ndb.ndb_dd_backuprestore       [ pass ]         109412
ndb.ndb_dd_basic               [ pass ]          31566
ndb.ndb_dd_ddl                 [ pass ]          48220
ndb.ndb_dd_disk2memory         [ pass ]          48331
ndb.ndb_dd_dump                [ pass ]          53559
ndb.ndb_dd_sql_features        [ pass ]          64883
ndb.ndb_gis                    [ pass ]         107332
ndb.ndb_index                  [ pass ]           3805
ndb.ndb_index_ordered          [ pass ]          51167
ndb.ndb_index_unique           [ pass ]          68071
ndb.ndb_insert                 [ pass ]          49264
ndb.ndb_limit                  [ pass ]           7598
ndb.ndb_load                   [ pass ]           4075
ndb.ndb_loaddatalocal          [ pass ]          20182
ndb.ndb_lock                   [ pass ]          32789
ndb.ndb_lock_table             [ pass ]           1642
ndb.ndb_minmax                 [ pass ]           8013
ndb.ndb_multi                  [ pass ]          21813
ndb.ndb_multi_row              [ pass ]          15968
ndb.ndb_partition_error        [ pass ]           5564
ndb.ndb_partition_error2       [ disabled ]  HF is not sure if the test can work as internded on all the platforms
ndb.ndb_partition_key          [ pass ]          45420
ndb.ndb_partition_list         [ pass ]           4523
ndb.ndb_partition_range        [ pass ]          42703
ndb.ndb_read_multi_range       [ pass ]          39661
ndb.ndb_rename                 [ pass ]           6438
ndb.ndb_replace                [ pass ]          17004
ndb.ndb_restore                [ pass ]         147640
ndb.ndb_restore_compat         [ pass ]          85921
ndb.ndb_restore_different_endian_data [ pass ]          55145
ndb.ndb_restore_partition      [ pass ]         138520
mysql-test-run: WARNING: Forcing kill of process 11569
ndb.ndb_restore_print          [ pass ]          24748
ndb.ndb_row_format             [ pass ]          18082
ndb.ndb_single_user            [ pass ]          28824
ndb.ndb_sp                     [ pass ]           2308
ndb.ndb_subquery               [ pass ]          17509
ndb.ndb_temporary              [ pass ]           2191
ndb.ndb_transaction            [ pass ]          17766
ndb.ndb_trigger                [ pass ]          25627
ndb.ndb_truncate               [ pass ]           5042
ndb.ndb_types                  [ pass ]           3113
ndb.ndb_update                 [ pass ]          13896
ndb.ndb_update_no_read         [ pass ]           3530
ndb.ndb_view                   [ pass ]           1754
ndb.ndbapi                     [ skipped ]   Test requires: 'have_ndbapi_examples'
-------------------------------------------------------
All 70 tests were successful.

mysqldev@sol10-x86-b:~/magnus/mysql-5.1.22-rc-pb589/mysql-test> ./mtr --do-test=rpl_ndb
Logging: ./mtr --do-test=rpl_ndb
MySQL Version 5.1.22
TEST                           RESULT         TIME (ms)
-------------------------------------------------------
rpl_ndb.rpl_ndb_2innodb        [ disabled ]  BUG#19227 2006-04-20 pekka pk delete apparently not replicated
rpl_ndb.rpl_ndb_2myisam        [ disabled ]  BUG#19227 Seems to pass currently
rpl_ndb.rpl_ndb_2other         [ disabled ]  BUG#21842 2007-08-30 tsmith test has never worked on bigendian (sol10-sparc-a, powermacg5
mysql-test-run: WARNING: Forcing kill of process 11780
rpl_ndb.rpl_ndb_UUID           [ pass ]          14276
rpl_ndb.rpl_ndb_auto_inc       [ pass ]          29394
rpl_ndb.rpl_ndb_bank           [ skipped ]   Test need 'ndb_extra' option
rpl_ndb.rpl_ndb_basic          [ pass ]          66279
mysql-test-run: WARNING: Forcing kill of process 13011
rpl_ndb.rpl_ndb_blob           [ pass ]          41649
mysql-test-run: WARNING: Forcing kill of process 13024
rpl_ndb.rpl_ndb_blob2          [ pass ]          43229
mysql-test-run: WARNING: Forcing kill of process 13037
rpl_ndb.rpl_ndb_charset        [ pass ]          38437
rpl_ndb.rpl_ndb_circular       [ pass ]           9101
rpl_ndb.rpl_ndb_circular_simplex [ pass ]           8145
rpl_ndb.rpl_ndb_commit_afterflush [ pass ]          40010
rpl_ndb.rpl_ndb_ctype_ucs2_def [ disabled ]  BUG#27404 util thd mysql_parse sig11 when mysqld default multibyte charset
rpl_ndb.rpl_ndb_dd_advance     [ pass ]         204607
rpl_ndb.rpl_ndb_dd_basic       [ pass ]          23105
rpl_ndb.rpl_ndb_dd_partitions  [ disabled ]  BUG#19259 2006-04-21 rpl_ndb_dd_partitions fails on s/AMD
rpl_ndb.rpl_ndb_ddl            [ disabled ]  BUG#28798 2007-05-31  lars  Valgrind failure in NDB
mysql-test-run: WARNING: Forcing kill of process 13152
rpl_ndb.rpl_ndb_delete_nowhere [ pass ]           8529
mysql-test-run: WARNING: Forcing kill of process 13165
rpl_ndb.rpl_ndb_do_db          [ pass ]          24917
rpl_ndb.rpl_ndb_do_table       [ pass ]          13047
rpl_ndb.rpl_ndb_extraCol       [ pass ]         188616
mysql-test-run: WARNING: Forcing kill of process 13205
rpl_ndb.rpl_ndb_extraColMaster [ pass ]         373078
rpl_ndb.rpl_ndb_func003        [ pass ]          19528
mysql-test-run: WARNING: Forcing kill of process 13231
rpl_ndb.rpl_ndb_idempotent     [ pass ]          18330
rpl_ndb.rpl_ndb_innodb2ndb     [ disabled ]  Bug#29549 rpl_ndb_myisam2ndb,rpl_ndb_innodb2ndb failed on Solaris for pack_length issue
rpl_ndb.rpl_ndb_innodb_trans   [ pass ]          13860
rpl_ndb.rpl_ndb_insert_ignore  [ pass ]          16315
rpl_ndb.rpl_ndb_load           [ skipped ]   Test need 'ndb_extra' option
rpl_ndb.rpl_ndb_log            [ skipped ]   Requiring binlog format 'row'
rpl_ndb.rpl_ndb_mix_innodb     [ disabled ]  BUG#28123 rpl_ndb_mix_innodb.test casue slave to core on sol10-sparc-a
mysql-test-run: WARNING: Forcing kill of process 13274
rpl_ndb.rpl_ndb_multi          [ pass ]           9215
rpl_ndb.rpl_ndb_multi_update2  [ pass ]          16499
mysql-test-run: WARNING: Forcing kill of process 13301
rpl_ndb.rpl_ndb_multi_update3  [ pass ]          13014
rpl_ndb.rpl_ndb_myisam2ndb     [ disabled ]  Bug#29549 rpl_ndb_myisam2ndb,rpl_ndb_innodb2ndb failed on Solaris for pack_length issue
rpl_ndb.rpl_ndb_relayrotate    [ skipped ]   Test need 'ndb_extra' option
rpl_ndb.rpl_ndb_rep_ignore     [ pass ]          22887
rpl_ndb.rpl_ndb_row_001        [ pass ]          20507
mysql-test-run: WARNING: Forcing kill of process 14512
rpl_ndb.rpl_ndb_sp003          [ pass ]         116793
rpl_ndb.rpl_ndb_sp006          [ pass ]          16078
rpl_ndb.rpl_ndb_stm_innodb     [ skipped ]   Requiring binlog format 'statement'
mysql-test-run: WARNING: Forcing kill of process 14537
rpl_ndb.rpl_ndb_sync           [ pass ]          34069
mysql-test-run: WARNING: Forcing kill of process 15341
rpl_ndb.rpl_ndb_trig004        [ pass ]          20017
mysql-test-run: WARNING: Forcing kill of process 15755
rpl_ndb.rpl_ndbapi_multi       [ skipped ]   Test requires: 'have_ndbapi_examples'
-------------------------------------------------------
All 29 tests were successful.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/41758

ChangeSet@1.2521, 2008-02-06 11:44:07+01:00, msvensson@pilot.mysql.com +1 -0
  Bug#19911 ndb does not work on sol10 x86

Pushed into 5.1.24-rc

Pushed into 6.0.5-alpha

Documented bugfix in the 5.1.24 and 6.0.5 changelogs.

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Bugfix also documented in the 5.1.23-ndb-6.3.11 changelog.