| Bug #81763 | ndb_desc output should be deterministic independent of platform | ||
|---|---|---|---|
| Submitted: | 7 Jun 2016 21:31 | Modified: | 27 Jun 2016 11:40 |
| Reporter: | Mauritz Sundell | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Tests: Cluster | Severity: | S3 (Non-critical) |
| Version: | OS: | Any | |
| Assigned to: | CPU Architecture: | Any | |
[14 Jun 2016 14:51]
Mauritz Sundell
Posted by developer: ndb_fully_replicated_restart fails also on Linux if it runs after ndb_index_stat_restart. Reason is that ndb_index_stat_restarts creates an extra hash map due to creating a table with one partition. Indexes created in ndb_fully_replicated_restart will get an id one higher than if the extra hashmap was not created, and the internal key in hash list of indexes will get another order. By listing index in id order the index order will be stable. Note, the extra hashmap is not visible from sql so mysqltest post test checks do not discover this change. And also there are currently no way to drop a hashmap.
[27 Jun 2016 11:40]
Jon Stephens
Documented fix in the NDB 7.5.3 changelog as follows:
Table indexes were listed in the output of ndb_desc in a
undefined order that could vary between platforms. Now these
indexes are ordered by ID.
Closed.
Description: ndb_desc are used in several MTR test for ndb. ndb_desc output is not deterministic, indexes can be listed in different order on different platforms. This becomes a problem for test there ndb_desc output is recorded. An other related problem is that table and index id might differ between runs of the test, and since the id is visible in different places in ndb_desc output that too is a source of indeterminism and a problem for recorded tests. Some of the indeterminism have been coped with in MTR using replace_regex, but that does not help for the current failures of ndb.ndb_fully_replicated_restart. How to repeat: On OS X 10 index occasionally are listed in different order than recorded on Linux. ndb.ndb_fully_replicated_restart w2 [ fail ] Test ended at 2016-06-04 11:52:32 CURRENT_TEST: ndb.ndb_fully_replicated_restart --- /hd3/pb2/test/sb_2-19177065-1465033011.14/mysql-cluster-gpl-7.5.3-osx10.11-x86_64/mysql-test/suite/ndb/r/ndb_fully_replicated_restart.result 2016-06-03 19:26:58.000000000 +0300 +++ /Volumes/ramdisk/mtr-29252/var-n_mix/2/log/ndb_fully_replicated_restart.reject 2016-06-04 12:52:31.000000000 +0300 @@ -42,10 +42,10 @@ c Int NULL AT=FIXED ST=MEMORY DYNAMIC -- Indexes -- PRIMARY KEY(a, b) - UniqueHashIndex +c$unique(c) - UniqueHashIndex b(b) - OrderedIndex c(c) - OrderedIndex PRIMARY(a, b) - OrderedIndex -c$unique(c) - UniqueHashIndex NDBT_ProgramExit: 0 - OK In mysql-test/suite/ndb/include/ndb_desc_print.inc there is some hacks making ndb_desc output more deterministic. # Use replace_regex to remove non-deterministic values from the # ndb_desc output # --replace_regex /Version: [0-9]*/Version: Any/ /t2_unique_index\(j\) - OrderedIndex/Index/ /t2_unique_index\$uni que\(j\) - UniqueHashIndex/Index/ /PRIMARY\(i\) - OrderedIndex/Index/ /NDB\$BLOB_[0-9]*_/NDB$BLOB_XX_/ /Length of frm data: [0-9]*/Length of frm data: XXX/ --exec $ndb_desc_cmd Suggested fix: Add options to ndb_desc making output more deterministic and use them in MTR tests. 1) allow sorting indexes for a table by name 2) replace visible id like for blob table names (NDB$BLOB_id_colname) in column definition with either some constant string like # or object name, that is NDB$BLOB_#_colname or NDB$BLOB_{db/def/tbl} An alternative would be to not use ndb_desc but some other tool or ndbinfo if they provide enough information.