MySQL Bugs: #43604: Backup: No native driver for NDB Backup

Bug #43604	Backup: No native driver for NDB Backup
Submitted:	12 Mar 2009 16:22	Modified:	8 May 2009 11:49
Reporter:	Chuck Bell	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S4 (Feature request)
Version:	mysql-6.0	OS:	Any (NDB supported OS only)
Assigned to:		CPU Architecture:	Any
Tags:	mysql-6.0.11

Description:
Backup does not recognize the NDB or NDBCLUSTER storage engines as supporting consistent snapshot.

When backup is performed on an NDB or NDBCLUSTER table like:

CREATE DATABASE ndbbup;
CREATE TABLE ndbbup.t1 (a INT NOT NULL PRIMARY KEY, b INT) ENGINE=NDBCLUSTER;
INSERT INTO ndbbup.t1 VALUES (1,1), (2,1), (3,3);
BACKUP DATABASE ndbbup TO 'n1.bak';
DROP DATABASE ndbbup;
RESTORE FROM 'n1.bak';
SELECT * FROM ndbbup.t1 ORDER BY a, b;

CREATE TABLE ndbbup.t2 (a INT NOT NULL PRIMARY KEY, c INT) ENGINE=NDB;
INSERT INTO ndbbup.t2 VALUES (2,3), (3, 4), (5, 6);
BACKUP DATABASE ndbbup TO 'n2.bak';
DROP DATABASE ndbbup;
RESTORE FROM 'n2.bak';
SHOW FULL TABLES FROM ndbbup;
SELECT * FROM ndbbup.t2 ORDER BY a, c;

--query_vertical SELECT * FROM mysql.backup_history

The result in the history log shows the default driver was used:

SELECT * FROM mysql.backup_history;
backup_id      270
...
command        BACKUP DATABASE ndbbup TO 'n1.bak'
drivers        Default

backup_id      271
...
command        RESTORE FROM 'n1.bak'
drivers        Default

backup_id      272
...
command        BACKUP DATABASE ndbbup TO 'n2.bak'
drivers        Default

backup_id      273
...
command        RESTORE FROM 'n2.bak'
drivers        Default

How to repeat:
Run MTR using the test above on a platform that supports NDB.

Suggested fix:
The storage engines that support consistent snapshot use a method in the handler named start_consistent_snapshot(). If that method is assigned, the backup snapshot driver identifies the storage engine as supporting consistent snapshot. This can be seen in be_snapshot.h @121:

  bool accept(const Table_ref&, const storage_engine_ref se)
  {
    ::handlerton *h= se_hton(se);

    return (h->start_consistent_snapshot != NULL);
  }; // accept all tables that support consistent read

If it is found NDB and/or NDBCLUSTER do indeed support consistent snapshot but not using the handler method above, some other method may be needed to enable the backup snapshot driver to properly identify this feature.

Another possible solution is to create a native backup driver for NDB.

Clarification: There is nothing wrong with how backup and/or NDB communication. On the contrary, both systems are working correctly.

The issue this bug report is reporting is simply that a backup of NDB should result in a backup that is similar to the consistent snapshot driver. That is, the backup is non-blocking.

Mikael Ronstrum has added some comments about how such a backup can take place:

The high level view on how online backup with NDB should happen is
something like this.

1) Start a NDB online backup

2) In the locking phase when other storage engines are locked for
updates, NDB should be locked for commits. One shouldn't enter
the locking phase until NDB has reached the point where all data
has been copied and only the logging is active.

3) After locking is completed one gathers the NDB online backup
files in some way appropriate for online backup.

Note: While this is reported as a deficiency, it could rated as either a S3 or S4.

Title changed to reflect true issue of the bug: backup cannot backup NDB with a non-blocking (like the consistent snapshot) driver. We need online backup for NDB.

See also BUG#36471

Workaround, use cluster on-line backup

For NDB to have a non-blocking backup, we will need to write a native driver
for it.  NDB will not be able to support the consistent snapshot driver.

So, this bug report is a feature request for a native driver for NDB.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/73616

2807 Hema Sridharan	2009-05-07
      WL#4834(Test ndb and backup)
      New test case ndb_backup.test is included in ndb suite.
      Note that this test will not satisfy BUG#43604 and BUG#36471
      Once these bugs are fixed backup will use consistent snapshot drivers
      for NDB / NDBCLUSTER storage engine.
      added:
        mysql-test/suite/ndb/r/ndb_backup.result
        mysql-test/suite/ndb/t/ndb_backup.test

Resetting to 'verified' since the patch above is a commit to WL#4834, not this bug.