Bug #54327 Restoring a NDB backup into another database using ndb_restore
Submitted: 8 Jun 2010 7:22 Modified: 2 Feb 2011 16:44
Reporter: Geert Vanderkelen Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S4 (Feature request)
Version:mysql-5.1-telco-6.3 OS:Any
Assigned to: Martin Zaun CPU Architecture:Any

[8 Jun 2010 7:22] Geert Vanderkelen
Description:
It would be useful for ndb_restore to be able to restore a backup into another database.
Handy when you want to rename a database for example.

How to repeat:
.

Suggested fix:
Example usage:

shell> ndb_restore --include-database=db1,db2 --restore_meta --rename-database=db1,new_db1 --rename-database=db2,new_db2
shell> ndb_restore --include-database=db1,db2 --restore_data --rename-database=db1,new_db1 --rename-database=db2,new_db2
[23 Nov 2010 10:51] Jonas Oreland
Bernie, can you assign martin z to fix this...
[22 Jan 2011 6:41] Martin Zaun
I'd like to discuss the (error) behaviour for corner cases.  Any comments?

Clearly, these cases should be handled as "Wrong arguments" errors:
--rename-database=
--rename-database=,
--rename-database=,a
--rename-database=a,
--rename-database=aaa

I suggest this is OK and handled transparently as (a->a):
--rename-database=a,a

This is OK (a->b):
--rename-database=a,b

I suggest this is OK and handled as (a->a):
--rename-database=a,b --rename-database=a,a

I suggest this is OK and handled as (a->b):
--rename-database=a,b --rename-database=a,b

I suggest this is OK and handled as (a->b, b->a):
--rename-database=a,b --rename-database=b,a

I suggest this is an error, cannot do (a->b, b->b):
--rename-database=a,b --rename-database=b,b

I suggest this is OK and handled as (a->b, b->c):
--rename-database=a,b --rename-database=b,c

I suggest this is OK and handled as (a->b, b->c, c->a):
--rename-database=a,b --rename-database=b,c --rename-database=c,a

Regards, Martin
[23 Jan 2011 0:05] Martin Zaun
More testing with multiple databases in the backup has convinced me
that this case is not as special as I'd thought:

> I suggest this is an error, cannot do (a->b, b->b):
> --rename-database=a,b --rename-database=b,b

So, I'd like to change this to read instead:

I suggest this is OK and handled as (a->b, b->b)
--rename-database=a,b --rename-database=b,b

Further:

I suggest this is OK and handled as (a->b, b->b)
--rename-database=a,b --rename-database=c,b

Rationale: there are two ways how the data from two databases
can end up in single database:
a) user has said so, e.g. (a->b, c->b)
b) user has redirected one db (a->b), but backup also has data for (b)

Idea: ndb_restore should allow all these cases if there's no conflict
in the data (like constraint violations).
[23 Jan 2011 0:12] Martin Zaun
I'd like to suggest that this behaviour is OK: when data from two
databases is redirected in one database, e.g. (a->b, c->b), and
both source databases have an index on the same column defined,
then the current ndb_restore patch reports an error message

  Rebuilding index UNIQUE_a_0$unique on table a ...FAIL!                       
  Rebuilding index UNIQUE_a_0$unique on table a failed: 4244: Index or table with given name already exists                                                   

but continues to restore the backup.  Which succeeds, if there's
no constraint violation by the added data.

We could try to prohibit that error message, but I think it's
actually quite logical.  Since ndb_restore continues, it might
be OK to leave it as is.

Comments?
[23 Jan 2011 0:14] Martin Zaun
I'd like to suggest that
  --redirect-database
is a better name than
  --rename-database
since (a->b,b->a) is not a no-op, as a renaming op would suggest,
but actually swaps the content of the databases during restore.

Comments?
[2 Feb 2011 6:53] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/130168

3381 Martin Zaun	2011-02-01
      bug#54327 ndb_restore: added new option --rewrite-database=oldDb,newDb
[2 Feb 2011 7:26] Bugs System
Pushed into mysql-5.1-telco-7.0 5.1.51-ndb-7.0.22 (revid:martin.zaun@oracle.com-20110202071641-075u8ua13s1ckk47) (version source revid:martin.zaun@oracle.com-20110202071641-075u8ua13s1ckk47) (merge vers: 5.1.51-ndb-7.0.22) (pib:24)
[2 Feb 2011 7:27] Bugs System
Pushed into mysql-5.1-telco-6.3 5.1.51-ndb-6.3.41 (revid:martin.zaun@oracle.com-20110202065924-9yeppme4andl4z07) (version source revid:martin.zaun@oracle.com-20110202065924-9yeppme4andl4z07) (merge vers: 5.1.51-ndb-6.3.41) (pib:24)
[2 Feb 2011 7:52] Martin Zaun
I pushed the implementation and tests of this new feature.

After some discussion, we decided not to mimic replication's
rewrite-db feature.

Here's a summary of the implemented behaviour:
1) Usage is: --rewrite-database=oldDb,newDb
2) Later options override preceding ones.
3) Not an error when multiple databases are written to the same db.
4) If multiple databases with non-disjoint data are written to the
   same db, there is no detection of data conflicts; rows may get
   overwritten resulting in lost updates; there is no guarantee
   about the order in which databases are written.

Regards,
Martin
[2 Feb 2011 8:31] Martin Zaun
Hi Jon,

there still could be minor changes on the option's syntax...
We'll keep you updated.

Regards,
Martin
[2 Feb 2011 16:45] Jon Stephens
Documented as follows in the NDB-6.3.41, 7.0.22, and 7.1.11 changelogs:

        A new --rewrite-database option is added for ndb_restore, which 
        makes it possible to restore to a database having a different 
        name from that of the original database.

        For more information, see "ndb_restore — Restore a MySQL Cluster 
        Backup".

Also added info about the new option in the usual places -- see http://lists.mysql.com/commits/130260 for docs changes.

Closed.