Bug #30024 restore of backup from different endian does not work for datetime column
Submitted: 25 Jul 2007 7:44 Modified: 15 Sep 2007 12:30
Reporter: david li Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:5.1 OS:Linux ((Linux, Solaris))
Assigned to: li zhou CPU Architecture:Any

[25 Jul 2007 7:44] david li
Description:
Ndb backup/restore should be endian compatible.
For a cluster running on big-endian machine(e.g. SUN SPARC Solaris), if its table has 'datetime' type attribute, backup it on big-endian machine, and copy backuped data onto a little-endian machine(e.g. X86 Linux), then restore it, will find that the restored 'datetime' type data is not correct.

How to repeat:
1. start a 2 data nodes cluster on a SUN SPARC Solaris machine.
2. create a table with datetime type attribute, and insert values into it.
   mysql> use test;
   mysql> create table t1 (a int primary key, b datetime) engine=ndbcluster;
   mysql> insert into t1 values (1, '2012-03-12 18:35:04');
   mysql> select * from t1;
   +---+---------------------+
   | a | b                   |
   +---+---------------------+
   | 1 | 2012-03-12 18:35:04 |
   +---+---------------------+
   1 row in set (0.00 sec)
3. start backup on solaris
   ./bin/ndb_mgm -e "start backup"
4. copy the backuped data to a X86 Linux machine.
   scp -r data/node2/BACKUP/BACKUP-1/ x86-machine:/tmp/BACKUPS/
   scp -r data/node3/BACKUP/BACKUP-1/* x86-machine:/tmp/BACKUPS/BACKUP-1/
5. start a same configure cluster as SUN machine on X86 machine.
6. restore database.
   ./bin/ndb_restore -n 2 -b 1 -m -r /tmp/BACKUPS/BACKUP-1
   ./bin/ndb_restore -n 3 -b 1 -r /tmp/BACKUPS/BACKUP-1
7. select on linux.
   mysql> select * from t1;
   +---+---------------------+
   | a | b                   |
   +---+---------------------+
   | 1 | �('/--.-/+ 14:27:84 |
   +---+---------------------+
   1 row in set (0.07 sec)

Suggested fix:
Maybe it's the same cause as Bug #27543.
[26 Jul 2007 5:14] Stewart Smith
I completely recommend the test of the backup/restore for all column types.

i propose 2 tests:
1 - big endian backup being restored
2 - little endian backup being restored

with a couple of tables, columns making up all data types.

my guess is we'll find a few more bugs too...

check with David... he's probably at least partially solved some of these problems as part of the endian work.
[30 Jul 2007 1:15] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31788

ChangeSet@1.2523, 2007-07-30 09:01:11+00:00, lzhou@dev3-63.(none) +1 -0
  BUG#30024 Restore of backup from different endian can work for datetime
[30 Jul 2007 1:18] li zhou
Will commit a test case for all data type in bug#27543 because they are all relative of endian problem. 
We should test this bug using test case in bug#27543.
[2 Aug 2007 9:35] david li
why is datetime type attribute set to DictTabInfo::an8Bit instead of DictTabInfo::a64Bit ?
doubt that this is the real cause.
[10 Aug 2007 1:43] Stewart Smith
good point. Certainly this is worth investigating.

Otherwise patch looks okay... although could be better to just use Twiddle64 in place instead of two memcpys.
[10 Aug 2007 5:47] li zhou
"DictTabInfo::an8Bit" is only exist in -telco tree. 
So i guess set datatime's size to 8 bits is the historic problem. If we change its size to 64 bits, we need to change the usage of datetime also.
[14 Aug 2007 4:36] Stewart Smith
please check revision history and ask who it is who changed it about it.
[5 Sep 2007 2:05] li zhou
approved by stewart.
[5 Sep 2007 7:56] li zhou
pushed into 5.1.22 ndb-bj tree
[14 Sep 2007 16:26] Bugs System
Pushed into 5.1.23-beta
[15 Sep 2007 12:30] Jon Stephens
Thank you for your bug report. This issue has already been fixed in the latest released version of that product, which you can download at

  http://www.mysql.com/downloads/

Documented bugfix in 5.1.23 changelog.
[21 Sep 2007 8:25] Jon Stephens
Also documented in mysql-5.1.22-ndb-6.2.6 changelog.