Bug #27543 restore of backup from different endian does not work for blob column
Submitted: 30 Mar 2007 9:23 Modified: 15 Sep 2007 12:30
Reporter: david li Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:5.1.14 OS:Linux (linux, solaris)
Assigned to: li zhou CPU Architecture:Any

[30 Mar 2007 9:23] david li
Description:
Ndb backup/restore should be endian compatible.
For a cluster running on big-endian machine(e.g. SUN SPARC Solaris), if its table has 'blob' type attribute, backup it on big-endian machine, and copy backuped data onto a little-endian machine(e.g. X86 Linux), then restore it, will find that restore process can succeed, but when executing 'select' query for the table will cause mysqld to crash and restart. 

How to repeat:
1. start a 2 data nodes cluster on a SUN SPARC Solaris machine.
2. create a table with blob type attribute, and insert values into it.
   mysql> use test;
   mysql> create table t1 (a int unsigned primary key, b blob) engine ndb;
   mysql> insert into t1 values (1, 'abcd');
3. start backup
   ./bin/ndb_mgm -e "start backup"
4. copy the backuped data to a X86 Linux machine.
   scp -r data/node2/BACKUP/BACKUP-1/ x86-machine:/tmp/BACKUPS/
   scp -r data/node3/BACKUP/BACKUP-1/* x86-machine:/tmp/BACKUPS/BACKUP-1/
5. start a same configure cluster as SUN machine on X86 machine.
6. restore database.
   ./bin/ndb_restore -n 2 -b 1 -m -r /tmp/BACKUPS/BACKUP-1
   ./bin/ndb_restore -n 3 -b 1 -r /tmp/BACKUPS/BACKUP-1
7. show tables and select.
mysql> use test;
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| t1             |
+----------------+
1 row in set (0.06 sec)

mysql> select * from t1;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> show errors;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    2
Current database: test

Empty set (0.01 sec)

mysql>

Suggested fix:
Printed out some log of the attribute data, and found that the 'blob' attribute data after restored is wrong.
Found that the first 8 bytes(the length of the blob data) of the 'blob' attribute data is stored in big-endian, it should be little-endian on a LE machine.
So it seems that there is some problems in handling the first 8 bytes for blob data when backup/restore.
[5 Jun 2007 10:31] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/28116

ChangeSet@1.2552, 2007-06-05 18:21:57+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[18 Jun 2007 11:03] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/28979

ChangeSet@1.2552, 2007-06-18 18:53:01+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[19 Jun 2007 7:14] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29057

ChangeSet@1.2552, 2007-06-19 15:03:51+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[19 Jun 2007 9:55] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29084

ChangeSet@1.2552, 2007-06-19 17:44:23+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[10 Jul 2007 3:10] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/30581

ChangeSet@1.2552, 2007-07-10 10:58:25+00:00, lzhou@dev3-63.(none) +15 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[23 Jul 2007 8:04] li zhou
the backup files of new test case for blob between different endians

Attachment: ndb_backup_files.tar.gz (application/x-gzip, text), 69.91 KiB.

[31 Jul 2007 7:52] Stewart Smith
I think we should also test the GIS types
[1 Aug 2007 8:00] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31937

ChangeSet@1.2552, 2007-08-01 15:46:48+00:00, lzhou@dev3-63.(none) +15 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[1 Aug 2007 8:04] li zhou
the backup files of new test case for all data type between  different endians

Attachment: ndb_backup.tar.gz (application/x-gzip, text), 120.67 KiB.

[10 Aug 2007 1:40] Stewart Smith
tests look good. ok to push.
[20 Aug 2007 7:36] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32729

ChangeSet@1.2552, 2007-08-20 15:23:36+00:00, lzhou@dev3-63.(none) +16 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute
[5 Sep 2007 5:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/33686

ChangeSet@1.2548, 2007-09-05 13:01:30+00:00, lzhou@dev3-63.(none) +16 -0
  BUG#27543 Backup and restore can compatible with different endians for blob attribute
[5 Sep 2007 7:57] li zhou
pushed into 5.1.22 ndb-bj tree
[14 Sep 2007 16:26] Bugs System
Pushed into 5.1.23-beta
[15 Sep 2007 12:30] Jon Stephens
Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented bugfix in 5.1.23 changelog.
[21 Sep 2007 8:25] Jon Stephens
Also documented in mysql-5.1.22-ndb-6.2.6 changelog.