MySQL Bugs: #27543: restore of backup from different endian does not work for blob column

Bug #27543	restore of backup from different endian does not work for blob column
Submitted:	30 Mar 2007 9:23	Modified:	15 Sep 2007 12:30
Reporter:	david li	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S3 (Non-critical)
Version:	5.1.14	OS:	Linux (linux, solaris)
Assigned to:	li zhou	CPU Architecture:	Any

Description:
Ndb backup/restore should be endian compatible.
For a cluster running on big-endian machine(e.g. SUN SPARC Solaris), if its table has 'blob' type attribute, backup it on big-endian machine, and copy backuped data onto a little-endian machine(e.g. X86 Linux), then restore it, will find that restore process can succeed, but when executing 'select' query for the table will cause mysqld to crash and restart. 

How to repeat:
1. start a 2 data nodes cluster on a SUN SPARC Solaris machine.
2. create a table with blob type attribute, and insert values into it.
   mysql> use test;
   mysql> create table t1 (a int unsigned primary key, b blob) engine ndb;
   mysql> insert into t1 values (1, 'abcd');
3. start backup
   ./bin/ndb_mgm -e "start backup"
4. copy the backuped data to a X86 Linux machine.
   scp -r data/node2/BACKUP/BACKUP-1/ x86-machine:/tmp/BACKUPS/
   scp -r data/node3/BACKUP/BACKUP-1/* x86-machine:/tmp/BACKUPS/BACKUP-1/
5. start a same configure cluster as SUN machine on X86 machine.
6. restore database.
   ./bin/ndb_restore -n 2 -b 1 -m -r /tmp/BACKUPS/BACKUP-1
   ./bin/ndb_restore -n 3 -b 1 -r /tmp/BACKUPS/BACKUP-1
7. show tables and select.
mysql> use test;
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| t1             |
+----------------+
1 row in set (0.06 sec)

mysql> select * from t1;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql> show errors;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    2
Current database: test

Empty set (0.01 sec)

mysql>

Suggested fix:
Printed out some log of the attribute data, and found that the 'blob' attribute data after restored is wrong.
Found that the first 8 bytes(the length of the blob data) of the 'blob' attribute data is stored in big-endian, it should be little-endian on a LE machine.
So it seems that there is some problems in handling the first 8 bytes for blob data when backup/restore.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/28116

ChangeSet@1.2552, 2007-06-05 18:21:57+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/28979

ChangeSet@1.2552, 2007-06-18 18:53:01+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29057

ChangeSet@1.2552, 2007-06-19 15:03:51+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/29084

ChangeSet@1.2552, 2007-06-19 17:44:23+00:00, lzhou@dev3-63.(none) +1 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/30581

ChangeSet@1.2552, 2007-07-10 10:58:25+00:00, lzhou@dev3-63.(none) +15 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

the backup files of new test case for blob between different endians

Attachment: ndb_backup_files.tar.gz (application/x-gzip, text), 69.91 KiB.

I think we should also test the GIS types

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31937

ChangeSet@1.2552, 2007-08-01 15:46:48+00:00, lzhou@dev3-63.(none) +15 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

the backup files of new test case for all data type between different endians

Attachment: ndb_backup.tar.gz (application/x-gzip, text), 120.67 KiB.

tests look good. ok to push.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/32729

ChangeSet@1.2552, 2007-08-20 15:23:36+00:00, lzhou@dev3-63.(none) +16 -0
  Bug#27543 Backup and Restore can compatible with different endians for blob attribute

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/33686

ChangeSet@1.2548, 2007-09-05 13:01:30+00:00, lzhou@dev3-63.(none) +16 -0
  BUG#27543 Backup and restore can compatible with different endians for blob attribute

pushed into 5.1.22 ndb-bj tree

Pushed into 5.1.23-beta

Thank you for your bug report. This issue has been committed to our source repository of that product and will be incorporated into the next release.

If necessary, you can access the source repository and build the latest available version, including the bug fix. More information about accessing the source trees is available at

    http://dev.mysql.com/doc/en/installing-source.html

Documented bugfix in 5.1.23 changelog.

Also documented in mysql-5.1.22-ndb-6.2.6 changelog.