Description:
ndb_restore --print-data makes 0x... for multi-byte strings under some condition.
[some condition]
A strings end with multi-byte character.
ndb_restore --print-data makes 0x... for multi-byte strings under some condition.
Tokyo -> Tokyo
東京 -> 0xE69DB1E4BAAC
1東京 -> 0x31E69DB1E4BAAC
東京1 -> 東京1
0x... format is wrong for LOAD DATA, so ndb_restore always output multi-byte as binary.
# mysqldump's --tab option always make binary data for multi-byte strings.
How to repeat:
1. Craete test environment with ./ndb_setup.py
2. Login to SQL node,
mysql -uroot --port=3306 --protocol=tcp test
3. create ndb table
create table t1(i1 int not null primary key, v2 varchar(20)) ENGINE=ndbcluster DEFAULT CHARSET=utf8;
4. insert data, each 3-times.
mysql> insert into t1 values(1,'Tokyo');
Query OK, 1 row affected (0.01 sec)
mysql> insert into t1 values(2,'Tokyo');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(3,'Tokyo');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(4,'東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(5,'東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(6,'東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(7,'1東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(8,'1東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(9,'1東京');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(10,'東京1');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(11,'東京1');
Query OK, 1 row affected (0.00 sec)
mysql> insert into t1 values(12,'東京1');
Query OK, 1 row affected (0.00 sec)
4. make backup with ./ndb_mgm
start backup
5. Output tsv with ndb_restore
cd /path/to/BACKUP
ndb_restore -n 1 -b 1 --print-data --tab=/tmp --append ./BACKUP-1
6. See /tmp/t1.txt as belows. Line with PK 5,6,9 is output in 0x format.
2 Tokyo
3 Tokyo
5 0xE69DB1E4BAAC
6 0xE69DB1E4BAAC
9 0x31E69DB1E4BAAC
10 東京1 10
Suggested fix:
[Suggested fix]
Output data like this.
1 Tokyo
12 東京1
8 1東京
7 1東京
11 東京1
4 東京
3 Tokyo
5 東京
10 東京1
9 1東京
2 Tokyo
6 東京
The same result of this command line.
./mysqldump -uroot --port=3306 --protocol=tcp --tab=/tmp test
[Workaround]
We can convert 0x to binary string with this command.
perl -pi -e 's/(0x[0-9A-F]+)/pack("H*",substr($1,2))/eg' /tmp/*.txt
*CAUTION*
This command is provided as sample, not apply for all cases. Please modify & add error cases for your environment.
Description: ndb_restore --print-data makes 0x... for multi-byte strings under some condition. [some condition] A strings end with multi-byte character. ndb_restore --print-data makes 0x... for multi-byte strings under some condition. Tokyo -> Tokyo 東京 -> 0xE69DB1E4BAAC 1東京 -> 0x31E69DB1E4BAAC 東京1 -> 東京1 0x... format is wrong for LOAD DATA, so ndb_restore always output multi-byte as binary. # mysqldump's --tab option always make binary data for multi-byte strings. How to repeat: 1. Craete test environment with ./ndb_setup.py 2. Login to SQL node, mysql -uroot --port=3306 --protocol=tcp test 3. create ndb table create table t1(i1 int not null primary key, v2 varchar(20)) ENGINE=ndbcluster DEFAULT CHARSET=utf8; 4. insert data, each 3-times. mysql> insert into t1 values(1,'Tokyo'); Query OK, 1 row affected (0.01 sec) mysql> insert into t1 values(2,'Tokyo'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(3,'Tokyo'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(4,'東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(5,'東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(6,'東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(7,'1東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(8,'1東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(9,'1東京'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(10,'東京1'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(11,'東京1'); Query OK, 1 row affected (0.00 sec) mysql> insert into t1 values(12,'東京1'); Query OK, 1 row affected (0.00 sec) 4. make backup with ./ndb_mgm start backup 5. Output tsv with ndb_restore cd /path/to/BACKUP ndb_restore -n 1 -b 1 --print-data --tab=/tmp --append ./BACKUP-1 6. See /tmp/t1.txt as belows. Line with PK 5,6,9 is output in 0x format. 2 Tokyo 3 Tokyo 5 0xE69DB1E4BAAC 6 0xE69DB1E4BAAC 9 0x31E69DB1E4BAAC 10 東京1 10 Suggested fix: [Suggested fix] Output data like this. 1 Tokyo 12 東京1 8 1東京 7 1東京 11 東京1 4 東京 3 Tokyo 5 東京 10 東京1 9 1東京 2 Tokyo 6 東京 The same result of this command line. ./mysqldump -uroot --port=3306 --protocol=tcp --tab=/tmp test [Workaround] We can convert 0x to binary string with this command. perl -pi -e 's/(0x[0-9A-F]+)/pack("H*",substr($1,2))/eg' /tmp/*.txt *CAUTION* This command is provided as sample, not apply for all cases. Please modify & add error cases for your environment.