| Bug #49353 | Memory tables + max_heap_table_size can cause backup/restore problems | ||
|---|---|---|---|
| Submitted: | 2 Dec 2009 15:13 | Modified: | 23 Mar 2010 15:56 |
| Reporter: | Victor Kirkebo | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | MySQL Server: Backup | Severity: | S3 (Non-critical) |
| Version: | 6.0-backup, 6.0.14 | OS: | Any |
| Assigned to: | Chuck Bell | CPU Architecture: | Any |
| Tags: | ERROR 1687, restore | ||
[2 Dec 2009 15:13]
Victor Kirkebo
[2 Dec 2009 15:43]
Valeriy Kravchuk
Verified just as described with recent 6.0.14 brom bzr (mysql-6.0-codebase tree) on Linux:
openxs@suse:/home2/openxs/dbs/6.0-code> bin/mysql -uroot test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 6.0.14-alpha-debug Source distribution
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database mydb;
Query OK, 1 row affected (0.00 sec)
mysql> use mydb;
Database changed
mysql> set max_heap_table_size=1024*1024*2;
Query OK, 0 rows affected (0.01 sec)
mysql> create table tb_mem(f1 char(255)) engine=memory;
Query OK, 0 rows affected (0.02 sec)
mysql> insert into tb_mem values('1'),('2'),('3'),('4'),('5'),('6'),('7'),('8');
Query OK, 8 rows affected (0.00 sec)
Records: 8 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 8 rows affected (0.01 sec)
Records: 8 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 16 rows affected (0.00 sec)
Records: 16 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 32 rows affected (0.00 sec)
Records: 32 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 64 rows affected (0.00 sec)
Records: 64 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 128 rows affected (0.01 sec)
Records: 128 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 256 rows affected (0.01 sec)
Records: 256 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 512 rows affected (0.01 sec)
Records: 512 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 1024 rows affected (0.02 sec)
Records: 1024 Duplicates: 0 Warnings: 0
mysql> insert into tb_mem select * from tb_mem;
Query OK, 2048 rows affected (0.02 sec)
Records: 2048 Duplicates: 0 Warnings: 0
mysql> backup database mydb to '/tmp/mydb.bck';
+-----------+
| backup_id |
+-----------+
| 1378 |
+-----------+
1 row in set (0.37 sec)
mysql> set max_heap_table_size=1024*1024;
Query OK, 0 rows affected (0.00 sec)
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> drop database mydb;
Query OK, 1 row affected (0.03 sec)
mysql> restore from '/tmp/mydb.bck';
ERROR 1687 (HY000): Error when sending data (for table #1) to Default restore driver
[4 Dec 2009 23:38]
Hema Sridharan
This error might be because of disk space problem. Looks like this bug is related to BUG#37249
[17 Dec 2009 17:56]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/94821 2917 Chuck Bell 2009-12-17 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems The default restore driver does not communicate the table full error message. In fact, the default drivers do not communicate the last known error by design. Rather than design a new error solution, the default drivers have been changed to report this specific error. This is accomplished by realizing the logger functionality that was included in the original design but not used. In this case, the default driver issues the error directly to the Logger class. The original error cited in the bug report is still issued but is listed second on the error list. Note: A redesign for error reporting is set for future work. This patch addresses the immediate problem. Similar solutions may be needed but will be considered on a case-by-case basis. @ mysql-test/suite/backup/r/backup_max_heap.result New result file. @ mysql-test/suite/backup/t/backup_max_heap.test New test to test condition where a memory table runs out of space (exceeds the size specified in max_heap_table_size). @ sql/backup/be_default.cc Added logger class to constructors. Error called from send_data() method. @ sql/backup/be_default.h Added logger class (already specified in original design) to the default driver classes. @ sql/backup/be_snapshot.h Added logger class (already specified in original design) to the default driver classes.
[17 Dec 2009 18:02]
Chuck Bell
DESIGN ------ The design for this solution is in two parts. The first part is to address the need to report errors from within the backup drivers. The second is a solution to the bug report using the first part. The goal of this design is to minimize changes to the kernel code while providing an exception to the error reporting stance taken with backup drivers. We consider the default drivers a bit lower level than native drivers and as such are (now) permitted to report errors. Rather than design a specific general-purpose error reporting mechanism (which is beyond the scope of this bug report), the goal is to use existing mechanism and avoiding code that deviates from established practices. Part A) The solution for this bug shall use the existing logger class that is passed to the Default_snapshot class. It was defined in the constructor but ignored until now. The solution shall pass this logger class to the default drivers. The default drivers can then use the logger class to report errors directly. Part B) The solution shall detect the error when a table is full and report it to the user via the logger class.
[22 Dec 2009 10:01]
Ingo Strüwing
Approved. Please find minor suggestions/questions in email.
[4 Jan 2010 15:46]
Rafal Somla
Good to push. Disclaimer: I think that this patch is introducing a change in the error reporting paradigm which is not good and is not required to fix this bug. The change is that now backup driver is reporting errors to backup logs, while before only backup kernel could do that. Since my concern was overruled by Lars' decision, my approval only means that I agree that the patch correctly implements the solution which I think is moving us in a wrong direction.
[5 Jan 2010 15:38]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/95994 2917 Chuck Bell 2010-01-05 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems The default restore driver does not communicate the table full error message or any errors for that matter. The original design included the logger as a parameter in the constructor for the snapshot parent class for the default driver classes but the parameter was not used. This patch now passes the logger class to the default drivers for reporting errors. The default drivers now issue the table full error message on restore. The original error that was issued by the kernel is still present but listed second on the error list. Note: A redesign for error reporting is set for future work. This patch addresses the immediate problem. Similar solutions may be needed but will be considered on a case-by-case basis. @ mysql-test/suite/backup/r/backup_max_heap.result New result file. @ mysql-test/suite/backup/t/backup_max_heap.test New test to test condition where a memory table runs out of space (exceeds the size specified in max_heap_table_size). @ sql/backup/be_default.cc Added logger class to constructors. Error called from send_data() method. @ sql/backup/be_default.h Added logger class (already specified in original design) to the default driver classes. @ sql/backup/be_snapshot.h Added logger class (already specified in original design) to the default driver classes.
[5 Jan 2010 16:41]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/96004 2928 Chuck Bell 2010-01-05 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems The default restore driver does not communicate the table full error message or any errors for that matter. The original design included the logger as a parameter in the constructor for the snapshot parent class for the default driver classes but the parameter was not used. This patch now passes the logger class to the default drivers for reporting errors. The default drivers now issue the table full error message on restore. The original error that was issued by the kernel is still present but listed second on the error list. Note: A redesign for error reporting is set for future work. This patch addresses the immediate problem. Similar solutions may be needed but will be considered on a case-by-case basis. @ mysql-test/suite/backup/r/backup_max_heap.result New result file. @ mysql-test/suite/backup/t/backup_max_heap.test New test to test condition where a memory table runs out of space (exceeds the size specified in max_heap_table_size). @ sql/backup/be_default.cc Added logger class to constructors. Error called from send_data() method. @ sql/backup/be_default.h Added logger class (already specified in original design) to the default driver classes. @ sql/backup/be_snapshot.h Added logger class (already specified in original design) to the default driver classes.
[6 Jan 2010 20:59]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/96177 2928 Chuck Bell 2010-01-06 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems This patch fixes a pushbuild failure on WinX64 machines. @ sql/backup/be_default.cc Referenced string (char *) member.
[7 Jan 2010 19:49]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/96314 3045 Chuck Bell 2010-01-07 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems The default restore driver does not communicate the table full error message or any errors for that matter. The original design included the logger as a parameter in the constructor for the snapshot parent class for the default driver classes but the parameter was not used. This patch now passes the logger class to the default drivers for reporting errors. The default drivers now issue the table full error message on restore. The original error that was issued by the kernel is still present but listed second on the error list. Note: A redesign for error reporting is set for future work. This patch addresses the immediate problem. Similar solutions may be needed but will be considered on a case-by-case basis. original changeset: 2925.1.3
[7 Jan 2010 20:03]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/96318 3046 Chuck Bell 2010-01-07 BUG#49353 : Memory tables + max_heap_table_size can cause backup/restore problems This patch fixes a pushbuild failure on WinX64 machines. original changeset: 2928 (mysql-6.0-backup)
[20 Feb 2010 9:18]
Bugs System
Pushed into 6.0.14-alpha (revid:ingo.struewing@sun.com-20100218152520-s4v1ld76bif06eqn) (version source revid:ingo.struewing@sun.com-20100119103538-wtp5alpz4p2jayl5) (merge vers: 6.0.14-alpha) (pib:16)
[23 Mar 2010 15:56]
Paul DuBois
Noted in 6.0.14 changelog. If a MEMORY table in a backup image exeeded the maximum size allowed by the server for that storage engine, a RESTORE of the image resulted in a general "Error when sending data" message from the default restore driver. Now the driver additionally reports a more specific "table full" message that better reflects the cause of the error.
