Bug #41642 | The purpose for the backup_engines suite isn't clear. | ||
---|---|---|---|
Submitted: | 19 Dec 2008 15:23 | Modified: | 26 Mar 2009 23:44 |
Reporter: | Chuck Bell | Email Updates: | |
Status: | Closed | Impact on me: | |
Category: | MySQL Server: Backup | Severity: | S3 (Non-critical) |
Version: | 6.0.9 | OS: | Any |
Assigned to: | Chuck Bell | CPU Architecture: | Any |
[19 Dec 2008 15:23]
Chuck Bell
[2 Feb 2009 15:59]
Chuck Bell
The backup_engines suite is designed to run a test once for each of the storage engines listed in the combinations file (stored in the root folder of the suite). The engines listed are falcon, myisam, innodb, and memory. Thus, a given test runs once substituting the storage engine for the default-storage-engine setting. The tests written for this suite make assumptions about the default storage engine and the presumption that running the same test through each storage engine is in some way beneficial. Problems -------- * There are only 4 storage engines listed. All storage engines should be listed. * Running the same test for each storage engine is inefficient and of little benefit. However, if we are testing backup for how it behaves for different storage engines, then a discrete test for each storage may (under very special cases) be beneficial. * If you write a test that should be run against 'all' storage engines, then this suite will not meet your needs. A more elaborate test is needed to test the use of a single and multiple storage engines in the same backup or restore operation. PROPOSAL -------- The following is what I think should be done: Option 1 -------- We agree the suite has some limited benefits. * Expand the combinations file to include all storage engines. * Rename the suite to something else like backup_single_engine so that it reflects the true nature of the test suite. * Examine the tests in the suite for compatibility with the existing limitations moving most if not all back to the backup suite. * (optional and probably not worth the effort) Rework the suite to make it more efficient. Option 2 -------- Eliminate the suite -- it is too limited and inefficient. Option X -------- Does anyone have another option?
[2 Feb 2009 16:06]
Chuck Bell
Additional problem with the backup_engines suite: * The --default-storage-engine parameter can be overridden and|or ignored.
[19 Feb 2009 14:59]
Chuck Bell
The intital analysis is complete. There were some tests in the backup_engines suite that were not using the combinations file. Initial pass at reorganization of tests (on Windows) show: BEFORE: 105 tests executing in 622 seconds. AFTER: 77 tests executing in 317 seconds. Moved most tests out of backup_engines leaving only partition and trigger tests. Created a new backup_ptr suite for testing point in time over replication modes rather than engines and replication modes (this was the greatest savings for number of tests). Patch forthcoming. With all of the renames it is a challenge to create a patch that can be applied correctly. Worked 20 hours. Estimate 8 hours max remain (including reviews).
[19 Feb 2009 19:20]
Chuck Bell
There are now 74 tests down from 105. :)
[19 Feb 2009 19:21]
Chuck Bell
Commands needed to apply patch
Attachment: 41642_linux.txt (text/plain), 1.95 KiB.
[19 Feb 2009 19:24]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/66957 2773 Chuck Bell 2009-02-19 BUG#41642 : The purpose for the backup_engines suite isn't clear. This patch reorganizes the backup_engines tests. A new suite is created for those tests that must iterate over replication modes (backup_ptr). The storage engines list is extended to include all storage engines except for MERGE and FEDERATED which require special commands to utilize. A description for each of the suites has been added to clarify the usage of the suites and what sort of tests should be included. Note: The archive storage engine is disabled for the partition tests. See BUG#43028 and BUG#43030 for details. removed: mysql-test/suite/backup_engines/r/backup_procedures.result mysql-test/suite/backup_engines/r/backup_ptr_commit_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_commit_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_row.result mysql-test/suite/backup_engines/r/backup_ptr_objects_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_row.result mysql-test/suite/backup_engines/r/backup_ptr_stmt.result mysql-test/suite/backup_engines/t/backup_procedures.test mysql-test/suite/backup_engines/t/backup_ptr_commit_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_commit_row.test mysql-test/suite/backup_engines/t/backup_ptr_commit_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_row.test mysql-test/suite/backup_engines/t/backup_ptr_objects_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_row.test mysql-test/suite/backup_engines/t/backup_ptr_stmt.test mysql-test/suite/backup_engines/t/disabled.def added: mysql-test/suite/backup/README mysql-test/suite/backup_engines/README mysql-test/suite/backup_engines/include/not_archive.inc mysql-test/suite/backup_engines/include/not_csv.inc mysql-test/suite/backup_ptr/ mysql-test/suite/backup_ptr/README mysql-test/suite/backup_ptr/include/ mysql-test/suite/backup_ptr/r/ mysql-test/suite/backup_ptr/r/backup_ptr.result mysql-test/suite/backup_ptr/r/backup_ptr_objects.result mysql-test/suite/backup_ptr/t/ renamed: mysql-test/suite/backup_engines/include/backup_ptr.inc => mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_engines/include/backup_ptr_commit.inc => mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_engines/include/backup_ptr_objects.inc => mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_have_falcon.inc => mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_have_innodb.inc => mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_have_memory.inc => mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_have_myisam.inc => mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup_engines/r/backup_functions.result => mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_engines/r/backup_online_testing.result => mysql-test/suite/backup/r/backup_online_testing.result mysql-test/suite/backup_engines/r/backup_ptr_commit_row.result => mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup_engines/r/backup_tmp_tables.result => mysql-test/suite/backup/r/backup_tmp_tables.result mysql-test/suite/backup_engines/t/backup_functions.test => mysql-test/suite/backup/t/backup_functions.test mysql-test/suite/backup_engines/t/backup_online_testing.test => mysql-test/suite/backup/t/backup_online_testing.test mysql-test/suite/backup_engines/t/backup_tmp_tables.test => mysql-test/suite/backup/t/backup_tmp_tables.test modified: mysql-test/suite/backup/r/backup_procedures.result mysql-test/suite/backup/t/backup_procedures.test mysql-test/suite/backup_engines/combinations mysql-test/suite/backup_engines/r/backup_partition.result mysql-test/suite/backup_engines/t/backup_partition.test mysql-test/suite/backup_engines/t/backup_partitioning.test mysql-test/suite/backup_engines/t/backup_triggers.test mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup/t/backup_functions.test
[23 Feb 2009 13:59]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/67194 2773 Chuck Bell 2009-02-23 BUG#41642 : The purpose for the backup_engines suite isn't clear. This patch reorganizes the backup_engines tests. A new suite is created for those tests that must iterate over replication modes (backup_ptr). The storage engines list is extended to include all storage engines except for MERGE and FEDERATED which require special commands to utilize. A description for each of the suites has been added to clarify the usage of the suites and what sort of tests should be included. removed: mysql-test/suite/backup_engines/r/backup_procedures.result mysql-test/suite/backup_engines/r/backup_ptr_commit_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_commit_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_row.result mysql-test/suite/backup_engines/r/backup_ptr_objects_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_row.result mysql-test/suite/backup_engines/r/backup_ptr_stmt.result mysql-test/suite/backup_engines/t/backup_procedures.test mysql-test/suite/backup_engines/t/backup_ptr_commit_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_commit_row.test mysql-test/suite/backup_engines/t/backup_ptr_commit_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_row.test mysql-test/suite/backup_engines/t/backup_ptr_objects_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_row.test mysql-test/suite/backup_engines/t/backup_ptr_stmt.test mysql-test/suite/backup_engines/t/disabled.def added: mysql-test/suite/backup/README mysql-test/suite/backup_engines/README mysql-test/suite/backup_engines/include/not_archive.inc mysql-test/suite/backup_engines/include/not_csv.inc mysql-test/suite/backup_ptr/ mysql-test/suite/backup_ptr/README mysql-test/suite/backup_ptr/combinations mysql-test/suite/backup_ptr/include/ mysql-test/suite/backup_ptr/r/ mysql-test/suite/backup_ptr/r/backup_ptr.result mysql-test/suite/backup_ptr/r/backup_ptr_objects.result mysql-test/suite/backup_ptr/t/ renamed: mysql-test/suite/backup_engines/include/backup_ptr.inc => mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_engines/include/backup_ptr_commit.inc => mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_engines/include/backup_ptr_objects.inc => mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_have_falcon.inc => mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_have_innodb.inc => mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_have_memory.inc => mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_have_myisam.inc => mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup_engines/r/backup_functions.result => mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_engines/r/backup_online_testing.result => mysql-test/suite/backup/r/backup_online_testing.result mysql-test/suite/backup_engines/r/backup_ptr_commit_row.result => mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup_engines/r/backup_tmp_tables.result => mysql-test/suite/backup/r/backup_tmp_tables.result mysql-test/suite/backup_engines/t/backup_functions.test => mysql-test/suite/backup/t/backup_functions.test mysql-test/suite/backup_engines/t/backup_online_testing.test => mysql-test/suite/backup/t/backup_online_testing.test mysql-test/suite/backup_engines/t/backup_tmp_tables.test => mysql-test/suite/backup/t/backup_tmp_tables.test modified: mysql-test/suite/backup/r/backup_procedures.result mysql-test/suite/backup/t/backup_procedures.test mysql-test/suite/backup_engines/combinations mysql-test/suite/backup_engines/r/backup_partition.result mysql-test/suite/backup_engines/t/backup_partition.test mysql-test/suite/backup_engines/t/backup_partitioning.test mysql-test/suite/backup_engines/t/backup_triggers.test mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup/t/backup_functions.test
[5 Mar 2009 6:50]
Rafal Somla
Patch approved.
[5 Mar 2009 20:05]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/68417 2784 Chuck Bell 2009-03-05 BUG#41642 : The purpose for the backup_engines suite isn't clear. This patch reorganizes the backup_engines tests. A new suite is created for those tests that must iterate over replication modes (backup_ptr). The storage engines list is extended to include all storage engines except for MERGE and FEDERATED which require special commands to utilize. A description for each of the suites has been added to clarify the usage of the suites and what sort of tests should be included. Note: The archive storage engine is disabled for the partition tests. See BUG#43028 and BUG#43030 for details. The following are commands needed to apply this patch. These create the new folders and move the test files. The changes to the test files are included in the patch. These should be executed from the root of the source tree. Added backup_ptr to the default list in new mtr. removed: mysql-test/suite/backup_engines/r/backup_procedures.result mysql-test/suite/backup_engines/r/backup_ptr_commit_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_commit_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_mixed.result mysql-test/suite/backup_engines/r/backup_ptr_objects_row.result mysql-test/suite/backup_engines/r/backup_ptr_objects_stmt.result mysql-test/suite/backup_engines/r/backup_ptr_row.result mysql-test/suite/backup_engines/r/backup_ptr_stmt.result mysql-test/suite/backup_engines/t/backup_procedures.test mysql-test/suite/backup_engines/t/backup_ptr_commit_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_commit_row.test mysql-test/suite/backup_engines/t/backup_ptr_commit_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_mixed.test mysql-test/suite/backup_engines/t/backup_ptr_objects_row.test mysql-test/suite/backup_engines/t/backup_ptr_objects_stmt.test mysql-test/suite/backup_engines/t/backup_ptr_row.test mysql-test/suite/backup_engines/t/backup_ptr_stmt.test mysql-test/suite/backup_engines/t/disabled.def added: mysql-test/suite/backup_ptr/ mysql-test/suite/backup_ptr/include/ mysql-test/suite/backup_ptr/r/ mysql-test/suite/backup_ptr/t/ renamed: mysql-test/suite/backup_engines/include/backup_ptr.inc => mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_engines/include/backup_ptr_commit.inc => mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_engines/include/backup_ptr_objects.inc => mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_have_falcon.inc => mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_have_innodb.inc => mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_have_memory.inc => mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_have_myisam.inc => mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup_engines/r/backup_functions.result => mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_engines/r/backup_online_testing.result => mysql-test/suite/backup/r/backup_online_testing.result mysql-test/suite/backup_engines/r/backup_ptr_commit_row.result => mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup_engines/r/backup_tmp_tables.result => mysql-test/suite/backup/r/backup_tmp_tables.result mysql-test/suite/backup_engines/t/backup_functions.test => mysql-test/suite/backup/t/backup_functions.test mysql-test/suite/backup_engines/t/backup_online_testing.test => mysql-test/suite/backup/t/backup_online_testing.test mysql-test/suite/backup_engines/t/backup_tmp_tables.test => mysql-test/suite/backup/t/backup_tmp_tables.test modified: mysql-test/mysql-test-run.pl mysql-test/suite/backup/r/backup_procedures.result mysql-test/suite/backup/t/backup_procedures.test mysql-test/suite/backup_engines/combinations mysql-test/suite/backup_engines/r/backup_partition.result mysql-test/suite/backup_engines/t/backup_partition.test mysql-test/suite/backup_engines/t/backup_partitioning.test mysql-test/suite/backup_engines/t/backup_triggers.test mysql-test/suite/backup_ptr/t/backup_ptr.test mysql-test/suite/backup_ptr/t/backup_ptr_commits.test mysql-test/suite/backup_ptr/t/backup_ptr_objects.test mysql-test/suite/backup_engines/include/not_falcon.inc mysql-test/suite/backup_engines/include/not_innodb.inc mysql-test/suite/backup_engines/include/not_memory.inc mysql-test/suite/backup_engines/include/not_myisam.inc mysql-test/suite/backup/r/backup_functions.result mysql-test/suite/backup_ptr/r/backup_ptr_commits.result mysql-test/suite/backup/t/backup_functions.test
[5 Mar 2009 20:41]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/68420 2785 Chuck Bell 2009-03-05 BUG#41642 : The purpose for the backup_engines suite isn't clear. Added missing includes. added: mysql-test/suite/backup_engines/include/not_archive.inc mysql-test/suite/backup_engines/include/not_csv.inc
[5 Mar 2009 20:51]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/68422 2786 Chuck Bell 2009-03-05 BUG#41642 : The purpose for the backup_engines suite isn't clear. Added missing files from patch. added: mysql-test/suite/backup/README mysql-test/suite/backup_engines/README mysql-test/suite/backup_ptr/README mysql-test/suite/backup_ptr/combinations mysql-test/suite/backup_ptr/r/backup_ptr.result
[5 Mar 2009 20:57]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/68424 2787 Chuck Bell 2009-03-05 BUG#41642 : The purpose for the backup_engines suite isn't clear. Ok, I think I found them all now. added: mysql-test/suite/backup_ptr/r/backup_ptr_objects.result
[26 Mar 2009 12:33]
Bugs System
Pushed into 6.0.11-alpha (revid:alik@sun.com-20090326121822-pt84kzxxayzho4mn) (version source revid:charles.bell@sun.com-20090309134019-vr480npdp30f2gvh) (merge vers: 6.0.11-alpha) (pib:6)
[26 Mar 2009 23:44]
Paul DuBois
Test suite changes. No changelog entry needed.