Bug #52637 Fix existing test cases in mtr for the innodb plugin
Submitted: 6 Apr 2010 21:32 Modified: 13 May 2010 18:30
Reporter: Mark Callaghan Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: InnoDB Plugin storage engine Severity:S2 (Serious)
Version:5.1.45 OS:Any
Assigned to: Vasil Dimov CPU Architecture:Any
Tags: innodb, mtr, plugin

[6 Apr 2010 21:32] Mark Callaghan
Description:
I will have more faith in the InnoDB plugin when it passes existing test cases in mtr. Code in mysql-test/lib/mtr_cases.pm is hardwired to skip several tests that fail for the plugin. Some of these are big and I prefer we don't skip all of the test cases in them because one fails.

How to repeat:
This is the code:

      next if ($test->{'name'} eq 'main.innodb'); # Failed with wrong errno (fk)
      next if ($test->{'name'} eq 'main.index_merge_innodb'); # Explain diff
      # innodb_file_per_table is rw with innodb_plugin
      next if ($test->{'name'} eq 'sys_vars.innodb_file_per_table_basic');
      # innodb_lock_wait_timeout is rw with innodb_plugin
      next if ($test->{'name'} eq 'sys_vars.innodb_lock_wait_timeout_basic');
      # Diff around innodb_thread_concurrency variable
      next if ($test->{'name'} eq 'sys_vars.innodb_thread_concurrency_basic');
      # Can't work with InnoPlug. Test framework needs to be re-designed.
      next if ($test->{'name'} eq 'main.innodb_bug46000');
      # Fails with innodb plugin
      next if ($test->{'name'} eq 'main.innodb-autoinc');
      # Fails with innodb plugin: r6185 Testcases changes not included
      next if ($test->{'name'} eq 'main.innodb_bug44369');

Suggested fix:
Split the big test cases into to parts -- one that passes for the plugin and one that does not. Then make everything pass for the plugin.
[6 Apr 2010 22:06] Mark Callaghan
More fun. I am trying to run the full suite with the plugin:

./mysql-test-run.pl --mysqld=--innodb \
     --mysqld=--plugin-load=innodb=ha_innodb_plugin.so --force 

It gives up after 10 failures. I assume the problem is my use of '--mysqld...' above, but I must do that to enable the InnoDB plugin. 

Too many failed: Failed 10/11 tests, 9.09% were successful.

Failing test(s): main.plugin_load main.plugin main.fulltext_plugin main.udf binlog.binlog_stm_unsafe_warning binlog.binlog_incident binlog.binlog_killed_simulate

An example failure is:

main.plugin_load                         [ fail ]
        Test ended at 2010-04-06 14:48:31

Server log is:
100407  0:48:31 [Warning] '--default-character-set' is deprecated and will be removed in a future release. Please use '--character-set-server' instead.
100407  0:48:31 [Warning] /s/bld/5145orig/sql/mysqld: unknown variable 'loose-innodb_data_file_path=ibdata1:10M:autoextend'
100407  0:48:31 [Warning] /s/bld/5145orig/sql/mysqld: unknown option '--loose-skip-innodb'
100407  0:48:31 [ERROR] /s/bld/5145orig/sql/mysqld: unknown option '--innodb'
100407  0:48:31 [ERROR] Aborting

100407  0:48:31 [Note] Debug sync points hit:                   2
100407  0:48:31 [Note] Debug sync points executed:              0
100407  0:48:31 [Note] Debug sync points max active per thread: 0
100407  0:48:31 [Note] /s/bld/5145orig/sql/mysqld: Shutdown complete
[6 Apr 2010 22:10] Mark Callaghan
Wait, I can use --max-test-fail=0, I think.
[7 Apr 2010 15:39] Harrison Fisk
For more information see:

http://www.facebook.com/#!/note.php?note_id=382618805932

Notably, the command which can be used to run all tests is:

./mysql-test-run.pl --mysqld=--innodb \
  --mysqld=--plugin-load=innodb=ha_innodb_plugin.so \
  --force --max-test-fail=0

And the failing tests listed as:

main.plugin_load main.plugin main.fulltext_plugin main.udf main.innodb_ignore_builtin rpl.rpl_plugin_load rpl.rpl_udf

These are in addition to the one that are skipped automatically due to the plugin being used as mentioned previously in the bug report.
[13 May 2010 17:26] Vasil Dimov
This has been fixed and will be released in the next 5.1 release.

The fix consists of using a separate test suites for the plugin and the builtin.

Thanks!
[13 May 2010 17:53] Vasil Dimov
Originally the InnoDB Plugin had its own test suite, but when it was integrated into MySQL an assumption was made that the builtin InnoDB tests should be also run against the plugin. This has now been fixed and the plugin tests are in suite/innodb_plugin, while the builtin tests are in suite/innodb.
[13 May 2010 18:30] Paul DuBois
Changes to test suite. No changelog entry needed.