Bug #28563 Test suite "jp" fails completely
Submitted: 21 May 2007 15:10 Modified: 22 May 2008 12:03
Reporter: Joerg Bruehe Email Updates:
Status: Closed Impact on me:
None 
Category:Tests: Server Severity:S3 (Non-critical)
Version:5.1.19 OS:Any
Assigned to: Matthias Leich CPU Architecture:Any

[21 May 2007 15:10] Joerg Bruehe
Description:
Noted in a test build of 5.1.19-beta,
where the "jp" suite is run for the very first time:

The suite fails completely:

1) Every "create table ... engine=innodb" fails with a warning
      Warning       1286    Unknown table engine 'innodb'
and MyISAM is used.
This is due to the lack of a line
      source include/have_innodb.inc;
at the start of each test.
(I verified locally that this helps.)

2) When I add that line, the next class of differences is about the output of "desc" and "show create table":
capitalization of "default", showing "NO" for a "NOT NULL" column, ...

3) The suite still tries to use the BDB table handler which we don't configure any more.

There may be several other classes of issues to which I did not proceed.

How to repeat:
Run the "jp" test suite:

cd mysql-test && ./mysql-test-run.pl --suite=jp

As most of the suite is to deal with Japanese identifiers, an environment that can display these will be very helpful.

Suggested fix:
As a short-term measure, I will remove the "jp" suite from the list of suites to be run when doing a release build.

When it has been fixed, this must be re-enabled.
[21 May 2007 15:34] Joerg Bruehe
Next class of differences:

4) The "show database" output in the result files
   does not list the "information_schema" database.
[9 Oct 2007 12:05] Matthias Leich
Conclusion after checking two tests:
The "jp" testsuite needs a refactoring in such a way that
only one storage engine is checked per test case =
script suite/jp/t/<whatever>test.
[7 May 2008 19:40] Matthias Leich
Next problems found:
- Some tests claim to test some storage engines
  <> InnoDB but assign InnoDB to every table used
- The current expected results within Join tests
  show a historical server error (no coalesce
  of columns in case of "USING")
- Several tests earn warnings like
  Note 1265 Data truncated for column 'c1' at row 30
  when loading data from input files.
  I assume that these warnings did not came up when
  the "jp" testsuite was created.
- It is to be expected that many of the current tests
  will show a random row order within result sets
  if they are applied to NDB.
[8 May 2008 16:59] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46524

ChangeSet@1.2607, 2008-05-08 18:57:48+02:00, mleich@five.local.lan +222 -0
  Fix for
    Bug#28563 Test suite "jp" fails completely
  - The main goal of this fix is to make the "jp" suite runnable in general
    and to make it more robust for use in pushbuild under build team conditions.
  - It was decided to fix the remaining heavy issues within the
    architecture of this test later
    (-> WL 4327 Redesign the 'jp' testsuite for build team testing conditions)
    Only the test jp_trim_sjis got some significant improvements of its
    architecture (Just as proof of concept for the other tests).
  - Tests suffering from
    Bug 36597 Testsuite "jp": Suspicious results for some tests
        (warnings during load data and partially unexpected result sets)
    - were disabled if they show differences to the old expected result sets
    - simply got an updated file with expected results if they only show
      the warnings during load data
[9 May 2008 21:11] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46583

ChangeSet@1.2607, 2008-05-09 23:11:01+02:00, mleich@five.local.lan +222 -0
  Fix for
    Bug#28563 Test suite "jp" fails completely
  - The main goal of this fix is to make the "jp" suite runnable in general
    and to make it more robust for use in pushbuild under build team conditions.
  - It was decided to fix the remaining heavy issues within the
    architecture of this test later
    (-> WL 4327 Redesign the 'jp' testsuite for build team testing conditions)
    Only the test jp_trim_sjis got some significant improvements of its
    architecture (Just as proof of concept for the other tests).
  - Tests suffering from
    Bug 36597 Testsuite "jp": Suspicious results for some tests
        (warnings during load data and partially unexpected result sets)
    - were disabled if they show differences to the old expected result sets
    - simply got an updated file with expected results if they only show
      the warnings during load data
[9 May 2008 21:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46584

ChangeSet@1.2607, 2008-05-09 23:17:04+02:00, mleich@five.local.lan +226 -0
  Fix for
    Bug#28563 Test suite "jp" fails completely
  - The main goal of this fix is to make the "jp" suite runnable in general
    and to make it more robust for use in pushbuild under build team conditions.
  - It was decided to fix the remaining heavy issues within the
    architecture of this test later
    (-> WL 4327 Redesign the 'jp' testsuite for build team testing conditions)
    Only the test jp_trim_sjis got some significant improvements of its
    architecture (Just as proof of concept for the other tests).
  - Tests suffering from
    Bug 36597 Testsuite "jp": Suspicious results for some tests
        (warnings during load data and partially unexpected result sets)
    - were disabled if they show differences to the old expected result sets
    - simply got an updated file with expected results if they only show
      the warnings during load data
[15 May 2008 16:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/46732

ChangeSet@1.2635, 2008-05-15 18:01:13+02:00, mleich@five.local.lan +226 -0
  Fix for
    Bug#28563 Test suite "jp" fails completely
  - The main goal of this fix is to make the "jp" suite runnable in general
    and to make it more robust for use in pushbuild under build team conditions.
  - It was decided to fix the remaining heavy issues within the
    architecture of this test later
    (-> WL 4327 Redesign the 'jp' testsuite for build team testing conditions)
    Only the test jp_trim_sjis got some significant improvements of its
    architecture (Just as proof of concept for the other tests).
  - Tests suffering from
       Bug 36597 Testsuite "jp": Suspicious results for some tests
                 (unexpected result sets)
    were disabled
  - *.opt files were used to avoid differences on OS (Windows) because of
    case sensitive tablenames within the tests
[15 May 2008 20:15] Matthias Leich
Patch pushed into
mysql-5.1-bugteam
mysql-6.0-bugteam

The difference to the patch reviewed by Patrick Crews is
that some files with expected results had to be updated
(= They do no more contain the warnings during LOAD DATA
 = They are more similar to the state before starting to
   fix Bug#28563).
[19 May 2008 8:25] Bugs System
Pushed into 5.1.25-rc
[20 May 2008 0:22] Paul DuBois
Noted in 5.1.25 changelog.

The jp test suite was not working. 

Setting report to Need Doc Info pending push into 6.0.x.
[22 May 2008 9:49] Bugs System
Pushed into 6.0.6-alpha
[22 May 2008 12:03] Paul DuBois
Noted in 6.0.6 changelog.
[28 Jul 2008 16:57] Bugs System
Pushed into 5.1.25-rc  (revid:sp1r-pcrews@pcrews-mac-local.local-20080515233115-03273) (version source revid:sp1r-pcrews@pcrews-mac-local.local-20080515233115-03273) (pib:3)