Bug #48236 InnoDB fails in "os_file_readdir_next_file()" on AIX and i5os
Submitted: 22 Oct 17:22 Modified: 23 Oct 21:57
Reporter: Joerg Bruehe
Status: Verified
Category:Server: InnoDB Severity:S3 (Non-critical)
Version:5.1.40 + 5.5.0 OS:Other (AIX + i5os)
Assigned to: Vasil Dimov Target Version:
Triage: Triaged: D3 (Medium)

[22 Oct 17:22] Joerg Bruehe
Description:
All runs of test "crash_commit_before" fail on AIX and IBM i5os (all configurations) for
both the built-in and the plugin version of InnoDB.

The scheme is:
     1  main.crash_commit_before                 [ retry-fail ]  Found warnings/errors in
server log file!
     2          Test ended at YYYY-MM-DD HH:MM:SS
     3  mysqltest: Logging to ''.
     4  mysqltest: Results saved in ''.
     5  mysqltest: Connecting to server localhost:13000 (socket
/PATH/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
     6  mysqltest: ... Connected.
     7  mysqltest: Start processing test commands from './include/check-warnings.test'
...
     8  file_name       line
     9 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
    10 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
    11 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
    ... ...
   406 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
   407 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
   408 
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysqld.1.err
     InnoDB: Error: os_file_readdir_next_file() returned -1 in
   409  ^ Found warnings!!
   410  mysqltest: ... Done processing test commands.
   411  mysqltest: Test has succeeded!
   412  ok

   413   - saving '/PATH/mysql-test/var/log/main.crash_commit_before/' to
'/PATH/mysql-test/var/log/main.crash
_commit_before/'

   414  Test has failed 2 times, no more retries!
   415  ===---===---===---===---===---===--- BUGDB:
   416  ===---===---===---===---===---===--- BUILD-ID: 2009100621142320647
   417  ===---===---===---===---===---===--- OCCURRED:
   418  i5os-02.mysql.com,i5os-power-32bit,5.1-community.log   debug

Similar for other configurations, whether on AIX or i5os, 32 or 64 bit, built-in or
plugin.

How to repeat:
Run the test suite on a debug server.
[23 Oct 13:14] Michael Izioumtchenko
just to make sure:
the problem is that on specified platforms there are error messages in
mysqld log:

InnoDB: Error: os_file_readdir_next_file() returned -1

but the execution continues, no crash or assert?
[23 Oct 19:06] Joerg Bruehe
I have quoted everything that got logged for this test,
just eliminated the lines 12 through 405 which are identical to 9-11 and 406-408.

So yes, there was no crash or assert logged, but MTR considered these lines to indicate
enough of a problem to mark that as a test failure.

Maybe these messages are to be expected on these platforms, and the code handles all this
just fine (you sure can check that), then we need some way to tell this to MTR.
[23 Oct 19:38] Michael Izioumtchenko
Joerg, as I don't have this kind of hardware readily available could
you provide the entire mysqld log, in your example this would be
/home/mysqldev/tmp-200910061726-5.1.40-20647/i5os-power-32bit/test/mysql-test/var/log/mysq
ld.1.err? In fact I'm mostly interested in a few lines
before and after the "InnoDB: Error: os_file_readdir_next_file() returned -1 in
" message which hopefully will provide some additional information.
[23 Oct 21:48] hery ramilison
mysqld.1.err error file

Attachment: mysqld.1.err (application/octet-stream, text), 3.65 KiB.

[23 Oct 21:52] hery ramilison
We just ran on aix entreprise in debug mode the crash_commit_before. Error only occurs
when running in debug mode. Complete log in mysqld.1.err file.
[26 Oct 13:54] Michael Izioumtchenko
Vasil, could you try to figure this out? IBM likes to do things a bit differently, maybe
we are missing an error code here.
[19 Nov 17:26] Joerg Bruehe
Same problem in the build of 5.5.0-beta,
tests "crash_commit_before" and "myisam_crash_before_flush_keys".