Bug #16635 Error messages wrong: absolute path names, "%s" format code
Submitted: 19 Jan 2006 14:52 Modified: 3 Aug 2007 1:44
Reporter: Joerg Bruehe Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Embedded Library ( libmysqld ) Severity:S3 (Non-critical)
Version:4.1.17 OS:Any (all Unix)
Assigned to: Joerg Bruehe

[19 Jan 2006 14:52] Joerg Bruehe
Description:
Release build of 4.1.17, based on ChangeSet
  1.2470.8.1 06/01/17 01:03:03 konstantin@mysql.com +4 -0
  A fix for Bug#13337 "ps test fails if configure wo/ usc2"

Test case "innodb" fails when run in "embedded" mode, output typically is like:

=====
innodb                         [ fail ]

Errors are (from /mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/var/log/mysqltest-time) :
060118 15:35:46  InnoDB: Error: in ALTER TABLE `test/t2`
InnoDB: has or is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
060118 15:35:46  InnoDB: Error: table `test/t2` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?
InnoDB: You can look for further help from
InnoDB: http://dev.mysql.com/doc/mysql/en/InnoDB_troubleshooting_datadict.html
060118 15:35:55  InnoDB: Error: in ALTER TABLE `test/t1`
InnoDB: has or is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
060118 15:35:55  InnoDB: Error: table `test/t1` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?
InnoDB: You can look for further help from
InnoDB: http://dev.mysql.com/doc/mysql/en/InnoDB_troubleshooting_datadict.html
060118 15:35:55  InnoDB: Error: in RENAME TABLE table `test/t1`
InnoDB: is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
mysqltest: Result length mismatch
(the last lines may be the most important ones)
Below are the diffs between actual and expected results:
-------------------------------------------------------
*** r/innodb.result     Tue Jan 17 17:17:39 2006
--- r/innodb.reject     Wed Jan 18 15:35:56 2006
***************
*** 1725,1737 ****
  set foreign_key_checks=0;
  create table t2 (a int primary key, b int, foreign key (b) references t1(a)) engine = innodb;
  create table t1(a char(10) primary key, b varchar(20)) engine = innodb;
! ERROR HY000: Can't create table './test/t1.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t2;
  set foreign_key_checks=0;
  create table t1(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=latin1;
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=utf8;
! ERROR HY000: Can't create table './test/t2.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t1;
  set foreign_key_checks=0;
--- 1725,1737 ----
  set foreign_key_checks=0;
  create table t2 (a int primary key, b int, foreign key (b) references t1(a)) engine = innodb;
  create table t1(a char(10) primary key, b varchar(20)) engine = innodb;
! ERROR HY000: Can't create table '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t1.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t2;
  set foreign_key_checks=0;
  create table t1(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=latin1;
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=utf8;
! ERROR HY000: Can't create table '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t2.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t1;
  set foreign_key_checks=0;
***************
*** 1751,1757 ****
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of './test/t3' to './test/t1' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
--- 1751,1757 ----
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t3' to '%s' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
-------------------------------------------------------

This effect was not noted in 4.1.16.

How to repeat:
Build "standard", test "embedded".
[26 Jul 2006 1:31] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/9567
[31 Jul 2006 11:14] Magnus Blåudd
Ian, can you explain why we need this. I.e why isn't the embedded server using the same paths as normal mysqld?
[31 Jul 2006 17:58] Ian Greenhoe
The reason the embedded server is using absolute paths and not relative paths is that the embedded server does not have control of what the current directory is (specifically, the app in which the embedded server is embedded has that control.)  Since the server can't control the current directory, it has to prepend the server directory to all of the paths that it uses.  (There is a global variable specifically for this purpose.)

This in turn means that the error messages produced by the embedded server contain the full path to the file in question.
[15 Aug 2006 18:02] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/10503

ChangeSet@1.2538, 2006-08-15 11:01:40-07:00, igreenhoe@anubis.greendragongames.com +1 -0
  Fix for bug #16635 (Error messages wrong: absolute path names, "%s"
  format code)
  
  Problem:  The embedded server is more verbose when reporting path
  names than the standalone server.  This is for two reasons:  First,
  since the embedded server has no control over the current working
  directory, it must prepend the path to all files.  Second, the error
  messages that caused a problem with the test case are reporting the
  error from the paths given to the function (with that directory
  prepended.)
  
  Solution:  Fix the test cases so that they ignore the paths returned
  in the error messages if, and only if, they match the path to what
  would otherwise be the current working directory of the server.
  ($MYSQLTEST_VARDIR)
  
  Note:  The origional solution I came up with was to add an
  additional argument to the functions that were reporting the error
  which contained a sanitized version of the path.  Upon a review of
  this code, it was established that this was not a feasable solution
  since the functions were potentially used in code outside of the
  server.  Since the error messages are reporting a superset of the
  required information to the user, it was decided that a better
  solution would be to fix the tests.
[1 Nov 2006 5:11] Ian Greenhoe
Putting this back to "In Progress" since patch is broken for 5.0 and up.
[12 Jun 2007 17:30] Joerg Bruehe
Same problem still in 4.1.23, in embedded runs everywhere:

-------------------------------------------------------
*** r/innodb.result
--- r/innodb.reject
***************
*** 1570,1576
  ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  drop table t2;
  create table t2 (id int(11) not null, id2 int(11) not null, constraint t1_id_fk foreign key (id2,id) references t1 (id))
engine = innodb;
! ERROR HY000: Can't create table './test/t2.frm' (errno: 150)
  create table t2 (a int auto_increment primary key, b int, index(b), foreign key (b) references t1(id), unique(b)) engine=
innodb;
  show create table t2;
  Table Create Table
--- 1570,1576
  ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  drop table t2;
  create table t2 (id int(11) not null, id2 int(11) not null, constraint t1_id_fk foreign key (id2,id) references t1 (id))
engine = innodb;
! ERROR HY000: Can't create table '/data0/mysqldev/tmp-200706072110-4.1.23-14857/production-icc-gli' (errno: 150)
  create table t2 (a int auto_increment primary key, b int, index(b), foreign key (b) references t1(id), unique(b)) engine=
innodb;
  show create table t2;
  Table Create Table
***************
...
***************
*** 1751,1757
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of './test/t3' to './test/t1' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
--- 1751,1757
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of '/data0/mysqldev/tmp-200706072110-4.1.23-14857/production-icc-gli' to '/data0/mysqldev/tm
p-200706072110-4.1.23-14857/production-icc-gli' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
-------------------------------------------------------

This is a test suite failure, increasing priority.
[12 Jun 2007 17:33] Joerg Bruehe
For any string replacement to work,
IMHO the absolute path names must not be truncated.
[24 Jul 2007 18:31] Joerg Bruehe
The patch proposed by Ian is part of the solution,
but an equivalent fix has already been applied before 4.1.23.

It still fails in release builds because the path names get truncated,
our directory structure for builds creates path names longer than the 64 characters in the error message formats.

In 5.0 (and up) these lengths have been increased to 150 characters.

Doing this in 4.1 may work with path names which do not really use that size, but will fail if they approach that limit.
This happens because the message formatting happens in a buffer which is limited to 256 characters.
If this buffer would be exceeded, values are just represented by their format code (like '%s' for a path name).

I found that a limit of 107 characters can still be handled within that work area (using the English message texts).
[26 Jul 2007 10:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31607

ChangeSet@1.2676, 2007-07-26 12:52:13+02:00, joerg@debian.(none) +1 -0
  Raise the 64 character limit for path names in the error messages
  to 150 or 107 characters for those messages which are generated
  by the embedded server during release builds.
  
  This fixes bug#16635:
     Error messages wrong: absolute path names, "%s" format code
  
  See the bug report or the changelog for "sql/share/english/errmsg.txt"
  for instructions how to do that with other languages, 
  even at the customer site, and for the restrictions to keep.
[26 Jul 2007 12:06] Chad MILLER
I'm happy with applying this to remove the test failure symptom.

I don't want this bug closed, though.  There's a real problem, and updating the English message with new lengths doesn't address it.  If you must close this bug, create a new one (p3 s2?) to describe the real problem.
[26 Jul 2007 13:06] Joerg Bruehe
I don't object to keeping it open / splitting off a new one,
but it is no crashing problem:
Our EOL policy will not let us fix that in 4.1

The only resaon to do this limited change now in 4.1
is to make builds go smoothly.

*If* some user wants to fix this for other languages,
or if a similar problem shows up with other messages,
this can be done even in a binary installation at the
customer site by following these steps:

       cd <<install-root>>/share
       $EDITOR <<lang>>/errmsg.txt
       ../../bin/comp_err -C./charsets/ <<lang>>/errmsg.txt <<lang>>/errmsg.sys

and then restarting the server.

The restriction is that the expanded message
(all format codes replaced by the respective values)
must fit in a 256 byte buffer within the server.

(If the buffer would overflow, a format specification will not be
expanded but just copied with its code, and the message output
will just contain '%s' or '%d' where a value is expected.
The code is protecting against buffer overflow, both by the
constant text and by the values to be expanded.)
[26 Jul 2007 19:42] Joerg Bruehe
Note this patch is specific to 4.1,

we currently do not build "embedded" in 5.0,
and in 5.1 this is fixed.
[2 Aug 2007 8:03] Magnus Blåudd
Joerg, can you check if this isn't a duplicate of BUG#13933 Path name replacement fails in case-insignificant systems
[2 Aug 2007 10:32] Joerg Bruehe
The two are not related.

See my entry of June 12, "fails everywhere":
The problem described here (and solved in 4.1) also occurred
on hosts with case-significant file systems,
whereas bug#13933 is specific to case-insignificant file systems.

They have in common that it only shows up in "embedded" tests
where the message contains the absolute path, 
not just the relative as in messages from a separate server.
[2 Aug 2007 19:13] Bugs System
Pushed into 5.1.21-beta
[2 Aug 2007 19:15] Bugs System
Pushed into 5.0.48
[2 Aug 2007 19:16] Bugs System
Pushed into 4.1.24
[3 Aug 2007 1:44] Paul Dubois
Noted in 4.1.24, 5.0.48, 5.1.21 changelogs.

Format strings in error messages were insufficiently wide for pathnames printed in those messages by the embedded server.
[6 Aug 2007 19:29] Paul Dubois
Correction: Change applies only to 4.1.24, and only for English messages.

Format strings in English error messages were insufficiently wide for
pathnames printed in those messages by the embedded server.