MySQL Bugs: #16635: Error messages wrong: absolute path names, "%s" format code

Bug #16635	Error messages wrong: absolute path names, "%s" format code
Submitted:	19 Jan 2006 14:52	Modified:	3 Aug 2007 1:44
Reporter:	Joerg Bruehe	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Embedded Library ( libmysqld )	Severity:	S3 (Non-critical)
Version:	4.1.17	OS:	Any (all Unix)
Assigned to:	Joerg Bruehe	CPU Architecture:	Any

Description:
Release build of 4.1.17, based on ChangeSet
  1.2470.8.1 06/01/17 01:03:03 konstantin@mysql.com +4 -0
  A fix for Bug#13337 "ps test fails if configure wo/ usc2"

Test case "innodb" fails when run in "embedded" mode, output typically is like:

=====
innodb                         [ fail ]

Errors are (from /mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/var/log/mysqltest-time) :
060118 15:35:46  InnoDB: Error: in ALTER TABLE `test/t2`
InnoDB: has or is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
060118 15:35:46  InnoDB: Error: table `test/t2` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?
InnoDB: You can look for further help from
InnoDB: http://dev.mysql.com/doc/mysql/en/InnoDB_troubleshooting_datadict.html
060118 15:35:55  InnoDB: Error: in ALTER TABLE `test/t1`
InnoDB: has or is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
060118 15:35:55  InnoDB: Error: table `test/t1` does not exist in the InnoDB internal
InnoDB: data dictionary though MySQL is trying to drop it.
InnoDB: Have you copied the .frm file of the table to the
InnoDB: MySQL database directory from another database?
InnoDB: You can look for further help from
InnoDB: http://dev.mysql.com/doc/mysql/en/InnoDB_troubleshooting_datadict.html
060118 15:35:55  InnoDB: Error: in RENAME TABLE table `test/t1`
InnoDB: is referenced in foreign key constraints
InnoDB: which are not compatible with the new table definition.
mysqltest: Result length mismatch
(the last lines may be the most important ones)
Below are the diffs between actual and expected results:
-------------------------------------------------------
*** r/innodb.result     Tue Jan 17 17:17:39 2006
--- r/innodb.reject     Wed Jan 18 15:35:56 2006
***************
*** 1725,1737 ****
  set foreign_key_checks=0;
  create table t2 (a int primary key, b int, foreign key (b) references t1(a)) engine = innodb;
  create table t1(a char(10) primary key, b varchar(20)) engine = innodb;
! ERROR HY000: Can't create table './test/t1.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t2;
  set foreign_key_checks=0;
  create table t1(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=latin1;
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=utf8;
! ERROR HY000: Can't create table './test/t2.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t1;
  set foreign_key_checks=0;
--- 1725,1737 ----
  set foreign_key_checks=0;
  create table t2 (a int primary key, b int, foreign key (b) references t1(a)) engine = innodb;
  create table t1(a char(10) primary key, b varchar(20)) engine = innodb;
! ERROR HY000: Can't create table '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t1.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t2;
  set foreign_key_checks=0;
  create table t1(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=latin1;
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=utf8;
! ERROR HY000: Can't create table '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t2.frm' (errno: 150)
  set foreign_key_checks=1;
  drop table t1;
  set foreign_key_checks=0;
***************
*** 1751,1757 ****
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of './test/t3' to './test/t1' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
--- 1751,1757 ----
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of '/mysqldev/mysqldev/aix43/test/mysql-standard-4.1.17-ibm-aix4.3.3.0-powerpc/mysql-test/va
r/master-data/test/t3' to '%s' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
-------------------------------------------------------

This effect was not noted in 4.1.16.

How to repeat:
Build "standard", test "embedded".

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/9567

Ian, can you explain why we need this. I.e why isn't the embedded server using the same paths as normal mysqld?

The reason the embedded server is using absolute paths and not relative paths is that the embedded server does not have control of what the current directory is (specifically, the app in which the embedded server is embedded has that control.)  Since the server can't control the current directory, it has to prepend the server directory to all of the paths that it uses.  (There is a global variable specifically for this purpose.)

This in turn means that the error messages produced by the embedded server contain the full path to the file in question.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/10503

ChangeSet@1.2538, 2006-08-15 11:01:40-07:00, igreenhoe@anubis.greendragongames.com +1 -0
  Fix for bug #16635 (Error messages wrong: absolute path names, "%s"
  format code)
  
  Problem:  The embedded server is more verbose when reporting path
  names than the standalone server.  This is for two reasons:  First,
  since the embedded server has no control over the current working
  directory, it must prepend the path to all files.  Second, the error
  messages that caused a problem with the test case are reporting the
  error from the paths given to the function (with that directory
  prepended.)
  
  Solution:  Fix the test cases so that they ignore the paths returned
  in the error messages if, and only if, they match the path to what
  would otherwise be the current working directory of the server.
  ($MYSQLTEST_VARDIR)
  
  Note:  The origional solution I came up with was to add an
  additional argument to the functions that were reporting the error
  which contained a sanitized version of the path.  Upon a review of
  this code, it was established that this was not a feasable solution
  since the functions were potentially used in code outside of the
  server.  Since the error messages are reporting a superset of the
  required information to the user, it was decided that a better
  solution would be to fix the tests.

Putting this back to "In Progress" since patch is broken for 5.0 and up.

Same problem still in 4.1.23, in embedded runs everywhere:

-------------------------------------------------------
*** r/innodb.result
--- r/innodb.reject
***************
*** 1570,1576
  ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  drop table t2;
  create table t2 (id int(11) not null, id2 int(11) not null, constraint t1_id_fk foreign key (id2,id) references t1 (id))
engine = innodb;
! ERROR HY000: Can't create table './test/t2.frm' (errno: 150)
  create table t2 (a int auto_increment primary key, b int, index(b), foreign key (b) references t1(id), unique(b)) engine=
innodb;
  show create table t2;
  Table Create Table
--- 1570,1576
  ) ENGINE=InnoDB DEFAULT CHARSET=latin1
  drop table t2;
  create table t2 (id int(11) not null, id2 int(11) not null, constraint t1_id_fk foreign key (id2,id) references t1 (id))
engine = innodb;
! ERROR HY000: Can't create table '/data0/mysqldev/tmp-200706072110-4.1.23-14857/production-icc-gli' (errno: 150)
  create table t2 (a int auto_increment primary key, b int, index(b), foreign key (b) references t1(id), unique(b)) engine=
innodb;
  show create table t2;
  Table Create Table
***************
...
***************
*** 1751,1757
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of './test/t3' to './test/t1' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
--- 1751,1757
  create table t2 (a varchar(10), foreign key (a) references t1(a)) engine = innodb DEFAULT CHARSET=latin1;
  create table t3(a varchar(10) primary key) engine = innodb DEFAULT CHARSET=utf8;
  rename table t3 to t1;
! ERROR HY000: Error on rename of '/data0/mysqldev/tmp-200706072110-4.1.23-14857/production-icc-gli' to '/data0/mysqldev/tm
p-200706072110-4.1.23-14857/production-icc-gli' (errno: 150)
  set foreign_key_checks=1;
  drop table t2,t3;
  create table t2 (
-------------------------------------------------------

This is a test suite failure, increasing priority.

For any string replacement to work,
IMHO the absolute path names must not be truncated.

The patch proposed by Ian is part of the solution,
but an equivalent fix has already been applied before 4.1.23.

It still fails in release builds because the path names get truncated,
our directory structure for builds creates path names longer than the 64 characters in the error message formats.

In 5.0 (and up) these lengths have been increased to 150 characters.

Doing this in 4.1 may work with path names which do not really use that size, but will fail if they approach that limit.
This happens because the message formatting happens in a buffer which is limited to 256 characters.
If this buffer would be exceeded, values are just represented by their format code (like '%s' for a path name).

I found that a limit of 107 characters can still be handled within that work area (using the English message texts).

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/31607

ChangeSet@1.2676, 2007-07-26 12:52:13+02:00, joerg@debian.(none) +1 -0
  Raise the 64 character limit for path names in the error messages
  to 150 or 107 characters for those messages which are generated
  by the embedded server during release builds.
  
  This fixes bug#16635:
     Error messages wrong: absolute path names, "%s" format code
  
  See the bug report or the changelog for "sql/share/english/errmsg.txt"
  for instructions how to do that with other languages, 
  even at the customer site, and for the restrictions to keep.

I'm happy with applying this to remove the test failure symptom.

I don't want this bug closed, though.  There's a real problem, and updating the English message with new lengths doesn't address it.  If you must close this bug, create a new one (p3 s2?) to describe the real problem.

I don't object to keeping it open / splitting off a new one,
but it is no crashing problem:
Our EOL policy will not let us fix that in 4.1

The only resaon to do this limited change now in 4.1
is to make builds go smoothly.

*If* some user wants to fix this for other languages,
or if a similar problem shows up with other messages,
this can be done even in a binary installation at the
customer site by following these steps:

       cd <<install-root>>/share
       $EDITOR <<lang>>/errmsg.txt
       ../../bin/comp_err -C./charsets/ <<lang>>/errmsg.txt <<lang>>/errmsg.sys

and then restarting the server.

The restriction is that the expanded message
(all format codes replaced by the respective values)
must fit in a 256 byte buffer within the server.

(If the buffer would overflow, a format specification will not be
expanded but just copied with its code, and the message output
will just contain '%s' or '%d' where a value is expected.
The code is protecting against buffer overflow, both by the
constant text and by the values to be expanded.)

Note this patch is specific to 4.1,

we currently do not build "embedded" in 5.0,
and in 5.1 this is fixed.

Joerg, can you check if this isn't a duplicate of BUG#13933 Path name replacement fails in case-insignificant systems

The two are not related.

See my entry of June 12, "fails everywhere":
The problem described here (and solved in 4.1) also occurred
on hosts with case-significant file systems,
whereas bug#13933 is specific to case-insignificant file systems.

They have in common that it only shows up in "embedded" tests
where the message contains the absolute path, 
not just the relative as in messages from a separate server.

Pushed into 5.1.21-beta

Pushed into 5.0.48

Pushed into 4.1.24

Noted in 4.1.24, 5.0.48, 5.1.21 changelogs.

Format strings in error messages were insufficiently wide for pathnames printed in those messages by the embedded server.

Correction: Change applies only to 4.1.24, and only for English messages.

Format strings in English error messages were insufficiently wide for
pathnames printed in those messages by the embedded server.