| Bug #72150 | mysqlcheck utility on Windows fails with spurious error on non-latin table names | ||
|---|---|---|---|
| Submitted: | 28 Mar 2014 12:10 | Modified: | 4 Aug 2014 9:15 |
| Reporter: | Peter Laursen (Basic Quality Contributor) | Email Updates: | |
| Status: | No Feedback | Impact on me: | |
| Category: | MySQL Server: Installing | Severity: | S2 (Serious) |
| Version: | 5.1.73, 5.6.16 | OS: | Windows (7/64) |
| Assigned to: | CPU Architecture: | Any | |
[28 Mar 2014 12:17]
Peter Laursen
With hindsight, I think the problem may be the table name 'русский'. This is cyrillic and my system LOCALE is Western (Danish) If this means that mysql_upgrade is not capable of handling Unicode file names on Windows, it is a pretty serious bug IMO. So assuming that I located the problem I am reasing severity to "S2".
[28 Mar 2014 13:10]
Peter Laursen
I can run "CHECK TABLE `test`.`русский`;" from inside a utf8 GUI-client (SQLyog) with no issues. But mysqlcheck from commandline: C:\Program Files\MySQL\MySQL Server 5.1\bin>mysqlcheck -uroot -p test Enter password: ******** test.??????? Error : Can't find file: '.\test\@003f@003f@003f@003f@003f@003f@003f.frm' (er rno: 22) status : Operation failed test.tableb OK test.testtable1 OK test.dansk OK test.dup OK test.foo OK test.mysqlbug OK test.pl OK test.some_table OK test.tablea OK test.tableinventory OK test.test OK test.testrcaai OK test.test_table OK test.testtable2 OK test.tst OK test.u8 OK test.x'02' OK C:\Program Files\MySQL\MySQL Server 5.1\bin>
[28 Mar 2014 13:45]
Peter Laursen
I tried to check this on 5.6, but was stopped by another bug! http://bugs.mysql.com/bug.php?id=72152
[28 Mar 2014 19:32]
Peter Laursen
the 'showstopper' in bug http://bugs.mysql.com/bug.php?id=72152 was resolved (thanks to Sveta). But I can now confirm that also on MySQL 5.6.16, the 'mysqlcheck' program fails to handle a tabel named with cyrillic characters (`русский`) also on this version (in Windows using Western/Danish LOCALE).
[28 Mar 2014 20:25]
Peter Laursen
Why is it "Looking for 'mysql.exe'" BTW? There is is no interactive input required.
[28 Mar 2014 20:36]
MySQL Verification Team
mysql_upgrade runs mysql.exe to check version, datadir, and do that mysql_fix_privilege_tables in batch mode...
[28 Mar 2014 21:28]
Peter Laursen
Thanks @Shane. I am learning every day! :-)
[28 Mar 2014 21:34]
MySQL Verification Team
About the weird error 22, I wonder if it is addressed in 5.7 by: Bug 14642248 - NONSENSE ERROR ON WINDOWS (ERRNO: 22 - INVALID ARGUMENT) Also: http://bugs.mysql.com/bug.php?id=68618
[28 Mar 2014 21:39]
Peter Laursen
I have one serious concern!
If the table name 'русский' ('russian' in Russian language) maps to the .frm file '@003f@003f@003f@003f@003f@003f@003f.frm' what will then the table name 'український' ('ukrainian' in Ukrainian language) map to? If it also maps to '@003f@003f@003f@003f@003f@003f@003f.frm' this means then when creating the 2nd table (in any order - not commenting on the European security situation!) the .frm file of the 1st table will be overwritten when the 2nd is created.
This would be very serious. But it is too late now here and I am too tired now to check this tonight. I have created mistaken reports before when it was too late and I was bombed. So I think I should rather wait till after the weekend!
[28 Mar 2014 21:41]
Peter Laursen
@Shane .. maybe. You can better verify than I (as you read code, and I don't). But if so it should definitely be backported to 5.6 (and 5.5 if also affected).
[29 Mar 2014 13:21]
Peter Laursen
My concern was not jsutified. .frm files are created as expected.
Anyway mysqlcheck fails. It looks for incorrect file names only containing sequences of '@003f'.
Here is a clean testcase on MySQL 5.6.16:
-- prepare tables
CREATE DATABASE frmtest;
USE frmtest;
CREATE TABLE `русский` (`id` INT);
CREATE TABLE `український` (`id` INT);
/* check file system
C:\ProgramData\mysql\MySQL Server 5.6\data\frmtest>dir
Disken i drev C har ikke noget navn.
Diskens serienummer er 40AF-D553
Indhold af C:\ProgramData\mysql\MySQL Server 5.6\data\frmtest
29-03-2014 14:04 <DIR> .
29-03-2014 14:04 <DIR> ..
29-03-2014 14:02 8.556 @w0@z0@x0@x0@q0@o0@p0.frm
29-03-2014 14:02 98.304 @w0@z0@x0@x0@q0@o0@p0.ibd
29-03-2014 14:04 8.556 @z0@q0@w0@g0@z1@t0@x0@o1@q0@o0@p0.frm
29-03-2014 14:04 98.304 @z0@q0@w0@g0@z1@t0@x0@o1@q0@o0@p0.ibd
29-03-2014 14:00 61 db.opt
5 fil(er) 213.781 byte
2 mappe(r) 22.679.900.160 byte ledig
C:\ProgramData\mysql\MySQL Server 5.6\data\frmtest>
*/
/*running mysqltest
C:\Program Files\MySQL\MySQL Server 5.6\bin>mysqlcheck -uroot -p frmtest
Enter password:
frmtest.???????
Error : Can't find file: '.\frmtest\@003f@003f@003f@003f@003f@003f@003f.frm'
(errno: 22 - Invalid argument)
status : Operation failed
frmtest.???????????
Error : Can't find file: '.\frmtest\@003f@003f@003f@003f@003f@003f@003f@003f@
003f@003f@003f.frm' (errno: 22 - Invalid argument)
status : Operation failed
C:\Program Files\MySQL\MySQL Server 5.6\bin>
*/
(and BTW .. it would be relevant to check if the situation is the same on a system running various non-western locales, I think. But I don't have this option myself)
I have updated synopsis according to my findings. I also added OS details as "Windows" as I can only state it is a problem here.
[29 Mar 2014 13:47]
Peter Laursen
On 5.0.96 it also fails, but error displays a little different:
C:\Program Files\MySQL\MySQL Server 5.0\bin>mysqlcheck -uroot -p --port=3309 frm
test
Enter password: ********
frmtest.???????
Error : Can't find file: '.\frmtest\???????.frm' (errno: 22)
error : Corrupt
frmtest.???????????
Error : Can't find file: '.\frmtest\???????????.frm' (errno: 22)
error : Corrupt
C:\Program Files\MySQL\MySQL Server 5.0\bin>
.. and the .frm file names are also different:
C:\Program Files\MySQL\MySQL Server 5.0\data\frmtest>dir
Disken i drev C har ikke noget navn.
Diskens serienummer er 40AF-D553
Indhold af C:\Program Files\MySQL\MySQL Server 5.0\data\frmtest
29-03-2014 14:42 <DIR> .
29-03-2014 14:42 <DIR> ..
29-03-2014 14:42 61 db.opt
29-03-2014 14:42 8.556 укÑ?аÑ-нÑ?ÑOкий.frm
29-03-2014 14:42 8.556 Ñ?уÑ?Ñ?кий.frm
3 fil(er) 17.173 byte
2 mappe(r) 22.678.491.136 byte ledig
C:\Program Files\MySQL\MySQL Server 5.0\data\frmtest>
[1 Apr 2014 19:23]
Peter Laursen
Besides .. I do not really understand what this encoding of non-ASCCI characters (@***) is for. Is there any supported OS that does not have full unicode support in the file system? I don't think so. (but I realize that for 10+ years MySQL developers believed that Windows doesn't support Unicode. But that is a stupid mistake (*Cygwin thinking of Windows*, so to say!). There is absolutely no need to encode any file name using Unicode (within BMP at least) as far as I can understand. Just create files (.ibd, .frm, whatever) with unicode file names!
[1 Apr 2014 19:27]
Peter Laursen
Just create folders and files (.ibd, .frm, whatever) with unicode file names (using the default Unicode encoding - UTF8, UTF16LE or UTF16BE specific for the platform)! With hindsight this *encoding* (sic!) is no less than an extremely bad joke!
[4 Jul 2014 9:15]
MySQL Verification Team
Thank you for the report and test case.
I could not repeat this with latest GA version. Could you please confirm if it is still repeatable at your end on 5.6.19?
// 5.6.19 on Windows server 2008 R2
F:\ushastry\mysql5.6.19>chcp 65001
Active code page: 65001
Your MySQL connection id is 4
Server version: 5.6.19-enterprise-commercial-advanced MySQL Enterprise Server - Advanced Edition (Commercial)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> drop database frmtest;
Query OK, 2 rows affected (0.07 sec)
mysql> CREATE DATABASE frmtest;
Query OK, 1 row affected (0.01 sec)
mysql> USE frmtest;
Database changed
mysql> CREATE TABLE `русский` (`id` INT);
Query OK, 0 rows affected (0.04 sec)
mysql> CREATE TABLE `український` (`id` INT);
Query OK, 0 rows affected (0.04 sec)
mysql> \q
Bye
F:\ushastry\mysql5.6.19>bin\mysqlcheck -uroot -p --port=5619 frmtest
Enter password:
frmtest.русский OK
frmtest.український OK
F:\ushastry\mysql5.6.19>dir data/
F:\ushastry\mysql5.6.19>dir data\frmtest
Volume in drive F is Data
Volume Serial Number is 882C-3C6D
Directory of F:\ushastry\mysql5.6.19\data\frmtest
07/04/2014 11:04 AM <DIR> .
07/04/2014 11:04 AM <DIR> ..
07/04/2014 11:04 AM 8,556 @w0@z0@x0@x0@q0@o0@p0.frm
07/04/2014 11:04 AM 98,304 @w0@z0@x0@x0@q0@o0@p0.ibd
07/04/2014 11:04 AM 8,556 @z0@q0@w0@g0@z1@t0@x0@o1@q0@o0@p0.frm
07/04/2014 11:04 AM 98,304 @z0@q0@w0@g0@z1@t0@x0@o1@q0@o0@p0.ibd
07/04/2014 11:04 AM 65 db.opt
5 File(s) 213,785 bytes
2 Dir(s) 1,757,532,979,200 bytes free
[5 Aug 2014 1:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
Description: I was upgrading my local 5.1 server instance from 5.1.72 to 5.1.73. After running the Windows installer (mysql-5.1.73-winx64.msi) I ran 'mysql_upgrade'. C:\Program Files\MySQL\MySQL Server 5.1\bin>mysql_upgrade -uroot -p --port=3306 Enter password: ******** Looking for 'mysql.exe' as: C:\Program Files\MySQL\MySQL Server 5.1\bin\mysql.ex e Looking for 'mysqlcheck.exe' as: C:\Program Files\MySQL\MySQL Server 5.1\bin\mys qlcheck.exe Running 'mysqlcheck' with connection arguments: "--port=3306" "--port=3306" Running 'mysqlcheck' with connection arguments: "--port=3306" "--port=3306" mydb.t OK mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log Error : You can't use locks with log tables. status : OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.servers OK mysql.slow_log Error : You can't use locks with log tables. status : OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK peter.u8 OK test.??????? Error : Can't find file: '.\test\@003f@003f@003f@003f@003f@003f@003f.frm' (er rno: 22) status : Operation failed test.tableb OK test.testtable1 OK test.dansk OK test.dup OK test.foo OK test.mysqlbug OK test.pl OK test.some_table OK test.tablea OK test.tableinventory OK test.test OK test.testrcaai OK test.test_table OK test.testtable2 OK test.tst OK test.u8 OK test.x'02' OK test_db.test_innodb_table OK times.times OK vtest.v1 OK Repairing tables test.??????? Error : Can't find file: '.\test\@003f@003f@003f@003f@003f@003f@003f.frm' (er rno: 22) status : Operation failed Running 'mysql_fix_privilege_tables'... OK C:\Program Files\MySQL\MySQL Server 5.1\bin> Tables recognized from interactive client in test database are: USE test; SHOW FULL TABLES FROM `test` WHERE table_type = 'BASE TABLE'; /* returns Tables_in_test Table_type -------------- ------------ русский BASE TABLE TableB BASE TABLE TestTable1 BASE TABLE dansk BASE TABLE dup BASE TABLE foo BASE TABLE mysqlbug BASE TABLE pl BASE TABLE some_table BASE TABLE tableA BASE TABLE tableinventory BASE TABLE test BASE TABLE testRCAAI BASE TABLE test_table BASE TABLE testtable2 BASE TABLE tst BASE TABLE u8 BASE TABLE x'02' BASE TABLE */ Checking from file system in /datadir/test: C:\ProgramData\mysql\MySQL Server 5.1\data\test>dir Disken i drev C har ikke noget navn. Diskens serienummer er 40AF-D553 Indhold af C:\ProgramData\mysql\MySQL Server 5.1\data\test 23-01-2013 05:23 <DIR> . 23-01-2013 05:23 <DIR> .. 02-02-2012 20:26 8.584 @w0@z0@x0@x0@q0@o0@p0.frm 02-02-2012 16:42 8.584 dansk.frm 03-12-2012 13:29 8.584 dup.frm 06-10-2012 08:54 8.604 foo.frm 23-09-2012 15:06 8.594 mysqlbug.frm 23-09-2012 15:06 7 mysqlbug.MYD 23-09-2012 15:14 1.024 mysqlbug.MYI 30-06-2012 14:54 8.556 pl.frm 30-06-2012 14:54 0 pl.MYD 30-06-2012 14:54 1.024 pl.MYI 02-09-2012 20:01 8.612 some_table.frm 23-01-2013 05:23 8.556 tableA.frm 29-09-2012 09:20 8.590 TableB.frm 16-03-2012 16:26 8.604 tableinventory.frm 15-11-2012 12:51 8.562 test.frm 14-03-2012 09:25 8.630 testRCAAI.frm 14-03-2012 09:25 30 testRCAAI.MYD 14-03-2012 09:25 3.072 testRCAAI.MYI 02-02-2012 17:43 8.590 TestTable1.frm 02-02-2012 17:43 0 TestTable1.MYD 02-02-2012 17:43 1.024 TestTable1.MYI 02-02-2012 17:43 8.590 testtable2.frm 02-02-2012 17:43 0 testtable2.MYD 02-02-2012 17:43 1.024 testtable2.MYI 07-02-2012 16:03 8.592 test_table.frm 02-08-2012 18:47 8.558 tst.frm 29-11-2012 21:12 8.584 u8.frm 29-09-2012 09:20 535 viewb.frm 14-04-2012 09:07 8.556 x@002702@0027.frm 29 fil(er) 162.270 byte 2 mappe(r) 24.843.960.320 byte ledig C:\ProgramData\mysql\MySQL Server 5.1\data\test> How to repeat: see above! I assume that in "???????" the "?"s are just placeholders for characters that cannot be printed. I am not able to decode the filename '@003f@003f@003f@003f@003f@003f@003f.frm'. Is this/should this be '@w0@z0@x0@x0@q0@o0@p0.frm' rather? It is possible that I have created some table (for testing) using complete nonsense characters - maybe even non-printable characters. But every Unicode character except for for the NULL-character should be valid from version 5.1 and mysql_upgrade should handle them! Or maybe this is another example of what Kolbe Kegel from SkySQL blogged about here http://www.skysql.com/blogs/kolbe/get-rid-orphaned-innodb-temporary-tables-right-way recently? Except for this error during mysql_upgrade, the database seems to work 100% fine, so this is likely more a cosmetical than a real problem. Anyway I am not sure and I think this should be checked (and also with more recent server versions). Suggested fix: Check what happens here! Is mysql_upgrade (on Windows) not able to handle valid (even though stupid) table names, for instance? (and also ignore the error "You can't use locks with log tables" in this discussion. It is reported elsewhere and is irrelevant for the discussion here)