| Bug #52019 | main.mysqltest fails on new tests for lowercase_result | ||
|---|---|---|---|
| Submitted: | 12 Mar 2010 21:07 | Modified: | 14 Nov 2010 2:52 |
| Reporter: | Karen Langford | Email Updates: | |
| Status: | Closed | Impact on me: | |
| Category: | Tools: MTR / mysql-test-run | Severity: | S3 (Non-critical) |
| Version: | 5.1.45 | OS: | IBM AIX |
| Assigned to: | Bjørn Munch | CPU Architecture: | Any |
| Tags: | test_failure | ||
[25 Mar 2010 18:31]
[ name withheld ]
I observe this test hanging on Fedora 12 PPC/PPC64 as well; confirm the misbehavior is new as of .45.
[26 Mar 2010 1:50]
[ name withheld ]
I've looked into this problem and identified multiple related bugs in mysqltest.cc. The direct cause of the infinite loop is that this code is wrong: /* It was not a multiline char, push back the characters */ /* We leave first 'c', i.e. pretend it was a normal char */ while (p > mb_start) my_ungetc(*--p); Contrary to what the comment claims, the loop pushes back all bytes of a multi-byte character; so the next iteration of the outer loop of read_line() starts at the same place, does the same thing again, lather rinse repeat forever. Fix the loop test so that it actually does leave the first byte behind, and the infinite loop goes away. Having seen this, the question is not why it fails on PPC, but how come it manages to avoid failing everywhere. Further investigation showed that the culprit is the preceding call to my_mbcharlen(), which is passing a "char" to an argument that is "uint". On PPC, where char is unsigned by default, this behaves as intended and you get a charlen that is > 1, and then if the multibyte character isn't actually validly coded (ie, exactly the test case added in main.mysqltest in 5.1.45), hello infinite loop. But if char is signed by default, then something like \304 promotes to a negative integer and then to a very large unsigned integer, causing my_mbcharlen_utf8() to punt and return charlen = 0. So on signed-char machines the intended skipping of additional bytes of a multibyte char fails to happen. In most encodings this is fairly harmless since the outer loop won't recognize those bytes as anything in particular, but there are probably cases where bad things would happen. While I was sniffing around to find these problems, I noticed a few other bugs. The feof() test inside the loop that accumulates extra bytes of a multibyte character ought to be after the my_getc call not before it. This is relatively unlikely to be a problem in practice, but it's still wrong; and someone might copy the erroneous coding pattern to somewhere else where it's more likely to cause real problems. Also, in replace_dynstr_append_mem(), the temporary array lower[] is being declared in an inner block but the code relies on it remaning valid until the end of the function. You need to declare it at function scope instead to ensure the compiler doesn't optimize it out from under you. Since that code is brand new in 5.1.45, I don't think you should assume the lack of problem reports so far proves its portable. The attached patch covers the above problems. There were also two things I noticed but didn't bother to change. On a machine where char is signed, my_getc will return different values for the original return of a high-bit-set character (eg, \304) versus when it is returned again after being pushed back through line_buffer[] (then you'll get a sign-extended version). This seems to be harmless for now because most callers just store the value right back into a "char" variable, but it's going to bite you on the rear someday. I'd suggest a cast, or maybe change line_buffer to be unsigned char[] altogether. Another thing that's dangerous with signed chars is that match_delimiter() is comparing ints to chars. I think it will accidentally fail to malfunction as long as delimiter strings never contain any high-bit-set characters, but if they ever do, look out. (Oh, and you'd better think about the case of hitting EOF while trying to match a multibyte delimiter string, too.) I wonder BTW whether this particular call of my_mbcharlen() is the only one with this sort of bug. Perhaps you ought to consider changing its argument type to char, instead of expecting every caller to be careful about casting.
[26 Mar 2010 1:52]
[ name withheld ]
patch for some of the problems identified here
Attachment: mysql-lowercase-bug.patch (application/octet-stream, text), 1.59 KiB.
[11 Oct 2010 7:58]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/120472 2940 Bjorn Munch 2010-10-11 Bug #52019 main.mysqltest fails on new tests for lowercase_result Fixed as suggested above Confirmed on Linux PPC and on AIX 5.3
[20 Oct 2010 9:24]
Bugs System
A patch for this bug has been committed. After review, it may be pushed to the relevant source trees for release in the next version. You can access the patch from: http://lists.mysql.com/commits/121268 2943 Bjorn Munch 2010-10-20 Bug #52019 main.mysqltest fails on new tests for lowercase_result Limited to actual bug fix, fixing a while condition Again confirmed on Linux PPC and on AIX 5.3
[20 Oct 2010 10:24]
Bjørn Munch
Pushed to -mtr branches. I also fixed each of the separate identified problems with separate commits.
[29 Oct 2010 15:35]
Paul DuBois
Changes to test suite. No changelog entry needed.
[13 Nov 2010 16:22]
Bugs System
Pushed into mysql-trunk 5.6.99-m5 (revid:alexander.nozdrin@oracle.com-20101113155825-czmva9kg4n31anmu) (version source revid:alexander.nozdrin@oracle.com-20101113152450-2zzcm50e7i4j35v7) (merge vers: 5.6.1-m4) (pib:21)
[13 Nov 2010 16:29]
Bugs System
Pushed into mysql-next-mr (revid:alexander.nozdrin@oracle.com-20101113160336-atmtmfb3mzm4pz4i) (version source revid:vasil.dimov@oracle.com-20100629074804-359l9m9gniauxr94) (pib:21)
[18 Nov 2010 15:55]
Bugs System
Pushed into mysql-5.1 5.1.54 (revid:build@mysql.com-20101118153531-693taxtxyxpt037i) (version source revid:build@mysql.com-20101118153531-693taxtxyxpt037i) (merge vers: 5.1.54) (pib:21)
[16 Dec 2010 22:27]
Bugs System
Pushed into mysql-5.5 5.5.9 (revid:jonathan.perkin@oracle.com-20101216101358-fyzr1epq95a3yett) (version source revid:jonathan.perkin@oracle.com-20101216101358-fyzr1epq95a3yett) (merge vers: 5.5.9) (pib:24)
Description: This bug has only just appeared in 5.1.45, occurring on aix5.3 powerpc 32/64 bit platforms. Test seems to be having a problem with the 'test for lowercase_result' tests added to mysqltest: possibly test 7? (just guessing, its the next test but the results don't appear) # 7. Test missing lower casing of "unknown" characters --character_set utf8 --lowercase_result SELECT 0 as "WILL NOT lower case \304\313\320"; --character_set latin1 main.mysqltest [ retry-fail ] timeout after 900 seconds Test ended at YYYY-MM-DD HH:MM:SS Test case timeout after 900 seconds == /PATH/mysql-test/var/log/mysqltest.log == select "500g bl\345b\346rsyltet\370y" as "will be lower cased"; will be lower cased 500g bl\345b\346rsyltet\370y SELECT "UPPER" AS "WILL NOT BE lower cased"; WILL NOT BE lower cased UPPER UP SELECT 0 as "UP AGAIN"; UP AGAIN 0 select "abcdef" as "uvwxyz"; uvwxyz abcdef select "xyz" as name union select "abc" as name order by name desc; name abc xyz select 1 as "some new text"; some new text 1 == /PATH/mysql-test/var/tmp/analyze-timeout-mysqld.1.err == mysqltest: Could not open connection 'default' after 500 attempts: 2002 Can't connect to local MySQL server through socket '/data0/mysqldev/tmp-201003011009-5.1.45-6768/aix5.3-powerpc-64bit/test/mysql-test/var/tmp/mysqld.1.s' (79) How to repeat: Found when running the full test suite.