| Bug #56872 | Greek characters display issue | ||
|---|---|---|---|
| Submitted: | 20 Sep 2010 14:28 | Modified: | 11 Aug 2013 6:53 |
| Reporter: | Bassu Bassu | Email Updates: | |
| Status: | No Feedback | Impact on me: | |
| Category: | Connector / ODBC | Severity: | S1 (Critical) |
| Version: | 3.51/5.1 | OS: | Windows (Server 2003) |
| Assigned to: | Assigned Account | CPU Architecture: | Any |
| Tags: | Fetching, ODBC dirver, utf8 characters cut off | ||
[21 Sep 2010 5:52]
Tonci Grgin
Hi Bassu and thanks for your report. There is no test case attached so I'm asking you to: o Attach small but complete test case demonstrating your problem. o Be sure to include connection string or complete description of DNS (type, options...) o Start mysql.exe command line client and provide me with output of SHOW VARIABLES LIKE "%char%"; statement. Only after this I will be able to check on problem reported.
[22 Sep 2010 6:50]
Bassu Bassu
Screenshot demonstrating the problem
Attachment: screenshot1.png (image/png, text), 123.74 KiB.
[22 Sep 2010 6:52]
Bassu Bassu
Just 1K classic asp file using odbc driver to show the problem
Attachment: test.asp (application/octet-stream, text), 659 bytes.
[22 Sep 2010 6:54]
Bassu Bassu
Using odbc with PHP, same problem
Attachment: test2.php (application/x-httpd-php, text), 340 bytes.
[22 Sep 2010 7:13]
Bassu Bassu
Hi Tonci, Thanks for replying back. I believe with the screenshot attached to this bug now, demonstrates actual problem. Sample test code including connection strings is attached as well. This is output of charset variables you were looking for. mysql> SHOW VARIABLES LIKE '%char%'; +--------------------------+------------------------------------------------------------------+ | Variable_name | Value | +--------------------------+------------------------------------------------------------------+ | character_set_client | utf8 | | character_set_connection | utf8 | | character_set_database | utf8 | | character_set_filesystem | binary | | character_set_results | utf8 | | character_set_server | latin1 | | character_set_system | utf8 | | character_sets_dir | ...\MySQL\share\charsets\ +--------------------------+------------------------------------------------------------------+ 8 rows in set (0.02 sec)
[22 Sep 2010 17:16]
Sveta Smirnova
Bug #56847 was marked as duplicate of this one.
[27 Sep 2010 7:36]
Tonci Grgin
Bassu, what happens if you start your server with --default-character-set=utf8? Also, please do use c/ODBC 5.1 and tell me which DNS options you checked.
[28 Sep 2010 7:30]
Bassu Bassu
Well, same thing happens when mysql is started with default charset utf8. I already tested it out with latest odbc connector 5.1 as mentioned in opening statement of this bug. The only option I checked in ODBC connection properites or in its connection string is the connecting character set that of course is set to utf8. As a matter of fact, Rick has narrowed down the problem to following. ODBC drivers 3.51 and 5.1 cut off Greek characters depending upon the current limit of varchar characters set in field properties of a table. If a field has a limit of varchar(255), the odbc connector only shows first 139 characters of it and cuts of rest of them. Details of Rick's findings are as under. Number of varchar characters set --- Number of characters downloaded 255 139 355 191 400 216 450 244 500 251 600 327 700 382 750 409 800 433+
[1 Oct 2010 4:00]
Bassu Bassu
This problem seems to be worse than it was initially thought to be. Also is happening to the Russian characters. So I increased severity level. Number of Russian characters set -- Number of characters downloaded 450 239 600 299 700 379 800 404+
[2 Oct 2010 5:15]
Bassu Bassu
This has become critical as its effecting more unicode languages. Has been duplicated with Persian. That means it would also be happening to Arabic and Urdu. I now assume that all utf8 encoded languages may be suffering the same characters' cut off behaviour with odbc connector.
[5 Oct 2010 10:29]
Lawrenty Novitsky
Hi Bassu, Rick in the Bug#56847 says that "...The above code works fine with Danish, Dutch, English, Farsi, and Russian." Now you say that both farsi and russian don't work too. What has changed since then? On other hand i would imagine that if there is problem, it would affect all languages(well, almost all in fact). Can you get odbc trace log for the php script(the one using odbc of course) run? It would be possible if you run php under apache, and not iis.
[6 Oct 2010 9:09]
Lawrenty Novitsky
Bassu, I have reasons to think that problem is in your old server version. There was a bug in metadata supplied by server in your version. It could cause the problem you face. Please test against latest 5.0 version(i would suggest you to consider 5.1 too) and if you still observe the problem, then we will continue to work on that bug. This test is even more important then trace log i asked before.
[6 Oct 2010 10:05]
Bogdan Degtyariov
Greek characters displayed correctly for me after setting the code page in ASP script as follows (instead of 1252): Response.CodePage = 65001
[6 Oct 2010 10:13]
Lawrenty Novitsky
one more thing, if you still see the problem with latest server, then please provide us with sql dump of the table you use for testing. and trace log would still help.
[7 Oct 2010 8:42]
Lawrenty Novitsky
oh, I missed somehow Bogdan's comment. Disregard my requests -I think his advice should fix your problem and there is no bug here.
[7 Oct 2010 12:19]
Bassu Bassu
Bogdan, using ASP code page of 65001 doesn't make any difference. Instead it grumbles most of the characters depending upon current collation. Lawrin, I just tested out with latest 5.0 version on a different server machine, which is having 5.0.91-community-nt and it has the same behaviour there as well. Attached are both test table dump and the trace logs. The test table has one row having 496 Persian characters and field limit of 500 varchar. If you'll perform a select query with odbc, you'll see its only showing 380 characters and rest of 116 characters are cut off. This of course doesn't happen when I run query directly through phpmyadmin or with myql_connect.
[7 Oct 2010 12:24]
Bassu Bassu
One thing more, when I ran my mysql in debug mode with mysql-debug.exe, I didn't see any output except 'incorrect information in tablename' errors.
[11 Oct 2010 8:53]
Lawrenty Novitsky
Bassu, No, that log is not of much help. We need ODBC driver manager trace log since we cannot reproduce the bug. Note, that one gets nothing in that log if he runs asp script under IIS. That's why i told use c/odbce in php under apache for that.
[11 Jul 2013 6:53]
Bogdan Degtyariov
Hello Bassu, We have been checking the bugs database and stumbled upon this report, which is still waiting on your feedback. Have you managed to resolve the issue? Thanks.
[12 Aug 2013 1:00]
Bugs System
No feedback was provided for this bug for over a month, so it is being suspended automatically. If you are able to provide the information that was originally requested, please do so and change the status of the bug back to "Open".
Description: Tables having greek characters are not getting fully displayed and some of characters from end of each row are being cut off and replaced by a grumbled character ('�') when fetching data from a db using standard odbc connector. Some of the facts: - Tables/fields have proper encoding (utf8), type (varchar) and char limit set. - Fetching characters of other languages works just fine from the same table. - ODBC connectors 3.51 and latest 5.1, both have same behaviour. - ODBC connection encoding is set to 'utf8' of course! - Setting odbc advanced connection properties options like 'allow big chars' and 'display full results' also don't make it work. Why I think its a bug in odbc connector: - Testing with odbc-less connection and PHP's mysql_connect, full line shows up displaying all greek characters. Encoding is set with 'SET NAMES' here. - Testing with classic ASP and odbc cuts off last few characters. Encoding is set with Response.CharSet. - Testing with odbc and PHP's odbc_connect does the same and cuts off last few characters of selection. How to repeat: 1. Create a table having column of type varchar and collation utf8. 2. Insert this line of greek characters into it Σχεδóν σε óλα óσα κάνω, το φυσικότερο πράγμα για μένα είναι να αναπτυχθεί μία ρουτίνα ή προσωπική τελετουργία-ένα κανονικό σχέδιο δράσης που λειτσμργεί αποτελεσματικά για μένα, και που επιθυμώ να επαναλάβω. 3. Do a SELECT using odbc connector either with PHP or with classic ASP and you'll see that last few characters are missing from the full row. In this case the characters missing are: "να επαναλάβω"