Bug #6536 ODBC connect function of PHP randomly fails to initialize latin1 character set
Submitted: 10 Nov 2004 1:25 Modified: 7 Dec 2007 10:39
Reporter: Eric Wilde Email Updates:
Status: Duplicate Impact on me:
None 
Category:Connector / ODBC Severity:S2 (Serious)
Version:3.51, 5.1 OS:Linux (RedHat Linux 8)
Assigned to:

[10 Nov 2004 1:25] Eric Wilde
Description:
The odbc_connect function call in a PHP script fails with the error:

SQL error: [unixODBC][MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/local/share/mysql/charsets/), SQL state S1000 in SQLConnect

MySQL was compiled by me with the following configure flag: --with-extra-charsets=complex.  Furthermore, inspection of the config.log file shows:

configure:34751: checking "character sets"
configure:35281: result: default: latin1, collation: latin1_swedish_ci; compiled in: latin1 latin1 utf8 big5 cp1250 euckr gb2312 gbk latin1 latin2 sjis tis620 ucs2 ujis utf8.

Note that the latin1 character set appears in the list of compiled in character sets a few times.

Also, the /usr/local/share/mysql/charsets/Index.xml file contains an apparently valid definition of the latin1 character set and there is an equally apparently valid latin1.xml in the same directory.

The failure occurs between 50% and 65% of the time, on a random basis. 

The Apache error log shows: Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/share/mysql/charsets/Index.xml' file

How to repeat:
On RedHat 8, under Apache 2.0.49 and PHP 5.0.2, using MySQL 4.1.7 and MyODBC 3.51.09, the following page will fail randomly:

<html>
<head>
<title>ODBC test script</title>
</head>
<p align="center"><b>ODBC Test Script</b></p>
<p>
<?
// Get a connection to the database.
if (($TrackingConn = odbc_connect("Tracking", "xyz", "xyzpass")) == 0)
  {
  exit("Couldn't connect to database Tracking, ODBC error = " .
    odbc_error());
  }

odbc_close($TrackingConn);
?>
</body>
</html>
[10 Nov 2004 1:37] Miguel Solorzano
Please test with Connector/ODBC 3.51.10

Thanks in advance.
[10 Nov 2004 5:14] Eric Wilde
I'd be more than happy to try this with Connector/ODBC 3.51.10 but I am unable to locate the source tarball for it anywhere (at present, the highest level of the source that I can find is 3.51.09, which is what I'm using).  If you'd be so kind as to point me to it, I'll download it, compile it and give it a spin.
[14 Feb 2005 22:54] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[24 Feb 2005 4:50] Eric Wilde
At the time, you asked me to download and try Connector/ODBC 3.51.10 but the source was not available.  I have since downloaded 3.51.11 and built it.  The ODBC connect function segfaults consistenly as well as failing to initialize latin1.  There is no workaround for this.  I put a loop around ODBC connect and it fails ten times in a row.  Basically, it works about 20% of the time, if at all.  The functionality of the product is seriously impacted.
[3 Mar 2005 6:47] Jorge del Conde
Hi!

Can you please provide us with a test-case that shows this behaviour ?

Thanks!
[3 Mar 2005 15:34] Eric Wilde
If you look under "How To Repeat", you will see an HTML page that must be processed by PHP.  When I run this test page on my system, it fails in the manner described.  That is all there is to it.  Nothing complicated.

In case you need configuration information, I'm running Apache 2.0.49, built with

./configure --prefix=/usr/share/httpd-2.0 --enable-proxy

PHP is 5.0.2, built with

./configure  --with-apxs2=/usr/share/httpd-2.0/bin/apxs --with-ldap --with-mysql --with-unixODBC=shared,/usr/local/unixODBC --disable-libxml

UnixODBC is 2.2.10, built with

./configure --prefix=/usr/local/unixODBC

MySQL is 4.1.7, built with

./configure --enable-local-infile --with-extra-charsets=complex

And, finally, MyODBC is 3.51.11, built with

./configure --prefix=/usr/local/unixODBC --with-mysql-path=/usr/local --with-unixODBC=/usr/local/unixODBC --disable-thread-safe

I realize that this is a pretty convoluted  set of services to be passing your requests through (Apache/PHP, UnixODBC, MyODBC and finally MySQL), hence my reason for supplying the exact build information.  The problem is pretty consistent (although random) and it also segfaults regularly as well as returning the character set error.  I would have taken a look at the fault dump but Apache seems to cover it up pretty well.  The task fails, is restarted and the Web page just doesn't display (no error message).  The problem is always in the PHP odbc_connect call.
[22 Mar 2005 15:39] Cristian Amitroaie
I do experience the same problem with the latin1 charset:
---
SQLError:2019
SQLStatus:HYT00
SQLMessage:[MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/share/mysql/charsets/)
---
   I use unixODBC-2.2.8-5.i386 (and devel for headers) + MyODBC-3.51.11-1.i586 on Linux lapc 2.6.5-1.358 #1 Sat May 8 09:04:50 EDT 2004 i686 i686 i386 GNU/Linux.

   I created some C library to connect and query the database. Indeed the connection fails quite often and randomly with the same error message.

   I could not reproduce the problem in a simple testcase. I'll try more since this is quite annoying already.
[3 Apr 2005 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[12 Jul 2005 19:43] Landon Fuller
I was witnessing the same behavior, so I tracked down the bug.

SQLFreeConnect() causes libiodbc to deallocate the associated database environment handle, at which point my_SQLFreeEnv() calls myodbc_end().

myodbc_end() calls libmysql's my_end(), and in turn, my_end() deallocates the "once" memory pool with my_once_free().

A backtrace may make this more clear:
#0  my_once_free () at my_once.c:111
#1  0x281e4aad in my_end (infoflag=0) at my_init.c:143
#2  0x281afe87 in myodbc_end () at dll.c:118
#3  0x281b3327 in my_SQLFreeEnv (henv=0x2817da30) at handle.c:109
#4  0x281b3e50 in SQLFreeHandle (HandleType=1, Handle=0x2817db0c) at handle.c:487
#5  0x28084f92 in _iodbcdm_driverunload (hdbc=0x804c000) at connect.c:640
#6  0x2808b1c9 in SQLFreeConnect_Internal (hdbc=0x804c000) at hdbc.c:277
#7  0x2808b28b in SQLFreeConnect (hdbc=0x804c000) at hdbc.c:297
#8  0x080486c4 in main ()

The crash is ultimately due to corruption of the charset data stored in the "once" memory pool. The pool's memory is freed by my_end() and eventually reallocated for other uses by malloc. However, the charset data is never re-initialized, and pointers to the old memory pool remain. This quickly results in the corruption of MySQL's charset data structures, and the crashes and failures outlined in this bug.

The solution for MySQL is to ensure that my_init() also re-initializes the charset data if necessary.

I'm not sure of the best solution for MyODBC. I'm inclined to say that my_end() should not be called; the memory that will be leaked is only allocated on libmysql initialization, and will not grow. If MyODBC continues to call my_end(), the cumulative overhead of repeatedly re-initializing the charset data is significant.

I've attached a simple test case.
[12 Jul 2005 19:45] Landon Fuller
The bug system will not allow me to attach a file. Test case inlined below:
#include <stdio.h>
#include <iodbcinst.h>

int main(void) {
        SQLRETURN ret;
        SQLHDBC env;
        SQLHDBC dbh;
        int i;

        SQLAllocEnv(&env);

        for (i = 0; i < 50; i++) {
                SQLAllocConnect(env, &dbh);
                ret = SQLConnect(dbh, "mysql", SQL_NTS, "root", SQL_NTS, "", SQL_NTS);
                if (ret != SQL_SUCCESS) {
                        printf("SQLConnect() failed!\n");
                        exit(1);
                }
                SQLDisconnect(dbh);
                SQLFreeConnect(dbh);
        }
        exit (0);
}
[25 Jul 2005 21:59] Mark Summers
Is there an update on this bug -- I am using odbc connector for Information Builders webfocus and see the error consistantly.
     : [S1T00] [MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (
     : path: /usr/share/mysql/charsets/)
[25 Jul 2005 22:46] Landon Fuller
I created a follow-up bug, #11892.

As a work-around, you can remove the call to my_end() in the myodbc_end() function, in driver/dll.c

I am using this change in production.
[26 Jul 2005 2:20] Mark Summers
Landon thank you very much -- that worked great!
[16 Aug 2005 15:20] M T
I have this same problem on FC4.

Environment:

Server version: Apache/2.0.54
Server built:   May 23 2005 08:12:24

PHP 4.3.11 (cgi) (built: Jul  5 2005 04:52:43)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

MyODBC-3.51.11-2.i586.rpm 

/usr/libexec/mysqld  Ver 4.1.11 for redhat-linux-gnu on i386 (Source distribution)

Problem:
[client 220.245.178.135] PHP Warning:  odbc_connect(): SQL error: [unixODBC][MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/share/mysql/charsets/), SQL state S1T00 in SQLConnect in

OR

random segmentation fault.

Problem occurs on both PHP5 and PHP43.

Is there a simple proper fix that does not require me to recompile the source code?

Will patches be released for this bug?

Has a fix been released for the problem? Where can i get it?

This bug is *very* annoying, as it is very random.

thanks.
[3 Sep 2005 13:21] Martin Kögler
If libmysqlclient is linked to a program, which uses the myodbc driver, the same error message happens in the second SQL Connect:
#include <stdio.h>
#include <odbcinst.h>

int main(void) {
        SQLRETURN ret;
        SQLHDBC env;
        SQLHDBC dbh;
        int i;

        for(i=0;i<2;i++)
          {
        SQLAllocEnv(&env);

        SQLAllocConnect(env, &dbh);
        ret = SQLConnect(dbh, "mysql", SQL_NTS, "root", SQL_NTS, "",
                         SQL_NTS);
        if (ret != SQL_SUCCESS) {
          printf("SQLConnect() failed!\n");
          exit(1);
        }

        SQLDisconnect(dbh);
        SQLFreeConnect(dbh);
          }

        exit (0);
}
If I compile this program with -lodbc -lmysqlclient on Suse 9.3 x86_64 (mysql-shared-4.1.10a-3,unixODBC-2.2.10-3), it aborts every time with the message "Character set 'latin1' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file". 

If SQLFreeEnv(env) is called directly after SQLFreeConnect(dhb); every time a Segmentation fault occurs.
The backtrace is:
#0  0x00002aaaaad48054 in my_strcasecmp_8bit () from /usr/lib64/libmysqlclient.so.14
#1  0x00002aaaaad40a1b in get_charset_number () from /usr/lib64/libmysqlclient.so.14
#2  0x00002aaaaad410b6 in get_charset_by_csname () from /usr/lib64/libmysqlclient.so.14
#3  0x00002aaaaad58302 in mysql_real_connect () from /usr/lib64/libmysqlclient.so.14
#4  0x00002aaaab941159 in SQLConnect () from /usr/lib64/unixODBC/libmyodbc3.so
#5  0x00002aaaaabd1fe1 in SQLConnect () from /usr/lib64/libodbc.so.1
#6  0x00000000004008e3 in main () at x.c:15
[7 Sep 2005 21:53] [ name withheld ]
How can this bug still exist if it was discovered in November 2004?

I am migrating an application from MS SQL Server and am running into this same random charset error.
[7 Sep 2005 21:58] Landon Fuller
Please attach your comments to the follow-up bug, #11892.
[8 Sep 2005 5:55] Martin Kögler
Consider switching to the libraries of MySQL 4.0.X on the client side, if applying the patch of #11892 is not an option. 

On a Debian Sarge i686 system (myodbc 3.51.09-1, MySQL Client Libraries 4.0.24-10, unixodbc 2.2.4-11), I can not reproduce this bug.
[25 Nov 2005 23:42] Peter Harvey
If this is a client bug then perhaps it is fixed for v5 of mysqlclient - otherwise we need to get this reclassified as a mysqlclient bug.

If his is a c/odbc bug and still exists then lets get a fix for it.
[26 Nov 2005 0:08] M T
The bug still exists on ODBC351 drivers. The bug can be reproduced on fedora core 2/4 easily. It has something to do with latin chars. Its very annoying. We have now started recoding all the code with direct mysql calls (eg: mysql_connect) rather than the odbc (eg: odbc_connect.) odbc_connect randomly crashes.

A proper fix will be great.
[1 Dec 2005 20:17] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/32930
[27 Dec 2005 13:14] Valerii Kravchuk
There is the same bug when connecting to MySQL 5.0.x. Please, check bug #16011 that is marked as a duplicate of this one. 

Looks like similar patch should be applied to 5.0.x.
[7 Feb 2006 16:08] Sinisa Milivojevic
This bug is worked upon and fix for it will be soon pushed.
[13 Feb 2006 14:34] Sergey Vlasenko
Fix is available in 4.1.19
[14 Feb 2006 2:36] Paul Dubois
Noted in 4.1.19 changelog.

        <para>
          Repeated invocation of <literal>my_init()</literal> and
          <literal>my_end() </literal> caused corruption of character
          set data and connection failure. (Bug #6536) 
        </para>
[16 Feb 2006 19:32] Trudy Pelzer
No unmerged changes on 2006-02-16; fix will
also be in MySQL 5.0.19.
[22 Jun 2006 15:29] Curtis Beattie
I can still reproduce this problem using MySQL 5.0.22 and MyODBC 3.51.12. It doesn't appear to have been fixed!
[7 Dec 2007 10:39] Tonci Grgin
Even this is an older report I'm setting it to a duplicate of Bug#15547.