MySQL Bugs: #6536: ODBC connect function of PHP randomly fails to initialize latin1 character set

Bug #6536	ODBC connect function of PHP randomly fails to initialize latin1 character set
Submitted:	10 Nov 2004 1:25	Modified:	7 Dec 2007 10:39
Reporter:	Eric Wilde	Email Updates:
Status:	Duplicate	Impact on me:	None
Category:	Connector / ODBC	Severity:	S2 (Serious)
Version:	3.51, 5.1	OS:	Linux (RedHat Linux 8)
Assigned to:		CPU Architecture:	Any

Description:
The odbc_connect function call in a PHP script fails with the error:

SQL error: [unixODBC][MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/local/share/mysql/charsets/), SQL state S1000 in SQLConnect

MySQL was compiled by me with the following configure flag: --with-extra-charsets=complex.  Furthermore, inspection of the config.log file shows:

configure:34751: checking "character sets"
configure:35281: result: default: latin1, collation: latin1_swedish_ci; compiled in: latin1 latin1 utf8 big5 cp1250 euckr gb2312 gbk latin1 latin2 sjis tis620 ucs2 ujis utf8.

Note that the latin1 character set appears in the list of compiled in character sets a few times.

Also, the /usr/local/share/mysql/charsets/Index.xml file contains an apparently valid definition of the latin1 character set and there is an equally apparently valid latin1.xml in the same directory.

The failure occurs between 50% and 65% of the time, on a random basis. 

The Apache error log shows: Character set 'latin1' is not a compiled character set and is not specified in the '/usr/local/share/mysql/charsets/Index.xml' file

How to repeat:
On RedHat 8, under Apache 2.0.49 and PHP 5.0.2, using MySQL 4.1.7 and MyODBC 3.51.09, the following page will fail randomly:

<html>
<head>
<title>ODBC test script</title>
</head>
<p align="center"><b>ODBC Test Script</b></p>
<p>
<?
// Get a connection to the database.
if (($TrackingConn = odbc_connect("Tracking", "xyz", "xyzpass")) == 0)
  {
  exit("Couldn't connect to database Tracking, ODBC error = " .
    odbc_error());
  }

odbc_close($TrackingConn);
?>
</body>
</html>

Please test with Connector/ODBC 3.51.10

Thanks in advance.

I'd be more than happy to try this with Connector/ODBC 3.51.10 but I am unable to locate the source tarball for it anywhere (at present, the highest level of the source that I can find is 3.51.09, which is what I'm using).  If you'd be so kind as to point me to it, I'll download it, compile it and give it a spin.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

At the time, you asked me to download and try Connector/ODBC 3.51.10 but the source was not available.  I have since downloaded 3.51.11 and built it.  The ODBC connect function segfaults consistenly as well as failing to initialize latin1.  There is no workaround for this.  I put a loop around ODBC connect and it fails ten times in a row.  Basically, it works about 20% of the time, if at all.  The functionality of the product is seriously impacted.

Hi!

Can you please provide us with a test-case that shows this behaviour ?

Thanks!

If you look under "How To Repeat", you will see an HTML page that must be processed by PHP.  When I run this test page on my system, it fails in the manner described.  That is all there is to it.  Nothing complicated.

In case you need configuration information, I'm running Apache 2.0.49, built with

./configure --prefix=/usr/share/httpd-2.0 --enable-proxy

PHP is 5.0.2, built with

./configure  --with-apxs2=/usr/share/httpd-2.0/bin/apxs --with-ldap --with-mysql --with-unixODBC=shared,/usr/local/unixODBC --disable-libxml

UnixODBC is 2.2.10, built with

./configure --prefix=/usr/local/unixODBC

MySQL is 4.1.7, built with

./configure --enable-local-infile --with-extra-charsets=complex

And, finally, MyODBC is 3.51.11, built with

./configure --prefix=/usr/local/unixODBC --with-mysql-path=/usr/local --with-unixODBC=/usr/local/unixODBC --disable-thread-safe

I realize that this is a pretty convoluted  set of services to be passing your requests through (Apache/PHP, UnixODBC, MyODBC and finally MySQL), hence my reason for supplying the exact build information.  The problem is pretty consistent (although random) and it also segfaults regularly as well as returning the character set error.  I would have taken a look at the fault dump but Apache seems to cover it up pretty well.  The task fails, is restarted and the Web page just doesn't display (no error message).  The problem is always in the PHP odbc_connect call.

I do experience the same problem with the latin1 charset:
---
SQLError:2019
SQLStatus:HYT00
SQLMessage:[MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/share/mysql/charsets/)
---
   I use unixODBC-2.2.8-5.i386 (and devel for headers) + MyODBC-3.51.11-1.i586 on Linux lapc 2.6.5-1.358 #1 Sat May 8 09:04:50 EDT 2004 i686 i686 i386 GNU/Linux.

   I created some C library to connect and query the database. Indeed the connection fails quite often and randomly with the same error message.

   I could not reproduce the problem in a simple testcase. I'll try more since this is quite annoying already.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

I was witnessing the same behavior, so I tracked down the bug.

SQLFreeConnect() causes libiodbc to deallocate the associated database environment handle, at which point my_SQLFreeEnv() calls myodbc_end().

myodbc_end() calls libmysql's my_end(), and in turn, my_end() deallocates the "once" memory pool with my_once_free().

A backtrace may make this more clear:
#0  my_once_free () at my_once.c:111
#1  0x281e4aad in my_end (infoflag=0) at my_init.c:143
#2  0x281afe87 in myodbc_end () at dll.c:118
#3  0x281b3327 in my_SQLFreeEnv (henv=0x2817da30) at handle.c:109
#4  0x281b3e50 in SQLFreeHandle (HandleType=1, Handle=0x2817db0c) at handle.c:487
#5  0x28084f92 in _iodbcdm_driverunload (hdbc=0x804c000) at connect.c:640
#6  0x2808b1c9 in SQLFreeConnect_Internal (hdbc=0x804c000) at hdbc.c:277
#7  0x2808b28b in SQLFreeConnect (hdbc=0x804c000) at hdbc.c:297
#8  0x080486c4 in main ()

The crash is ultimately due to corruption of the charset data stored in the "once" memory pool. The pool's memory is freed by my_end() and eventually reallocated for other uses by malloc. However, the charset data is never re-initialized, and pointers to the old memory pool remain. This quickly results in the corruption of MySQL's charset data structures, and the crashes and failures outlined in this bug.

The solution for MySQL is to ensure that my_init() also re-initializes the charset data if necessary.

I'm not sure of the best solution for MyODBC. I'm inclined to say that my_end() should not be called; the memory that will be leaked is only allocated on libmysql initialization, and will not grow. If MyODBC continues to call my_end(), the cumulative overhead of repeatedly re-initializing the charset data is significant.

I've attached a simple test case.

The bug system will not allow me to attach a file. Test case inlined below:
#include <stdio.h>
#include <iodbcinst.h>

int main(void) {
        SQLRETURN ret;
        SQLHDBC env;
        SQLHDBC dbh;
        int i;

        SQLAllocEnv(&env);

        for (i = 0; i < 50; i++) {
                SQLAllocConnect(env, &dbh);
                ret = SQLConnect(dbh, "mysql", SQL_NTS, "root", SQL_NTS, "", SQL_NTS);
                if (ret != SQL_SUCCESS) {
                        printf("SQLConnect() failed!\n");
                        exit(1);
                }
                SQLDisconnect(dbh);
                SQLFreeConnect(dbh);
        }
        exit (0);
}

Is there an update on this bug -- I am using odbc connector for Information Builders webfocus and see the error consistantly.
     : [S1T00] [MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (
     : path: /usr/share/mysql/charsets/)

I created a follow-up bug, #11892.

As a work-around, you can remove the call to my_end() in the myodbc_end() function, in driver/dll.c

I am using this change in production.

Landon thank you very much -- that worked great!

I have this same problem on FC4.

Environment:

Server version: Apache/2.0.54
Server built:   May 23 2005 08:12:24

PHP 4.3.11 (cgi) (built: Jul  5 2005 04:52:43)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

MyODBC-3.51.11-2.i586.rpm 

/usr/libexec/mysqld  Ver 4.1.11 for redhat-linux-gnu on i386 (Source distribution)

Problem:
[client 220.245.178.135] PHP Warning:  odbc_connect(): SQL error: [unixODBC][MySQL][ODBC 3.51 Driver]Can't initialize character set latin1 (path: /usr/share/mysql/charsets/), SQL state S1T00 in SQLConnect in

OR

random segmentation fault.

Problem occurs on both PHP5 and PHP43.

Is there a simple proper fix that does not require me to recompile the source code?

Will patches be released for this bug?

Has a fix been released for the problem? Where can i get it?

This bug is *very* annoying, as it is very random.

thanks.

If libmysqlclient is linked to a program, which uses the myodbc driver, the same error message happens in the second SQL Connect:
#include <stdio.h>
#include <odbcinst.h>

int main(void) {
        SQLRETURN ret;
        SQLHDBC env;
        SQLHDBC dbh;
        int i;

        for(i=0;i<2;i++)
          {
        SQLAllocEnv(&env);

        SQLAllocConnect(env, &dbh);
        ret = SQLConnect(dbh, "mysql", SQL_NTS, "root", SQL_NTS, "",
                         SQL_NTS);
        if (ret != SQL_SUCCESS) {
          printf("SQLConnect() failed!\n");
          exit(1);
        }

        SQLDisconnect(dbh);
        SQLFreeConnect(dbh);
          }

        exit (0);
}
If I compile this program with -lodbc -lmysqlclient on Suse 9.3 x86_64 (mysql-shared-4.1.10a-3,unixODBC-2.2.10-3), it aborts every time with the message "Character set 'latin1' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file". 

If SQLFreeEnv(env) is called directly after SQLFreeConnect(dhb); every time a Segmentation fault occurs.
The backtrace is:
#0  0x00002aaaaad48054 in my_strcasecmp_8bit () from /usr/lib64/libmysqlclient.so.14
#1  0x00002aaaaad40a1b in get_charset_number () from /usr/lib64/libmysqlclient.so.14
#2  0x00002aaaaad410b6 in get_charset_by_csname () from /usr/lib64/libmysqlclient.so.14
#3  0x00002aaaaad58302 in mysql_real_connect () from /usr/lib64/libmysqlclient.so.14
#4  0x00002aaaab941159 in SQLConnect () from /usr/lib64/unixODBC/libmyodbc3.so
#5  0x00002aaaaabd1fe1 in SQLConnect () from /usr/lib64/libodbc.so.1
#6  0x00000000004008e3 in main () at x.c:15

How can this bug still exist if it was discovered in November 2004?

I am migrating an application from MS SQL Server and am running into this same random charset error.

Please attach your comments to the follow-up bug, #11892.

Consider switching to the libraries of MySQL 4.0.X on the client side, if applying the patch of #11892 is not an option. 

On a Debian Sarge i686 system (myodbc 3.51.09-1, MySQL Client Libraries 4.0.24-10, unixodbc 2.2.4-11), I can not reproduce this bug.

If this is a client bug then perhaps it is fixed for v5 of mysqlclient - otherwise we need to get this reclassified as a mysqlclient bug.

If his is a c/odbc bug and still exists then lets get a fix for it.

The bug still exists on ODBC351 drivers. The bug can be reproduced on fedora core 2/4 easily. It has something to do with latin chars. Its very annoying. We have now started recoding all the code with direct mysql calls (eg: mysql_connect) rather than the odbc (eg: odbc_connect.) odbc_connect randomly crashes.

A proper fix will be great.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/internals/32930

There is the same bug when connecting to MySQL 5.0.x. Please, check bug #16011 that is marked as a duplicate of this one. 

Looks like similar patch should be applied to 5.0.x.

This bug is worked upon and fix for it will be soon pushed.

Fix is available in 4.1.19

Noted in 4.1.19 changelog.

        <para>
          Repeated invocation of <literal>my_init()</literal> and
          <literal>my_end() </literal> caused corruption of character
          set data and connection failure. (Bug #6536) 
        </para>

No unmerged changes on 2006-02-16; fix will
also be in MySQL 5.0.19.

I can still reproduce this problem using MySQL 5.0.22 and MyODBC 3.51.12. It doesn't appear to have been fixed!

Even this is an older report I'm setting it to a duplicate of Bug#15547.