MySQL Bugs: #73709: MyODBC crashes in SQLConnect() when the application is linked with -lmysqlclient

Bug #73709	MyODBC crashes in SQLConnect() when the application is linked with -lmysqlclient
Submitted:	25 Aug 2014 10:31	Modified:	20 Sep 6:54
Reporter:	Aleksandrs Saveljevs	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	Connector / ODBC	Severity:	S1 (Critical)
Version:	5.3.4	OS:	Linux (Debian Jessie)
Assigned to:		CPU Architecture:	Any
Tags:	crash, libmyodbc, libmysqlclient

Description:
We have noticed that when our ODBC application is not linked with -lmysqlclient, it works well. However, when our application is linked with -lmysqlclient, it crashes in MyODBC's SQLConnect().

Please see https://support.zabbix.com/browse/ZBX-7665 for the exact problem we are investigating. There was also a related problem with conflicting symbols and dynamic linking, https://support.zabbix.com/browse/ZBX-8629, which might or might not lead you in the right direction.

How to repeat:
Attached "report.c" is a short example to reproduce the problem. The program connects to the database, performs a simple query, and prints out the answer.

In order to reproduce, we first download a precompiled 64-bit version of MyODBC 5.3.4 for Debian from http://dev.mysql.com/downloads/connector/odbc/ and set it up as follows:

$ cat /etc/odbcinst.ini 
[mysql]
Driver=/home/zabbix/download/mysql-connector-odbc-5.3.4-linux-debian6.0-x86-64bit/lib/libmyodbc5a.so
UsageCount=1

We also set up a DSN in unixODBC and verify that it works:

$ cat /etc/odbc.ini
[zabbix_2_2]
Description = Zabbix 2.2
Driver      = mysql
Server      = 127.0.0.1
User        = root
Password    =
Port        = 3306
Database    = zabbix_2_2

$ echo 'select count(*) from hosts' | isql -b zabbix_2_2
+---------------------+
| count(*)            |
+---------------------+
| 87                  |
+---------------------+
SQLRowCount returns 1
1 rows fetched

Now, if we compile "report.c" without linking it with -lmysqlclient, it works:

$ gcc -g -rdynamic -lodbc report.c -o report
$ ./report 
87

However, if we compile "report.c" and link it with -lmysqlclient, it crashes with the following backtrace:

$ gcc -g -rdynamic -lodbc -lmysqlclient report.c -o report
$ ./report 
10: ./report() [0x400bdc]
9: /lib/x86_64-linux-gnu/libc.so.6(+0x35480) [0x7fc1bf1fb480]
8: /usr/lib/x86_64-linux-gnu/libmysqlclient.so.18(+0x3f6e9) [0x7fc1bf5ae6e9]
7: /usr/lib/x86_64-linux-gnu/libmysqlclient.so.18(+0x3fa49) [0x7fc1bf5aea49]
6: /home/zabbix/software/mysql-connector-odbc-5.3.4-linux-debian6.0-x86-64bit/lib/libmyodbc5a.so(copy_and_convert+0xd7) [0x7fc1bd20bae7]
5: /home/zabbix/software/mysql-connector-odbc-5.3.4-linux-debian6.0-x86-64bit/lib/libmyodbc5a.so(sqlchar_as_sqlwchar+0x13f) [0x7fc1bd20c66d]
4: /home/zabbix/software/mysql-connector-odbc-5.3.4-linux-debian6.0-x86-64bit/lib/libmyodbc5a.so(SQLConnect+0x60) [0x7fc1bd176fea]
3: /usr/lib/x86_64-linux-gnu/libodbc.so.2(SQLConnect+0x1f0) [0x7fc1bfab7890]
2: ./report(main+0xb5) [0x400d0f]
1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc1bf1e7b45]
0: ./report() [0x400af9]

On our system, the issue manifests with unixODBC 2.3.1, MyODBC 5.3.4 (both the precompiled version downloaded from the website and compiled manually), and libmysqlclient 5.5.37. If, however, we use MyODBC 5.1.10, which is currently available in Debian packages, it works well.

Example program to reproduce the problem.

Attachment: report.c (text/x-csrc), 1.37 KiB.

There is also another related issue. Suppose we replace the ANSI version of MyODBC driver with the Unicode one:

$ cat /etc/odbcinst.ini 
[mysql]
Driver=/home/zabbix/software/mysql-connector-odbc-5.3.4-linux-debian6.0-x86-64bit/lib/libmyodbc5w.so
UsageCount=1

Compiling and linking our program without -lmysqlclient works well:

$ gcc -g -rdynamic -lodbc report.c -o report
$ ./report 
87

However, if we link our program with -lmysqlclient, it goes into an infinite loop and uses 100% CPU, but this time in SQLExecDirect() function. Backtrace in the debugger follows:

$ ./report &
[1] 26973

$ gdb -p 26973
...
(gdb) bt
#0  is_param_marker (parser=0x7ffff04e17b0) at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/parse.c:623
#1  0x00007ff31e6db87d in tokenize (parser=0x7ffff04e17b0) at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/parse.c:745
#2  0x00007ff31e6db8d9 in parse (pq=<optimized out>) at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/parse.c:843
#3  0x00007ff31e6e1c83 in prepare (stmt=0x1cf7760, query=0x1cf14d0 "select count(*) from hosts", query_length=<optimized out>)
    at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/my_stmt.c:376
#4  0x00007ff31e6dc607 in my_SQLPrepare (hstmt=<optimized out>, szSqlStr=0x1cf14d0 "select count(*) from hosts", cbSqlStr=26, dupe=1 '\001')
    at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/prepare.c:103
#5  0x00007ff31e6dc677 in MySQLPrepare (hstmt=0x1cf7760, query=0x1cf14d0 "select count(*) from hosts", len=26, dupe=0 '\000')
    at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/prepare.c:74
#6  0x00007ff31e6e7da8 in SQLPrepareWImpl (hstmt=0x1cf7760, str=<optimized out>, str_len=26) at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/unicode.c:859
#7  0x00007ff31e6e894a in SQLExecDirectW (hstmt=0x7ffff04e17b0, str=0x0, str_len=-263317584) at /export/home/pb2/build/sb_0-12449490-1403132068.74/mysql-connector-odbc-5.3.4-src/driver/unicode.c:354
#8  0x00007ff32102fd6f in SQLExecDirect () from /usr/lib/x86_64-linux-gnu/libodbc.so.2
#9  0x0000000000400d3c in main () at report.c:53

Looks like `rdynamic` is creating error here, removing it resolves issue. Still working on it.

Add -fvisibility=hidden while compiling will resolve clashing issue for GCC 4.0 or newer.

I still have the same problem, so what is the solution for this issue.
Please explain in details

Hello,

I see that issue is in verified status, could you please be so kind and give an update, does it mean that it's going to be fixed eventually ?

This bug is 4 years old but it is still around.  We use zabbix 3.4.12 on CentOS 7.5.1804 (unixODBC 2.3.1-11) with ODBC database queries, and most versions of mysql-connector-odbc still cause zabbix to crash.  We have seen this problem for years under earlier versions, when changing OS or zabbix major versions it has always been painful to find an odbc connector that works.

Crashes:
mysql-connector-odbc-8.0.12-1.el7.x86_64.rpm
mysql-connector-odbc-5.3.11-1.el7.x86_64.rpm

Works:
mysql-connector-odbc-5.2.7-1.el7.x86_64.rpm

Posted by developer:
 
Please note that the scenario with both ODBC and C API connection from the same process is not supported.
Mostly because of possible conflicts between libmyqlclient used by the program and libmysqlclient used by ODBC Driver.
There are a few workarounds, but if you use them it will be at your own risk.

 * Build ODBC driver that links libmysqlclient.so dynamically, so it would be the same library, which your program is loading

OR

 * Use ODBC 8.0.24 or newer and libmysqlclient 8.0.24 or newer trying to keep the versions of libmysqlclient and ODBC driver in sync. (I tested with versions 8.0.24 and 8.0.25)

These workarounds are not officially supported and therefore no guarantees can be giving about them. Also, currently there are no plans to 

If I may ask, why do you need C API and ODBC together in the same program?

Thank you for your reply, we are aware of workarounds and recommend using it like this or using different database than used for UnixODBC, but this creates a breading ground for errors because users can compile program with any library and use any UnixODBC library.

There is a program that connects to database during startup using C API, then disconnects and forks processes. There is one process that read configuration from database into shared memory using C API, but there are processes that read shared memory and depending on configuration perform different user configurable checks and among those UnixODBC checks are possible. This is why both C API and UnixODBC are used at once in a program.
Note that recently C API usage was removed from processes that use UnixODBC, however it was not tested if it helps with the issue.

Posted by developer:
 
Added a few lines to connect to MySQL Server using mysql_real_connect() along with ODBC connection.
The bug could not be repeated with MySQL 9.1.0 and ODBC Driver 9.1.0:

$ ldd ./report 
        linux-vdso.so.1 (0x00007fff52b7d000)
        libodbc.so.2 => /lib/x86_64-linux-gnu/libodbc.so.2 (0x000078712723e000)
        libmysqlclient.so.24 => /lib/x86_64-linux-gnu/libmysqlclient.so.24 (0x0000787126a00000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x0000787126600000)
        libltdl.so.7 => /lib/x86_64-linux-gnu/libltdl.so.7 (0x0000787127233000)
        libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x0000787127189000)
        libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x0000787126000000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x0000787127174000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x0000787125c00000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x0000787126917000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x0000787127147000)
        /lib64/ld-linux-x86-64.so.2 (0x00007871272cf000)

$ ./report 
Connected via mysql_real_connect()
Result from ODBC connection: 9.1.0-commercial