Bug #31220 SQLFetch or SQLFetchScroll returns negative data length using SQL_C_WCHAR

• Submitted: 27 Sep 2007 0:22
• Modified: 14 Mar 2008 18:41
• Reporter: Viktor Ferenczi
• Status: Closed
• Category: Connector / ODBC
• Severity: S2 (Serious)
• Version: 3.51.20r750
• OS: Linux (Ubuntu Feisty, up-to-date)
• Assigned to: Jess Balint
• CPU Architecture: Any
• Tags: buffer, crash, LENGTH, negative, overflow, segfault, segmentation fault, SQLFetch, SQLFetchScroll, string, varchar

[27 Sep 2007 0:22] Viktor Ferenczi

Description:
Calling SQLFetch or SQLFetchScroll returns negative data length in certain circumstances. I've attached the complete unixODBC trace of my test. At the end my application segfaulted due to the unexpected negative length. The core dump has been analysed to point out the problem. Strings are encoded with 16bit unicode characters (SQL_C_WCHAR).

CPU: Intel Core Duo T5500
Memory: 1.5Gbytes
OS: Ubuntu Feisty 7.04, up-to-date, 32 bit
MySQL: 5.0.38
ODBC manager: unixODBC 2.2.11-13
MySQL ODBC connector: mysql-connector-odbc-3.51.20r750.tar.gz
The ODBC driver is compiled from source against the development headers installed by Ubuntu's package manager (apt).

Driver settings:

[mysql]
Description             = mysql
Driver          = /usr/local/lib/libmyodbc3.so
Driver64                = /usr/lib
Setup           = /usr/local/lib/libmyodbc3S.so
Setup64         = /usr/lib
UsageCount              = 1
CPTimeout               =
CPReuse         =

System DSN used:

[test_mysql]
Driver          = mysql
DATABASE                = test
DESCRIPTION             = mysql
PWD             = test
SERVER          = localhost
UID             = test

How to repeat:
Construct a test case to reproduce the SQL commands related to the last table used accoring to the attached trace.

[27 Sep 2007 0:25] Viktor Ferenczi

unixODBC trace output

Attachment: mysql-unixODBC-trace.log.bz2 (application/x-bzip, text), 25.33 KiB.

[3 Oct 2007 14:53] Bogdan Degtyariov

Test case with _W functions

Attachment: bug31220.c (text/plain), 2.73 KiB.

[3 Oct 2007 14:54] Bogdan Degtyariov

The test case above returns non-negative buffer length

[3 Oct 2007 15:50] Bogdan Degtyariov

Tested the program in Windows: got the correct length, in HP-UX the Buffer length is 1073745752.

[6 Oct 2007 0:12] Jim Winstead

Bogdan, one problem with your test case is that cLength is the wrong type. It should be SQLLEN, not SQLINTEGER.

[8 Oct 2007 16:56] Bogdan Degtyariov

Jim, I agree. Thanks for your note.
Unfortunately, SQLLEN has not changed the situation. Results are the same.

[23 Feb 2008 5:37] Jess Balint

fix + test

Attachment: bug31220.diff (application/octet-stream, text), 1.91 KiB.

[3 Mar 2008 10:39] Jess Balint

Committed as rev1054, will be released in 3.51.24.

[14 Mar 2008 18:41] MC Brown

A note has been added to the 3.51.24 changelog: 

Calling SQLFetch or SQLFetchScroll would return negative data lengths when using SQL_C_WCHAR.