Bug #31220 SQLFetch or SQLFetchScroll returns negative data length using SQL_C_WCHAR
Submitted: 27 Sep 2007 0:22 Modified: 14 Mar 2008 18:41
Reporter: Viktor Ferenczi Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / ODBC Severity:S2 (Serious)
Version:3.51.20r750 OS:Linux (Ubuntu Feisty, up-to-date)
Assigned to: Jess Balint CPU Architecture:Any
Tags: buffer, crash, LENGTH, negative, overflow, segfault, segmentation fault, SQLFetch, SQLFetchScroll, string, varchar
Triage: D3 (Medium)

[27 Sep 2007 0:22] Viktor Ferenczi
Description:
Calling SQLFetch or SQLFetchScroll returns negative data length in certain circumstances. I've attached the complete unixODBC trace of my test. At the end my application segfaulted due to the unexpected negative length. The core dump has been analysed to point out the problem. Strings are encoded with 16bit unicode characters (SQL_C_WCHAR).

CPU: Intel Core Duo T5500
Memory: 1.5Gbytes
OS: Ubuntu Feisty 7.04, up-to-date, 32 bit
MySQL: 5.0.38
ODBC manager: unixODBC 2.2.11-13
MySQL ODBC connector: mysql-connector-odbc-3.51.20r750.tar.gz
The ODBC driver is compiled from source against the development headers installed by Ubuntu's package manager (apt).

Driver settings:

[mysql]
Description             = mysql
Driver          = /usr/local/lib/libmyodbc3.so
Driver64                = /usr/lib
Setup           = /usr/local/lib/libmyodbc3S.so
Setup64         = /usr/lib
UsageCount              = 1
CPTimeout               =
CPReuse         =

System DSN used:

[test_mysql]
Driver          = mysql
DATABASE                = test
DESCRIPTION             = mysql
PWD             = test
SERVER          = localhost
UID             = test

How to repeat:
Construct a test case to reproduce the SQL commands related to the last table used accoring to the attached trace.
[27 Sep 2007 0:25] Viktor Ferenczi
unixODBC trace output

Attachment: mysql-unixODBC-trace.log.bz2 (application/x-bzip, text), 25.33 KiB.

[3 Oct 2007 14:53] Bogdan Degtyariov
Test case with _W functions

Attachment: bug31220.c (text/plain), 2.73 KiB.

[3 Oct 2007 14:54] Bogdan Degtyariov
The test case above returns non-negative buffer length
[3 Oct 2007 15:50] Bogdan Degtyariov
Tested the program in Windows: got the correct length, in HP-UX the Buffer length is 1073745752.
[6 Oct 2007 0:12] Jim Winstead
Bogdan, one problem with your test case is that cLength is the wrong type. It should be SQLLEN, not SQLINTEGER.
[8 Oct 2007 16:56] Bogdan Degtyariov
Jim, I agree. Thanks for your note.
Unfortunately, SQLLEN has not changed the situation. Results are the same.
[23 Feb 2008 5:37] Jess Balint
fix + test

Attachment: bug31220.diff (application/octet-stream, text), 1.91 KiB.

[3 Mar 2008 10:39] Jess Balint
Committed as rev1054, will be released in 3.51.24.
[14 Mar 2008 18:41] MC Brown
A note has been added to the 3.51.24 changelog: 

Calling SQLFetch or SQLFetchScroll would return negative data lengths when using SQL_C_WCHAR.