Bug #26024 improper casting for 64-bit

• Submitted: 1 Feb 2007 18:38
• Modified: 22 Feb 2007 11:03
• Reporter: Lee Stigile
• Status: Closed
• Category: Connector / ODBC
• Severity: S2 (Serious)
• Version: 3.51.13r146
• OS: Linux (RedHat Linux x64)
• Assigned to: Jim Winstead
• CPU Architecture: Any
• Tags: 64-bit, Contribution, ODBC

[1 Feb 2007 18:38] Lee Stigile

Description:
similar to bug 16259.  Buffer overflow due to improper casting on 64-bit machines.  I'm submitting on hehalf of customer.    Memory errors occur when running ODBC on RedHat on 64-bit system.

How to repeat:

      I don't have the exact application that was executed to produce the behavior.  However, the suggested fix identifies the improper casting.

Suggested fix:
  2) in results.c
        
        931,933c931,933
         *((long*) rgbValue)= ((long) atol(value)*10000L+
        (long) atol(value+5)*100L+
        (long) atol(value+8));
        ---
       *((SQLINTEGER*) rgbValue)= ((SQLINTEGER) atol(value)*10000L+
                                                       (SQLINTEGER) atol(value+5)*100L+
                                                     (SQLINTEGER) atol(value+8));
        936c936
        *((long*) rgbValue)= (long) atol(value);
        ---  *((SQLINTEGER*) rgbValue)= (SQLINTEGER) atol(value);
        938c938
                        *pcbValue= sizeof(long);
        ---     *pcbValue= sizeof(SQLINTEGER);

        944c944
                    *((ulong*) rgbValue)= strtoul(value,&end_ptr,10);
        ---         *((SQLUINTEGER*) rgbValue)= strtoul(value,&end_ptr,10);

        946c946
       *pcbValue= sizeof(long);
        ---       *pcbValue= sizeof(SQLUINTEGER);

[1 Feb 2007 18:40] Lee Stigile

The same changes need to be made further in results.c
       
        3) in results.c
      
        
        1056c1056
        *pcrow= (SQLINTEGER) mysql_affected_rows(&stmt->dbc->mysql);
        ---  *((long*)pcrow)= (SQLINTEGER) mysql_affected_rows(&stmt->dbc->mysql);
        1061c1061
         *pcrow= (SQLINTEGER) stmt->affected_rows;
        --- *((long*)pcrow)= (SQLINTEGER) stmt->affected_rows;

[13 Feb 2007 0:39] Jim Winstead

The first patch appears to have already been applied. The second is incorrect. The correct change would be:

--- driver/results.c    (revision 161)
+++ driver/results.c    (working copy)
@@ -1196,12 +1196,12 @@
 
     if ( stmt->result )
     {
-        *pcrow= (SQLINTEGER) mysql_affected_rows(&stmt->dbc->mysql);
+        *pcrow= (SQLLEN) mysql_affected_rows(&stmt->dbc->mysql);
         MYODBCDbgInfo( "Rows in set: %ld", *pcrow );
     }
     else
     {
-        *pcrow= (SQLINTEGER) stmt->affected_rows;
+        *pcrow= (SQLLEN) stmt->affected_rows;
         MYODBCDbgInfo( "Affected rows: %ld", *pcrow );
     }
     MYODBCDbgReturnReturn(SQL_SUCCESS);

(Please use unified or context diffs. Thanks!)

[21 Feb 2007 17:46] Jim Winstead

Committed revision 191.

[22 Feb 2007 11:03] MC Brown

A note has been added to the 3.51.13 changelog.