MySQL Bugs: #74998: readRemainingMultiPackets not computed correctly for rows larger than 16 MB

Bug #74998	readRemainingMultiPackets not computed correctly for rows larger than 16 MB
Submitted:	25 Nov 2014 18:11	Modified:	27 Jan 2015 17:10
Reporter:	Spencer Woodworth (OCA)	Email Updates:
Status:	Closed	Impact on me:	None
Category:	Connector / J	Severity:	S2 (Serious)
Version:	5.1.34	OS:	Any
Assigned to:	Filipe Silva	CPU Architecture:	Any

Description:
By attaching the debugger to the jdbc client (TestCursor) when it is retrieving the rows
we found the following:

MysqlIO.java 

private int readRemainingMultiPackets(Buffer reuse, byte multiPacketSeq,
			int packetEndPoint) throws IOException, SQLException {
......
else if (packetLength < this.maxThreeBytes) {
				byte newPacketSeq = this.packetHeaderBuf[3];

				if (newPacketSeq != (multiPacketSeq + 1)) {
					throw new IOException(Messages.getString(
					"MysqlIO.49")); //$NON-NLS-1$
				}
				
				
Both newPacketSeq and multiPacketSeq are of type 'byte'. When comparing newPacketSeq to 
(multiPacketSeq + 1), it is not casting ((multiPacketSeq + 1) to byte. Debugger shows at 
the time of exception multiPacketSeq is of value 127 and newPacketSeq is of value -128. 
(byte)(multiPacketSeq + 1) would give a value of -128 but ((multiPacketSeq + 1) does NOT.

How to repeat:
Replace the host, port, database, select query, username, and password

import java.sql.*;

public class TestCursor {
   // JDBC driver name and database URL
   static final String JDBC_DRIVER = "com.mysql.jdbc.Driver";  

    static final String URL_PROTOCOL = "jdbc:mysql://";
    static final String URL_AUTO_RECONNECT = "autoReconnect";
    static final String URL_USE_CURSOR_FETCH = "useCursorFetch";
    static final String URL_CACHE_PREPARED_STMTS = "cachePrepStmts";
    static final String URL_REWRITE_BATCHED_STMTS = "rewriteBatchedStatements";
    static final String URL_DEFAULT_FETCH_SIZE = "defaultFetchSize";

   //  Database credentials
   static final String USER = "xxxxxx";
   static final String PASS = "xxxxxxxxxxxx";
   
public static void main(String[] args) {
   Connection conn = null;
   Statement stmt = null;
   try{
      //STEP 2: Register JDBC driver
      Class.forName("com.mysql.jdbc.Driver");

      final StringBuilder builder = new StringBuilder(128);
        builder.append(URL_PROTOCOL);
        builder.append("").append(":"); //host
        builder.append("").append("/"); //port
        builder.append(""); //database
        builder.append("?").append(URL_AUTO_RECONNECT).append("=true");
        builder.append("&").append(URL_USE_CURSOR_FETCH).append("=false");
        builder.append("&").append(URL_CACHE_PREPARED_STMTS).append("=true");
        builder.append("&").append(URL_REWRITE_BATCHED_STMTS).append("=true");
        //builder.append("&").append(URL_DEFAULT_FETCH_SIZE).append("=").append("10000");

      //STEP 3: Open a connection
      System.out.println("Connecting to database...");
      conn = DriverManager.getConnection(builder.toString(),USER,PASS);

      //STEP 4: Execute a query to create statment with
      // required arguments for RS example.
      System.out.println("Creating statement...");
      stmt = conn.createStatement(
                           ResultSet.TYPE_FORWARD_ONLY,
                           ResultSet.CONCUR_READ_ONLY);
      String sql;
      sql = ""; // Select statement here
      ResultSet rs = stmt.executeQuery(sql);

      int pCnt = 0;
      while( rs.next()) {
        pCnt++;
        //Read results
      }
      
      //STEP 8: Clean-up environment
      rs.close();
      stmt.close();
      conn.close();
   }catch(SQLException se){
      //Handle errors for JDBC
      se.printStackTrace();
   }catch(Exception e){
      e.printStackTrace();
   }finally{
      //finally block used to close resources
      try{
         if(stmt!=null)
            stmt.close();
      }catch(SQLException se2){
      }// nothing we can do
      try{
         if(conn!=null)
            conn.close();
      }catch(SQLException se){
         se.printStackTrace();
      }//end finally try
   }//end try
   System.out.println("Goodbye!");
}//end main
}//end JDBCExample

Suggested fix:
Changes to MysqlIO.java of 5.1.34 are *(multiPacketSeq + 1) * is not casted to byte.

@@ -3477,7 +3477,7 @@
             } else if (packetLength < this.maxThreeBytes) {
                 byte newPacketSeq = this.packetHeaderBuf[3];
 
+                if (newPacketSeq != (byte) (multiPacketSeq + 1)) {
-                if (newPacketSeq != (multiPacketSeq + 1)) {
                     throw new IOException(Messages.getString("MysqlIO.49"));
                 }
 
@@ -3508,7 +3508,7 @@
 
             byte newPacketSeq = this.packetHeaderBuf[3];
 
+            if (newPacketSeq != (byte) (multiPacketSeq + 1)) {
-            if (newPacketSeq != (multiPacketSeq + 1)) {
                 throw new IOException(Messages.getString("MysqlIO.53"));
             }

Create Table

Attachment: sharedTest.sql.gz (application/x-tar-gz, text), 64.61 KiB.

Test Program

Attachment: TestCursor.java (application/octet-stream, text), 2.57 KiB.

I have uploaded a test program and some cleansed data with the correct shape to reproduce the error.

Hi Spencer,

Thank you for this bug report. Nice catch!

The Connector/J behaves as you described, however your test case + data doesn't hit the exact point of failure in my environment. I'm sure that's because you're getting data in different order but with some changes this can be reproduced.

Added the following entry to the Connector/J 5.1.35 changelog:

"The readRemainingMultiPackets method in the MysqlIO class returned incorrect results when a row was larger than 16MB in size. This fix corrects the wrong type conversion occurred during the calculation, which caused the problem."

Hi Daniel, Do you have any time-frame for the release of version 5.1.35?

Filipe or Daniel, can you please provide an update on the official release of this fix. The issue has been 'closed' for about 6 weeks. There are still no entries in the 5.1.35 changelog - http://dev.mysql.com/doc/relnotes/connector-j/en/news-5-1-35.html

Hi Spencer,

You should expect that this bug will be fixed in Connector/J 5.1.35 when it's released. But, I'm sorry, I can't add anything more that that. You should subscribe our announcement list (http://lists.mysql.com/announce) to get notified when this happens.

Thank you,