Bug #28078 CompressInputStream Bug when record size > 10MBytes

• Submitted: 24 Apr 2007 16:31
• Modified: 7 May 2007 13:50
• Reporter: Alejandro Soto
• Status: Not a Bug
• Category: Connector / J
• Severity: S1 (Critical)
• Version: 5.0.5
• OS: Any
• CPU Architecture: Any
• Tags: ConnectorJ crash when record size is very long

[24 Apr 2007 16:31] Alejandro Soto

Description:
Sometimes, not always, but always using Compression with Connector/J, the driver crash when the query report some records with a GEOMETRY/BLOB field and its size is very big ( >10MBytes ). In this case, in the method

private void getNextPacketIfRequired(int numBytes) throws IOException{
		if ((this.buffer == null)
				|| ((this.pos + numBytes) > this.buffer.length)) {
			getNextPacketFromServer();
		}
	}

only one packet from server is read. In some case, it's need read more than one packet because a System.arraycopy fails.

24/04/07 16:56:38 ERROR: OraSQLIterator.iterame:: Message:Communications link failure due to underlying exception: 

** BEGIN NESTED EXCEPTION ** 

java.lang.ArrayIndexOutOfBoundsException

STACKTRACE:

java.lang.ArrayIndexOutOfBoundsException
	at java.lang.System.arraycopy(Native Method)
	at com.mysql.jdbc.CompressedInputStream.read(CompressedInputStream.java:286)
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1960)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2428)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2874)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:868)
	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1351)
	at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:367)
	at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:354)
	at com.mysql.jdbc.ResultSet.next(ResultSet.java:7330)
	at ** END NESTED EXCEPTION **

Last packet sent to the server was 15 ms ago.
com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception: 

** BEGIN NESTED EXCEPTION ** 

java.lang.ArrayIndexOutOfBoundsException

STACKTRACE:

java.lang.ArrayIndexOutOfBoundsException
	at java.lang.System.arraycopy(Native Method)
	at com.mysql.jdbc.CompressedInputStream.read(CompressedInputStream.java:286)
	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1960)
	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2428)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2874)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:868)
	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1351)
	at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:367)
	at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:354)
	at com.mysql.jdbc.ResultSet.next(ResultSet.java:7330)
	at ** END NESTED EXCEPTION **

Last packet sent to the server was 15 ms ago.
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2880)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:868)
	at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1351)
	at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:367)
	at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:354)
	at com.mysql.jdbc.ResultSet.next(ResultSet.java:7330)
======================================================

How to repeat:
create a table with geometry field and fill with large geometries. Make a preparedStatement querying all records.

Suggested fix:
To change the method to check if it's needed to read more than one packet from server.

CompressedInputStream.java:233

private void getNextPacketIfRequired(int numBytes) throws IOException {
  int i=0;
  while((this.buffer == null)	|| ((this.pos + numBytes) > this.buffer.length)) {
       getNextPacketFromServer();
   }
		
}

[3 May 2007 9:50] Tonci Grgin

Hi Alejandro and sorry for the delay in processing. Thanks for your report.

[7 May 2007 13:25] Tonci Grgin

Alejandro, just to be on a safe side, what is your max_allowed_packet value? Is it bigger than "large geometry"?

[7 May 2007 13:42] Alejandro Soto

Hi Tonci,
The max geometry size was around 15 MB, but the max_allowed_packet value was set to 4MB. After that, I tried changing the max_allowed_packet value to 30MB and then it's work. Also I tried with blobs fields and all was OK.
So, it fails when the max_allowed_packet value is lower than the max length of the geometry field.

Bye,
Alex.

[7 May 2007 13:50] Tonci Grgin

Alejandro, AFAIK this is then intentional.