MySQL Bugs: #72494: EndOfStreamException when inserting large blob values using Compression=True

Bug #72494	EndOfStreamException when inserting large blob values using Compression=True
Submitted:	30 Apr 2014 19:41	Modified:	16 Nov 2015 9:44
Reporter:	Marco Kummer	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	Connector / NET	Severity:	S2 (Serious)
Version:	6.7.5	OS:	Windows
Assigned to:		CPU Architecture:	Any

Description:
I'm getting the following exception while trying to insert a record into MySQL containing a long blob field of 14MB in size when using Compression=True in my connection string:

MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered during command execution. --->
MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered attempting to read the resultset. --->
MySql.Data.MySqlClient.MySqlException (0x80004005): Reading from the stream has failed. --->
System.IO.EndOfStreamException

How to repeat:
1. Set max_allowed_packet to a large value, like 100M

2. Connect to DB via socket using the .NET connector. Make sure your connection string contains "Compression=True"

3. Insert large data (~15-30MB) into a longblob field using a regular command object.

4. When executing the query (command), you will experience a EndOfStreamException.

5. Repeat the same with "Compression=False". Everything should work fine now.

Suggested fix:
There must be a bug in the client libray. I think what happens under the hood is the following:

1. Source data is written into a local buffer on the client, along with an int containing the data size

2. Source data is compressed locally before sending it over the wire. However, the library doesn't update the blob field's size to the new (compressed) size (--> bug)

3. The client library then transmits to the server and keeps reading from the buffer, which is now smaller than the size-int. So we run into an EndOfStreamException.

This assumption would explain the experienced symptoms.

Hello Marco Kummer,

Thank you for the bug report.
I could not repeat the issue on Visual Studio 2013 (C#.Net) with  MySQL Connector/Net 6.9.5.
Could you please list out exact steps you tried out at your end, this would help us to reproduce the issue at our end.

Thanks,
Chiranjeevi.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

I become almost crazy for hours, and was to use compression

INSERT INTO `GEM_999_2015`.`archivos`
(TIPO,COD_CTA,LINEA,CODIGO,MARCADO,CATEGORIA,GRUPOS,RUTA,NOMBRE,DESCRIPCION,OBSERVACIONES,URL_DOWN,ARCHIVO,COMPRIMIDO,TEXTO_IMG,USUARIO,IS_WEB,TAM_FILE,FECHA_ALTA,FECHA_MODIF)
VALUES
('GENER','EMPRESA','','00207','0','00006','','','','','','',

0x00 <- LONG BLOB HERE

,'S','','','N','23503348','2014/07/04','    /  /  ')ON DUPLICATE KEY UPDATE
MARCADO=VALUES(MARCADO),CATEGORIA=VALUES(CATEGORIA),GRUPOS=VALUES(GRUPOS),RUTA=VALUES(RUTA),NOMBRE=VALUES(NOMBRE),
DESCRIPCION=VALUES(DESCRIPCION),OBSERVACIONES=VALUES(OBSERVACIONES),URL_DOWN=VALUES(URL_DOWN),ARCHIVO=VALUES(ARCHIVO),
COMPRIMIDO=VALUES(COMPRIMIDO),TEXTO_IMG=VALUES(TEXTO_IMG),USUARIO=VALUES(USUARIO),IS_WEB=VALUES(IS_WEB),
TAM_FILE=VALUES(TAM_FILE),FECHA_ALTA=VALUES(FECHA_ALTA),FECHA_MODIF=VALUES(FECHA_MODIF)

I have been used .NetConnector 6.9.7.0

ConnectionString=server=localhost;port=5551;password=***;user id=***;compress=True;ConnectionTimeout=3000

'CREATE TABLE `archivos` (
  `MARCADO` char(1) NOT NULL DEFAULT ''0'',
  `TIPO` char(5) NOT NULL DEFAULT '''',
  `COD_CTA` varchar(70) NOT NULL DEFAULT '''',
  `LINEA` varchar(5) NOT NULL DEFAULT '''',
  `CODIGO` char(5) NOT NULL DEFAULT '''',
  `CATEGORIA` varchar(5) NOT NULL DEFAULT '''',
  `GRUPOS` varchar(255) NOT NULL DEFAULT '''',
  `RUTA` varchar(255) NOT NULL DEFAULT '''',
  `NOMBRE` varchar(255) NOT NULL DEFAULT '''',
  `DESCRIPCION` varchar(70) NOT NULL DEFAULT '''',
  `OBSERVACIONES` varchar(255) NOT NULL DEFAULT '''',
  `URL_DOWN` varchar(255) NOT NULL DEFAULT '''',
  `ARCHIVO` longblob NOT NULL,
  `COMPRIMIDO` char(1) NOT NULL DEFAULT ''N'',
  `TEXTO_IMG` longtext NOT NULL,
  `USUARIO` varchar(20) NOT NULL DEFAULT '''',
  `IS_WEB` char(1) NOT NULL DEFAULT ''N'',
  `TAM_FILE` int(12) NOT NULL DEFAULT ''0'',
  `FECHA_ALTA` varchar(10) NOT NULL DEFAULT ''    /  /  '',
  `FECHA_MODIF` varchar(10) NOT NULL DEFAULT ''    /  /  '',
  PRIMARY KEY (`TIPO`,`COD_CTA`,`LINEA`,`CODIGO`),
  KEY `INDICE_1` (`TIPO`,`COD_CTA`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 CHECKSUM=1'

Hello Fernando Diaz Toledano,

Thank you for your feedback.
I could not repeat the issue on Visual Studio 2013 (C#.Net) with  MySQL Connector/Net 6.9.7 and 6.9.8 versions.
If you can provide more information, feel free to add it to this bug and change the status back to 'Open'.

Thanks,
Chiranjeevi.

Hi

I was able to replicate this bug. I am sending via SFTP a project and binary file (25MB) that can replicate problem. If you remove the "UseCompression=true" from connection string the problem disappears.

My setup is:

Server version: 5.6.31
Connector version: 6.9.9
max_allowed_packet=128M

I may have found a consistent repro for this bug here: https://bugs.mysql.com/bug.php?id=83330