Bug #83330 MySqlException (EndOfStreamException) inserting large blob with UseCompression

• Submitted: 11 Oct 2016 4:58
• Modified: 7 Apr 2020 16:19
• Reporter: Bradley Grainger (OCA)
• Status: Not a Bug
• Category: Connector / NET
• Severity: S2 (Serious)
• Version: 6.9.9, 7.0.5
• OS: Windows (10.0.14393 x64)
• CPU Architecture: Any

[11 Oct 2016 4:58] Bradley Grainger

Description:
Windows 10.0.14393 x64
MySQL Server mysqld 5.7.15-log
Microsoft .NET Framework 4.6.2
Visual Studio 2015
MySql.Data 6.9.9

This is similar to http://bugs.mysql.com/bug.php?id=83329 but the exception is different, with a different call stack, indicating that it may reflect a different underlying bug.

When MySqlConnectionStringBuilder.UseCompression=true is set, inserting a sufficiently large random blob will throw the following exception:

Unhandled Exception: MySql.Data.MySqlClient.MySqlException: Fatal error encountered during command execution. ---> MySql.Data.MySqlClient.MySqlException: Fatal error encountered attempting to read the resultset. ---> MySql.Data.MySqlClient.MySqlException: Reading from the stream has failed. ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.CompressedStream.PrepareNextPacket()
   at MySql.Data.MySqlClient.CompressedStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
   at MySqlCompressBlob.Program.Main(String[] args) in Program.cs:line 33

I've pasted code to reproduce this at https://gist.github.com/bgrainger/76b3125bd643801e11285851c62da730

In the example, creating a random blob of length 16519388 doesn't reproduce the problem, a blob of length 16519389 causes http://bugs.mysql.com/bug.php?id=83329, and a blob of length 16519390 causes this exception.

Setting UseCompression=false will work around the problem.

How to repeat:
Create a new C# console application with the source code at: https://gist.github.com/bgrainger/76b3125bd643801e11285851c62da730

I've also reproduced the source below:

using System;
using MySql.Data.MySqlClient;

namespace MySqlCompressBlob
{
	class Program
	{
		static void Main(string[] args)
		{
			var csb = new MySqlConnectionStringBuilder
			{
				Server = "localhost",
				Database = "test",
				UserID = "root",
				Password = /* redacted */
				UseCompression = true,
			};

			using (var connection = new MySqlConnection(csb.ConnectionString))
			{
				connection.Open();

				using (var cmd = new MySqlCommand("drop table if exists test.blobs; create table test.blobs(data longblob)", connection))
					cmd.ExecuteNonQuery();

				var bytes = new byte[16519390]; // causes http://bugs.mysql.com/bug.php?id=83329 with new byte[16519389]
				var random = new Random(1);
				random.NextBytes(bytes);

				using (var cmd = new MySqlCommand("insert into test.blobs(data) values(@data);", connection))
				{
					cmd.Parameters.AddWithValue("data", bytes);
					cmd.ExecuteNonQuery();
				}
			}
		}
	}
}

[11 Oct 2016 5:18] Bradley Grainger

With a buffer size of 16MiB (`var bytes = new byte[16777216];`) I can get the following exception:

MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered during command execution. ---> MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered attempting to read the resultset. ---> MySql.Data.MySqlClient.MySqlException (0x80004005): Reading from the stream has failed. ---> System.IO.IOException: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. ---> System.Net.Sockets.SocketException: An established connection was aborted by the software in your host machine
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at MySql.Data.Common.MyNetworkStream.HandleOrRethrowException(Exception e)
   at MySql.Data.Common.MyNetworkStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.TimedStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.CompressedStream.PrepareNextPacket()
   at MySql.Data.MySqlClient.CompressedStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
   at MySqlCompressBlob.Program.Main(String[] args) in Program.cs:line 33

[11 Oct 2016 5:24] Bradley Grainger

With a buffer of `new byte[16748422]` (in the preceding code sample), I can sometimes reproduce the following exception (infrequently) as well as the other two exceptions. It doesn't seem to be predictable: running a new instance of a console app may throw any one of the three exceptions given so far in this bug.

MySql.Data.MySqlClient.MySqlException (0x80004005): Fatal error encountered during command execution. ---> System.IO.IOException: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at MySql.Data.Common.MyNetworkStream.HandleOrRethrowException(Exception e)
   at MySql.Data.Common.MyNetworkStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.TimedStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.CompressedStream.CompressAndSendCache()
   at MySql.Data.MySqlClient.CompressedStream.Flush()
   at MySql.Data.MySqlClient.MySqlStream.SendPacket(MySqlPacket packet)
   at MySql.Data.MySqlClient.NativeDriver.ExecutePacket(MySqlPacket packetToExecute)
   at MySql.Data.MySqlClient.NativeDriver.SendQuery(MySqlPacket queryPacket)
   at MySql.Data.MySqlClient.Driver.SendQuery(MySqlPacket p)
   at MySql.Data.MySqlClient.Statement.ExecuteNext()
   at MySql.Data.MySqlClient.PreparableStatement.ExecuteNext()
   at MySql.Data.MySqlClient.PreparableStatement.Execute()
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
   at MySqlCompressBlob.Program.Main(String[] args) in Program.cs:line 33

[12 Oct 2016 9:43] Chiranjeevi Battula

Hello Bradley Grainger,

Thank you for the bug report and test case.
Verified this behavior on Visual Studio 2013 (C#.Net) and Connector/NET 6.9.9 version.

Thanks,
Chiranjeevi.

[12 Oct 2016 9:44] Chiranjeevi Battula

Screenshot

Attachment: 83330.JPG (image/jpeg, text), 212.84 KiB.

[7 Apr 2020 16:19] Daniel Valdez

Posted by developer:
 
This test was done using all the specified versions and the mentioned error didn't came up, instead of that, we received the following exception:
 
MySql.Data.MySqlClient.MySqlException: 'Packets larger than max_allowed_packet are not allowed.'
 
Which is properly triggered, only when we modify the value of the variable max_allowed_packet to be higher than the value to be inserted the test works fine, the blob value is properly inserted in the database.
 
Regards