Bug #43678 Reading from a CompressedStream may result in deadlock
Submitted: 16 Mar 2009 15:08 Modified: 11 Aug 2009 9:19
Reporter: Yvan Rodrigues Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / NET Severity:S1 (Critical)
Version:5.2.5 OS:Microsoft Windows
Assigned to: Vladislav Vaintroub CPU Architecture:Any
Tags: Compressed Stream, compression, deadlock, freeze, GetValues, hang, ReadNextPacket
Triage: D2 (Serious)

[16 Mar 2009 15:08] Yvan Rodrigues
Description:
When reading data by any means such as with a MySqlDataAdapter on a MySqlConnection where compression is enabled, the Connector may end up in an endless loop in CompressedStream.ReadNextpacket(). This will appear to the user as a frozen application if reading on the UI thread or multi-threaded application that can never synchronize without killing the database thread.

A stack trace might look like this:
>	MySql.Data.dll!MySql.Data.MySqlClient.CompressedStream.ReadNextPacket(int len = 2567) Line 171	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.CompressedStream.PrepareNextPacket() Line 151 + 0xb bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.CompressedStream.Read(byte[] buffer = {byte[10207]}, int offset = 6285, int count = 3922) Line 111 + 0x8 bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.MySqlStream.Read(byte[] buffer = {byte[10207]}, int offset = 6285, int count = 3922) Line 355 + 0x23 bytes	C#
 	MySql.Data.dll!MySql.Data.Types.MySqlBinary.MySql.Data.Types.IMySqlValue.ReadValue(MySql.Data.MySqlClient.MySqlStream stream = {MySql.Data.MySqlClient.MySqlStream}, long length = 10207, bool nullVal = false) Line 181 + 0x12 bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.NativeDriver.ReadColumnValue(int index = 9, MySql.Data.MySqlClient.MySqlField field = {MySql.Data.MySqlClient.MySqlField}, MySql.Data.Types.IMySqlValue valObject = {MySql.Data.Types.MySqlBinary}) Line 631 + 0x1e bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.MySqlDataReader.GetFieldValue(int index = 9, bool checkNull = false) Line 980 + 0x57 bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.MySqlDataReader.GetValue(int i = 9) Line 701 + 0xd bytes	C#
 	MySql.Data.dll!MySql.Data.MySqlClient.MySqlDataReader.GetValues(object[] values = {object[18]}) Line 731 + 0x1a bytes	C#
 	System.Data.dll!System.Data.ProviderBase.DataReaderContainer.CommonLanguageSubsetDataReader.GetValues(object[] values) + 0xe bytes	
 	System.Data.dll!System.Data.ProviderBase.SchemaMapping.LoadDataRow() + 0x2d bytes	
 	System.Data.dll!System.Data.Common.DataAdapter.FillLoadDataRow(System.Data.ProviderBase.SchemaMapping mapping) + 0x5c bytes	
 	System.Data.dll!System.Data.Common.DataAdapter.FillFromReader(System.Data.DataSet dataset = null, System.Data.DataTable datatable = {Production_Builder}, string srcTable = null, System.Data.ProviderBase.DataReaderContainer dataReader = {System.Data.ProviderBase.DataReaderContainer.CommonLanguageSubsetDataReader}, int startRecord = 0, int maxRecords = 0, System.Data.DataColumn parentChapterColumn = null, object parentChapterValue = null) + 0xa5 bytes	
 	System.Data.dll!System.Data.Common.DataAdapter.Fill(System.Data.DataTable[] dataTables = {System.Data.DataTable[1]}, System.Data.IDataReader dataReader = {MySql.Data.MySqlClient.MySqlDataReader}, int startRecord = 0, int maxRecords = 0) + 0x11b bytes	
 	System.Data.dll!System.Data.Common.DbDataAdapter.FillInternal(System.Data.DataSet dataset, System.Data.DataTable[] datatables, int startRecord, int maxRecords, string srcTable, System.Data.IDbCommand command = {MySql.Data.MySqlClient.MySqlCommand}, System.Data.CommandBehavior behavior) + 0xd3 bytes	
 	System.Data.dll!System.Data.Common.DbDataAdapter.Fill(System.Data.DataTable[] dataTables, int startRecord, int maxRecords, System.Data.IDbCommand command, System.Data.CommandBehavior behavior) + 0xa3 bytes	
 	System.Data.dll!System.Data.Common.DbDataAdapter.Fill(System.Data.DataTable dataTable) + 0x74 bytes	
 	MisMabel.DataAccess.dll!MisMabel.DataAccess.StoredProcedure.Call<System.Data.DataTable>(MisMabel.DataAccess.StoredProcedureReturnValue<System.Data.DataTable> RetVal = {Production_Builder}, MisMabel.DataAccess.IStoredProcedureParameter[] supplied = {MisMabel.DataAccess.StoredProcedureParameter[1]}) Line 113 + 0xd bytes	C#

How to repeat:
I do not know the specific circumstances under which this occurs, but if I enable compression and run my application, maybe 1 in 20 times a read with end up in this deadlock. Perhaps the connection has been broken while in this loop.

In SVN revision 1527, the programming error is at line 170 of Source/CompressedStream.cs. The context looks like this:

        private void ReadNextPacket(int len)
        {
            inBuffer = (byte[])inBufferRef.Target;
            if (inBuffer == null || inBuffer.Length < len)
                inBuffer = new byte[len];
            int numRead = 0;
            int numToRead = len;
            while (numToRead > 0)
            {
                int read = baseStream.Read(inBuffer, numRead, numToRead);
                numRead += read;
                numToRead -= read;
            }
        }

Note that on line 170 if read == 0, the while loop will loop infinitely. This condition needs to be evaluated and handled. From the platform documentation, "The Read method will return 0 only if the end of the stream is reached. In all other cases, Read always reads at least one byte from the stream before returning. By definition, if no data is available from the stream upon a call to Read, the Read method returns 0 (the end of the stream is reached automatically)." I also note that in the example provided in the platform documentation they have also made the same error.

Suggested fix:
        private void ReadNextPacket(int len)
        {
            inBuffer = (byte[])inBufferRef.Target;
            if (inBuffer == null || inBuffer.Length < len)
                inBuffer = new byte[len];
            int numRead = 0;
            int numToRead = len;
            while (numToRead > 0)
            {
                int read = baseStream.Read(inBuffer, numRead, numToRead);
                numRead += read;
                numToRead -= read;
                if (read == 0 && numToRead > 0)
                    throw new MySqlException("The end of the stream was unexpectedly reached.");
            }
        }
[17 Mar 2009 7:54] Tonci Grgin
Hi Yvan and thanks for spotting this.

Verified by looking into trunk sources, problem is still there.
[21 Jul 2009 6:35] Tonci Grgin
Wlad, probable duplicate in Bug#46308.
[11 Aug 2009 9:19] Tony Bedford
An entry was added to the 5.2.8, 6.0.5 and 6.1.1 changelogs:

When reading data, such as with a MySqlDataAdapter on a MySqlConnection, MySQL Connector/NET could potentially enter an infinite loop in CompressedStream.ReadNextpacket() if compression was enabled.