| Bug #101000 | Command.ExecuteReader hangs indefinitely | ||
|---|---|---|---|
| Submitted: | 29 Sep 2020 19:48 | Modified: | 7 Oct 2020 18:04 |
| Reporter: | Swikriti Jain | Email Updates: | |
| Status: | Can't repeat | Impact on me: | |
| Category: | Connector / NET | Severity: | S2 (Serious) |
| Version: | 8.0.21 | OS: | Windows |
| Assigned to: | CPU Architecture: | Any | |
[29 Sep 2020 19:48]
Swikriti Jain
Stack trace for the hang
Attachment: HangTrace.png (image/png, text), 33.09 KiB.
[29 Sep 2020 19:49]
Swikriti Jain
Attaching the call stack where it is stuck
[30 Sep 2020 6:37]
MySQL Verification Team
Hello Swikriti, Thank you for the report and feedback. May I request you to please provide exact test case(C# file or even better compressed VS project dump demonstrating the issue along with the exact schema involved) to reproduce this issue at our end? Thank you regards, Umesh
[30 Sep 2020 22:44]
Swikriti Jain
I have uploaded the dump to the file share. As we are testing the issue, I am using tpcc database schema.
[1 Oct 2020 6:21]
MySQL Verification Team
Please provide a c# script demonstrating the issue(memory dump not required) so that we can reproduce here. You can mark it as private after posting here. regards, Umesh
[1 Oct 2020 6:43]
Swikriti Jain
It is a big project and not just a c# file. We won't be able to share that. Is there any other way?
[1 Oct 2020 6:48]
MySQL Verification Team
Just a c# script which manifest the issue. For example, please see Bug #100846, Bug #100522, Bug #100327 etc
[7 Oct 2020 18:04]
Swikriti Jain
Hi,
This is not easily reproducible separately. This only happens when the server is under heavy load.
On further investigation I noticed that this is the error we get after the hang reaches the wait_timeout:
Inner exception type: MySql.Data.MySqlClient.MySqlException
Message:
Fatal error encountered attempting to read the resultset.
HResult : 0x80004005
FacilityCode : 0 (000)
ErrorCode : 16389 (4005)
Stack:
at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
Inner exception type: MySql.Data.MySqlClient.MySqlException
Message:
Reading from the stream has failed.
HResult : 0x80004005
FacilityCode : 0 (000)
ErrorCode : 16389 (4005)
Stack:
at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)
at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
Description: We have a multi threaded application where we are reading data from our MySQL server. There are about 15 threads in parallel. There comes a point when the command.ExecuteReader() hangs indefinitely. We have wait_timeout set to 8 hours. It finally errors out after 8 hours. using (var connection = new MySqlConnection()) { connection.ConnectionString = this.ConnectionString; try { connection.Open(); } using (var command = connection.CreateCommand()) { command.CommandText = queryText; command.CommandType = CommandType.Text; command.CommandTimeout = 60 * 60 * 2; // 2 hours try { // Execute command and process result using (var reader = command.ExecuteReader()) { result = resultHandler(reader); reader.Close(); } } catch (MySqlException exc) { throw exc; } catch (SocketException exc) { throw exc; } } if (connection.State == ConnectionState.Open) { connection.Close(); } } We also set connection pooling = false to see if that solves the problem, but no improvement. We looked at the number of connections open and there were about 30 and all of them for our database were in sleep mode. We were able to connect via workbench and run the same query manually. How to repeat: It repros every time we run our application. If we run the above piece of code from different threads and let it run for some time, it will eventually hang.