Bug #75604 Unable to access MySQL Server via Connector/NET after 24.9 days of uptime

Submitted: 23 Jan 2015 15:33 Modified: 19 Jun 2019 16:24
Reporter: Markus Nägele Email Updates:
Status: Closed Impact on me:
None 
Category:Connector / NET Severity:S2 (Serious)
Version:6.8.3.0 OS:Windows
Assigned to: Assigned Account CPU Architecture:Any

[23 Jan 2015 15:33] Markus Nägele
Description:
Hi there,

we were getting the following exception 24.9 days after our servers were restarted:

System.ArgumentOutOfRangeException: Timeout can be only be set to 'System.Threading.Timeout.Infinite' or a value > 0.
Parameter name: value
   at System.Net.Sockets.NetworkStream.set_ReadTimeout(Int32 value)
   at MySql.Data.MySqlClient.TimedStream.StartTimer(IOKind op)
   at MySql.Data.MySqlClient.TimedStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteScalar()

We tracked it down to the following line in MySql.Data.MySqlClient.TimedStream.StartTimer:

num = this.timeout - (int)this.stopwatch.ElapsedMilliseconds;

This line can return a negative value, if the elapsed time is negative. The problem is located in the Stop function of the LowResolutionStopwatch:

public void Stop()
{
	long num = (long)Environment.TickCount;
	long num2 = (num < this.startTime) ? (2147483647L - this.startTime + num) : (num - this.startTime);
	this.millis += num2;
}

If the stopwatch was started before the overflow of Environment.TickCount took place, and was stopped after the overflow, then startTime is round about 2^31 and num about -(2^31). The subtraction taking place in this case again returns a number near -(2^31), and not near 0 as expected.

Best regards,

Markus Nägele

How to repeat:
Restart the computer and wait 24.9 days.

Or create a test case as described above for the Stop function of the LowResolutionStopwatch.

Suggested fix:
To avoid this, use 2147483647L*2 instead.
[27 Feb 2015 10:05] Chiranjeevi Battula
Hello Markus Nägele,

Thank you for the bug report.
I could not repeat the issue on Visual Studio 2013 (C#.Net) with  MySQL Connector/Net 6.9.5. Could you please provide complete repeatable test case to confirm this issue at our end?

Thanks,
Chiranjeevi.
[10 Mar 2015 10:13] Johan Lindvall
We have been bit by this bug a couple of times in production too.

Note that on our servers (2012 R2, .Net 4.5.2), Environment.TickCount wraps around to a large negative number (Int32.MinValue).

One one of our affected machines I ran this simple program:

        static void Main(string[] args)
        {
            Console.WriteLine("Environment.TickCount is {0}.", Environment.TickCount);
        }

Environment.TickCount is -2099954984.
[4 Jun 2015 12:17] Chiranjeevi Battula
Hello Johan Lindvall,

Thank you for your feedback.
I tried to reproduce the issue at my end using Visual Studio 2013 (C#.Net), MySQL Connector/Net 6.9.6, but couldn't trace out any issue in "Environment.TickCount" and working fine as per "TickCount" property values.

Thanks,
Chiranjeevi.
[4 Jun 2015 12:26] Johan Lindvall
Hello,

I strongly suggest you read the documentation for Environment.TickCount then:

Compare the .net v1.1 and the current versions and see that your code is still using the obsolete 1.1 value ranges.

V1.1:
https://msdn.microsoft.com/en-us/library/system.environment.tickcount(v=vs.71).aspx

The TickCount property handles an overflow condition by resetting its value to zero. The minimum value returned by TickCount is 0.

THIS IS WHAT YOUR CODE ASSUMES

V2.0:
https://msdn.microsoft.com/en-us/library/system.environment.tickcount(v=vs.80).aspx

The value of this property is derived from the system timer and is stored as a 32-bit signed integer. Consequently, if the system runs continuously, TickCount will increment from zero to Int32.MaxValue for approximately 24.9 days, then jump to Int32.MinValue, which is a negative number, then increment back to zero during the next 24.9 days.

I know the bug is hard to reproduce, but we get bitten by it quite often.

I will also attached a proposed fix to the problem.

If you don't understand what is broken, then please assign this to a developer to look at. This is a serious bug that bites us every two months approximately!.
[4 Jun 2015 12:27] Johan Lindvall
Patch to fix the issue

Attachment: LowResolutionStopWatch.patch (application/octet-stream, text), 1003 bytes.

[8 Jun 2015 20:30] Gabriela Martinez Sanchez
Hi Johan,

I checked this issue and there had been some changes in the way this value is handled based on the version of the framework. Thus I think we need to update the calculations taking into account the framework version.

I'll provide more feedback after the changes and tests.

Thanks for your feedback.
[1 Sep 2015 10:55] Johan Lindvall
Any updates? We were just once again bitten by this bug.
[13 Sep 2015 7:53] Jahwan Kim
This bug has just hit me hard too. ANY UPDATE? This is extremely serious for long-running applications. This bug happens not when the app run for 25 days, but when the machine is up for 25 days (and then about every 50 days).
[29 Mar 2016 10:13] Tom Rathbone
Also just hit this problem in 6.9.8, any progress on this? Johan's patch looks like a suitable solution.
[19 Jul 2016 5:01] UIJONG CHOI
Is there any update regarding this issue? I'm also struggling with this issue with Connector v6.9.8 and it seems like there's still no fix at v7.0; is there any reason not to fix this?

I might apply the patch above and build MyDSQL.Data.DLL for myself.
[6 Oct 2017 2:27] MySQL Verification Team
Verified as described. Changed the synopsis as below:

OLD:
ArgumentOutOfRangeException in TimedStream.StartTimer()

NEW:
Unable to access MySQL Server via Connector/NET after 24.9 days of uptime
[19 Jun 2019 16:24] Christine Cole
Posted by developer:
 
Fixed as of the upcoming MySQL Connector/NET  6.10.9 and 8.0.17 release, and here's the changelog entry:

Connector/NET access to MySQL stopped working after the computer hosting
the server was started and continued to operate uninterrupted for a
defined period of time.

Thank you for the bug report.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.063 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.