Bug #75604 Unable to access MySQL Server via Connector/NET after 24.9 days of uptime
Submitted: 23 Jan 2015 15:33 Modified: 6 Oct 2017 2:27
Reporter: Markus Nägele Email Updates:
Status: Verified Impact on me:
None 
Category:Connector / NET Severity:S2 (Serious)
Version:6.8.3.0 OS:Microsoft Windows
Assigned to: Assigned Account CPU Architecture:Any

[23 Jan 2015 15:33] Markus Nägele
Description:
Hi there,

we were getting the following exception 24.9 days after our servers were restarted:

System.ArgumentOutOfRangeException: Timeout can be only be set to 'System.Threading.Timeout.Infinite' or a value > 0.
Parameter name: value
   at System.Net.Sockets.NetworkStream.set_ReadTimeout(Int32 value)
   at MySql.Data.MySqlClient.TimedStream.StartTimer(IOKind op)
   at MySql.Data.MySqlClient.TimedStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, Byte[] buffer, Int32 offset, Int32 count)
   at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.GetResult(Int32 statementId, Int32& affectedRows, Int64& insertedId)
   at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, Boolean force)
   at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
   at MySql.Data.MySqlClient.MySqlCommand.ExecuteScalar()

We tracked it down to the following line in MySql.Data.MySqlClient.TimedStream.StartTimer:

num = this.timeout - (int)this.stopwatch.ElapsedMilliseconds;

This line can return a negative value, if the elapsed time is negative. The problem is located in the Stop function of the LowResolutionStopwatch:

public void Stop()
{
	long num = (long)Environment.TickCount;
	long num2 = (num < this.startTime) ? (2147483647L - this.startTime + num) : (num - this.startTime);
	this.millis += num2;
}

If the stopwatch was started before the overflow of Environment.TickCount took place, and was stopped after the overflow, then startTime is round about 2^31 and num about -(2^31). The subtraction taking place in this case again returns a number near -(2^31), and not near 0 as expected.

Best regards,

Markus Nägele

How to repeat:
Restart the computer and wait 24.9 days.

Or create a test case as described above for the Stop function of the LowResolutionStopwatch.

Suggested fix:
To avoid this, use 2147483647L*2 instead.
[27 Feb 2015 10:05] Chiranjeevi Battula
Hello Markus Nägele,

Thank you for the bug report.
I could not repeat the issue on Visual Studio 2013 (C#.Net) with  MySQL Connector/Net 6.9.5. Could you please provide complete repeatable test case to confirm this issue at our end?

Thanks,
Chiranjeevi.
[10 Mar 2015 10:13] Johan Lindvall
We have been bit by this bug a couple of times in production too.

Note that on our servers (2012 R2, .Net 4.5.2), Environment.TickCount wraps around to a large negative number (Int32.MinValue).

One one of our affected machines I ran this simple program:

        static void Main(string[] args)
        {
            Console.WriteLine("Environment.TickCount is {0}.", Environment.TickCount);
        }

Environment.TickCount is -2099954984.
[4 Jun 2015 12:17] Chiranjeevi Battula
Hello Johan Lindvall,

Thank you for your feedback.
I tried to reproduce the issue at my end using Visual Studio 2013 (C#.Net), MySQL Connector/Net 6.9.6, but couldn't trace out any issue in "Environment.TickCount" and working fine as per "TickCount" property values.

Thanks,
Chiranjeevi.
[4 Jun 2015 12:26] Johan Lindvall
Hello,

I strongly suggest you read the documentation for Environment.TickCount then:

Compare the .net v1.1 and the current versions and see that your code is still using the obsolete 1.1 value ranges.

V1.1:
https://msdn.microsoft.com/en-us/library/system.environment.tickcount(v=vs.71).aspx

The TickCount property handles an overflow condition by resetting its value to zero. The minimum value returned by TickCount is 0.

THIS IS WHAT YOUR CODE ASSUMES

V2.0:
https://msdn.microsoft.com/en-us/library/system.environment.tickcount(v=vs.80).aspx

The value of this property is derived from the system timer and is stored as a 32-bit signed integer. Consequently, if the system runs continuously, TickCount will increment from zero to Int32.MaxValue for approximately 24.9 days, then jump to Int32.MinValue, which is a negative number, then increment back to zero during the next 24.9 days.

I know the bug is hard to reproduce, but we get bitten by it quite often.

I will also attached a proposed fix to the problem.

If you don't understand what is broken, then please assign this to a developer to look at. This is a serious bug that bites us every two months approximately!.
[4 Jun 2015 12:27] Johan Lindvall
Patch to fix the issue

Attachment: LowResolutionStopWatch.patch (application/octet-stream, text), 1003 bytes.

[8 Jun 2015 20:30] Gabriela Martinez Sanchez
Hi Johan,

I checked this issue and there had been some changes in the way this value is handled based on the version of the framework. Thus I think we need to update the calculations taking into account the framework version.

I'll provide more feedback after the changes and tests.

Thanks for your feedback.
[1 Sep 2015 10:55] Johan Lindvall
Any updates? We were just once again bitten by this bug.
[13 Sep 2015 7:53] Jahwan Kim
This bug has just hit me hard too. ANY UPDATE? This is extremely serious for long-running applications. This bug happens not when the app run for 25 days, but when the machine is up for 25 days (and then about every 50 days).
[29 Mar 2016 10:13] Tom Rathbone
Also just hit this problem in 6.9.8, any progress on this? Johan's patch looks like a suitable solution.
[19 Jul 2016 5:01] UIJONG CHOI
Is there any update regarding this issue? I'm also struggling with this issue with Connector v6.9.8 and it seems like there's still no fix at v7.0; is there any reason not to fix this?

I might apply the patch above and build MyDSQL.Data.DLL for myself.
[6 Oct 2017 2:27] Mikiya Okuno
Verified as described. Changed the synopsis as below:

OLD:
ArgumentOutOfRangeException in TimedStream.StartTimer()

NEW:
Unable to access MySQL Server via Connector/NET after 24.9 days of uptime