MySQL Bugs: #40611: MySQL cannot make a binary log after sequential number beyond unsigned long.

Bug #40611	MySQL cannot make a binary log after sequential number beyond unsigned long.
Submitted:	10 Nov 2008 9:43	Modified:	7 Mar 2010 18:20
Reporter:	Meiji KIMURA	Email Updates:
Status:	Closed	Impact on me:	None
Category:	MySQL Server: Replication	Severity:	S3 (Non-critical)
Version:	5.0.51	OS:	Any
Assigned to:	Luis Soares	CPU Architecture:	Any

Description:
A number of binary log's file name is ever-increasing. It become 1000000 after 999999. There is no documentation about this, so I looked into source codes.　(Windows 32-bit) A filename is created in find_uniq_filename() in log.cc like this.

sprintf(end,"%06ld",max_found+1);

I supposed that an upper limit of a sequential number is 4294967295 because of max_found is unsigned long. But it is created "%06ld", so a sequential number is displayed as signed long like these,

:
bin-log.2147483645
bin-log.2147483646
bin-log.2147483647
bin-log.-2147483648
bin-log.-2147483647
bin-log.-2147483646
:

After becoming bin-log.-2147483648 or later, it works well as before.

But internally max_found is unsigned long.
So if continue to increase a number of bin-log, it could not work well after bin-log.-00001. (If I issue FLUSH LOGS; at that time, MySQL tried to make bin-log.000000, but it faild.)

How to repeat:
Continue to issue "FLASH LOGS;" until exceed to an upper limit of singed long.

But it's distant idea.

To simulate that situation, operate like this.

(1) Shutdown mysqld.
(2) Rename latest bin-log's name to 'bin-log.-000002'.
(3) Edit the bottom line of bin-log.index to 'bin-log.-000002';
(4) Activate mysqld.
(5) issue "FLASH LOGS;" some times.

Suggested fix:
Display an error or stop mysqld.

Or make a code to deal with bin-log.000000 after bin-log.-00001.

[Workaround]
I know that I can avoid this situation and problem by operating bin-log from 000001 to 2147483647 only and I can use 'RESET MASTER' to reset 'bin-log.000001' before exceed bin-log.2147483647 in operation.

In real world, this bug is not severe because it will take 68-year even if I issue 'FLUSH LOGS;' per second. It's distant idea.

While fixing this bug, one should take into account the following situations to 
preventing wraparound failures:

1 - When the counter wraps around, the next value produced must be *.000001, *.000002, *.000003 and so one. This will require changes in the routine that calculates the next value.

2 - Note that the file *.000001 may still exist. In such case, the replication should fail and print out an error message. This situation is highly unlikely but one needs to consider it.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/71474

2835 Luis Soares	2009-04-06
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      
      The upper limit of the sequential number for binlog extension
      overflows and creates extensions with negative numbers. This can lead
      to problems when incrementing and reaching the extension: -00001 ->
      000000 .
      
      This patch addresses this by not allowing negative extensions and by
      returning an error when the limit is exceeded. Furthermore, warnings
      are printed to the error log when the limit is approaching. When FLUSH
      LOGS is issued a warning is returned to the user that something wrong
      happened with the logging.
     @ mysql-test/suite/binlog/r/binlog_max_extension.result
        result file.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the wrap around problem.
     @ sql/log.cc
        Changes to find_uniq_filename.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72045

2835 Luis Soares	2009-04-14
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
      
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72501

2835 Luis Soares	2009-04-20
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
            
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
            
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72502

2835 Luis Soares	2009-04-20
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
            
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
            
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72603

2836 Luis Soares	2009-04-21
      BUG#40611: MySQL cannot make a binary log after sequential number beyond 
      unsigned long.
      
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
      
      NOTE: incremental commit for fixing codestyle as asked by reviewer. and
            added comment for macros.

Pushed to 6.0-rpl.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/72786

2846 Luis Soares	2009-04-24
      BUG#40611: MySQL cannot make a binary log after sequential number 
      beyond unsigned long.
      
      Post-push fix. Test case failed on PB on windows machines because
      atol would truncate/overflow at maximum extension of 0x7FFFFFFF.
      
      Setting 0x7FFFFFFF as the maximum to be the same in all platforms.

Pushed into 6.0.12-alpha (revid:alik@sun.com-20090513032549-rxa73jbxd1qv09xc) (version source revid:aelkin@mysql.com-20090429125820-vu261kl1z4z5f0iv) (merge vers: 6.0.12-alpha) (pib:6)

Documented bugfix in the 6.0.12 changelog as follows:

        MySQL creates binary logs in a numbered sequence, with a maximum
        possible 4294967295 concurrent log files, 4294967295 being the
        maximum value for an unsigned long integer. However, binary log
        file extensions were turned into negative numbers once the
        variable used to hold the value reached the maximum value for a
        signed long integer (2147483647). Consequently, when the
        sequence value was incremented to the next (negative) number,
        this caused MySQL to try to create the file using the .000000
        extension, causing the server to fail since this file already
        existed.

        Negative file extensions are now disallowed, and an error is
        returned when the limit is reached. In addition, FLUSH LOGS now
        also reports warnings to the user, if the extension number has
        reached the limit, and warnings are printed to the error log
        when the limit is approaching.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85008

3113 Luis Soares	2009-09-29
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      
      NOTE: this is the backport to next-mr.
      
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
                  
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit. The limit has been
      set to 0x7FFFFFFF as the maximum.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85012

3113 Luis Soares	2009-09-29
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      BUG#44779: binlog.binlog_max_extension may be causing failure on 
      next test in PB
      
      NOTE1: this is the backport to next-mr.
      NOTE2: already includes patch for BUG#44779.
      
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
                  
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit. The limit has been
      set to 0x7FFFFFFF as the maximum.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/85030

3117 Luis Soares	2009-09-29
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      BUG#44779: binlog.binlog_max_extension may be causing failure on 
      next test in PB
            
      NOTE1: this is the backport to next-mr.
      NOTE2: already includes patch for BUG#44779.
            
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
                        
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit. The limit has been
      set to 0x7FFFFFFF as the maximum.
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/86062

3131 Luis Soares	2009-10-07
      BUG#40611, BUG#44779: reverted in mysql-5.1-rep+2.

Pushed into 6.0.14-alpha (revid:alik@sun.com-20091027094604-9p7kplu1vd2cvcju) (version source revid:zhenxing.he@sun.com-20091026140226-uhnqejkyqx1aeilc) (merge vers: 6.0.14-alpha) (pib:13)

Already documented in 6.0.12; closed.

Pushed into 5.5.0-beta (revid:alik@sun.com-20091110093229-0bh5hix780cyeicl) (version source revid:alik@sun.com-20091027095744-rf45u3x3q5d1f5y0) (merge vers: 5.5.0-beta) (pib:13)

Also documented in the 5.5.0 changelog, closed.

Removed 5.5.0 changelog entry per IRC discussion with Luís.

A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/91231

3145 Luis Soares	2009-11-22
      BUG#40611: MySQL cannot make a binary log after sequential number
      beyond unsigned long.
      BUG#44779: binlog.binlog_max_extension may be causing failure on 
      next test in PB
                  
      NOTE1: this is the backport to next-mr.
      NOTE2: already includes patch for BUG#44779.
                  
      Binlog file extensions would turn into negative numbers once the
      variable used to hold the value reached maximum for signed
      long. Consequently, incrementing value to the next (negative) number
      would lead to .000000 extension, causing the server to fail.
                              
      This patch addresses this issue by not allowing negative extensions
      and by returning an error on find_uniq_filename, when the limit is
      reached. Additionally, warnings are printed to the error log when the
      limit is approaching. FLUSH LOGS will also report warnings to the
      user, if the extension number has reached the limit. The limit has been
      set to 0x7FFFFFFF as the maximum.
     @ mysql-test/suite/binlog/r/binlog_max_extension.result
        
     @ mysql-test/suite/binlog/t/binlog_max_extension.test
        Test case added that checks the maximum available number for
        binlog extensions.
     @ sql/log.cc
        Changes to find_uniq_filename and test_if_number.
     @ sql/log.h
        Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
        LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

Pushed into 6.0.14-alpha (revid:aelkin@mysql.com-20091202145207-zjr6kdpwm5z5jj2z) (version source revid:aelkin@mysql.com-20091202145207-zjr6kdpwm5z5jj2z) (merge vers: 6.0.14-alpha) (pib:13)

Pushed into 5.6.0-beta (revid:aelkin@mysql.com-20091201190718-ls6a6i82bs4vovf9) (version source revid:aelkin@mysql.com-20091201190718-ls6a6i82bs4vovf9) (merge vers: 5.6.0-beta) (pib:13)

Bugfix also documented in the 5.6.0 changelog; closed.

Pushed into 5.5.3-m3 (revid:alik@sun.com-20100306103849-hha31z2enhh7jwt3) (version source revid:vvaintroub@mysql.com-20091202211633-8reso8l6h11gw9ot) (merge vers: 5.6.0-beta) (pib:16)

Moved 5.6.0 changelog entry to 5.5.3.