Bug #50018 binlog corruption when table has many columns
Submitted: 1 Jan 2010 8:33 Modified: 15 Mar 2010 5:05
Reporter: Shane Bester (Platinum Quality Contributor) Email Updates:
Status: Closed Impact on me:
None 
Category:MySQL Server: Row Based Replication ( RBR ) Severity:S1 (Critical)
Version:5.1.42 OS:Any
Assigned to: Luis Soares CPU Architecture:Any
Triage: Triaged: D2 (Serious)

[1 Jan 2010 8:33] Shane Bester
Description:
when inserting into a table with many columns, the binlog gets corrupted and attempts to read it show errors like this:

# at 7012
ERROR: Error in Log_event::read_log_event(): 'read error', data_len: 82444288, event_type: 61
ERROR: Could not read entry at offset 7487: Error in log format or read error.

How to repeat:
start server with --log-bin --binlog-format=row
import the privately attached file.
shutdown server.
try read the binlog with mysqlbinlog (or attach a slave)
[1 Jan 2010 9:20] Shane Bester
testcase. import and binlog gets corrupted.

Attachment: bug50018_public_testcase_v1.sql (, text), 5.61 KiB.

[1 Jan 2010 9:22] Shane Bester
a malicious (or not!) user can break all slaves and point in time recovery using binlogs.  this is a serious bug.
[6 Jan 2010 0:45] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/96050

3303 Luis Soares	2010-01-06
      BUG#50018: binlog corruption when table has many columns
      
      For tables with metadata sizes ranging from 251 to 255 the size
      of the event data (m_data_size) was being improperly calculated
      in the Table_map_log_event constructor. This was due to the fact
      that when writing the Table_map_log_event body (in
      Table_map_log_event::write_data_body) a call to net_store_length
      is made for packing the m_field_metadata_size. It happens that
      net_store_length uses *one* byte for storing
      m_field_metadata_size when it is smaller than 251 but *three*
      bytes when it exceeds that value. BUG 42749 had already
      pinpointed and fix this fact, but the fix was incomplete, as the
      calculation in the Table_map_log_event constructor considers 255
      instead of 251 as the threshold to increment m_data_size by
      three. Thence, the window for having a mismatch between the
      number of bytes written and the number of bytes accounted in the
      event length (m_data_size) was left open for
      m_field_metadata_size values between 251 and 255.
      
      We fix this by changing the condition in the Table_map_log_event
      constructor to match the one in the net_store_length, ie,
      increment one byte if m_field_metadata_size < 251 and three if it
      exceeds this value.
     @ mysql-test/suite/rpl/r/rpl_row_tbl_metadata.result
        Updated result file.
     @ mysql-test/suite/rpl/t/rpl_row_tbl_metadata.test
        Changes to the original test case: added slave and moved
        file into the rpl suite.
        
        New test case: replicates two tables one with 250 and 
        another with 252 metadata sizes. This exercises the usage
        of 1 or 3 bytes while packing the m_field_metadata_size.
     @ sql/log_event.cc
        Made the m_data_size calculation for the table map log event
        to match the number of bytes used while packing the 
        m_field_metadata_size value (according to net_store_length
        function in pack.c).
[7 Jan 2010 17:46] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/96285

3304 Luis Soares	2010-01-07
      BUG#50018: binlog corruption when table has many columns
      
      Some improvements on the test case as suggested during review.
[14 Jan 2010 10:48] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/96869

3305 Luis Soares	2010-01-14 [merge]
      BUG#50018: automerge from 5.1-bt local --> 5.1-bt local latest.
[14 Jan 2010 10:49] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/96870

3306 Luis Soares	2010-01-14
      BUG#50018: binlog corruption when table has many columns
      
      Small fix in the test case. Changed the UNLOCK tables 
      to happen after each insert.
[14 Jan 2010 11:09] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/96873

3827 Luis Soares	2010-01-14 [merge]
      BUG#50018: automerge 5.1-bugteam latest --> mysql-pe latest.
[4 Feb 2010 10:18] Bugs System
Pushed into 5.1.44 (revid:joro@sun.com-20100204101444-2j32mhqroo0iiio6) (version source revid:joro@sun.com-20100115090646-0g4tjrmqf20axlpv) (merge vers: 5.1.43) (pib:16)
[5 Feb 2010 11:45] Bugs System
Pushed into mysql-next-mr (revid:alik@sun.com-20100204063540-9czpdmpixi3iw2yb) (version source revid:alik@sun.com-20100119163614-172adculixyu26j5) (pib:16)
[5 Feb 2010 11:52] Bugs System
Pushed into 6.0.14-alpha (revid:alik@sun.com-20100205113942-oqovjy0eoqbarn7i) (version source revid:alik@sun.com-20100204064210-ljwanqvrjs83s1gq) (merge vers: 6.0.14-alpha) (pib:16)
[5 Feb 2010 11:57] Bugs System
Pushed into 5.5.2-m2 (revid:alik@sun.com-20100203172258-1n5dsotny40yufxw) (version source revid:alexey.kopytov@sun.com-20100115112653-e3a24041ag1cv6v3) (merge vers: 5.5.1-m2) (pib:16)
[10 Feb 2010 7:00] Jon Stephens
Documented bugfix in the 5.1.44, 5.5.2, and 6.0.14 changelogs, as follows:

        In some cases, inserting into a table with many columns could
        cause the binary log to become corrupted.

Closed.
[12 Mar 2010 14:20] Bugs System
Pushed into 5.1.44-ndb-7.0.14 (revid:jonas@mysql.com-20100312135944-t0z8s1da2orvl66x) (version source revid:jonas@mysql.com-20100312115609-woou0te4a6s4ae9y) (merge vers: 5.1.44-ndb-7.0.14) (pib:16)
[12 Mar 2010 14:35] Bugs System
Pushed into 5.1.44-ndb-6.2.19 (revid:jonas@mysql.com-20100312134846-tuqhd9w3tv4xgl3d) (version source revid:jonas@mysql.com-20100312060623-mx6407w2vx76h3by) (merge vers: 5.1.44-ndb-6.2.19) (pib:16)
[12 Mar 2010 14:51] Bugs System
Pushed into 5.1.44-ndb-6.3.33 (revid:jonas@mysql.com-20100312135724-xcw8vw2lu3mijrhn) (version source revid:jonas@mysql.com-20100312103652-snkltsd197l7q2yg) (merge vers: 5.1.44-ndb-6.3.33) (pib:16)
[15 Mar 2010 5:05] Jon Stephens
No additional changelog entries required. Returning to Closed state.