Bug #47014 rpl_drop_temp fails on PB-2 with results mismatch
Submitted: 31 Aug 2009 11:43 Modified: 19 Dec 2009 11:11
Reporter: Andrei Elkin Email Updates:
Status: Closed Impact on me:
None 
Category:Tests: Replication Severity:S3 (Non-critical)
Version:5.1 OS:Any
Assigned to: Luis Soares CPU Architecture:Any
Tags: pb-2, test failure

[31 Aug 2009 11:43] Andrei Elkin
Description:
rpl.rpl_drop_temp                        w4 [ fail ]
        Test ended at 2009-08-31 07:08:56

CURRENT_TEST: rpl.rpl_drop_temp
--- /export/home/pb2/test/sb_1-720448-1251690576.1/mysql-5.1.39-solaris10-sparc-test/mysql-test/suite/rpl/r/rpl_drop_temp.result	2009-08-31 06:41:54.000000000 +0300
+++ /export/home/pb2/test/sb_1-720448-1251690576.1/mysql-5.1.39-solaris10-sparc-test/mysql-test/suite/rpl/r/rpl_drop_temp.reject	2009-08-31 08:08:55.438886000 +0300
http://pb2.norway.sun.com/web.py?template=mysql_show_test_failure&test_failure_id=2297928

shows

@@ -10,5 +10,5 @@
 create temporary table mysqltest.t2 (n int)ENGINE=MyISAM;
 show status like 'Slave_open_temp_tables';
 Variable_name	Value
-Slave_open_temp_tables	0
+Slave_open_temp_tables	1
 drop database mysqltest;

mysqltest: Result content mismatch

 - saving '/export/home/pb2/test/sb_1-720448-1251690576.1/mysql-5.1.39-solaris10-sparc-test/mysql-test/var/4/log/rpl.rpl_drop_temp/' to '/export/home/pb2/test/sb_1-720448-1251690576.1/mysql-5.1.39-solaris10-sparc-test/mysql-test/var/log/rpl.rpl_drop_temp/'

How to repeat:
Look at PB-2 logs.
[4 Sep 2009 10:48] Luis Soares
The test case creates two temporary tables, then closes the
connection, waits for it to disconnect, then syncs the slave with
the master, checks for remaining opened temporary tables on
slave (which should be 0) and finally drops the used
database (mysqltest).

Unfortunately, sometimes, the test fails with one open table on
the slave. This seems to be caused by the fact that waiting for
the connection to be closed is not sufficient. The test needs to
wait for the DROP event to be logged and only then synchronize
the slave with the master and proceed with the check. This is
caused by the asynchronous nature of the disconnect wrt
binlogging of the DROP temporary table statement.

Backing this analysis is the binlog contents of a failed test
case run. There one may find that: 

DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `t2`,`t1`

appears last in the binlog, whereas in a successful run, it shows
up before dropping the database mysqltest (see test case file for
details on test implementation). Since the check for open tables
happens before dropping the database, this means that in a failed
run, the slave has not executed the drop temporary table at the
time the check for open temp tables is performed, ultimately,
causing the test to fail.

Given the analysis, this bug should be fixed by deploying a wait
for DROP event before the first synchronization between master
and slave.
[4 Sep 2009 10:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/82445

3094 Luis Soares	2009-09-04
      BUG#47014: rpl_drop_temp fails on PB-2 with results mismatch
      
      The test case creates two temporary tables, then closes the
      connection, waits for it to disconnect, then syncs the slave with
      the master, checks for remaining opened temporary tables on
      slave (which should be 0) and finally drops the used
      database (mysqltest).
      
      Unfortunately, sometimes, the test fails with one open table on
      the slave. This seems to be caused by the fact that waiting for
      the connection to be closed is not sufficient. The test needs to
      wait for the DROP event to be logged and only then synchronize
      the slave with the master and proceed with the check. This is
      caused by the asynchronous nature of the disconnect wrt
      binlogging of the DROP temporary table statement.
      
      We fix this by deploying a call to wait_for_binlog_event.inc
      on the test case, which makes execution to wait for the DROP
      temp tables event before synchronizing master and slave.
[13 Sep 2009 20:52] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/83122

3122 Luis Soares	2009-09-13
      BUG#47014: rpl_drop_temp fails on PB-2 with results mismatch
            
      The test case creates two temporary tables, then closes the
      connection, waits for it to disconnect, then syncs the slave with
      the master, checks for remaining opened temporary tables on
      slave (which should be 0) and finally drops the used
      database (mysqltest).
            
      Unfortunately, sometimes, the test fails with one open table on
      the slave. This seems to be caused by the fact that waiting for
      the connection to be closed is not sufficient. The test needs to
      wait for the DROP event to be logged and only then synchronize
      the slave with the master and proceed with the check. This is
      caused by the asynchronous nature of the disconnect wrt
      binlogging of the DROP temporary table statement.
            
      We fix this by deploying a call to wait_for_binlog_event.inc
      on the test case, which makes execution to wait for the DROP
      temp tables event before synchronizing master and slave.
[13 Sep 2009 21:07] Bugs System
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/83123

3603 Luis Soares	2009-09-13 [merge]
      BUG#47014: merge: mysql-5.1-bugteam --> mysql-pe
[13 Sep 2009 21:12] Luis Soares
Queued in mysql-5.1-bugteam and mysql-pe.
[14 Sep 2009 16:04] Bugs System
Pushed into 5.4.4-alpha (revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (version source revid:alik@sun.com-20090914155317-m1g9wodmndzdj4l1) (merge vers: 5.4.4-alpha) (pib:11)
[15 Sep 2009 7:27] Jon Stephens
Test failure only; no user-facing changes to document; closed without further action.
[6 Oct 2009 8:59] Bugs System
Pushed into 5.1.40 (revid:joro@sun.com-20091006073316-lea2cpijh9r6on7c) (version source revid:ingo.struewing@sun.com-20090916070128-6053el2ucp5z7pyn) (merge vers: 5.1.39) (pib:11)
[18 Dec 2009 10:32] Bugs System
Pushed into 5.1.41-ndb-7.1.0 (revid:jonas@mysql.com-20091218102229-64tk47xonu3dv6r6) (version source revid:jonas@mysql.com-20091218095730-26gwjidfsdw45dto) (merge vers: 5.1.41-ndb-7.1.0) (pib:15)
[18 Dec 2009 10:48] Bugs System
Pushed into 5.1.41-ndb-6.2.19 (revid:jonas@mysql.com-20091218100224-vtzr0fahhsuhjsmt) (version source revid:jonas@mysql.com-20091217101452-qwzyaig50w74xmye) (merge vers: 5.1.41-ndb-6.2.19) (pib:15)
[18 Dec 2009 11:03] Bugs System
Pushed into 5.1.41-ndb-6.3.31 (revid:jonas@mysql.com-20091218100616-75d9tek96o6ob6k0) (version source revid:jonas@mysql.com-20091217154335-290no45qdins5bwo) (merge vers: 5.1.41-ndb-6.3.31) (pib:15)
[18 Dec 2009 11:17] Bugs System
Pushed into 5.1.41-ndb-7.0.11 (revid:jonas@mysql.com-20091218101303-ga32mrnr15jsa606) (version source revid:jonas@mysql.com-20091218064304-ezreonykd9f4kelk) (merge vers: 5.1.41-ndb-7.0.11) (pib:15)
[19 Dec 2009 11:11] Jon Stephens
Setting back to Closed state; see my earlier comments.