Bug #46597 MySQL Replication problems HA_ERR_END_OF_FILE
Submitted: 7 Aug 2009 8:07 Modified: 18 Jun 2010 21:39
Reporter: SubbbyBoy Subby Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Server: General Severity:S1 (Critical)
Version:5.1.34 OS:Linux (Centos 5.0)
Assigned to: CPU Architecture:Any

[7 Aug 2009 8:07] SubbbyBoy Subby
Description:
there is continuous this errors mysqld.log file. My Replication Type Master<->Master.
 

How to repeat:
3 minutes interval

Suggested fix:
I'dont know.
[7 Aug 2009 9:51] Sveta Smirnova
Thank you for the report.

Please provide full error log file and configuration files for both MySQL servers.
[7 Sep 2009 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[8 Sep 2009 15:22] SubbbyBoy Subby
Master-1 Conf File

Attachment: My1Cnf.txt (text/plain), 1.73 KiB.

[8 Sep 2009 15:23] SubbbyBoy Subby
Master-2 Conf File

Attachment: My2Cnf.txt (text/plain), 1.76 KiB.

[8 Sep 2009 18:01] Sveta Smirnova
Thank you for the update.

Please send us full error log files from both servers.
[9 Sep 2009 6:31] SubbbyBoy Subby
090908 16:17:33 [ERROR] Slave SQL: Could not execute Update_rows event on table community.conversations; Can't find record in 'conversations', Error_code: 1032; handler error HA_ERR_END_OF_FILE; the event's master log mysql-bin.000192, end_log_pos 876831852, Error_code: 1032
[9 Sep 2009 6:33] SubbbyBoy Subby
each servers same errors
[9 Sep 2009 6:45] Sveta Smirnova
Thank you for the feedback.

Please provide output of SHOW CREATE TABLE conversations on both servers.
[9 Sep 2009 7:15] SubbbyBoy Subby
CREATE TABLE `conversations` (
  `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `OWNER_ID` int(10) unsigned NOT NULL,
  `CITY_ID` tinyint(3) unsigned NOT NULL DEFAULT '1',
  `USER2_ID` int(10) unsigned NOT NULL,
  `SUBJECT` varchar(128) DEFAULT NULL,
  `BODY_IN` varchar(128) DEFAULT NULL,
  `BODY_OUT` varchar(128) DEFAULT NULL,
  `SENT_TIME` datetime NOT NULL DEFAULT '2000-01-01 00:00:00',
  `RECIEVED_TIME` datetime NOT NULL DEFAULT '2000-01-01 00:00:00',
  `STARRED` enum('Y','N') NOT NULL DEFAULT 'N',
  `MESSAGE_COUNT` smallint(5) unsigned NOT NULL DEFAULT '1',
  `SIBLING_ID` bigint(20) unsigned NOT NULL,
  `TRASHED` enum('Y','N') NOT NULL DEFAULT 'N',
  `STATUS_IN` enum('NEW','DISPLAYED','READ') NOT NULL DEFAULT 'NEW',
  `STATUS_OUT` enum('NEW','READ') NOT NULL DEFAULT 'NEW',
  `SYSTEM_MESSAGE` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`ID`,`OWNER_ID`),
  KEY `index_1` (`OWNER_ID`,`CITY_ID`) USING BTREE,
  KEY `Index_2` (`ID`) USING BTREE,
  KEY `owner_city_trash` (`OWNER_ID`,`CITY_ID`,`TRASHED`)
) ENGINE=InnoDB AUTO_INCREMENT=992904306 DEFAULT CHARSET=latin5
/*!50100 PARTITION BY HASH (OWNER_ID)
PARTITIONS 101 */

CREATE TABLE `conversations` (
  `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  `OWNER_ID` int(10) unsigned NOT NULL,
  `CITY_ID` tinyint(3) unsigned NOT NULL DEFAULT '1',
  `USER2_ID` int(10) unsigned NOT NULL,
  `SUBJECT` varchar(128) DEFAULT NULL,
  `BODY_IN` varchar(128) DEFAULT NULL,
  `BODY_OUT` varchar(128) DEFAULT NULL,
  `SENT_TIME` datetime NOT NULL DEFAULT '2000-01-01 00:00:00',
  `RECIEVED_TIME` datetime NOT NULL DEFAULT '2000-01-01 00:00:00',
  `STARRED` enum('Y','N') NOT NULL DEFAULT 'N',
  `MESSAGE_COUNT` smallint(5) unsigned NOT NULL DEFAULT '1',
  `SIBLING_ID` bigint(20) unsigned NOT NULL,
  `TRASHED` enum('Y','N') NOT NULL DEFAULT 'N',
  `STATUS_IN` enum('NEW','DISPLAYED','READ') NOT NULL DEFAULT 'NEW',
  `STATUS_OUT` enum('NEW','READ') NOT NULL DEFAULT 'NEW',
  `SYSTEM_MESSAGE` int(10) unsigned DEFAULT NULL,
  PRIMARY KEY (`ID`,`OWNER_ID`),
  KEY `index_1` (`OWNER_ID`,`CITY_ID`) USING BTREE,
  KEY `Index_2` (`ID`) USING BTREE,
  KEY `owner_city_trash` (`OWNER_ID`,`CITY_ID`,`TRASHED`)
) ENGINE=InnoDB AUTO_INCREMENT=992904387 DEFAULT CHARSET=latin5
/*!50100 PARTITION BY HASH (OWNER_ID)
PARTITIONS 101 */
[9 Sep 2009 8:14] Sveta Smirnova
Thank you for the feedback.

Tables are same, my guesswork about this can be duplicate of one of existing bugs not confirmed.

Please also provide extract from mysql-bin.000192 where end_log_pos 876831852 (use mysqlbinlog --stop-position=876831852)
[9 Sep 2009 9:09] SubbbyBoy Subby
#090908 16:17:32 server id 14035  end_log_pos 876830793         Query   thread_id=609426        exec_time=0     error_code=0
SET TIMESTAMP=1252415852/*!*/;
BEGIN
/*!*/;
# at 876830793
# at 876830854
# at 876830941
#090908 16:17:32 server id 14035  end_log_pos 876830854         Table_map: `community`.`message_counts` mapped to number 15
#090908 16:17:32 server id 14035  end_log_pos 876830941         Table_map: `community`.`conversations` mapped to number 16
#090908 16:17:32 server id 14035  end_log_pos 876831167         Update_rows: table id 16 flags: STMT_END_F

BINLOG '
bFmmShPTNgAAPQAAAIZgQzQAAA8AAAAAAAAACWNvbW11bml0eQAObWVzc2FnZV9jb3VudHMABAMD
AwwAAA==
bFmmShPTNgAAVwAAAN1gQzQAABAAAAAAAAAACWNvbW11bml0eQANY29udmVyc2F0aW9ucwAQCAMB
Aw8PDwwM/gII/v7+Aw6AAIAAgAD3AfcB9wH3AXCA
bFmmShjTNgAA4gAAAL9hQzQQABAAAAAAAAEAEP////9AgOK8KDsAAAAAyC5AAATqYygABVNlbGFt
KVRJSyBUSUsuLi5LaU1TRSBZT0tNVT9CRU4gRGlMaSBCQUxEQU4gLi4ubLZxx0USAABstnHHRRIA
AAIBAMGfeEQAAAAAAgECQIDivCg7AAAAAMguQAAE6mMoAAVTZWxhbSlUSUsgVElLLi4uS2lNU0Ug
WU9LTVU/QkVOIERpTGkgQkFMREFOIC4uLmy2ccdFEgAAbLZxx0USAAACAQDBn3hEAAAAAAICAg==
'/*!*/;
# at 876831167
#090908 16:17:32 server id 14035  end_log_pos 876831194         Xid = 272382602
COMMIT/*!*/;
# at 876831194
#090908 16:17:33 server id 14021  end_log_pos 876831272         Query   thread_id=981710        exec_time=0     error_code=0
use community/*!*/;
SET TIMESTAMP=1252415853/*!*/;
SET @@session.auto_increment_increment=8, @@session.auto_increment_offset=1/*!*/;
BEGIN
/*!*/;
# at 876831272
#090908 16:17:33 server id 14021  end_log_pos 876831461         Query   thread_id=981710        exec_time=0     error_code=0
SET TIMESTAMP=1252415853/*!*/;
/*  */ REPLACE INTO wink_counts (USER_ID, WINK_COUNT, CITY_ID,LAST_UPDATE) values(3804373,0,1,'2009-09-08 16:17:33')
/*!*/;
# at 876831461
#090908 16:17:33 server id 14021  end_log_pos 876831488         Xid = 272382631
COMMIT/*!*/;
# at 876831488
#090908 16:17:33 server id 14021  end_log_pos 876831566         Query   thread_id=981708        exec_time=0     error_code=0
SET TIMESTAMP=1252415853/*!*/;
BEGIN
/*!*/;
# at 876831566
# at 876831653
# at 876831714
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
[11 Sep 2009 7:44] Sveta Smirnova
See also bug #44360
[11 Sep 2009 8:18] Sveta Smirnova
Thank you for the feedback.

Nothing obvious in this part of binary log.

Error 1032 means "No such row", so this can be application logic when one of rows somehow doesn't exists on another server. With master-master replication this is possible.

Please compare if both slaves contain same data. Also you can temporarily switch binary log format to STATEMENT to get more human-readable error, when check if such a row exists in the server where replication is stopped.
[11 Oct 2009 23:00] Bugs System
No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".
[18 Jun 2010 21:43] Roel Van de Paar
Likely duplicate of Bug #40638 which is a duplicate of Bug #39648