Bug #18596 binlog schema event auto push causes mysqld errors
Submitted: 28 Mar 2006 23:38 Modified: 4 Apr 2006 6:51
Reporter: Jonathan Miller Email Updates:
Status: Duplicate Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S2 (Serious)
Version:5.1.9 OS:Linux (Linux 32 Bit OS)
Assigned to: Assigned Account CPU Architecture:Any

[28 Mar 2006 23:38] Jonathan Miller 
Description:
Running cid_ndb_dd2.pl causes errors on all mysqld (5) processes that are running:

Errors:

060328 23:16:10 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:22:04 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:22:27 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:22:56 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:23:46 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:24:09 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:24:38 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:25:01 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:25:49 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Table definition has changed, please retry transaction
060328 23:25:49 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Sort aborted
060328 23:26:20 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:26:41 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Got error 155393856 from storage engine
060328 23:26:41 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Sort aborted
060328 23:26:41 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Got error 152729952 from storage engine
060328 23:26:41 [ERROR] /home/ndbdev/jmiller/builds/libexec/mysqld: Sort aborted
060328 23:27:11 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:27:34 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:31:50 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:32:15 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:32:44 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:33:07 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:35:17 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:35:41 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:39:35 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:39:58 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:43:01 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:43:24 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:43:52 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:44:18 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060328 23:47:21 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 7
060328 23:47:43 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060329  0:37:48 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE TESTER2.t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 8
060329  0:38:11 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1
060329  0:39:31 [ERROR] Could not discover table 'TESTER2.t1' from binlog schema event 'CREATE TABLE TESTER2.t1 (c1 INT, c2 CHAR(4), c3 FLOAT,
                         c4 DOUBLE, c5 BIT(4), c6 VARCHAR(100), PRIMARY KEY (c1))
                         TABLESPACE TESTER2_TS STORAGE DISK ENGINE=NDB' from node 8
060329  0:39:54 [ERROR] NDB: DROP DATABASE TESTER2: error Can't drop database 'TESTER2'; database doesn't exist 1008 1 1

How to repeat:
run attached script to cluster with mutipal MySQLD process attached.
[28 Mar 2006 23:39] Jonathan Miller 
cid_ndb_dd2.pl

Attachment: cid_ndb_dd2.pl (application/x-perl, text), 6.98 KiB.
[29 Mar 2006 9:14] Tomas Ulin 
these errors all seem related to a missing database on this mysqld

there is no autodiscovery of databases.  A missing database TESTER2 would give the errors below.

If on the other hand the database TESTER2 was created while both mysqld's where connected it is a bug that this database was not created on the other mysqld.

But I don't see such an error message
[29 Mar 2006 12:26] Jonathan Miller 
Hi,

So the cid_ndb_dd2.pl script creates table space, creates the databases tester2. creates a table t1, inserts some data, deletes from t1, drops t1, drops the table space, and drops the database.

All of the mysqld error logs except for the one the script run on have error logs filled with these errors. Not sure what to tell you. 

I wonder why we can not discover table 'TESTER2.t1' from binlog schema event? 

Best wishes,
[29 Mar 2006 15:49] Jonathan Miller 
Error: NdbObjectIdMap::::unmap(0, 0x8b47218) obj=0x8938fe8
[3 Apr 2006 15:18] Bugs System 
A patch for this bug has been committed. After review, it may
be pushed to the relevant source trees for release in the next
version. You can access the patch from:

  http://lists.mysql.com/commits/4415
[4 Apr 2006 6:51] Tomas Ulin 
duplicate with 18490

The root cause are statements being dropped/lost when distributed to other mysqld's.  In this case it is a "create database" that is lost every now and then.

This because these statements are (before this patch) distributed asynchronously, and the blob "event code" may collapse the statements into one...

It has now been made synchronous.