Bug #36689 error: 723, No such table existed
Submitted: 13 May 2008 13:29 Modified: 20 Nov 2008 13:53
Reporter: ws lee Email Updates:
Status: Can't repeat Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:5.1.23 OS:Solaris (5.10)
Assigned to: Tomas Ulin CPU Architecture:Any

[13 May 2008 13:29] ws lee
Description:
test> create table t1(a int);
ERROR 1050 (42S01): Table 'test' already exists

test> show tables;
Empty set (0.05 sec)

# ls /var/lib/mysql5.1.23/test
db.opt

# tail -f mysql01.log

080513 22:12:29 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:31 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:32 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:33 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:34 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:35 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:36 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:38 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:39 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:40 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:41 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080513 22:12:42 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
.....

i can't t1 table.
and now in mysql error log upper error message is showing.
thie error message is never-ending.

how can i repair this error?

How to repeat:
i have 2 data node and 2 mysql node.
since never-ending this error message, cpu load is 4% used continuously in idle.
[13 May 2008 17:22] Hartmut Holzgraefe
Could you provide the clusters message log (from the management node) and the data nodes error log and trace files (if any) together with your cluster configuration file?

It is next to impossible to tell what went wrong here from the mysqld error log alone ...
[14 May 2008 0:38] ws lee
thanks your confirm.

in yesterday, management and data log don't have error message.

management log is below.
only include checkpoint log.
...
2008-05-13 18:16:32 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109708 started. Keep GCI = 2826983 oldest restorable GCI = 282691
6
2008-05-13 18:18:14 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109709 started. Keep GCI = 2827017 oldest restorable GCI = 282699
5
2008-05-13 18:20:00 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109710 started. Keep GCI = 2827052 oldest restorable GCI = 282699
5
2008-05-13 18:21:44 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109711 started. Keep GCI = 2827088 oldest restorable GCI = 282699
5
2008-05-13 18:23:26 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109712 started. Keep GCI = 2827121 oldest restorable GCI = 282713
1
2008-05-13 18:25:10 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109713 started. Keep GCI = 2827155 oldest restorable GCI = 282716
2
2008-05-13 18:26:52 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109714 started. Keep GCI = 2827189 oldest restorable GCI = 282716
2
2008-05-13 18:28:36 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109715 started. Keep GCI = 2827223 oldest restorable GCI = 282724
9
2008-05-13 18:30:16 [MgmSrvr] INFO     -- Node 2: Local checkpoint 109716 started. Keep GCI = 2827258 oldest restorable GCI = 282727
4
...

alse, in yesterday, data log(ndb_?_error.log file) don't include error.
last error of data node is below. 

Time: Friday 9 May 2008 - 22:13:53
Status: Temporary error, restart node
Message: Internal program error (failed ndbrequire) (Internal error, programming error or missing error message, please report a bug
)
Error: 2341
Error data: lgman.cpp
Error object: LGMAN (Line: 1446) 0x0000000a
Program: /usr/local/mysql5.1.23/libexec/ndbd
Pid: 11247
Trace: /usr/local/mysql5.1.23/data/ndb_2_trace.log.3
Version: Version 5.1.23 (rc)
***EOM***

last log was not related with this err.
[14 May 2008 15:51] Jonathan Miller
Can you run ndb_show_tables and see if the tables already exist in the cluster?

If so, you can run ndb_drop_table to remove them.
[15 May 2008 0:17] ws lee
thanks Jonathan Miller, your reply.

# ndb_desc -c 192.x.x.x t_provisioning -d test
No such object: t1

NDBT_ProgramExit: 0 - OK

# ndb_drop_table -c 192.x.x.x t_1 -d test
Dropping table t1...
723: No such table existed

NDBT_ProgramExit: 1 - Failed

t1 tables is not likely exist alreay.

The error continues today also.
# tail -f mysql01.err
....
080515  9:08:23 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080515  9:08:24 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080515  9:08:26 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080515  9:08:26 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
....
[28 May 2008 18:00] Hartmut Holzgraefe
The repeated error message is emitted in ndbcluster_find_all_files()
which iterates over a list of tables retrieved using the NDBAPI call 

  dict->listObjects(list, NdbDictionary::Object::UserTable) 

So the t1 table seems to exist in the cluster dictionary in some form.

Does it show in any way when you execute 

  ndb_show_tables 

without any further options? This usese listObjects(), too, 
as far as i know so i would assume that it should show the t1
object somehow ...
[29 May 2008 1:20] ws lee
To. Hartmut Holzgraefe 
thanks your reply.

in ndb_show_tables, t1 table is shown.

328   UserTable            Online   Yes     test         def      t1

P.S)
in my system, this error is continuing....
error log file size is becoming larger shockingly.
[29 May 2008 6:26] Tomas Ulin
please do a "flush table t1;"
[29 May 2008 6:35] ws lee
To. Tomas Ulin
thanks!
but, errro was not repair.

mysql> use test;
mysql> flush table t1;
mysql> select now();
+---------------------+
| now()               |
+---------------------+
| 2008-05-29 15:29:11 | 
+---------------------+
1 row in set (0.00 sec)

# tail -f mysql01.err
.....
080529 15:29:55 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:29:56 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:29:58 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:29:58 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:30:00 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:30:01 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:30:02 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:30:03 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 15:30:05 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
......
[29 May 2008 6:39] Tomas Ulin
are you able to set up another mysqld and connect it to the cluster?  and does it show the same symptoms?

T
[29 May 2008 6:44] Tomas Ulin
can you show the full ndb_show_tables?
[29 May 2008 7:12] ws lee
To.Tomas Ulin
thanks for your fast reply.

i done right now new machine in mysql03.
but same error.

# hostname
mysql03
# tail -f mysql03.err 
080529 16:01:11 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 16:01:12 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 16:01:13 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 16:01:14 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
080529 16:01:15 [ERROR] NDB: failed to setup table test.t1, error: 723, No such table existed
[29 May 2008 7:35] Tomas Ulin
So what I'm trying to figure out is if t1 is broken.  table t1 consists of several objects shown in ndb_show_tables, e.g. indexes.  That is why I want to see the full list.

What is the SQL table definition of t1

T
[20 Nov 2008 13:53] Bernd Ocklin
Probably fixed in later versions.
[24 Jan 2010 9:00] gilberto migliavacca
Hi all

I have the same error and scenario of Lee Wonseok and I think that
the answer of Tomas Ulin is correct
 
"So what I'm trying to figure out is if t1 is broken.  table t1 consists of 
several objects shown in ndb_show_tables, e.g. indexes.  
That is why I want to see the full list."

this is my ndb_show_tables results

[root@custdev05 ~]# ndb_show_tables
id    type                 state    logging database     schema   name
3     UserTable            Online   Yes     mysql        def      NDB$BLOB_2_3
2     UserTable            Online   Yes     mysql        def      ndb_schema
4     UserTable            Online   Yes     mysql        def      ndb_apply_status
116   UserTable            Online   Yes     fnbl         def      fnbl_pim_contact
69    UserTable            Online   Yes     fnbl         def      Layout
60    UserTable            Online   Yes     fnbl         def      Group_
89    UserTable            Online   Yes     fnbl         def      Role_
1     SystemTable          Online   Yes     sys          def      NDB$EVENTS_0
92    UserTable            Online   Yes     fnbl         def      sp_sp_i18n
66    UserTable            Online   Yes     fnbl         def      Image
0     SystemTable          Online   Yes     sys          def      SYSTAB_0
8     UserTable            Online   Yes     fnbl         def      CalEvent
119   UserTable            Online   Yes     fnbl         def      fnbl_pim_contact_photo
1     TableEvent           Online   -                             REPL$mysql/ndb_schema
2389  TableEvent           Online   -                             REPL$test/Company
2     TableEvent           Online   -                             NDB$BLOBEVENT_REPL$mysql/ndb_schema_3
2390  TableEvent           Online   -                             NDB$BLOBEVENT_REPL$test/Company_1
3     TableEvent           Online   -                             REPL$mysql/ndb_apply_status

NDBT_ProgramExit: 0 - OK

when I try to execute ndb_drop_table fnbl_pim_contact -d fnbl
I get
Dropping table fnbl_pim_contact...
723: No such table existed

NDBT_ProgramExit: 1 - Failed

How can I force a delete/drop table?
is there a way to remove some files from the file system in order to remove this reference?

thanks in advance
[12 Oct 2011 13:13] ma hao
We have met a similar problem.
{code}
111012 16:10:42 [ERROR] NDB: failed to setup table crowddb_p68.cwd_property, error: 723, No such table existed
111012 16:10:46 [ERROR] NDB: failed to setup table crowddb_p68.cwd_property, error: 723, No such table existed
{code}

And when we try to delete the table with ndb_drop_table.
{code}
[root@esjirp64 database]# ndb_drop_table cwd_property                       
Dropping table cwd_property...
723: No such table existed

NDBT_ProgramExit: 1 - Failed

{code}

And now our problem is we can not create table named with "cwd_property" on database, also we can not delete any table named with "cwd_property".
{code}
mysql> use crowddb_p68;               
Database changed
mysql> create table cwd_property (id int (1));                
ERROR 1050 (42S01): Table 'cwd_property' already exists
mysql> delete from cwd_property;
ERROR 1146 (42S02): Table 'crowddb_p68.cwd_property' doesn't exist
mysql> drop table cwd_property;                         
ERROR 1051 (42S02): Unknown table 'cwd_property'
mysql> delete from cwd_property;
ERROR 1146 (42S02): Table 'crowddb_p68.cwd_property' doesn't exist
mysql> create table cwd_property (id int (1)); 
ERROR 1050 (42S01): Table 'cwd_property' already exists
{code}

Any advice for this?