MySQL Bugs: #19418: Impossible to read database after having restored it from backup of a full db

Bug #19418	Impossible to read database after having restored it from backup of a full db
Submitted:	28 Apr 2006 9:15	Modified:	4 Aug 2006 9:01
Reporter:	Daniel Stricot	Email Updates:
Status:	Not a Bug	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	5.1.8-beta	OS:	Linux (Linux RedHat9)
Assigned to:	Jonas Oreland	CPU Architecture:	Any

Description:
Temporary Error messages during the restoration and impossible to read the database after restoring it from a completely full database .
Error message :
[root@vm10 Src]# /usr/mysql/bin/ndb_restore --backupid=1 --nodeid=3 --restore_data /user/database/ndbd/BACKUP/BACKUP-1 -m
Backup Id = 1
Nodeid = 3
backup path = /user/database/ndbd/BACKUP/BACKUP-1
Ndb version in backup files: Version 5.1.8
Connected to ndb!!
Successfully restored table hlr/def/IMSI
Successfully restored table event REPL$hlr/IMSI
_____________________________________________________
Processing data in table: sys/def/NDB$EVENTS_0(1) fragment 0
_____________________________________________________
Processing data in table: hlr/def/IMSI(4) fragment 0
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
Temporary error: 1221: REDO buffers overloaded, consult online manual (increase RedoBuffer)
_____________________________________________________
Processing data in table: cluster/def/schema(3) fragment 0
_____________________________________________________
Processing data in table: cluster/def/apply_status(2) fragment 0
_____________________________________________________
Processing data in table: sys/def/SYSTAB_0(0) fragment 0
Restored 39772 tuples and 0 log entries
 
NDBT_ProgramExit: 0 - OK
 
mysql> select count(*) from IMSI;
ERROR 1015 (HY000): Can't lock file (errno: 4009)

How to repeat:
Launch the cluster (1 ndb)
Create the table in the database
Populate it with the maximum rows possible to get the database full.
Make a backup of the database.
Stop the ndbd processes and relaunch them with the option --initial
Try to restore the database with ndb_restore

Suggested fix:
.

example

Attachment: Pbbackuprestore.V51 (application/octet-stream, text), 8.28 KiB.

Do you get an error when trying to access other tables than IMSI?
And have you tried to increase the [NDBD]RedoBuffer setting
as suggested by the error message?

There was only one table called 'IMSI' in the database. We didn't try to increase the size of the REDO buffers.

No feedback was provided for this bug for over a month, so it is
being suspended automatically. If you are able to provide the
information that was originally requested, please do so and change
the status of the bug back to "Open".

With the default value for the size of the RedoBuffer (8MB), we had the Temporay Error 1221: REDO buffers overloaded (advice : increase RedoBuffer).
I changed this value in the config.ini file to 80MB and I replayed the test . This time, I got  the Temporary Error 1220 : REDO log files overloaded (advice : decrease TimeBetweenLocalCheckpoints, and|or increase NoOfFragmentLogFiles).
I moved the parameter TimeBetweenLocalCheckpoints from 20 to 4, I got the error 1220 when populating my database at the beginning  of the test (befor the backup).
So , I moved the parameter NoOfFragmentLogFiles from 4 to 40; the population is OK, but the restoration failed like before.
Finally, after many tries, with the value 80 of NoOfFragmentLogFiles , the restoration succeeded. It's a little bit difficult to find the good tuning for the configuration file ?!!!

With the default value for the size of the RedoBuffer (8MB), we had the Temporay Error 1221: REDO buffers overloaded (advice : increase RedoBuffer).
I changed this value in the config.ini file to 80MB and I replayed the test . This time, I got  the Temporary Error 1220 : REDO log files overloaded (advice : decrease TimeBetweenLocalCheckpoints, and|or increase NoOfFragmentLogFiles).
I moved the parameter TimeBetweenLocalCheckpoints from 20 to 4, I got the error 1220 when populating my database at the beginning  of the test (befor the backup).
So , I moved the parameter NoOfFragmentLogFiles from 4 to 40; the population is OK, but the restoration failed like before.
Finally, after many tries, with the value 80 of NoOfFragmentLogFiles , the restoration succeeded. It's a little bit difficult to find the good tuning for the configuration file ?!!!

1) tuning...yes that can be difficult
2) can't read(4009)...you have to close all your connection to a running mysql (or restart it), so that it can safely proceed wo/ any lingering caches.

  for this there is already an open bug report...