MySQL Bugs: #38264: Cluster Backup ends with error 2810

Bug #38264	Cluster Backup ends with error 2810
Submitted:	21 Jul 2008 16:15	Modified:	17 Jul 2010 18:36
Reporter:	Michael Neubert	Email Updates:
Status:	Can't repeat	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S2 (Serious)
Version:	mysql-5.1-telco-6.2	OS:	Any
Assigned to:		CPU Architecture:	Any
Tags:	Backup, cluster, disk space, mysql-5.1.23 ndb-6.2.14

Description:
Hello,

we are using a cluster setup of 4 nodes (number of replicas = 2). When the whole cluster is online a cluster backup writes approximately 12 GB data per node to the local storage disk, where more than 150 GB free disk space is available.

If we stop one node (e.g. of nodegroup 0) and then start a cluster backup, the one single node in nodegroup 0 should write 24 GB data to the local disk and the 2 nodes of nodegroup 1 each 12 GB.

With mysql-5.1.22 ndb-6.2.7 this was no problem, but now we get:

- ERROR 2810 "No space left on the device" -

...after writing approximately 16 GB of data to the single node in nodegroup 0 and the Backup is aborted.

After searching the bugs database I found the following bug report:

http://bugs.mysql.com/bug.php?id=28647

Maybe the patch, that was pushed to 5.1.23-ndb-6.2.11 lead to the described problem, because this new error code was just introduced in this revision.

Best wishes
Michael

How to repeat:
see scenario above

Suggested fix:
no fix available, just try to avoid disconnected nodes when cluster backup starts, that writes a lot of data

What are the physical memory of your machines.
Can you create a single file whose size is bigger than 16GB in the hosts of nodegroup 0.
Can you privide configuration files.

The machines have 16 GB of physical memory. Furthermore we are using disk data tables to increase the available memory.

Our backup space is using ext3 filesystem. Under kernel 2.4 the file size is limited to 16 GB, but we are using kernel 2.6, where 2 TB should be possible. And we already had files with a size of 20 GB+.

Thank you for the feedback.

I can not repeat described behavior. What is exact package name you use (file name you downloaded)? Or do you compile yourself? If so, please send us configure options.

Please also inform us which operating system do you use and if it is 64-bit or 32-bit binaries.

We don't use the Cluster Engine anymore. So no further Informations are possible.

Best wishes
Michael

Thank you for the feedback.

Report closed as "Can't repeat" as we were not able to repeat it. If you start using NDB engine again and able to repeat the problem feel free to add a comment and reopen the report.