MySQL Bugs: #42127: Incremental LCP

Bug #42127	Incremental LCP
Submitted:	15 Jan 2009 8:40	Modified:	15 Jan 2009 8:57
Reporter:	Mikiya Okuno	Email Updates:
Status:	Verified	Impact on me:	None
Category:	MySQL Cluster: Cluster (NDB) storage engine	Severity:	S4 (Feature request)
Version:	mysql-5.1-telco-7.0	OS:	Any
Assigned to:		CPU Architecture:	Any

Description:
When executing a local checkpoint, the algorithm flushes all data pages to disk.  However, a recent machine has a much larger RAM than before and DataMemory is getting huge, thus time taken to complete every LCP is increasing. It will take ages if a server host has several terabytes of RAM as well as filling up all I/O band width.

We need more efficient method to complete LCP, don't we?

How to repeat:
n/a

Suggested fix:
As each record has epoch value, ndbd can determine which record is updated from previous LCP or previous previous LCP, or previous previous previous LCP. The problem is that LCP cannot written incrementally. This RFE is aimed to change some file format so that ndbd can write LCP incrementally. Instead of flushing all data pages, keep the old successful LCP and update it where updated only.

Otherwise, implement completely different LCP file format? Like InnoDB table space?