Bug #120461 InnoDB Batch Commit for Faster Source and Replica Execution
Submitted: 12 May 19:56
Reporter: Nuno Carvalho Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Server Severity:S4 (Feature request)
Version:10.x OS:Any
Assigned to: CPU Architecture:Any

[12 May 19:56] Nuno Carvalho
Description:
This feature introduces a batch commit capability in InnoDB so that higher-level components can commit a group of transactions as a single coordinated unit. The primary motivation is to provide a stronger storage-engine primitive that can improve execution on both the source and the replica. On the source, the server can use batch commit to commit a full binary log group commit set of transactions in one go, reducing repeated commit overhead and aligning storage-engine commit work with the group already formed by the binary log pipeline. On the replica, replication can use the same capability as a foundation for reshaping replicated work before it is applied.

The high-level goal is to improve how MySQL handles workloads whose transaction shape is inefficient for commit processing or replication apply. Very small transactions suffer from high per-transaction overhead, including metadata updates, binary log coordination, and storage-engine commit costs. Very large transactions can create long replication lag and reduce parallelism because the applier must process a large unit before later work can proceed. A batch commit interface gives the server a way to reduce these bottlenecks by committing related transactions together when they are already coordinated, while still preserving correct visibility guarantees.

For source workloads, batch commit can make group commit more effective by allowing the transactions in a binary log group commit batch to be handed to InnoDB as a batch instead of being committed one by one. This can reduce duplicated commit-path work, improve cache and scheduling behavior, and make the source commit pipeline more efficient under high concurrency. The feature does not change transaction semantics: each transaction remains logically independent, but the storage engine receives enough context to process the group more efficiently.

For replica workloads, batch commit provides a foundation for applying work in a form better suited to the applier. Small transactions can be grouped into larger units to reduce per-transaction overhead, while very large transactions can be split into independently prepared units that are made visible together. This can reduce replica write amplification, improve parallel preparation, and shorten catch-up time after backup restore, maintenance, or topology changes. The result is a replication system that can use available CPU and I/O capacity more effectively without changing the logical results observed by users.

The source-side group commit improvement and the replica-side chunk batch commit improvement do not need to be delivered at the same time. Each can provide value independently: the source path can first use batch commit to optimize binary log group commit, while the replica path can later use the same primitive to optimize reshaped apply units. However, the overall design should consider both use cases from the beginning so that the handlerton interface, ordering guarantees, visibility rules, and recovery expectations are compatible with both improvements. This avoids introducing a narrow source-only or replica-only API that would need to be redesigned later.

The storage-engine contract should provide atomic visibility for the batch: other transactions must never observe a state where only part of the batch is committed. They should see either none of the batch effects or all of them. For durability, the feature should guarantee ordered persistence, where transactions are persisted according to the order supplied by the caller. Full all-or-nothing persistence across the batch remains the responsibility of higher-level two-phase commit and recovery integration, allowing this work to stay focused on the InnoDB primitive needed by the source commit path, replication, and other callers.

The implementation should expose the capability through the handlerton interface, advertise support from InnoDB, and reuse existing commit logic as much as possible. A single-transaction commit should remain a trivial special case of batch commit, keeping behavior consistent and minimizing duplicated code. The result is a low-level engine feature with broad performance value: it can make source commits more efficient and give replicas a safer, more flexible foundation for applying source workloads faster.

How to repeat:
Please see description.
[27 May 15:44] Nuno Carvalho
Presentation on Oracle MySQL Contributor Summit

Attachment: InnoDB_Batch_Commit.pdf (application/pdf, text), 233.18 KiB.