Bug #121785 Missing rmb() on moving to the next job buffer (aarch64), reopened by Bug#33650674

Submitted: 15 Nov 1:14
Reporter: Russell Wallace Email Updates:
Status: Open Impact on me:
None 
Category:MySQL Cluster: Cluster (NDB) storage engine Severity:S3 (Non-critical)
Version:trunk (06bcb0eb0ad2, October 2026) OS:Linux
Assigned to: CPU Architecture:ARM

[15 Nov 1:14] Russell Wallace
Description:
In execute_signals() (storage/ndb/src/kernel/vm/mt.cpp, trunk lines
6193-6208), when the consumer reaches the end of a job buffer that isn't
the last one, it moves to the next buffer:

    read_buffer = q->m_buffers[read_index];
    read_pos = 0;
    read_end = read_buffer->m_len;
    ...
and then goes on to execute the signals in read_buffer->m_data up to
read_end, with no rmb() between the m_len load and the m_data loads.

The producer may still be filling that buffer. publish_position() does
wmb() and then stores m_len, but nothing orders the consumer's side. The
data loads depend on read_buffer, not on m_len (only a control
dependency through read_pos < read_end, which doesn't order loads on
ARM). So on aarch64 the consumer can see the new m_len and the buffer's
old contents, and execute a garbage signal from a recycled buffer. On
x86 loads aren't reordered with loads, so it can't happen there.

Before commit 145987997 (Bug#33650674 "Missing memory barriers - Apple
M1", April 2022), the earlier workaround for bug#3589359 ran an rmb()
before every signal on aarch64, which covered this path. 145987997
removed that as too intrusive and added an rmb() where run_job_buffers()
re-reads the prio A state, which fixed the same-buffer case, but not
this one. Trunk's read_jba_state() and read_all_jbb_state() only order
the m_len of the current buffer; the move-to-next path loads a new m_len
on its own. The prio A and prio B queues both go through it.

I found this with armem, an ARMv8 memory model explorer that runs
compiled code: https://github.com/russellw/armem

How to repeat:
armem's c/mysql_ndb_jb.c holds excerpts of the producer
(publish_position, check_next_index_position, insert_signal) and
consumer (read_jba_state, execute_signals, run_job_buffers' prio A
loop) as in trunk, with a 4-word buffer so that each of two signals
fills one buffer. One thread sends signals (1, 11) and (2, 12); the
other runs the prio A part of run_job_buffers() once and checks that
each signal it executes has the contents sent.

    git clone https://github.com/russellw/armem && cd armem
    cargo build --release
    clang --target=aarch64-linux-gnu -fno-pic -O2 -ffreestanding \
        -c c/mysql_ndb_jb.c -o ndb.o
    ./target/release/armem --elf ndb.o -f 1000 c/mysql_ndb_jb.litmus

armem reports the bad outcome as reachable under ARM and not under
x86-TSO. It's the same at -O1, -Os and -O2, with and without
-march=armv8.1-a (LSE). With -DSAME, both signals go in one buffer, and
trunk holds, as does the code before 145987997 (-DOLD). With
-DNEXT_RMB (the fix below), the bad outcome is unreachable.

Suggested fix:
Add an rmb() after the next buffer's m_len load in execute_signals():

        read_buffer = q->m_buffers[read_index];
        read_pos = 0;
        read_end = read_buffer->m_len;
 +      /* Order the signal loads after m_len, see publish_position(). */
 +      rmb();

It runs once per 32 KB job buffer, not per signal, and on x86 rmb() is
a compiler barrier.

Portions of this website are copyright © 2001, 2002 The PHP Group

Page generated in 0.051 sec. using MySQL 8.4.7-u6-cloud

Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the respective copyright holders.
It is not reviewed in advance by Oracle and does not necessarily represent the opinion of Oracle or any other party.