Bug #121785 Missing rmb() on moving to the next job buffer (aarch64), reopened by Bug#33650674
Description:
In execute_signals() (storage/ndb/src/kernel/vm/mt.cpp, trunk lines
6193-6208), when the consumer reaches the end of a job buffer that isn't
the last one, it moves to the next buffer:
read_buffer = q->m_buffers[read_index];
read_pos = 0;
read_end = read_buffer->m_len;
...
and then goes on to execute the signals in read_buffer->m_data up to
read_end, with no rmb() between the m_len load and the m_data loads.
The producer may still be filling that buffer. publish_position() does
wmb() and then stores m_len, but nothing orders the consumer's side. The
data loads depend on read_buffer, not on m_len (only a control
dependency through read_pos < read_end, which doesn't order loads on
ARM). So on aarch64 the consumer can see the new m_len and the buffer's
old contents, and execute a garbage signal from a recycled buffer. On
x86 loads aren't reordered with loads, so it can't happen there.
Before commit 145987997 (Bug#33650674 "Missing memory barriers - Apple
M1", April 2022), the earlier workaround for bug#3589359 ran an rmb()
before every signal on aarch64, which covered this path. 145987997
removed that as too intrusive and added an rmb() where run_job_buffers()
re-reads the prio A state, which fixed the same-buffer case, but not
this one. Trunk's read_jba_state() and read_all_jbb_state() only order
the m_len of the current buffer; the move-to-next path loads a new m_len
on its own. The prio A and prio B queues both go through it.
I found this with armem, an ARMv8 memory model explorer that runs
compiled code: https://github.com/russellw/armem
How to repeat:
armem's c/mysql_ndb_jb.c holds excerpts of the producer
(publish_position, check_next_index_position, insert_signal) and
consumer (read_jba_state, execute_signals, run_job_buffers' prio A
loop) as in trunk, with a 4-word buffer so that each of two signals
fills one buffer. One thread sends signals (1, 11) and (2, 12); the
other runs the prio A part of run_job_buffers() once and checks that
each signal it executes has the contents sent.
git clone https://github.com/russellw/armem && cd armem
cargo build --release
clang --target=aarch64-linux-gnu -fno-pic -O2 -ffreestanding \
-c c/mysql_ndb_jb.c -o ndb.o
./target/release/armem --elf ndb.o -f 1000 c/mysql_ndb_jb.litmus
armem reports the bad outcome as reachable under ARM and not under
x86-TSO. It's the same at -O1, -Os and -O2, with and without
-march=armv8.1-a (LSE). With -DSAME, both signals go in one buffer, and
trunk holds, as does the code before 145987997 (-DOLD). With
-DNEXT_RMB (the fix below), the bad outcome is unreachable.
Suggested fix:
Add an rmb() after the next buffer's m_len load in execute_signals():
read_buffer = q->m_buffers[read_index];
read_pos = 0;
read_end = read_buffer->m_len;
+ /* Order the signal loads after m_len, see publish_position(). */
+ rmb();
It runs once per 32 KB job buffer, not per signal, and on x86 rmb() is
a compiler barrier.
Portions of this website are copyright © 2001, 2002 The PHP Group
Page generated in 0.051 sec. using MySQL 8.4.7-u6-cloud
Timestamp references displayed by the system are UTC.
Content reproduced on this site is the property of the
respective copyright holders.
It is not reviewed in advance
by Oracle and does not necessarily represent the opinion of
Oracle or any other party.
Description: In execute_signals() (storage/ndb/src/kernel/vm/mt.cpp, trunk lines 6193-6208), when the consumer reaches the end of a job buffer that isn't the last one, it moves to the next buffer: read_buffer = q->m_buffers[read_index]; read_pos = 0; read_end = read_buffer->m_len; ... and then goes on to execute the signals in read_buffer->m_data up to read_end, with no rmb() between the m_len load and the m_data loads. The producer may still be filling that buffer. publish_position() does wmb() and then stores m_len, but nothing orders the consumer's side. The data loads depend on read_buffer, not on m_len (only a control dependency through read_pos < read_end, which doesn't order loads on ARM). So on aarch64 the consumer can see the new m_len and the buffer's old contents, and execute a garbage signal from a recycled buffer. On x86 loads aren't reordered with loads, so it can't happen there. Before commit 145987997 (Bug#33650674 "Missing memory barriers - Apple M1", April 2022), the earlier workaround for bug#3589359 ran an rmb() before every signal on aarch64, which covered this path. 145987997 removed that as too intrusive and added an rmb() where run_job_buffers() re-reads the prio A state, which fixed the same-buffer case, but not this one. Trunk's read_jba_state() and read_all_jbb_state() only order the m_len of the current buffer; the move-to-next path loads a new m_len on its own. The prio A and prio B queues both go through it. I found this with armem, an ARMv8 memory model explorer that runs compiled code: https://github.com/russellw/armem How to repeat: armem's c/mysql_ndb_jb.c holds excerpts of the producer (publish_position, check_next_index_position, insert_signal) and consumer (read_jba_state, execute_signals, run_job_buffers' prio A loop) as in trunk, with a 4-word buffer so that each of two signals fills one buffer. One thread sends signals (1, 11) and (2, 12); the other runs the prio A part of run_job_buffers() once and checks that each signal it executes has the contents sent. git clone https://github.com/russellw/armem && cd armem cargo build --release clang --target=aarch64-linux-gnu -fno-pic -O2 -ffreestanding \ -c c/mysql_ndb_jb.c -o ndb.o ./target/release/armem --elf ndb.o -f 1000 c/mysql_ndb_jb.litmus armem reports the bad outcome as reachable under ARM and not under x86-TSO. It's the same at -O1, -Os and -O2, with and without -march=armv8.1-a (LSE). With -DSAME, both signals go in one buffer, and trunk holds, as does the code before 145987997 (-DOLD). With -DNEXT_RMB (the fix below), the bad outcome is unreachable. Suggested fix: Add an rmb() after the next buffer's m_len load in execute_signals(): read_buffer = q->m_buffers[read_index]; read_pos = 0; read_end = read_buffer->m_len; + /* Order the signal loads after m_len, see publish_position(). */ + rmb(); It runs once per 32 KB job buffer, not per signal, and on x86 rmb() is a compiler barrier.