s390/bpf: Emit a barrier for BPF_FETCH instructions
Subsystems (3)
Ilya Leoshkevich
maintainerPATCH v2 1/1 s390/bpf: Emit a barrier for BPF_FETCH instructions
BPF_ATOMIC_OP() macro documentation states that "BPF_ADD | BPF_FETCH" should be the same as atomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.
s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on BPF_FETCH acting as a barrier, for example:
stw [%r0], 1
afadd [%r1], %r2
ldxw %r3, [%r4]
may be broken. Fix it by emitting "bcr 14,0".
Note that a separate serialization instruction is not needed for BPF_XCHG and BPF_CMPXCHG, because COMPARE AND SWAP performs serialization itself.
---
v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2: Emit a barrier only for BPF_FETCH variants;
Add an example of the code that may break to the commit message (Puranjay).
arch/s390/net/bpf_jit_comp.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/s390/net/bpf_jit_comp.c +6 -2
@@ -1427,8 +1427,12 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,EMIT6_DISP_LH (0xeb000000 , is32 ? (op32) : (op64), \ (insn->imm &BPF_FETCH ) ? src_reg :REG_W0 , \ src_reg, dst_reg, off); \ -if (is32 && (insn->imm &BPF_FETCH )) \ -EMIT_ZERO (src_reg); \ +if (insn->imm &BPF_FETCH ) { \ +/* bcr 14,0 - see atomic_fetch_{add,and,or,xor}() */ \ +_EMIT2 (0x07e0 ); \ +if (is32) \ +EMIT_ZERO (src_reg); \ + } \ }while (0 )case BPF_ADD :case BPF_ADD |BPF_FETCH :
Signature · 1 line
2.45.0
Quoted 15 lines · Ilya Leoshkevich
BPF_ATOMIC_OP()macro documentation states that "BPF_ADD|BPF_FETCH" should be the same asatomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on
BPF_FETCHacting as a barrier, for example:stw [%r0], 1
afadd [%r1], %r2
ldxw %r3, [%r4]may be broken. Fix it by emitting "bcr 14,0".
Quoted 8 lines
Note that a separate serialization instruction is not needed for
BPF_XCHGandBPF_CMPXCHG, because COMPARE AND SWAP performs serialization itself.Fixes ba3b86b9cef0 ("s390/bpf: Implement new atomic ops")Reported-by Puranjay Mohan <[email protected]>Signed-off-by Ilya Leoshkevich <[email protected]>
Thanks,
Puranjay
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Tue, 7 May 2024 02:02:49 +0200 you wrote:
BPF_ATOMIC_OP()macro documentation states that "BPF_ADD|BPF_FETCH" should be the same asatomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on
BPF_FETCHacting as a barrier, for example:[...]
Here is the summary with links:
- [bpf-next,v2] s390/bpf: Emit a barrier for BPF_FETCH instructions
https://git.kernel.org/bpf/bpf-next/c/68378982f0b2
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html