Skip to contentSkip to conversation

s390/bpf: Emit a barrier for BPF_FETCH instructions

PATCH Applied Ilya Leoshkevich 3 participants 3 messages May 7, 2024 – May 13, 2024

Subsystems (3)
lore
Share & tools
lore

Ilya Leoshkevich

author maintainer
#0 lore raw

PATCH v2 1/1 s390/bpf: Emit a barrier for BPF_FETCH instructions

BPF_ATOMIC_OP() macro documentation states that "BPF_ADD | BPF_FETCH" should be the same as atomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.

s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on BPF_FETCH acting as a barrier, for example:

stw [%r0], 1
afadd [%r1], %r2
ldxw %r3, [%r4]

may be broken. Fix it by emitting "bcr 14,0".

Note that a separate serialization instruction is not needed for BPF_XCHG and BPF_CMPXCHG, because COMPARE AND SWAP performs serialization itself.

Fixes ba3b86b9cef0 ("s390/bpf: Implement new atomic ops")

---

v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2: Emit a barrier only for BPF_FETCH variants;
Add an example of the code that may break to the commit message (Puranjay).

 arch/s390/net/bpf_jit_comp.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
arch/s390/net/bpf_jit_comp.c +6 -2
index fa2f824e3b06..4be8f5cadd02 100644
@@ -1427,8 +1427,12 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp,
 	EMIT6_DISP_LH(0xeb000000, is32 ? (op32) : (op64),		\
 		      (insn->imm & BPF_FETCH) ? src_reg : REG_W0,	\
 		      src_reg, dst_reg, off);				\
-	if (is32 && (insn->imm & BPF_FETCH))				\
-		EMIT_ZERO(src_reg);					\
+	if (insn->imm & BPF_FETCH) {					\
+		/* bcr 14,0 - see atomic_fetch_{add,and,or,xor}() */	\
+		_EMIT2(0x07e0);						\
+		if (is32)                                               \
+			EMIT_ZERO(src_reg);				\
+	}								\
 } while (0)
 		case BPF_ADD:
 		case BPF_ADD | BPF_FETCH:
Signature · 1 line
2.45.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 15 lines · Ilya Leoshkevich

BPF_ATOMIC_OP() macro documentation states that "BPF_ADD | BPF_FETCH" should be the same as atomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.

s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on BPF_FETCH acting as a barrier, for example:

stw [%r0], 1
afadd [%r1], %r2
ldxw %r3, [%r4]

may be broken. Fix it by emitting "bcr 14,0".

Quoted 8 lines

Note that a separate serialization instruction is not needed for BPF_XCHG and BPF_CMPXCHG, because COMPARE AND SWAP performs serialization itself.

Fixes ba3b86b9cef0 ("s390/bpf: Implement new atomic ops")

Thanks,
Puranjay

#2 lore raw
In reply to Ilya Leoshkevich
This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 10 lines · On Tue, 7 May 2024 02:02:49 +0200 you wrote:

BPF_ATOMIC_OP() macro documentation states that "BPF_ADD | BPF_FETCH" should be the same as atomic_fetch_add(), which is currently not the case on s390x: the serialization instruction "bcr 14,0" is missing. This applies to "and", "or" and "xor" variants too.

s390x is allowed to reorder stores with subsequent fetches from different addresses, so code relying on BPF_FETCH acting as a barrier, for example:

[...]

Here is the summary with links:
- [bpf-next,v2] s390/bpf: Emit a barrier for BPF_FETCH instructions

    https://git.kernel.org/bpf/bpf-next/c/68378982f0b2

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html