Skip to contentSkip to conversation

libarena: Use compiler load-acquire/store-release in bpf_atomic.h

PATCH Applied Puranjay Mohan 3 participants 3 messages Jul 22

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 libarena: Use compiler load-acquire/store-release in bpf_atomic.h

Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises __BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.

Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.

Commit 880442305a39 ("bpf: Introduce load-acquire and store-release instructions") introduced support, hence kernels from 6.15 onwards are needed when compiling with compilers supporting these instructions. We have relatively recent kernel version requirements in libarena anyway, and have not cut first release, hence declare such a dependency.

---

Changelog:
v2: https://lore.kernel.org/bpf/[email protected]/
Changes in v3:
- Use consistent ({ }) for multi-line macros

v1: https://lore.kernel.org/bpf/[email protected]/
Changes in v2:
- Use __BPF_FEATURE_LOAD_ACQ_STORE_REL directly for guard.
- Use __unqual_typeof

---
 .../bpf/libarena/include/bpf_atomic.h         | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
tools/testing/selftests/bpf/libarena/include/bpf_atomic.h +20 -0
index b7b2304319295..43c306e17f198 100644
@@ -86,6 +86,25 @@ extern bool CONFIG_X86_64 __kconfig __weak;
 /* Control dependency provides LOAD->STORE, provide LOAD->LOAD */
 #define smp_acquire__after_ctrl_dep() ({ smp_rmb(); })
 
+#if defined(__BPF_FEATURE_LOAD_ACQ_STORE_REL)
+/*
+ * Clang advertises this feature when it can lower acquire/release atomic
+ * builtins to BPF_LOAD_ACQ/BPF_STORE_REL. Older compilers keep using the
+ * barrier-based fallback below. The generated instructions require kernel
+ * verifier/JIT support added in Linux 6.15; compile for an older BPF CPU to
+ * keep using the fallback when targeting older kernels.
+ */
+#define smp_load_acquire(p)								\
+	({										\
+		__unqual_typeof(*(p)) ___p1 = __atomic_load_n((p), __ATOMIC_ACQUIRE);	\
+		(typeof(*(p)))___p1;							\
+	})
+
+#define smp_store_release(p, val)							\
+	({										\
+		__atomic_store_n((p), (val), __ATOMIC_RELEASE);				\
+	})
+#else
 #define smp_load_acquire(p)                                  \
 	({                                                   \
 		__unqual_typeof(*(p)) __v = READ_ONCE(*(p)); \
@@ -102,6 +121,7 @@ extern bool CONFIG_X86_64 __kconfig __weak;
 		barrier();             \
 		WRITE_ONCE(*(p), val); \
 	})
+#endif
 
 #define smp_cond_load_relaxed_label(p, cond_expr, label)                \
 	({                                                              \

base-commit: a23a71823352e2d792dcaae25f1ebb744acbfc0b

Signature · 1 line
2.53.0-Meta
reviewer Reviewed-by on v3 1/1
#1 lore raw
Quoted 8 lines · Puranjay Mohan

Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises __BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.

Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.

Quoted 7 lines

Commit 880442305a39 ("bpf: Introduce load-acquire and store-release instructions") introduced support, hence kernels from 6.15 onwards are needed when compiling with compilers supporting these instructions. We have relatively recent kernel version requirements in libarena anyway, and have not cut first release, hence declare such a dependency.

Quoted 56 lines

---

Changelog:
v2: https://lore.kernel.org/bpf/[email protected]/
Changes in v3:
- Use consistent ({ }) for multi-line macros

v1: https://lore.kernel.org/bpf/[email protected]/
Changes in v2:
- Use __BPF_FEATURE_LOAD_ACQ_STORE_REL directly for guard.
- Use __unqual_typeof

---
 .../bpf/libarena/include/bpf_atomic.h         | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
tools/testing/selftests/bpf/libarena/include/bpf_atomic.h +20 -0
index b7b2304319295..43c306e17f198 100644
@@ -86,6 +86,25 @@ extern bool CONFIG_X86_64 __kconfig __weak;
 /* Control dependency provides LOAD->STORE, provide LOAD->LOAD */
 #define smp_acquire__after_ctrl_dep() ({ smp_rmb(); })
 
+#if defined(__BPF_FEATURE_LOAD_ACQ_STORE_REL)
+/*
+ * Clang advertises this feature when it can lower acquire/release atomic
+ * builtins to BPF_LOAD_ACQ/BPF_STORE_REL. Older compilers keep using the
+ * barrier-based fallback below. The generated instructions require kernel
+ * verifier/JIT support added in Linux 6.15; compile for an older BPF CPU to
+ * keep using the fallback when targeting older kernels.
+ */
+#define smp_load_acquire(p)								\
+	({										\
+		__unqual_typeof(*(p)) ___p1 = __atomic_load_n((p), __ATOMIC_ACQUIRE);	\
+		(typeof(*(p)))___p1;							\
+	})
+
+#define smp_store_release(p, val)							\
+	({										\
+		__atomic_store_n((p), (val), __ATOMIC_RELEASE);				\
+	})
+#else
 #define smp_load_acquire(p)                                  \
 	({                                                   \
 		__unqual_typeof(*(p)) __v = READ_ONCE(*(p)); \
@@ -102,6 +121,7 @@ extern bool CONFIG_X86_64 __kconfig __weak;
 		barrier();             \
 		WRITE_ONCE(*(p), val); \
 	})
+#endif
 
 #define smp_cond_load_relaxed_label(p, cond_expr, label)                \
 	({                                                              \

base-commit: a23a71823352e2d792dcaae25f1ebb744acbfc0b

#2 lore raw
In reply to Puranjay Mohan
This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 9 lines · On Wed, 22 Jul 2026 07:10:01 -0700 you wrote:

Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises __BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.

Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.

[...]

Here is the summary with links:
- [bpf-next,v3] libarena: Use compiler load-acquire/store-release in bpf_atomic.h

    https://git.kernel.org/bpf/bpf-next/c/87267b894598

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html