libarena: Use compiler load-acquire/store-release in bpf_atomic.h
Subsystems (3)
PATCH v3 1/1 libarena: Use compiler load-acquire/store-release in bpf_atomic.h
Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises __BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.
Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.
Commit 880442305a39 ("bpf: Introduce load-acquire and store-release instructions") introduced support, hence kernels from 6.15 onwards are needed when compiling with compilers supporting these instructions. We have relatively recent kernel version requirements in libarena anyway, and have not cut first release, hence declare such a dependency.
---
Changelog:
v2: https://lore.kernel.org/bpf/[email protected]/
Changes in v3:
- Use consistent ({ }) for multi-line macros
v1: https://lore.kernel.org/bpf/[email protected]/
Changes in v2:
- Use __BPF_FEATURE_LOAD_ACQ_STORE_REL directly for guard.
- Use __unqual_typeof
--- .../bpf/libarena/include/bpf_atomic.h | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+)
tools/testing/selftests/bpf/libarena/include/bpf_atomic.h +20 -0
@@ -86,6 +86,25 @@ extern bool CONFIG_X86_64 __kconfig __weak;/* Control dependency provides LOAD->STORE, provide LOAD->LOAD */ #define smp_acquire__after_ctrl_dep () ({ smp_rmb(); }) +#if defined(__BPF_FEATURE_LOAD_ACQ_STORE_REL) +/* +* Clang advertises this feature when it can lower acquire/release atomic +* builtins to BPF_LOAD_ACQ/BPF_STORE_REL. Older compilers keep using the +* barrier-based fallback below. The generated instructions require kernel +* verifier/JIT support added in Linux 6.15; compile for an older BPF CPU to +* keep using the fallback when targeting older kernels. +*/ +#define smp_load_acquire (p) \ + ({ \ + __unqual_typeof(*(p)) ___p1 = __atomic_load_n((p), __ATOMIC_ACQUIRE); \ + (typeof(*(p)))___p1; \ + }) + +#define smp_store_release (p, val) \ + ({ \ + __atomic_store_n((p), (val), __ATOMIC_RELEASE); \ + }) +#else #define smp_load_acquire (p) \ ({ \ __unqual_typeof(*(p)) __v = READ_ONCE(*(p)); \
@@ -102,6 +121,7 @@ extern bool CONFIG_X86_64 __kconfig __weak;barrier (); \WRITE_ONCE (*(p), val); \ }) +#endif #define smp_cond_load_relaxed_label (p, cond_expr, label) \ ({ \
base-commit: a23a71823352e2d792dcaae25f1ebb744acbfc0b
Signature · 1 line
2.53.0-Meta
Quoted 8 lines · Puranjay Mohan
Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises
__BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.
Quoted 7 lines
Commit
880442305a39("bpf: Introduce load-acquire and store-release instructions") introduced support, hence kernels from 6.15 onwards are needed when compiling with compilers supporting these instructions. We have relatively recent kernel version requirements in libarena anyway, and have not cut first release, hence declare such a dependency.Signed-off-by Puranjay Mohan <[email protected]>
Quoted 56 lines
---
Changelog:
v2: https://lore.kernel.org/bpf/[email protected]/
Changes in v3:
- Use consistent ({ }) for multi-line macrosv1: https://lore.kernel.org/bpf/[email protected]/
Changes in v2:
- Use__BPF_FEATURE_LOAD_ACQ_STORE_RELdirectly for guard.
- Use__unqual_typeof--- .../bpf/libarena/include/bpf_atomic.h | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+)tools/testing/selftests/bpf/libarena/include/bpf_atomic.h +20
-0@@ -86,6 +86,25 @@ extern bool CONFIG_X86_64 __kconfig __weak;/* Control dependency provides LOAD->STORE, provide LOAD->LOAD */ #define smp_acquire__after_ctrl_dep () ({ smp_rmb(); }) +#if defined(__BPF_FEATURE_LOAD_ACQ_STORE_REL) +/* +* Clang advertises this feature when it can lower acquire/release atomic +* builtins to BPF_LOAD_ACQ/BPF_STORE_REL. Older compilers keep using the +* barrier-based fallback below. The generated instructions require kernel +* verifier/JIT support added in Linux 6.15; compile for an older BPF CPU to +* keep using the fallback when targeting older kernels. +*/ +#define smp_load_acquire (p) \ + ({ \ + __unqual_typeof(*(p)) ___p1 = __atomic_load_n((p), __ATOMIC_ACQUIRE); \ + (typeof(*(p)))___p1; \ + }) + +#define smp_store_release (p, val) \ + ({ \ + __atomic_store_n((p), (val), __ATOMIC_RELEASE); \ + }) +#else #define smp_load_acquire (p) \ ({ \ __unqual_typeof(*(p)) __v = READ_ONCE(*(p)); \@@ -102,6 +121,7 @@ extern bool CONFIG_X86_64 __kconfig __weak;barrier (); \WRITE_ONCE (*(p), val); \ }) +#endif #define smp_cond_load_relaxed_label (p, cond_expr, label) \ ({ \base-commit:
a23a71823352e2d792dcaae25f1ebb744acbfc0b
This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 9 lines · On Wed, 22 Jul 2026 07:10:01 -0700 you wrote:
Teach libarena's BPF atomic primitives to use compiler builtins for load-acquire and store-release when Clang advertises
__BPF_FEATURE_LOAD_ACQ_STORE_REL. Older compilers continue to use the existing barrier-based fallback.Notably, as BPF programs begin running on arm64, it is better to use the more appropriate variants since we can no longer rely on x86 TSO ordering.
[...]
Here is the summary with links:
- [bpf-next,v3] libarena: Use compiler load-acquire/store-release in bpf_atomic.h
https://git.kernel.org/bpf/bpf-next/c/87267b894598
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html