Introduce arena library and runtime
Subsystems (4)
Emil Tsalapatis
reviewerv5 cover Introduce arena library and runtime
Add a new subdirectory to tools/testing/selftests/bpf called libarena, along with programs useful for writing arena-based BPF code. This patchset adds the following:
1) libarena, a subdirectory where arena BPF code that is generally useful to BPF arena programs can be easily added and tested.
2) An ASAN runtime for BPF arena programs. BPF arenas allow for accessing memory after it has been freed or if it is out of bounds, making it more difficult to triage bugs combined to regular BPF. Use LLVM's recently added support for address-space based sanitization to selectively sanitize just the arena accesses.
3) A buddy memory allocator that can be reused by BPF programs to handle memory allocation/deletion. The allocator uses the ASAN runtime to add address sanitization if requested.
The patch includes testing for the new allocators and ASAN features that can be built from the top directory using "make libarena_test" and "make libarena_test_asan". The generated binaries reside in libarena/. The patch also adds test-progs-based selftests to the codebase for the libarena code, so the new tests are run by ./test_progs.
The patchset has the following stucture:
1-2: Addresses a verification failure triggered by SCALAR += PTR_TO_ARENA instructions.
3-4: Minor changes to selftest headers to prepare for the introduction of libarena.
5-7: Add the libarena directory and testing scaffolding, and introduce the ASAN runtime.
8-9: Add the new buddy memory allocator along with self-contained and prog-tests-based selftests.
HISTORY
=======
v4->v5 (https://lore.kernel.org/bpf/[email protected])
Omitting various nits and fixups.
- Properly adjust subreg_def for scalar += ptr_to_arena calls (Sashiko)
- Remove extraneous definition from prog_tests/arena_spin_lock.c (Song)
- Trim extraneous comments from ASAN and buddy (Alexei)
- Remove asan_dummy call and replace with function pointer array (Alexei)
- Remove usersapi.h header and merge it into common.h (Alexei)
- Replace ASAN macros with function calls (Alexei)
- Embed buddy lock into the struct and move the buddy allocator to __arena_global (Alexei)
- Add commenting for buddy allocator constants (Alexei)
- Add default buddy allocator directly in common.bpf.c, so that the user does not need to define it.
- Expand test harnesses to dynamically find individual selftests. Now the selftests also reports each test individually (e.g., 5 entries for the buddy allocator instead of 1). This brings them to par with the rest of the test_progs.
v3->v4 (https://lore.kernel.org/bpf/[email protected])
- Add Acks by Song to patches 1-4.
- Expand the verifier's handling of scalar/arena operations to include all 3-operand operations in Patch 1 (Alexei)
- Add additional tests for arena/arena (allowed) and arena/pointer (not allowed) operations in Patch 2
- Remove ASAN version of the library from default compilation since it requires LLVM 22 and up (CI)
- Rework buddy allocator locking for clarity and add comments
- Fix From: email to be consistent with SOB
- Address (most) Sashiko comments
v2->v3 (https://lore.kernel.org/bpf/[email protected])
Nonexhaustive due to significant patch rework.
- Do not duplicate WRITE_ONCE macro (Mykyta, Kumar)
- Add SPDX headers (Alexei)
- Remove bump/stack allocators (Alexei)
- Integrate testing with test_progs (Kumar)
- Add short description of ASAN algorithm at the top of the file (Alexei)
v1->v2 (https://lore.kernel.org/bpf/[email protected]/)
- Added missing format string argument (AI)
- Fix outdated selftests prog name check (AI)
- Fixed stack allocation check for segment creation (AI)
- Fix errors in non-ASAN bump allocator selftests (AI)
- Propagate error value from individual selftests in selftest.c
- Removed embedded metadata from bump allocator as it was needlessly complicating its behavior
Emil Tsalapatis (9):
bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition selftests/bpf: Add test for scalar/arena pointer addition selftests/bpf: Move bpf_arena_spin_lock.h to the top level selftests/bpf: Deduplicate WRITE_ONCE macro between headers selftests/bpf: Add basic libarena scaffolding selftests/bpf: Add arena ASAN runtime to libarena selftests/bpf: Add ASAN support for libarena selftests selftests/bpf: Add buddy allocator for libarena selftests/bpf: Add selftests for libarena buddy allocator
kernel/bpf/verifier.c | 25 +-
tools/testing/selftests/bpf/.gitignore | 2 +
tools/testing/selftests/bpf/Makefile | 28 +-
.../testing/selftests/bpf/bpf_arena_common.h | 4 -
.../bpf/{progs => }/bpf_arena_spin_lock.h | 4 +-
tools/testing/selftests/bpf/bpf_atomic.h | 4 -
.../testing/selftests/bpf/bpf_experimental.h | 3 +
tools/testing/selftests/bpf/libarena/Makefile | 77 ++
.../selftests/bpf/libarena/include/asan.h | 105 +++
.../selftests/bpf/libarena/include/buddy.h | 92 ++
.../selftests/bpf/libarena/include/common.h | 68 ++
.../bpf/libarena/include/selftest_helpers.h | 121 +++
.../bpf/libarena/selftests/selftest.c | 184 ++++
.../libarena/selftests/st_asan_buddy.bpf.c | 243 +++++
.../bpf/libarena/selftests/st_asan_common.h | 47 +
.../bpf/libarena/selftests/st_buddy.bpf.c | 211 +++++
.../selftests/bpf/libarena/src/asan.bpf.c | 550 +++++++++++
.../selftests/bpf/libarena/src/buddy.bpf.c | 879 ++++++++++++++++++
.../selftests/bpf/libarena/src/common.bpf.c | 71 ++
.../bpf/prog_tests/arena_spin_lock.c | 7 -
.../selftests/bpf/prog_tests/libarena.c | 61 ++
.../selftests/bpf/progs/arena_spin_lock.c | 2 +-
.../selftests/bpf/progs/verifier_arena.c | 85 ++
23 files changed, 2852 insertions(+), 21 deletions(-)
rename tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h (99%)
create mode 100644 tools/testing/selftests/bpf/libarena/Makefile
create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.cSignature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 1/9 bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
The compiler sometimes stores the result of a PTR_TO_ARENA + SCALAR addition into the scalar register rather than the pointer register. Handle this case by upgrading the destination scalar register to PTR_TO_ARENA, matching the existing handling when the destination is already PTR_TO_ARENA.
--- kernel/bpf/verifier.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-)
kernel/bpf/verifier.c +23 -2
@@ -16641,11 +16641,32 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,int err; dst_reg = ®s[insn->dst_reg]; - src_reg =NULL ; +if (BPF_SRC (insn->code) ==BPF_X ) + src_reg = ®s[insn->src_reg]; +else + src_reg =NULL ; -if (dst_reg->type ==PTR_TO_ARENA ) { +/* Case where at least one operand is an arena. */ +if (dst_reg->type ==PTR_TO_ARENA || (src_reg && src_reg->type ==PTR_TO_ARENA )) {struct bpf_insn_aux_data *aux =cur_aux (env); +if (dst_reg->type !=PTR_TO_ARENA ) { +/* Can't do arena arithmetic with non-scalars. */ +if (dst_reg->type !=SCALAR_VALUE ) { +verbose (env,"R%d %s R%d: Invalid operation between " +"bpf_reg_state types %s and %s\n" , + insn->dst_reg, + bpf_alu_string[opcode >>4 ], + insn->src_reg, +reg_type_str (env, dst_reg->type), +reg_type_str (env, src_reg->type)); +return -EACCES ; + } + + *dst_reg = *src_reg; + dst_reg->subreg_def = env->insn_idx +1 ; + } +if (BPF_CLASS (insn->code) ==BPF_ALU64 )/* * 32-bit operations zero upper bits automatically.
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH bpf-next v5 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
reg_type_str() returns a pointer to the shared buffer env->tmp_str_buf. Its own comment in log.c says:
Quoted 5 lines
kernel/bpf/verifier.c +0
-0@@ -16655,11 +16655,32 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,
[ ... ]
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16662
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+if (dst_reg->type !=PTR_TO_ARENA ) { +/* Can't do arena arithmetic with non-scalars. */ +if (dst_reg->type !=SCALAR_VALUE ) { +verbose (env,"R%d %s R%d: Invalid operation between " +"bpf_reg_state types %s and %s\n" , + insn->dst_reg, + bpf_alu_string[opcode >>4 ], + insn->src_reg, +reg_type_str (env, dst_reg->type), +reg_type_str (env, src_reg->type));
reg_type_str() returns a pointer to the shared buffer env->tmp_str_buf. Its own comment in log.c says:
Note that reg_type_str() can not appear more than once in a single verbose() statement.
Calling it twice in the same verbose() here means both %s arguments point to the same buffer, so the error message will display one of the two type names twice and the other not at all.
Would it work to split this into two verbose() calls, or save one result to a local buffer before printing?
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16668
Quoted 6 lines
kernel/bpf/verifier.c +6
-0+return -EACCES ; + } + + *dst_reg = *src_reg; + dst_reg->subreg_def = env->insn_idx +1 ; + }
[ ... ]
This commit fixes a bug where the verifier rejects valid BPF programs when the compiler places the result of a PTR_TO_ARENA + SCALAR addition into the scalar register. The bug was introduced by commit 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.") which added the original PTR_TO_ARENA handling in the verifier.
This was also noted during review of the v3 series on lore, and the
Fixes: tag was acknowledged but appears to be missing from the final version. Should this carry:
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/24254055723
Emil Tsalapatis
reviewerRe: [PATCH bpf-next v5 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
Quoted 25 lines · bot+bpf-ci
Quoted 5 lines
kernel/bpf/verifier.c +0
-0@@ -16655,11 +16655,32 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,[ ... ]
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16662
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+if (dst_reg->type !=PTR_TO_ARENA ) { +/* Can't do arena arithmetic with non-scalars. */ +if (dst_reg->type !=SCALAR_VALUE ) { +verbose (env,"R%d %s R%d: Invalid operation between " +"bpf_reg_state types %s and %s\n" , + insn->dst_reg, + bpf_alu_string[opcode >>4 ], + insn->src_reg, +reg_type_str (env, dst_reg->type), +reg_type_str (env, src_reg->type));
reg_type_str()returns a pointer to the shared bufferenv->tmp_str_buf. Its own comment inlog.csays:Note that
reg_type_str()can not appear more than once in a singleverbose()statement.
Quoted 7 lines
Calling it twice in the same
verbose()here means both %s arguments point to the same buffer, so the error message will display one of the two type names twice and the other not at all.Would it work to split this into two
verbose()calls, or save one result to a local buffer before printing?
Ack. We can just split this into two verbose() statements.
Quoted 15 lines
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16668
Quoted 6 lines
kernel/bpf/verifier.c +6
-0+return -EACCES ; + } + + *dst_reg = *src_reg; + dst_reg->subreg_def = env->insn_idx +1 ; + }[ ... ]
This commit fixes a bug where the verifier rejects valid BPF programs when the compiler places the result of a
PTR_TO_ARENA+ SCALAR addition into the scalar register. The bug was introduced by commit6082b6c328b5("bpf: Recognizeaddr_space_castinstruction in the verifier.") which added the originalPTR_TO_ARENAhandling in the verifier.
Quoted 6 lines
This was also noted during review of the v3 series on lore, and the
Fixes: tag was acknowledged but appears to be missing from the final version. Should this carry:Fixes 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.")
Not sure actually. The original patch logic was overly restrictive, but it's not like it was broken. This code is more of an expansion than a fix imo.
Quoted 6 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/24254055723
Emil Tsalapatis
reviewerPATCH v5 2/9 selftests/bpf: Add test for scalar/arena pointer addition
In reply to Emil TsalapatisAdd a selftest that ensures scalar += ptr_to_arena instructions are accepted by the verifier.
--- .../selftests/bpf/progs/verifier_arena.c | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+)
tools/testing/selftests/bpf/progs/verifier_arena.c +85 -0
@@ -477,4 +477,89 @@ int arena_kfuncs_under_bpf_lock(void *ctx)return 0 ; } + +#if defined(__BPF_FEATURE_ADDR_SPACE_CAST) + +/* +* Test that scalar += PTR_TO_ARENA correctly upgrades the +* destination register to a PTR_TO_ARENA. +*/ +SEC("syscall") +__success __retval(0) +int scalar_add_arena_ptr (void *ctx) +{ + int __arena *scalar, *arena_ptr; + + volatile char __arena *base = arena_base(&arena); + + asmvolatile ( +"%[arena_ptr] = 8192;" +"%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);" +"%[scalar] = 12;" +"%[scalar] += %[arena_ptr];" + : [scalar]"=r" (scalar), + [arena_ptr]"=&r" (arena_ptr) + :"r" (base) + : + ); +return 0 ; +} + +/* +* Test that PTR_TO_ARENA + PTR_TO_ARENA is allowed. +*/ +SEC("syscall") +__success __retval(0) +int arena_ptr_add_arena_ptr(void *ctx) +{ + int __arena *arena_ptr2, *arena_ptr1; + +/* Needed for the verifier to link the arena to the subprog. */ + volatile char __arena *base = arena_base(&arena); + + asmvolatile ( +"%[arena_ptr1] = 8192;" +"%[arena_ptr1] = addr_space_cast(%[arena_ptr1], 0x0, 0x1);" +"%[arena_ptr2] = 4096;" +"%[arena_ptr2] = addr_space_cast(%[arena_ptr2], 0x0, 0x1);" +"%[arena_ptr2] += %[arena_ptr1];" + : [arena_ptr2]"=r" (arena_ptr2), + [arena_ptr1]"=&r" (arena_ptr1) + :"r" (base) + : + ); +return 0 ; +} + +/* +* Test that PTR_TO_ARENA + non-arena pointer is rejected. +*/ +SEC("syscall") +__failure __msg("R4 += R3: Invalid operation between bpf_reg_state types") +int arena_ptr_add_to_non_arena_ptr(void *ctx) +{ + register int __arena *arena_ptr asm("r3"); + registervoid *dst asm("r4" ); + +/* Needed for the verifier to link the arena to the subprog. */ + volatile char __arena *base = arena_base(&arena); + + asmvolatile ( +"%[arena_ptr] = 8192;" +"%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);" +"%[dst] = %[ctx];" +"%[dst] += %[arena_ptr];" + : [arena_ptr]"=&r" (arena_ptr), + [dst]"=&r" (dst) + : [ctx]"r" (ctx),"r" (base) + : + ); + + (void )ctx; + +return 0 ; +} + +#endif + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 3/9 selftests/bpf: Move bpf_arena_spin_lock.h to the top level
In reply to Emil TsalapatisThe bpf_arena_spin_lock.h header is useful for all programs and not just the selftests. Move it to the top level of the BPF selftests to make it more readily accessible.
---
.../selftests/bpf/{progs => }/bpf_arena_spin_lock.h | 4 ++--
tools/testing/selftests/bpf/prog_tests/arena_spin_lock.c | 7 -------
tools/testing/selftests/bpf/progs/arena_spin_lock.c | 2 +-
3 files changed, 3 insertions(+), 10 deletions(-)
rename tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h (99%)tools/testing/selftests/bpf/bpf_arena_spin_lock.h renamed from tools/testing/selftests/bpf/progs/bpf_arena_spin_lock.h +2 -2
@@ -107,7 +107,7 @@ struct arena_qnode {#define _Q_LOCKED_VAL (1U << _Q_LOCKED_OFFSET)#define _Q_PENDING_VAL (1U << _Q_PENDING_OFFSET) -struct arena_qnode __arena qnodes[_Q_MAX_CPUS][_Q_MAX_NODES]; +struct arena_qnode __weak __arena __hidden qnodes[_Q_MAX_CPUS][_Q_MAX_NODES];static inline u32 encode_tail (int cpu,int idx) {
@@ -240,7 +240,7 @@ static __always_inline int arena_spin_trylock(arena_spinlock_t __arena *lock)return likely (atomic_try_cmpxchg_acquire (&lock->val, &val, _Q_LOCKED_VAL)); } -__noinline +__noinline __weak int arena_spin_lock_slowpath(arena_spinlock_t __arena __arg_arena *lock, u32 val) { struct arena_mcs_spinlock __arena *prev, *next, *node0, *node;
tools/testing/selftests/bpf/prog_tests/arena_spin_lock.c +0 -7
@@ -5,13 +5,6 @@#include <sys/sysinfo.h> struct __qspinlock {int val; }; -typedef struct __qspinlock arena_spinlock_t ; - -struct arena_qnode { -unsigned long next; -int count; -int locked; -};#include "arena_spin_lock.skel.h"
tools/testing/selftests/bpf/progs/arena_spin_lock.c +1 -1
@@ -4,7 +4,7 @@#include <bpf/bpf_tracing.h> #include <bpf/bpf_helpers.h> #include "bpf_misc.h" -#include "bpf_arena_spin_lock.h" +#include "../bpf_arena_spin_lock.h" struct { __uint(type, BPF_MAP_TYPE_ARENA);
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 4/9 selftests/bpf: Deduplicate WRITE_ONCE macro between headers
In reply to Emil TsalapatisThe WRITE_ONCE macro is identically defined both in bpf_atomic.h and in bpf_arena_common.h. Deduplicate the definitions by moving the macro to the commonly used bpf_experimental.h header.
--- tools/testing/selftests/bpf/bpf_arena_common.h | 4 ---- tools/testing/selftests/bpf/bpf_atomic.h | 4 ---- tools/testing/selftests/bpf/bpf_experimental.h | 3 +++ 3 files changed, 3 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/bpf_arena_common.h +0 -4
@@ -2,10 +2,6 @@/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ #pragma once -#ifndef WRITE_ONCE -#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *) &(x)) = (val)) -#endif -#ifndef NUMA_NO_NODE #define NUMA_NO_NODE (-1)#endif
tools/testing/selftests/bpf/bpf_atomic.h +0 -4
@@ -40,10 +40,6 @@ extern bool CONFIG_X86_64 __kconfig __weak;/* No-op for BPF */ #define cpu_relax () ({}) -#define READ_ONCE (x) (*(volatile typeof(x) *)&(x)) - -#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *)&(x)) = (val)) -#define cmpxchg (p, old, new) __sync_val_compare_and_swap((p), old, new)#define try_cmpxchg (p, pold, new) \
tools/testing/selftests/bpf/bpf_experimental.h +3 -0
@@ -11,6 +11,9 @@/* Convenience macro to wrap over bpf_obj_new */ #define bpf_obj_new (type) ((type *)bpf_obj_new(bpf_core_type_id_local(type))) +#define READ_ONCE (x) (*(volatile typeof(x) *)&(x)) +#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *)&(x)) = (val)) +/* Convenience macro to wrap over bpf_percpu_obj_new */ #define bpf_percpu_obj_new (type) ((type __percpu_kptr *)bpf_percpu_obj_new(bpf_core_type_id_local(type)))
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 5/9 selftests/bpf: Add basic libarena scaffolding
In reply to Emil TsalapatisAdd initial code for an arena-based BPF library. The current commit introduces a test runner and Makefile for the library. Library code can be added just by including the source file in the library's src/ subdirectory. Future commits will introduce the library code itself.
The current commit also includes a standalone test runner. This is to keep libarena self-contained and testable even when copied out of the kernel tree. Subsequent commits add integration with test_progs.
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 15 ++ tools/testing/selftests/bpf/libarena/Makefile | 50 ++++++ .../selftests/bpf/libarena/include/common.h | 67 +++++++++ .../bpf/libarena/include/selftest_helpers.h | 86 +++++++++++ .../bpf/libarena/selftests/selftest.c | 142 ++++++++++++++++++ .../selftests/bpf/libarena/src/common.bpf.c | 59 ++++++++ 7 files changed, 420 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/Makefile create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -49,3 +49,4 @@ verification_cert.h *.BTF.base usdt_1 usdt_2 +libarena/test_libarena
tools/testing/selftests/bpf/Makefile +15 -0
@@ -153,6 +153,7 @@ override define CLEAN $(Q)$(RM) -r $(TEST_KMODS) $(Q)$(RM) -r $(EXTRA_CLEAN) $(Q)$(MAKE) -C test_kmods clean + $(Q)$(MAKE) -C libarena clean $(Q)$(MAKE) docs-clean endef
@@ -739,6 +740,17 @@ $(VERIFY_SIG_HDR): $(VERIFICATION_CERT) echo "};"; \ echo "unsigned int test_progs_verification_cert_len = $$(wc -c < $<);") > $@ +LIBARENA_MAKE_ARGS = \ + BPFTOOL="$(BPFTOOL)" \ + INCLUDE_DIR="$(HOST_INCLUDE_DIR)" \ + LIBBPF_INCLUDE="$(HOST_INCLUDE_DIR)" \ + BPFOBJ="$(BPFOBJ)" \ + LDLIBS="$(LDLIBS) -lzstd" \ + CLANG="$(CLANG)" \ + BPF_CFLAGS="$(BPF_CFLAGS) $(CLANG_CFLAGS)" \ + BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \ + Q="$(Q)" + # Define test_progs test runner. TRUNNER_TESTS_DIR := prog_tests TRUNNER_BPF_PROGS_DIR := progs
@@ -931,3 +943,6 @@ override define INSTALL_RULE rsync -a $(OUTPUT)/$$DIR/*.bpf.o $(INSTALL_PATH)/$$DIR;\ done endef + +test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
tools/testing/selftests/bpf/libarena/Makefile new file +50 -0
50 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/include/common.h new file +67 -0
67 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/include/selftest_helpers.h new file +86 -0
86 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/selftests/selftest.c new file +142 -0
142 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/src/common.bpf.c new file +59 -0
59 lines not on this page · raw message
Signature · 1 line
2.53.0
Song Liu
reviewerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
On Fri, Apr 10, 2026 at 9:31 AM Emil Tsalapatis <[email protected]> wrote:
[...]
Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +0
-0
tools/testing/selftests/bpf/libarena/include/common.h line 9
Quoted 9 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +9
-0@@ -0,0 +1,67 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#ifdef __BPF__ + +#include <vmlinux.h> + +#include "bpf_experimental.h"
bpf_experimental.h is not needed here.
tools/testing/selftests/bpf/libarena/include/common.h line 11
Quoted 2 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +2
-0+#include "bpf_arena_common.h" +#include "bpf_arena_spin_lock.h"
These two headers are indeed needed. However, what's the plan with them? They are not part of libarena at the moment. How should we include them in the future?
Thanks,
Song
Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +6
-0+#include <asm-generic/errno.h> + +#ifndef __BPF_FEATURE_ADDR_SPACE_CAST +#error "Arena allocators require bpf_addr_space_cast feature" +#endif +
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
Quoted 21 lines · Song Liu
On Fri, Apr 10, 2026 at 9:31 AM Emil Tsalapatis <[email protected]> wrote:
[...]Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +0
-0tools/testing/selftests/bpf/libarena/include/common.h line 9
Quoted 9 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +9
-0@@ -0,0 +1,67 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#ifdef __BPF__ + +#include <vmlinux.h> + +#include "bpf_experimental.h"
bpf_experimental.his not needed here.
Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h line 11
Quoted 2 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +2
-0+#include "bpf_arena_common.h" +#include "bpf_arena_spin_lock.h" These two headers are indeed needed. However, what's the plan with them? They are not part of libarena at the moment. How should we include them in the future?
I would propose making them part of libarena, and adding libarena's header directory to -I for all progs in selftest to simplify inclusion.
Quoted 11 lines
Thanks,
SongQuoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +6
-0+#include <asm-generic/errno.h> + +#ifndef __BPF_FEATURE_ADDR_SPACE_CAST +#error "Arena allocators require bpf_addr_space_cast feature" +#endif +[...]
Emil Tsalapatis
reviewerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
Quoted 29 lines · Kumar Kartikeya Dwivedi
Quoted 21 lines · Song Liu
On Fri, Apr 10, 2026 at 9:31 AM Emil Tsalapatis <[email protected]> wrote:
[...]Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +0
-0tools/testing/selftests/bpf/libarena/include/common.h line 9
Quoted 9 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +9
-0@@ -0,0 +1,67 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#ifdef __BPF__ + +#include <vmlinux.h> + +#include "bpf_experimental.h"
bpf_experimental.his not needed here.Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h line 11
Quoted 2 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +2
-0+#include "bpf_arena_common.h" +#include "bpf_arena_spin_lock.h" These two headers are indeed needed. However, what's the plan with them? They are not part of libarena at the moment. How should we include them in the future?
Quoted 4 lines
I would propose making them part of libarena, and adding libarena's header directory to -I for all progs in selftest to simplify inclusion.
+1 on this. We should have all arena related functionality in a single directory.
Quoted 11 lines
Quoted 11 lines
Thanks,
SongQuoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +6
-0+#include <asm-generic/errno.h> + +#ifndef __BPF_FEATURE_ADDR_SPACE_CAST +#error "Arena allocators require bpf_addr_space_cast feature" +#endif +[...]
Song Liu
reviewerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
Quoted 31 lines · Emil Tsalapatis
Quoted 29 lines · Kumar Kartikeya Dwivedi
Quoted 27 lines · Song Liu
On Fri, Apr 10, 2026 at 9:31 AM Emil Tsalapatis <[email protected]> wrote:
[...]Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +0
-0tools/testing/selftests/bpf/libarena/include/common.h line 9
Quoted 9 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +9
-0@@ -0,0 +1,67 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#ifdef __BPF__ + +#include <vmlinux.h> + +#include "bpf_experimental.h"
bpf_experimental.his not needed here.tools/testing/selftests/bpf/libarena/include/common.h line 11
Quoted 2 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +2
-0+#include "bpf_arena_common.h" +#include "bpf_arena_spin_lock.h" These two headers are indeed needed. However, what's the plan with them? They are not part of libarena at the moment. How should we include them in the future?
Quoted 7 lines
Quoted 4 lines
I would propose making them part of libarena, and adding libarena's header directory to -I for all progs in selftest to simplify inclusion.
+1 on this. We should have all arena related functionality in a single directory.
There is also tools/sched_ext/include/scx/bpf_arena_common.h, which is similar to bpf_arena_common.h in selftests/bpf. We can have sched_ext include this version of bpf_arena_common.h. However, it seems weird to have sched_ext code include a header in selftests/bpf. Therefore, I think it makes more sense to move libarena to tools/lib/. WDYT?
Thanks,
Song
Re: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
Quoted 38 lines · Song Liu
Quoted 36 lines · Emil Tsalapatis
Quoted 33 lines · Kumar Kartikeya Dwivedi
Quoted 21 lines · Song Liu
On Fri, Apr 10, 2026 at 9:31 AM Emil Tsalapatis <[email protected]> wrote:
[...]Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +0
-0tools/testing/selftests/bpf/libarena/include/common.h line 9
Quoted 9 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +9
-0@@ -0,0 +1,67 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#ifdef __BPF__ + +#include <vmlinux.h> + +#include "bpf_experimental.h"
bpf_experimental.his not needed here.Quoted 6 lines
tools/testing/selftests/bpf/libarena/include/common.h line 11
Quoted 2 lines
tools/testing/selftests/bpf/libarena/include/common.h new file +2
-0+#include "bpf_arena_common.h" +#include "bpf_arena_spin_lock.h" These two headers are indeed needed. However, what's the plan with them? They are not part of libarena at the moment. How should we include them in the future?
I would propose making them part of libarena, and adding libarena's header directory to -I for all progs in selftest to simplify inclusion.
Quoted 4 lines
Quoted 2 lines
+1 on this. We should have all arena related functionality in a single directory.
There is also
tools/sched_ext/include/scx/bpf_arena_common.h, which
imo tools/sched_ext are bitrotting. I wouldn't be surprised if they're removed. It's not really a library or a tool.
Quoted 4 lines
is similar to
bpf_arena_common.hin selftests/bpf. We can havesched_extinclude this version ofbpf_arena_common.h. However, it seems weird to havesched_extcode include a header in selftests/bpf. Therefore, I think it makes more sense to move libarena totools/lib/. WDYT?
I wouldn't. In general tools/lib/ is not a place for generic libraries. libbpf is there because it's needed to build the kernel. libarena is in selftest/bpf, because it's needed for selftests. The kernel itself doesn't need it.
I would remove some of the libs from tools/lib/.
Song Liu
reviewerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
On Fri, Apr 10, 2026 at 3:50 PM Alexei Starovoitov <[email protected]> wrote:
[...]
Quoted 5 lines
Quoted 2 lines
There is also
tools/sched_ext/include/scx/bpf_arena_common.h, whichimo
tools/sched_extare bitrotting. I wouldn't be surprised if they're removed. It's not really a library or a tool.
Hmm.. In this case, we can probably ignore tools/sched_ext. I guess we can also ignore selftests/sched_ext? It is not getting a lot of active development either.
Quoted 10 lines
Quoted 4 lines
is similar to
bpf_arena_common.hin selftests/bpf. We can havesched_extinclude this version ofbpf_arena_common.h. However, it seems weird to havesched_extcode include a header in selftests/bpf. Therefore, I think it makes more sense to move libarena totools/lib/. WDYT?I wouldn't. In general
tools/lib/is not a place for generic libraries. libbpf is there because it's needed to build the kernel. libarena is in selftest/bpf, because it's needed for selftests. The kernel itself doesn't need it.
I would remove some of the libs fromtools/lib/.
Assuming both tools/sched_ext and selftest/sched_ext are bitrotting, selftests/bpf is likely the only user of libarena in the longer term. In this case, I agree it makes sense to keep libarena inside selftests/bpf.
Thanks,
Song
Emil Tsalapatis
reviewerRe: [PATCH bpf-next v5 5/9] selftests/bpf: Add basic libarena scaffolding
Quoted 24 lines · Song Liu
On Fri, Apr 10, 2026 at 3:50 PM Alexei Starovoitov <[email protected]> wrote:
[...]Quoted 5 lines
Quoted 2 lines
There is also
tools/sched_ext/include/scx/bpf_arena_common.h, whichimo
tools/sched_extare bitrotting. I wouldn't be surprised if they're removed. It's not really a library or a tool.Hmm.. In this case, we can probably ignore
tools/sched_ext. I guess we can also ignore selftests/sched_ext? It is not getting a lot of active development either.Quoted 10 lines
Quoted 4 lines
is similar to
bpf_arena_common.hin selftests/bpf. We can havesched_extinclude this version ofbpf_arena_common.h. However, it seems weird to havesched_extcode include a header in selftests/bpf. Therefore, I think it makes more sense to move libarena totools/lib/. WDYT?I wouldn't. In general
tools/lib/is not a place for generic libraries. libbpf is there because it's needed to build the kernel. libarena is in selftest/bpf, because it's needed for selftests. The kernel itself doesn't need it.
I would remove some of the libs fromtools/lib/.
Quoted 4 lines
Assuming both
tools/sched_extand selftest/sched_ext are bitrotting, selftests/bpf is likely the only user of libarena in the longer term. In this case, I agree it makes sense to keep libarena inside selftests/bpf.
The selftest/sched_ext selftests are being actively tested on the CI, but for libarena specifically they don't use arenas.
I think keeping stale BPF headers in tools/sched_ext is not an issue - in fact we can remove the header from there right now. The only user of bpf_arena_common.h in tools/sched_ext is scx_sdt.c and is mostly there to demonstrate how to use arenas in the first place. Most of the contents of the header are for compatibility with older Clang and pahole versions, but we explicitly don't support those and can remove them. The scheduler is also not actively being expanded either, so we don't need to worry about importing new features from selftests/bpf.
Quoted 2 lines
Thanks,
Song
Emil Tsalapatis
reviewerPATCH v5 6/9 selftests/bpf: Add arena ASAN runtime to libarena
In reply to Emil TsalapatisAdd an address sanitizer (ASAN) runtime to the arena library. The ASAN runtime implements the functions injected into BPF binaries by LLVM sanitization when ASAN is enabled during compilation.
The runtime also includes functions called explicitly by memory allocation code to mark memory as poisoned/unpoisoned to ASAN. This code is a no-op when sanitization is turned off.
--- .../selftests/bpf/libarena/include/asan.h | 105 ++++ .../selftests/bpf/libarena/include/common.h | 1 + .../selftests/bpf/libarena/src/asan.bpf.c | 550 ++++++++++++++++++ 3 files changed, 656 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
tools/testing/selftests/bpf/libarena/include/asan.h new file +105 -0
@@ -0,0 +1,105 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +struct asan_init_args { +u64 arena_all_pages; +u64 arena_globals_pages; +}; + +int asan_init (struct asan_init_args *args); + +extern volatile u64 __asan_shadow_memory_dynamic_address; +extern volatile u32 asan_reported; +extern volatile bool asan_inited; +extern volatile bool asan_report_once; +extern volatile bool asan_emit_stack; + +#ifdef __BPF__ + +#define ASAN_SHADOW_SHIFT 3 +#define ASAN_SHADOW_SCALE (1ULL << ASAN_SHADOW_SHIFT) +#define ASAN_GRANULE_MASK ((1ULL << ASAN_SHADOW_SHIFT) - 1) +#define ASAN_GRANULE (addr) ((s8)((u32)(u64)((addr)) & ASAN_GRANULE_MASK)) + +#define __noasan __attribute__((no_sanitize("address"))) + +#ifdef BPF_ARENA_ASAN + +typedef s8 __arena s8a; + +static inline +s8a *mem_to_shadow(void __arena __arg_arena *addr) +{ +return (s8a *)(((u32 )(u64 )addr >>ASAN_SHADOW_SHIFT ) + + __asan_shadow_memory_dynamic_address); +} + +__weak __noasan +bool asan_ready(void) +{ +return __asan_shadow_memory_dynamic_address; +} + +int asan_poison(void __arena *addr, s8 val, size_t size); +int asan_unpoison(void __arena *addr, size_t size); +bool asan_shadow_set(void __arena *addr); + +/* +* Dummy calls to ensure the ASAN runtime's BTF information is present +* in every object file when compiling the runtime and local BPF code +* separately. The runtime calls are injected into the LLVM IR file +*/ +#define DECLARE_ASAN_LOAD_STORE_SIZE (size) \ + void __asan_store##size(void *addr); \ + void __asan_store##size##_noabort(void *addr); \ + void __asan_load##size(void *addr); \ + void __asan_load##size##_noabort(void *addr); \ + void __asan_report_store##size(void *addr); \ + void __asan_report_store##size##_noabort(void *addr); \ + void __asan_report_load##size(void *addr); \ + void __asan_report_load##size##_noabort(void *addr); + +DECLARE_ASAN_LOAD_STORE_SIZE (1 ); +DECLARE_ASAN_LOAD_STORE_SIZE (2 ); +DECLARE_ASAN_LOAD_STORE_SIZE (4 ); +DECLARE_ASAN_LOAD_STORE_SIZE (8 ); + +/* +* Force LLVM to emit BTF information for the stubs, +* because the ASAN pass in LLVM by itself doesn't. +*/ +#define ASAN_LOAD_STORE_SIZE (size) \ + __asan_store##size, \ + __asan_store##size##_noabort, \ + __asan_load##size, \ + __asan_load##size##_noabort, \ + __asan_report_store##size, \ + __asan_report_store##size##_noabort, \ + __asan_report_load##size, \ + __asan_report_load##size##_noabort + +/* +* Force LLVM to generate BTF information for the ASAN function +* declarations without invoking them. The used attribute prevents +* the function pointer table from being optimized out by the +* compiler. +*/ +__attribute__((used)) +static void (*__asan_btf_anchors[])(void *) = { +ASAN_LOAD_STORE_SIZE (1 ), +ASAN_LOAD_STORE_SIZE (2 ), +ASAN_LOAD_STORE_SIZE (4 ), +ASAN_LOAD_STORE_SIZE (8 ) +}; + +#else /* BPF_ARENA_ASAN */ + +static inline int asan_poison(void __arena *addr, s8 val, size_t size) { return 0; } +static inline int asan_unpoison(void __arena *addr, size_t size) { return 0; } +static inline bool asan_shadow_set(void __arena *addr) { return 0; } +__weak bool asan_ready (void ) {return true; } + +#endif /* BPF_ARENA_ASAN */ + +#endif /* __BPF__ */
tools/testing/selftests/bpf/libarena/include/common.h +1 -0
@@ -39,6 +39,7 @@ struct { } arena __weak SEC(".maps");extern const volatile u32 zero; +extern volatile u64 asan_violated;int arena_fls (__u64 word);
tools/testing/selftests/bpf/libarena/src/asan.bpf.c new file +550 -0
550 lines not on this page · raw message
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 7/9 selftests/bpf: Add ASAN support for libarena selftests
In reply to Emil TsalapatisExpand the arena library selftest infrastructure to support address sanitization. Add the compiler flags necessary to compile the library under ASAN when supported.
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 13 ++++- tools/testing/selftests/bpf/libarena/Makefile | 31 +++++++++++- .../bpf/libarena/include/selftest_helpers.h | 28 +++++++++++ .../bpf/libarena/selftests/selftest.c | 38 ++++++++++++++- .../bpf/libarena/selftests/st_asan_common.h | 47 +++++++++++++++++++ 6 files changed, 153 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -50,3 +50,4 @@ verification_cert.h usdt_1 usdt_2 libarena/test_libarena +libarena/test_libarena_asan
tools/testing/selftests/bpf/Makefile +12 -1
@@ -751,6 +751,13 @@ LIBARENA_MAKE_ARGS = \ BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \ Q="$(Q)" +# libarena_asan.skel.h currently requires LLVM 22. For now, +# only integrate the non-ASAN version with test_progs. +LIBARENA_SKEL := libarena/libarena.skel.h + +$(LIBARENA_SKEL): $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena libarena.skel.h $(LIBARENA_MAKE_ARGS) + # Define test_progs test runner. TRUNNER_TESTS_DIR := prog_tests TRUNNER_BPF_PROGS_DIR := progs
@@ -774,7 +781,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \ flow_dissector_load.h \ ip_check_defrag_frags.h \ bpftool_helpers.c \ - usdt_1.c usdt_2.c + usdt_1.c usdt_2.c \ + $(LIBARENA_SKEL) TRUNNER_LIB_SOURCES := find_bit.c TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \ $(OUTPUT)/liburandom_read.so \
@@ -946,3 +954,6 @@ endef test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS) + +test_libarena_asan: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
tools/testing/selftests/bpf/libarena/Makefile +29 -2
@@ -8,6 +8,7 @@ LIBARENA=$(abspath .) LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c) LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o)) +LIBARENA_OBJECTS_ASAN = $(notdir $(LIBARENA_SOURCES:.bpf.c=_asan.bpf.o)) INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/.. ifneq ($(INCLUDE_DIR),)
@@ -17,6 +18,13 @@ ifneq ($(LIBBPF_INCLUDE),) INCLUDES += -I$(LIBBPF_INCLUDE) endif +ASAN_FLAGS = -fsanitize=kernel-address -fno-stack-protector -fno-builtin +ASAN_FLAGS += -mllvm -asan-instrument-address-spaces=1 -mllvm -asan-shadow-addr-space=1 +ASAN_FLAGS += -mllvm -asan-use-stack-safety=0 -mllvm -asan-stack=0 +ASAN_FLAGS += -mllvm -asan-kernel=1 +ASAN_FLAGS += -mllvm -asan-constructor-kind=none +ASAN_FLAGS += -mllvm -asan-destructor-kind=none + # ENABLE_ATOMICS_TESTS required because we use arena spinlocks override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
@@ -28,23 +36,42 @@ CFLAGS += $(INCLUDES) vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests -all: test_libarena +all: test_libarena test_libarena_asan + +test_libarena_asan: selftest.c $(BPFOBJ) libarena_asan.skel.h + $(call msg,BINARY,libarena,$@) + $(Q)$(CLANG) $(CFLAGS) -DBPF_ARENA_ASAN $< $(BPFOBJ) $(LDLIBS) -o $@ test_libarena: selftest.c $(BPFOBJ) libarena.skel.h $(call msg,BINARY,libarena,$@) $(Q)$(CLANG) $(CFLAGS) $< $(BPFOBJ) $(LDLIBS) -o $@ +skeletons: libarena.skel.h libarena_asan.skel.h +.PHONY: skeletons + +libarena_asan.skel.h: main_asan.bpf.o + $(call msg,GEN-SKEL,libarena,$@) + $(Q)$(BPFTOOL) gen skeleton $< name "libarena_asan" > $@ + libarena.skel.h: main.bpf.o $(call msg,GEN-SKEL,libarena,$@) $(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@ +main_asan.bpf.o: $(LIBARENA_OBJECTS_ASAN) + $(call msg,GEN-OBJ,libarena,$@) + $(Q)$(BPFTOOL) gen object $@ $^ + main.bpf.o: $(LIBARENA_OBJECTS) $(call msg,GEN-OBJ,libarena,$@) $(Q)$(BPFTOOL) gen object $@ $^ +%_asan.bpf.o: %.bpf.c + $(call msg,CLNG-BPF,libarena,$@) + $(Q)$(CLANG) $(BPF_CFLAGS) $(ASAN_FLAGS) -DBPF_ARENA_ASAN $(BPF_TARGET_ENDIAN) -c $< -o $@ + %.bpf.o: %.bpf.c $(call msg,CLNG-BPF,libarena,$@) $(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@ clean: - $(Q)rm -f *.skel.h *.bpf.o test_libarena + $(Q)rm -f *.skel.h *.bpf.o *.linked*.o test_libarena test_libarena_asan
tools/testing/selftests/bpf/libarena/include/selftest_helpers.h +28 -0
@@ -84,3 +84,31 @@ static inline int libarena_get_globals_pages(int arena_get_base_fd,free (vec);return 0 ; } + +static inline int libarena_asan_init (int arena_get_base_fd, +int asan_init_fd, +size_t arena_all_pages) +{ +LIBBPF_OPTS (bpf_test_run_opts, opts); +struct asan_init_args args; +u64 globals_pages; +int ret; + + ret =libarena_get_globals_pages (arena_get_base_fd, + arena_all_pages, &globals_pages); +if (ret) +return ret; + + args = (struct asan_init_args ){ + .arena_all_pages = arena_all_pages, + .arena_globals_pages = globals_pages, + }; + + opts.ctx_in = &args; + opts.ctx_size_in =sizeof (args); + + ret =bpf_prog_test_run_opts (asan_init_fd, &opts); +if (ret) +return ret; +return opts.retval; +}
tools/testing/selftests/bpf/libarena/selftests/selftest.c +36 -2
@@ -16,8 +16,18 @@#include <sys/sysinfo.h> #include <common.h> +#include <asan.h> #include <selftest_helpers.h> +#ifdef BPF_ARENA_ASAN +#include "../libarena_asan.skel.h" +typedef struct libarena_asan selftest ; +#define selftest__open libarena_asan__open +#define selftest__open_and_load libarena_asan__open_and_load +#define selftest__load libarena_asan__load +#define selftest__attach libarena_asan__attach +#define selftest__destroy libarena_asan__destroy +#else #include "../libarena.skel.h" typedef struct libarena selftest ;#define selftest__open libarena__open
@@ -25,6 +35,7 @@ typedef struct libarena selftest;#define selftest__load libarena__load#define selftest__attach libarena__attach#define selftest__destroy libarena__destroy +#endif static bool verbose;static int testno =1 ;
@@ -67,6 +78,26 @@ static int libbpf_print_fn(enum libbpf_print_level level,return vfprintf (stderr, format, args); } +static int init_arena (selftest *skel) +{ +int ret; + + ret =libarena_run_prog (bpf_program__fd (skel->progs.arena_alloc_reserve)); +if (ret) +return ret; + +#ifdef BPF_ARENA_ASAN + ret =libarena_asan_init ( +bpf_program__fd (skel->progs.arena_get_base), +bpf_program__fd (skel->progs.asan_init), + (1ULL <<32 ) /sysconf (_SC_PAGESIZE)); +if (ret) +return ret; +#endif + +return 0 ; +} +static int run_test (selftest *skel,struct bpf_program *prog) {int prog_fd;
@@ -82,10 +113,13 @@ static void banner(const char *progpath) {char *name =basename (progpath); +bool is_asan; +/* Check if our BPF programs are ASAN-capable using strstr on the prog name. */ printf ("%s\n" , name); + is_asan =strstr (name,"_asan" ); -printf ("=== %s ===\n" ,"libarena selftests" ); +printf ("=== %s %s===\n" ,"libarena selftests" , is_asan ?"(asan) " :"" ); }int main (int argc,char *argv[])
@@ -116,7 +150,7 @@ int main(int argc, char *argv[])return 1 ; } - ret =libarena_run_prog (bpf_program__fd (skel->progs.arena_alloc_reserve)); + ret =init_arena (skel);if (ret) {fprintf (stderr,"Failed to initialize arena: %d\n" , ret);selftest__destroy (skel);
tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 8/9 selftests/bpf: Add buddy allocator for libarena
In reply to Emil TsalapatisAdd a byte-oriented buddy allocator for libarena. The buddy allocator provides an alloc/free interface for small arena allocations ranging from 16 bytes to 512 KiB. Lower allocations values are rounded up to 16 bytes. The buddy allocator does not handle larger allocations that can instead use the existing bpf_arena_{alloc, free}_pages() kfunc.
--- .../selftests/bpf/libarena/include/buddy.h | 92 ++ .../selftests/bpf/libarena/src/buddy.bpf.c | 879 ++++++++++++++++++ .../selftests/bpf/libarena/src/common.bpf.c | 12 + 3 files changed, 983 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
tools/testing/selftests/bpf/libarena/include/buddy.h new file +92 -0
@@ -0,0 +1,92 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +struct buddy_chunk ; +typedef struct buddy_chunk __arena buddy_chunk_t; + +struct buddy_header ; +typedef struct buddy_header __arena buddy_header_t; + +enum buddy_consts { +/* +* Minimum allocation is 1 << BUDDY_MIN_ALLOC_SHIFT. +* Larger sizes increase internal fragmentation, but smaller +* sizes increase the space overhead of the block metadata. +*/ +BUDDY_MIN_ALLOC_SHIFT =4 , +BUDDY_MIN_ALLOC_BYTES =1 <<BUDDY_MIN_ALLOC_SHIFT , + +/* +* How many orders the buddy allocator can serve. Minimum block +* size is 1 << BUDDY_MIN_ALLOC_SHIFT, maximum block size is +* 1 << (BUDDY_MIN_ALLOC_SHIFT + BUDDY_CHUNK_NUM_ORDERS - 1): +* Each block has size 1 << BUDDY_MIN_ALLOC_SHIFT, and the +* allocation orders are in [0, BUDDY_CHUNK_NUM_ORDERS). +* We keep two blocks of the maximum size to retain the +* property in the code that all blocks have a buddy. +* values increase the maximum allocation size, but +* also the size of the metadata for each block. +*/ +BUDDY_CHUNK_NUM_ORDERS =1 <<4 , +BUDDY_CHUNK_BYTES =BUDDY_MIN_ALLOC_BYTES << (BUDDY_CHUNK_NUM_ORDERS ), + +/* Offset off the buddy header within a free block, see buddy.bpf.c for details */ +BUDDY_HEADER_OFF =8 , + +/* The maximum number of blocks a chunk may have to track. */ +BUDDY_CHUNK_ITEMS =1 << (BUDDY_CHUNK_NUM_ORDERS ), +BUDDY_CHUNK_OFFSET_MASK =BUDDY_CHUNK_BYTES -1 , + +/* +* Alignment for chunk allocations based on bpf_arena_alloc_pages. +* The arena allocation kfunc does not ahave an alignment argument, +* but that is required all the block calculations in the chunk to +* work. +*/ +BUDDY_VADDR_OFFSET =BUDDY_CHUNK_BYTES , + +/* Total arena virtual address space the allocator can consume. */ +BUDDY_VADDR_SIZE =BUDDY_CHUNK_BYTES <<10 +}; + +struct buddy_header { +u32 prev_index;/* "Pointer" to the previous available allocation of the same size. */ +u32 next_index;/* Same for the next allocation. */ +}; + +/* +* We bring memory into the allocator 1 MiB at a time. +*/ +struct buddy_chunk { +/* The order of the current allocation for a item. 4 bits per order. */ +u8 orders[BUDDY_CHUNK_ITEMS /2 ]; +/* +* Bit to denote whether chunk is allocated. Size of the allocated/free +* chunk found from the orders array. +*/ +u8 allocated[BUDDY_CHUNK_ITEMS /8 ]; +/* Freelists for O(1) allocation. */ +u64 freelists[BUDDY_CHUNK_NUM_ORDERS ]; +buddy_chunk_t *next; +}; + +struct buddy { +buddy_chunk_t *first_chunk;/* Pointer to the chunk linked list. */ +arena_spinlock_t lock;/* Allocator lock */ +u64 vaddr;/* Allocation into reserved vaddr */ +}; + +typedef struct buddy __arena buddy_t; + +#ifdef __BPF__ + +int buddy_init (buddy_t *buddy); +int buddy_destroy (buddy_t *buddy); +int buddy_free_internal (buddy_t *buddy,u64 free); +#define buddy_free (buddy, ptr) buddy_free_internal((buddy), (u64)(ptr)) +u64 buddy_alloc_internal (buddy_t *buddy,size_t size); +#define buddy_alloc (alloc, size) ((void __arena *)buddy_alloc_internal((alloc), (size))) + + +#endif /* __BPF__ */
tools/testing/selftests/bpf/libarena/src/buddy.bpf.c new file +879 -0
879 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/src/common.bpf.c +12 -0
@@ -1,9 +1,13 @@// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause /* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ #include <common.h> +#include <asan.h> +#include <buddy.h> const volatile u32 zero =0 ; +buddy_t buddy; +/* How many pages do we reserve at the beginning of the arena segment? */ #define RESERVE_ALLOC (8)
@@ -56,4 +60,12 @@ __weak int arena_alloc_reserve(void)return bpf_arena_reserve_pages (&arena,NULL ,RESERVE_ALLOC ); } +SEC("syscall") +__weak int arena_buddy_reset (void ) +{ +buddy_destroy (&buddy); + +return buddy_init (&buddy); +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v5 9/9 selftests/bpf: Add selftests for libarena buddy allocator
In reply to Emil TsalapatisIntroduce selftests for the buddy allocator without ASAN. Add the libarena selftests both to the libarena test runner and to test_progs, so that they are a) available when libarena is pulled as a standalone library, and b) exercised along with all other test programs in this directory.
Currently ASAN for libarena requires LLVM 22, which is not available in the CI. Delay testing the ASAN case directly through test_progs until that is the case. The ASAN version is available for testing on demand through the library's own harness.
--- .../bpf/libarena/include/selftest_helpers.h | 7 + .../bpf/libarena/selftests/selftest.c | 8 + .../libarena/selftests/st_asan_buddy.bpf.c | 243 ++++++++++++++++++ .../bpf/libarena/selftests/st_buddy.bpf.c | 211 +++++++++++++++ .../selftests/bpf/prog_tests/libarena.c | 61 +++++ 5 files changed, 530 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.c
tools/testing/selftests/bpf/libarena/include/selftest_helpers.h +7 -0
@@ -112,3 +112,10 @@ static inline int libarena_asan_init(int arena_get_base_fd,return ret;return opts.retval; } + +static inline bool libarena_must_setup_alloc (struct bpf_program *prog) +{ +const char *name =bpf_program__name (prog); + +return !strstr (name,"test_buddy" ); +}
tools/testing/selftests/bpf/libarena/selftests/selftest.c +8 -0
@@ -17,6 +17,7 @@#include <common.h> #include <asan.h> +#include <buddy.h> #include <selftest_helpers.h> #ifdef BPF_ARENA_ASAN
@@ -101,6 +102,13 @@ static int init_arena(selftest *skel)static int run_test (selftest *skel,struct bpf_program *prog) {int prog_fd; +int ret; + +if (libarena_must_setup_alloc (prog)) { + ret =libarena_run_prog (bpf_program__fd (skel->progs.arena_buddy_reset)); +if (ret) +return ret; + } prog_fd =bpf_program__fd (prog);if (prog_fd <0 )
tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c new file +243 -0
243 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c new file +211 -0
211 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/libarena.c new file +61 -0
61 lines not on this page · raw message
Signature · 1 line
2.53.0