Introduce arena library and runtime
Subsystems (4)
Emil Tsalapatis
reviewerv3 cover Introduce arena library and runtime
Add a new subdirectory to tools/testing/selftests/bpf called libarena, along with programs useful for writing arena-based BPF code. This patchset adds the following:
1) libarena, a subdirectory where arena BPF code that is generally useful to BPF arena programs can be easily added and tested.
2) An ASAN runtime for BPF arena programs. BPF arenas allow for accessing memory after it has been freed or if it is out of bounds, making it more difficult to triage bugs combined to regular BPF. Use LLVM's recently added support for address-space based sanitization to selectively sanitize just the arena accesses.
3) A buddy memory allocator that can be reused by BPF programs to handle memory allocation/deletion. The allocator uses the ASAN runtime to add address sanitization if requested.
The patch includes testing for the new allocators and ASAN features that can be built from the top directory using "make libarena_test" and "make libarena_test_asan". The generated binaries reside in libarena/. The patch also adds test-progs-based selftests to the codebase for the libarena code, so the new tests are run by ./test_progs.
The patchset has the following stucture:
1-2: Addresses a verification failure triggered by SCALAR += PTR_TO_ARENA instructions.
3-4: Minor changes to selftest headers to prepare for the introduction of libarena.
5-7: Add the libarena directory and testing scaffolding, and introduce the ASAN runtime.
8-9: Add the new buddy memory allocator along with self-contained and prog-tests-based selftests.
HISTORY
=======
v2->v3 (https://lore.kernel.org/bpf/[email protected])
Nonexhaustive due to significant patch rework.
- Do not duplicate WRITE_ONCE macro (Mykyta, Kumar)
- Add SPDX headers (Alexei)
- Remove bump/stack allocators (Alexei)
- Integrate testing with test_progs (Kumar)
- Add short description of ASAN algorithm at the top of the file (Alexei)
v1->v2 (https://lore.kernel.org/bpf/[email protected]/)
- Added missing format string argument (AI)
- Fix outdated selftests prog name check (AI)
- Fixed stack allocation check for segment creation (AI)
- Fix errors in non-ASAN bump allocator selftests (AI)
- Propagate error value from individual selftests in selftest.c
- Removed embedded metadata from bump allocator as it was needlessly complicating its behavior
Emil Tsalapatis (9):
bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition selftests/bpf: Add test for scalar/arena pointer addition selftests/bpf: Move bpf_arena_spin_lock.h to the top level selftests/bpf: Deduplicate WRITE_ONCE macro between headers selftests/bpf: Add basic libarena scaffolding selftests/bpf: Add arena ASAN runtime to libarena selftests/bpf: Add ASAN support for libarena selftests selftests/bpf: Add buddy allocator for libarena selftests/bpf: Add selftests for libarena buddy allocator
kernel/bpf/verifier.c | 18 +
tools/testing/selftests/bpf/.gitignore | 2 +
tools/testing/selftests/bpf/Makefile | 31 +-
.../testing/selftests/bpf/bpf_arena_common.h | 4 -
.../bpf/{progs => }/bpf_arena_spin_lock.h | 4 +-
tools/testing/selftests/bpf/bpf_atomic.h | 4 -
.../testing/selftests/bpf/bpf_experimental.h | 3 +
tools/testing/selftests/bpf/libarena/Makefile | 77 ++
.../selftests/bpf/libarena/include/asan.h | 124 +++
.../selftests/bpf/libarena/include/buddy.h | 64 ++
.../selftests/bpf/libarena/include/common.h | 50 ++
.../bpf/libarena/include/selftest_helpers.h | 109 +++
.../selftests/bpf/libarena/include/userapi.h | 27 +
.../bpf/libarena/selftests/selftest.c | 190 +++++
.../bpf/libarena/selftests/selftest.h | 12 +
.../libarena/selftests/st_asan_buddy.bpf.c | 251 ++++++
.../bpf/libarena/selftests/st_asan_common.h | 46 +
.../bpf/libarena/selftests/st_buddy.bpf.c | 229 +++++
.../selftests/bpf/libarena/src/asan.bpf.c | 524 ++++++++++++
.../selftests/bpf/libarena/src/buddy.bpf.c | 783 ++++++++++++++++++
.../selftests/bpf/libarena/src/common.bpf.c | 66 ++
.../selftests/bpf/prog_tests/libarena.c | 79 ++
.../selftests/bpf/progs/arena_spin_lock.c | 2 +-
.../selftests/bpf/progs/verifier_arena.c | 29 +
24 files changed, 2716 insertions(+), 12 deletions(-)
rename tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h (99%)
create mode 100644 tools/testing/selftests/bpf/libarena/Makefile
create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h
create mode 100644 tools/testing/selftests/bpf/libarena/include/userapi.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.cSignature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v3 1/9 bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
From: Emil Tsalapatis <[email protected]>
The compiler sometimes stores the result of a PTR_TO_ARENA + SCALAR addition into the scalar register rather than the pointer register. Handle this case by upgrading the destination scalar register to PTR_TO_ARENA, matching the existing handling when the destination is already PTR_TO_ARENA.
--- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
kernel/bpf/verifier.c +18 -0
@@ -16333,6 +16333,24 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, bpf_alu_string[opcode >> 4]);return -EACCES ; }else { +/* +* The compiler sometimes stores the result of +* PTR_TO_ARENA + SCALAR addition to the scalar +* register. Upgrade it to a PTR_TO_ARENA. +*/ +if (src_reg->type ==PTR_TO_ARENA && opcode ==BPF_ADD ) { +struct bpf_insn_aux_data *aux =cur_aux (env); + +__mark_reg_unknown (env, dst_reg); + dst_reg->type =PTR_TO_ARENA ; + dst_reg->subreg_def = env->insn_idx +1 ; + +if (BPF_CLASS (insn->code) ==BPF_ALU64 ) + aux->needs_zext = true; + +return 0 ; + } +/* scalar += pointer * This is legal, but we have to reverse our * src/dest handling in computing the range
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH bpf-next v3 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
This appears to fix a gap in the original PTR_TO_ARENA support added by commit 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier."), which handled arithmetic when the…
Quoted 14 lines
kernel/bpf/verifier.c +7
-0@@ -16354,6 +16354,24 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,return -EACCES ; }else { +/* +* The compiler sometimes stores the result of +* PTR_TO_ARENA + SCALAR addition to the scalar +* register. Upgrade it to a PTR_TO_ARENA. +*/ +if (src_reg->type ==PTR_TO_ARENA && opcode ==BPF_ADD ) { +struct bpf_insn_aux_data *aux =cur_aux (env);
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16372
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+ +__mark_reg_unknown (env, dst_reg); + dst_reg->type =PTR_TO_ARENA ; + dst_reg->subreg_def = env->insn_idx +1 ; + +if (BPF_CLASS (insn->code) ==BPF_ALU64 ) + aux->needs_zext = true; + +return 0 ; + }
This appears to fix a gap in the original PTR_TO_ARENA support added by commit 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier."), which handled arithmetic when the destination was already PTR_TO_ARENA but missed the commutative case where the scalar register is the destination.
Should this carry a Fixes: tag?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/23934176047
Emil Tsalapatis
reviewerRe: [PATCH bpf-next v3 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
Quoted 31 lines · bot+bpf-ci
Quoted 14 lines
kernel/bpf/verifier.c +7
-0@@ -16354,6 +16354,24 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env,return -EACCES ; }else { +/* +* The compiler sometimes stores the result of +* PTR_TO_ARENA + SCALAR addition to the scalar +* register. Upgrade it to a PTR_TO_ARENA. +*/ +if (src_reg->type ==PTR_TO_ARENA && opcode ==BPF_ADD ) { +struct bpf_insn_aux_data *aux =cur_aux (env);kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16372
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+ +__mark_reg_unknown (env, dst_reg); + dst_reg->type =PTR_TO_ARENA ; + dst_reg->subreg_def = env->insn_idx +1 ; + +if (BPF_CLASS (insn->code) ==BPF_ALU64 ) + aux->needs_zext = true; + +return 0 ; + }This appears to fix a gap in the original
PTR_TO_ARENAsupport added by commit6082b6c328b5("bpf: Recognizeaddr_space_castinstruction in the verifier."), which handled arithmetic when the destination was alreadyPTR_TO_ARENAbut missed the commutative case where the scalar register is the destination.
Quoted 4 lines
Should this carry a Fixes: tag?
Fixes 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.")
Ack, I will add the tag in subsequent versions.
Quoted 6 lines
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/23934176047
Re: [PATCH bpf-next v3 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
In reply to Emil TsalapatisQuoted 10 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
The compiler sometimes stores the result of a
PTR_TO_ARENA+ SCALAR addition into the scalar register rather than the pointer register. Handle this case by upgrading the destination scalar register toPTR_TO_ARENA, matching the existing handling when the destination is alreadyPTR_TO_ARENA.Signed-off-by Emil Tsalapatis <[email protected]>
Re: [PATCH bpf-next v3 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
In reply to Emil TsalapatisQuoted 19 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
The compiler sometimes stores the result of a
PTR_TO_ARENA+ SCALAR addition into the scalar register rather than the pointer register. Handle this case by upgrading the destination scalar register toPTR_TO_ARENA, matching the existing handling when the destination is alreadyPTR_TO_ARENA.Signed-off-by Emil Tsalapatis <[email protected]>--- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)kernel/bpf/verifier.c +0
-0
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16341
Quoted 9 lines
kernel/bpf/verifier.c +6
-0@@ -16333,6 +16333,24 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, bpf_alu_string[opcode >> 4]);return -EACCES ; }else { +/* +* The compiler sometimes stores the result of +* PTR_TO_ARENA + SCALAR addition to the scalar +* register. Upgrade it to a PTR_TO_ARENA. +*/ +if (src_reg->type ==PTR_TO_ARENA && opcode ==BPF_ADD ) {
This is too specific and doesn't fully address the issue. How about something like this:
kernel/bpf/verifier.c +0 -0
@@ -16572,11 +16572,16 @@ static int adjust_reg_min_max_vals(struct
bpf_verifier_env *env,
int err;
dst_reg = ®s[insn->dst_reg];- src_reg = NULL;
+ if (BPF_SRC(insn->code) == BPF_X)
+ src_reg = ®s[insn->src_reg];
+ else
+ src_reg = NULL;
- if (dst_reg->type == PTR_TO_ARENA) {
+ if (dst_reg->type == PTR_TO_ARENA || (src_reg && src_reg->type
== PTR_TO_ARENA)) {
struct bpf_insn_aux_data *aux = cur_aux(env);+ if (dst_reg->type != PTR_TO_ARENA)
+ *dst_reg = *src_reg;
if (BPF_CLASS(insn->code) == BPF_ALU64)
/*
* 32-bit operations zero upper bits automatically.
@@ -16592,7 +16597,6 @@ static int adjust_reg_min_max_vals(struct
bpf_verifier_env *env,
ptr_reg = dst_reg;
if (BPF_SRC(insn->code) == BPF_X) {
- src_reg = ®s[insn->src_reg];
if (src_reg->type != SCALAR_VALUE) {
if (dst_reg->type != SCALAR_VALUE) {
/* Combining two pointers by any ALU op yieldsEmil Tsalapatis
reviewerRe: [PATCH bpf-next v3 1/9] bpf: Upgrade scalar to PTR_TO_ARENA on arena pointer addition
Quoted 24 lines · Alexei Starovoitov
Quoted 23 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
The compiler sometimes stores the result of a
PTR_TO_ARENA+ SCALAR addition into the scalar register rather than the pointer register. Handle this case by upgrading the destination scalar register toPTR_TO_ARENA, matching the existing handling when the destination is alreadyPTR_TO_ARENA.Signed-off-by Emil Tsalapatis <[email protected]>--- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)kernel/bpf/verifier.c +1
-0@@ -16333,6 +16333,24 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, bpf_alu_string[opcode >> 4]);return -EACCES ; }else { +/*
Quoted 8 lines
kernel/bpf/verifier.c static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, · line 16341
Quoted 5 lines
kernel/bpf/verifier.c +5
-0+* The compiler sometimes stores the result of +* PTR_TO_ARENA + SCALAR addition to the scalar +* register. Upgrade it to a PTR_TO_ARENA. +*/ +if (src_reg->type ==PTR_TO_ARENA && opcode ==BPF_ADD ) {This is too specific and doesn't fully address the issue. How about something like this:
Makes sense, and keeps the SCALAR -> PTR_TO_ARENA in a single site. My initial rationale for doing this just for ADD was that it was the only two-operand operation that would make sense in this context, but that is irrelevant to whether the instructions are safe.
Quoted 35 lines
kernel/bpf/verifier.c +7
-2@@ -16572,11 +16572,16 @@ static int adjust_reg_min_max_vals(structbpf_verifier_env *env, int err; dst_reg = ®s[insn->dst_reg]; - src_reg =NULL ; +if (BPF_SRC (insn->code) ==BPF_X ) + src_reg = ®s[insn->src_reg]; +else + src_reg =NULL ; -if (dst_reg->type ==PTR_TO_ARENA ) { +if (dst_reg->type ==PTR_TO_ARENA || (src_reg && src_reg->type ==PTR_TO_ARENA )) {struct bpf_insn_aux_data *aux =cur_aux (env); +if (dst_reg->type !=PTR_TO_ARENA ) + *dst_reg = *src_reg;if (BPF_CLASS (insn->code) ==BPF_ALU64 )/*
* 32-bit operations zero upper bits automatically.kernel/bpf/verifier.c +0
-1@@ -16592,7 +16597,6 @@ static int adjust_reg_min_max_vals(structbpf_verifier_env *env, ptr_reg = dst_reg;if (BPF_SRC (insn->code) ==BPF_X ) { - src_reg = ®s[insn->src_reg];if (src_reg->type !=SCALAR_VALUE ) {if (dst_reg->type !=SCALAR_VALUE ) {/* Combining two pointers by any ALU op yields
Emil Tsalapatis
reviewerPATCH v3 2/9 selftests/bpf: Add test for scalar/arena pointer addition
In reply to Emil TsalapatisFrom: Emil Tsalapatis <[email protected]>
Add a selftest that ensures scalar += ptr_to_arena instructions are accepted by the verifier.
--- .../selftests/bpf/progs/verifier_arena.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+)
tools/testing/selftests/bpf/progs/verifier_arena.c +29 -0
@@ -477,4 +477,33 @@ int arena_kfuncs_under_bpf_lock(void *ctx)return 0 ; } +/* +* Test that scalar += PTR_TO_ARENA correctly upgrades the +* destination register to a PTR_TO_ARENA. +*/ +SEC("syscall") +__success __retval(0) +int scalar_add_arena_ptr(void *ctx) +{ +#if defined(__BPF_FEATURE_ADDR_SPACE_CAST) + int __arena *result, *arena_ptr; + +/* Needed for the verifier to link the arena to the subprog. */ + volatile char __arena *base = arena_base(&arena); + +/* Passing base here to avoid optimizing it out during compilation. */ + asmvolatile ( +"%[arena_ptr] = 8192;" +"%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);" +"%[result] = 12;" +"%[result] += %[arena_ptr];" + : [result]"=r" (result), + [arena_ptr]"=&r" (arena_ptr) + :"r" (base) + : + ); +#endif +return 0 ; +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.53.0
Re: [PATCH bpf-next v3 2/9] selftests/bpf: Add test for scalar/arena pointer addition
Quoted 7 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
Add a selftest that ensures scalar +=
ptr_to_arenainstructions are accepted by the verifier.Signed-off-by Emil Tsalapatis <[email protected]>
Re: [PATCH bpf-next v3 2/9] selftests/bpf: Add test for scalar/arena pointer addition
In reply to Emil TsalapatisQuoted 18 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
Add a selftest that ensures scalar +=
ptr_to_arenainstructions are accepted by the verifier.Signed-off-by Emil Tsalapatis <[email protected]>--- .../selftests/bpf/progs/verifier_arena.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+)tools/testing/selftests/bpf/progs/verifier_arena.c +0
-0@@ -477,4 +477,33 @@ int arena_kfuncs_under_bpf_lock(void *ctx)return 0 ;
tools/testing/selftests/bpf/progs/verifier_arena.c int arena_kfuncs_under_bpf_lock(void *ctx) · line 488
Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +9
-0} +/* +* Test that scalar += PTR_TO_ARENA correctly upgrades the +* destination register to a PTR_TO_ARENA. +*/ +SEC("syscall") +__success __retval(0) +int scalar_add_arena_ptr (void *ctx) +{ +#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)
I think the #if can be removed, since the whole test is in asm?
Quoted 24 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +20
-1+ int __arena *result, *arena_ptr; + +/* Needed for the verifier to link the arena to the subprog. */ + volatile char __arena *base = arena_base(&arena); + +/* Passing base here to avoid optimizing it out during compilation. */ + asmvolatile ( +"%[arena_ptr] = 8192;" +"%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);" +"%[result] = 12;" +"%[result] += %[arena_ptr];" + : [result]"=r" (result), + [arena_ptr]"=&r" (arena_ptr) + :"r" (base) + : + ); +#endif +return 0 ; +} + char _license[] SEC("license") = "GPL"; --2.53.0
Emil Tsalapatis
reviewerRe: [PATCH bpf-next v3 2/9] selftests/bpf: Add test for scalar/arena pointer addition
Quoted 18 lines · Alexei Starovoitov
Quoted 17 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
Add a selftest that ensures scalar +=
ptr_to_arenainstructions are accepted by the verifier.Signed-off-by Emil Tsalapatis <[email protected]>--- .../selftests/bpf/progs/verifier_arena.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+)tools/testing/selftests/bpf/progs/verifier_arena.c +0
-0@@ -477,4 +477,33 @@ int arena_kfuncs_under_bpf_lock(void *ctx)
Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_arena.c int arena_kfuncs_under_bpf_lock(void *ctx) · line 488
Quoted 11 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +9
-0return 0 ; } +/* +* Test that scalar += PTR_TO_ARENA correctly upgrades the +* destination register to a PTR_TO_ARENA. +*/ +SEC("syscall") +__success __retval(0) +int scalar_add_arena_ptr(void *ctx) +{ +#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)I think the #if can be removed, since the whole test is in asm?
With Clang 18 the assembly fails to load for me with "Invalid operand for instruction". AFAICT this is because addr_space_cast is only recognized as a valid instruction from the BPF backend starting Clang >= 19, which is also the first version that defines the feature flag.
Quoted 25 lines
Quoted 25 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +20
-1+ int __arena *result, *arena_ptr; + +/* Needed for the verifier to link the arena to the subprog. */ + volatile char __arena *base = arena_base(&arena); + +/* Passing base here to avoid optimizing it out during compilation. */ + asmvolatile ( +"%[arena_ptr] = 8192;" +"%[arena_ptr] = addr_space_cast(%[arena_ptr], 0x0, 0x1);" +"%[result] = 12;" +"%[result] += %[arena_ptr];" + : [result]"=r" (result), + [arena_ptr]"=&r" (arena_ptr) + :"r" (base) + : + ); +#endif +return 0 ; +} + char _license[] SEC("license") = "GPL"; --2.53.0
Re: [PATCH bpf-next v3 2/9] selftests/bpf: Add test for scalar/arena pointer addition
Quoted 34 lines · Emil Tsalapatis
Quoted 18 lines · Alexei Starovoitov
Quoted 17 lines · Emil Tsalapatis
From: Emil Tsalapatis <[email protected]>
Add a selftest that ensures scalar +=
ptr_to_arenainstructions are accepted by the verifier.Signed-off-by Emil Tsalapatis <[email protected]>--- .../selftests/bpf/progs/verifier_arena.c | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+)tools/testing/selftests/bpf/progs/verifier_arena.c +0
-0@@ -477,4 +477,33 @@ int arena_kfuncs_under_bpf_lock(void *ctx)Quoted 13 lines
tools/testing/selftests/bpf/progs/verifier_arena.c int arena_kfuncs_under_bpf_lock(void *ctx) · line 488
Quoted 11 lines
tools/testing/selftests/bpf/progs/verifier_arena.c +9
-0return 0 ; } +/* +* Test that scalar += PTR_TO_ARENA correctly upgrades the +* destination register to a PTR_TO_ARENA. +*/ +SEC("syscall") +__success __retval(0) +int scalar_add_arena_ptr(void *ctx) +{ +#if defined(__BPF_FEATURE_ADDR_SPACE_CAST)I think the #if can be removed, since the whole test is in asm?
Quoted 4 lines
With Clang 18 the assembly fails to load for me with "Invalid operand for instruction". AFAICT this is because
addr_space_castis only recognized as a valid instruction from the BPF backend starting Clang >= 19, which is also the first version that defines the feature flag.
Replace the whole thing with hard coded registers
and hex representation for addr_space_cast ?
See define bpf_addr_space_cast
maybe it's overkill just to test it.
I'm ok keeping it as-is.
Emil Tsalapatis
reviewerPATCH v3 3/9 selftests/bpf: Move bpf_arena_spin_lock.h to the top level
In reply to Emil TsalapatisThe bpf_arena_spin_lock.h header is useful for all programs and not just the selftests. Move it to the top level of the BPF selftests to make it more readily accessible.
---
tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h | 4 ++--
tools/testing/selftests/bpf/progs/arena_spin_lock.c | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
rename tools/testing/selftests/bpf/{progs => }/bpf_arena_spin_lock.h (99%)tools/testing/selftests/bpf/bpf_arena_spin_lock.h renamed from tools/testing/selftests/bpf/progs/bpf_arena_spin_lock.h +2 -2
@@ -107,7 +107,7 @@ struct arena_qnode {#define _Q_LOCKED_VAL (1U << _Q_LOCKED_OFFSET)#define _Q_PENDING_VAL (1U << _Q_PENDING_OFFSET) -struct arena_qnode __arena qnodes[_Q_MAX_CPUS][_Q_MAX_NODES]; +struct arena_qnode __weak __arena __hidden qnodes[_Q_MAX_CPUS][_Q_MAX_NODES];static inline u32 encode_tail (int cpu,int idx) {
@@ -240,7 +240,7 @@ static __always_inline int arena_spin_trylock(arena_spinlock_t __arena *lock)return likely (atomic_try_cmpxchg_acquire (&lock->val, &val, _Q_LOCKED_VAL)); } -__noinline +__noinline __weak int arena_spin_lock_slowpath(arena_spinlock_t __arena __arg_arena *lock, u32 val) { struct arena_mcs_spinlock __arena *prev, *next, *node0, *node;
tools/testing/selftests/bpf/progs/arena_spin_lock.c +1 -1
@@ -4,7 +4,7 @@#include <bpf/bpf_tracing.h> #include <bpf/bpf_helpers.h> #include "bpf_misc.h" -#include "bpf_arena_spin_lock.h" +#include "../bpf_arena_spin_lock.h" struct { __uint(type, BPF_MAP_TYPE_ARENA);
Signature · 1 line
2.53.0
Re: [PATCH bpf-next v3 3/9] selftests/bpf: Move bpf_arena_spin_lock.h to the top level
Quoted 7 lines · Emil Tsalapatis
The
bpf_arena_spin_lock.hheader is useful for all programs and not just the selftests. Move it to the top level of the BPF selftests to make it more readily accessible.Signed-off-by Emil Tsalapatis <[email protected]>
Emil Tsalapatis
reviewerPATCH v3 4/9 selftests/bpf: Deduplicate WRITE_ONCE macro between headers
In reply to Emil TsalapatisThe WRITE_ONCE macro is identically defined both in bpf_atomic.h and in bpf_arena_common.h. Deduplicate the definitions by moving the macro to the commonly used bpf_experimental.h header.
--- tools/testing/selftests/bpf/bpf_arena_common.h | 4 ---- tools/testing/selftests/bpf/bpf_atomic.h | 4 ---- tools/testing/selftests/bpf/bpf_experimental.h | 3 +++ 3 files changed, 3 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/bpf_arena_common.h +0 -4
@@ -2,10 +2,6 @@/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ #pragma once -#ifndef WRITE_ONCE -#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *) &(x)) = (val)) -#endif -#ifndef NUMA_NO_NODE #define NUMA_NO_NODE (-1)#endif
tools/testing/selftests/bpf/bpf_atomic.h +0 -4
@@ -40,10 +40,6 @@ extern bool CONFIG_X86_64 __kconfig __weak;/* No-op for BPF */ #define cpu_relax () ({}) -#define READ_ONCE (x) (*(volatile typeof(x) *)&(x)) - -#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *)&(x)) = (val)) -#define cmpxchg (p, old, new) __sync_val_compare_and_swap((p), old, new)#define try_cmpxchg (p, pold, new) \
tools/testing/selftests/bpf/bpf_experimental.h +3 -0
@@ -11,6 +11,9 @@/* Convenience macro to wrap over bpf_obj_new */ #define bpf_obj_new (type) ((type *)bpf_obj_new(bpf_core_type_id_local(type))) +#define READ_ONCE (x) (*(volatile typeof(x) *)&(x)) +#define WRITE_ONCE (x, val) ((*(volatile typeof(x) *)&(x)) = (val)) +/* Convenience macro to wrap over bpf_percpu_obj_new */ #define bpf_percpu_obj_new (type) ((type __percpu_kptr *)bpf_percpu_obj_new(bpf_core_type_id_local(type)))
Signature · 1 line
2.53.0
Re: [PATCH bpf-next v3 4/9] selftests/bpf: Deduplicate WRITE_ONCE macro between headers
Quoted 6 lines · Emil Tsalapatis
The
WRITE_ONCEmacro is identically defined both inbpf_atomic.hand inbpf_arena_common.h. Deduplicate the definitions by moving the macro to the commonly usedbpf_experimental.hheader.Signed-off-by Emil Tsalapatis <[email protected]>
Emil Tsalapatis
reviewerPATCH v3 5/9 selftests/bpf: Add basic libarena scaffolding
In reply to Emil TsalapatisAdd initial code for an arena-based BPF library. The current commit introduces a test runner and Makefile for the library. Library code can be added just by including the source file in the library's src/ subdirectory. Future commits will introduce the library code itself.
The current commit also includes a standalone test runner. This is to keep libarena self-contained and testable even when copied out of the kernel tree. Subsequent commits add integration with test_progs.
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 15 ++ tools/testing/selftests/bpf/libarena/Makefile | 50 ++++++ .../selftests/bpf/libarena/include/common.h | 49 ++++++ .../bpf/libarena/include/selftest_helpers.h | 81 ++++++++++ .../selftests/bpf/libarena/include/userapi.h | 26 +++ .../bpf/libarena/selftests/selftest.c | 153 ++++++++++++++++++ .../bpf/libarena/selftests/selftest.h | 12 ++ .../selftests/bpf/libarena/src/common.bpf.c | 65 ++++++++ 9 files changed, 452 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/Makefile create mode 100644 tools/testing/selftests/bpf/libarena/include/common.h create mode 100644 tools/testing/selftests/bpf/libarena/include/selftest_helpers.h create mode 100644 tools/testing/selftests/bpf/libarena/include/userapi.h create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.c create mode 100644 tools/testing/selftests/bpf/libarena/selftests/selftest.h create mode 100644 tools/testing/selftests/bpf/libarena/src/common.bpf.c
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -49,3 +49,4 @@ verification_cert.h *.BTF.base usdt_1 usdt_2 +libarena/test_libarena
tools/testing/selftests/bpf/Makefile +15 -0
@@ -153,6 +153,7 @@ override define CLEAN $(Q)$(RM) -r $(TEST_KMODS) $(Q)$(RM) -r $(EXTRA_CLEAN) $(Q)$(MAKE) -C test_kmods clean + $(Q)$(MAKE) -C libarena clean $(Q)$(MAKE) docs-clean endef
@@ -739,6 +740,17 @@ $(VERIFY_SIG_HDR): $(VERIFICATION_CERT) echo "};"; \ echo "unsigned int test_progs_verification_cert_len = $$(wc -c < $<);") > $@ +LIBARENA_MAKE_ARGS = \ + BPFTOOL="$(BPFTOOL)" \ + INCLUDE_DIR="$(HOST_INCLUDE_DIR)" \ + LIBBPF_INCLUDE="$(HOST_INCLUDE_DIR)" \ + BPFOBJ="$(BPFOBJ)" \ + LDLIBS="$(LDLIBS) -lzstd" \ + CLANG="$(CLANG)" \ + BPF_CFLAGS="$(BPF_CFLAGS) $(CLANG_CFLAGS)" \ + BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \ + Q="$(Q)" + # Define test_progs test runner. TRUNNER_TESTS_DIR := prog_tests TRUNNER_BPF_PROGS_DIR := progs
@@ -931,3 +943,6 @@ override define INSTALL_RULE rsync -a $(OUTPUT)/$$DIR/*.bpf.o $(INSTALL_PATH)/$$DIR;\ done endef + +test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
tools/testing/selftests/bpf/libarena/Makefile new file +50 -0
50 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/include/common.h new file +49 -0
49 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/include/selftest_helpers.h new file +81 -0
81 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/include/userapi.h new file +26 -0
@@ -0,0 +1,26 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#include <stdint.h> + +/* +* Header for the userspace C programs that load +* and initialize the BPF code. +*/ + +#define __arena + +typedef uint8_t u8 ; +typedef uint16_t u16 ; +typedef uint32_t u32 ; +typedef uint64_t u64 ; +typedef int8_t s8 ; +typedef int16_t s16 ; +typedef int32_t s32 ; +typedef int64_t s64 ; + +/* Dummy "definition" for userspace. */ +#define arena_spinlock_t u64 + +#include "common.h"
tools/testing/selftests/bpf/libarena/selftests/selftest.c new file +153 -0
153 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/selftests/selftest.h new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +#define ALLOC_SELFTEST (func, ...) \ + do { \ + int ret = func(__VA_ARGS__); \ + if (ret) { \ + arena_stderr("SELFTEST %s FAIL: %d", #func, ret); \ + return ret; \ + } \ + } while (0)
tools/testing/selftests/bpf/libarena/src/common.bpf.c new file +65 -0
65 lines not on this page · raw message
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v3 6/9 selftests/bpf: Add arena ASAN runtime to libarena
In reply to Emil TsalapatisAdd an address sanitizer (ASAN) runtime to the arena library. The ASAN runtime implements the functions injected into BPF binaries by LLVM sanitization when ASAN is enabled during compilation.
The runtime also includes functions called explicitly by memory allocation code to mark memory as poisoned/unpoisoned to ASAN. This code is a no-op when sanitization is turned off.
--- .../selftests/bpf/libarena/include/asan.h | 124 +++++ .../selftests/bpf/libarena/include/common.h | 1 + .../selftests/bpf/libarena/src/asan.bpf.c | 524 ++++++++++++++++++ .../selftests/bpf/libarena/src/common.bpf.c | 1 + 4 files changed, 650 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/include/asan.h create mode 100644 tools/testing/selftests/bpf/libarena/src/asan.bpf.c
tools/testing/selftests/bpf/libarena/include/asan.h new file +124 -0
@@ -0,0 +1,124 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +struct asan_init_args { +u64 arena_all_pages; +u64 arena_globals_pages; +}; + +int asan_init (struct asan_init_args *args); + +/* Parameters usable by userspace. */ +extern volatile u64 __asan_shadow_memory_dynamic_address; +extern volatile bool asan_reported; +extern volatile bool asan_inited; +extern volatile bool asan_report_once; +extern volatile bool asan_emit_stack; + +#ifdef __BPF__ + +#define ASAN_SHADOW_SHIFT 3 +#define ASAN_SHADOW_SCALE (1ULL << ASAN_SHADOW_SHIFT) +#define ASAN_GRANULE_MASK ((1ULL << ASAN_SHADOW_SHIFT) - 1) +#define ASAN_GRANULE (addr) ((s8)((u32)(u64)((addr)) & ASAN_GRANULE_MASK)) + +#define __noasan __attribute__((no_sanitize("address"))) + +#ifdef BPF_ARENA_ASAN + +/* +* Defined as char * to get 1-byte granularity for pointer arithmetic. +*/ +typedef s8 __arena s8a; + +/* +* Address to shadow map translation. +*/ +static inline +s8a *mem_to_shadow(void __arena __arg_arena *addr) +{ +return (s8a *)(((u32 )(u64 )addr >>ASAN_SHADOW_SHIFT ) + __asan_shadow_memory_dynamic_address); +} + +/* +* Helper for directly reading the shadow map. +*/ +static inline __noasan +s8 asan_shadow_value(void __arena __arg_arena *addr) +{ +return *(s8a *)mem_to_shadow (addr); +} + +__weak __noasan +bool asan_ready(void) +{ +return __asan_shadow_memory_dynamic_address; +} + +/* +* Shadow map manipulation helpers. +*/ +int asan_poison(void __arena *addr, s8 val, size_t size); +int asan_unpoison(void __arena *addr, size_t size); +bool asan_shadow_set(void __arena *addr); + +/* +* Dummy calls to ensure the ASAN runtime's BTF information is present +* in every object file when compiling the runtime and local BPF code +* separately. The runtime calls are injected into the LLVM IR file +*/ +#define DECLARE_ASAN_LOAD_STORE_SIZE (size) \ + void __asan_store##size(void *addr); \ + void __asan_store##size##_noabort(void *addr); \ + void __asan_load##size(void *addr); \ + void __asan_load##size##_noabort(void *addr); \ + void __asan_report_store##size(void *addr); \ + void __asan_report_store##size##_noabort(void *addr); \ + void __asan_report_load##size(void *addr); \ + void __asan_report_load##size##_noabort(void *addr); + +DECLARE_ASAN_LOAD_STORE_SIZE (1 ); +DECLARE_ASAN_LOAD_STORE_SIZE (2 ); +DECLARE_ASAN_LOAD_STORE_SIZE (4 ); +DECLARE_ASAN_LOAD_STORE_SIZE (8 ); + +#define ASAN_DUMMY_CALLS_SIZE (size, arg) \ +do { \ + __asan_store##size((arg)); \ + __asan_store##size##_noabort((arg)); \ + __asan_load##size((arg)); \ + __asan_load##size##_noabort((arg)); \ + __asan_report_store##size((arg)); \ + __asan_report_store##size##_noabort((arg)); \ + __asan_report_load##size((arg)); \ + __asan_report_load##size##_noabort((arg)); \ +} while (0) + +#define ASAN_DUMMY_CALLS_ALL (arg) \ +do { \ + ASAN_DUMMY_CALLS_SIZE(1, (arg)); \ + ASAN_DUMMY_CALLS_SIZE(2, (arg)); \ + ASAN_DUMMY_CALLS_SIZE(4, (arg)); \ + ASAN_DUMMY_CALLS_SIZE(8, (arg)); \ +} while (0) + +__weak __noasan +int asan_dummy_call(void) { +/* Use the shadow map base to prevent it from being optimized out. */ +if (__asan_shadow_memory_dynamic_address) +ASAN_DUMMY_CALLS_ALL (NULL ); + +return 0 ; +} +#else /* BPF_ARENA_ASAN */ + +static inline int asan_poison(void __arena *addr, s8 val, size_t size) { return 0; } +static inline int asan_unpoison(void __arena *addr, size_t size) { return 0; } +static inline bool asan_shadow_set(void __arena *addr) { return 0; } +static inline s8 asan_shadow_value(void __arena *addr) { return 0; } +__weak bool asan_ready (void ) {return true; } + +#endif /* BPF_ARENA_ASAN */ + +#endif /* __BPF__ */
tools/testing/selftests/bpf/libarena/include/common.h +1 -0
@@ -39,6 +39,7 @@ struct { } arena __weak SEC(".maps");extern const volatile u32 zero; +extern volatile u64 asan_violated;int arena_fls (__u64 word);
tools/testing/selftests/bpf/libarena/src/asan.bpf.c new file +524 -0
524 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/src/common.bpf.c +1 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause /* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ #include <common.h> +#include <asan.h> const volatile u32 zero =0 ;
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v3 7/9 selftests/bpf: Add ASAN support for libarena selftests
In reply to Emil TsalapatisExpand the arena library selftest infrastructure to support address sanitization. Add the compiler flags necessary to compile the library under ASAN when supported.
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 16 ++++++- tools/testing/selftests/bpf/libarena/Makefile | 31 ++++++++++++- .../bpf/libarena/include/selftest_helpers.h | 28 +++++++++++ .../selftests/bpf/libarena/include/userapi.h | 1 + .../bpf/libarena/selftests/selftest.c | 27 ++++++++++- .../bpf/libarena/selftests/st_asan_common.h | 46 +++++++++++++++++++ 7 files changed, 146 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -50,3 +50,4 @@ verification_cert.h usdt_1 usdt_2 libarena/test_libarena +libarena/test_libarena_asan
tools/testing/selftests/bpf/Makefile +15 -1
@@ -751,6 +751,16 @@ LIBARENA_MAKE_ARGS = \ BPF_TARGET_ENDIAN="$(BPF_TARGET_ENDIAN)" \ Q="$(Q)" +LIBARENA_SKELS := libarena/libarena.skel.h libarena/libarena_asan.skel.h + +# Use a stamp file so the submake is invoked only once for both skeletons. +libarena/.libarena_skels_stamp: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena skeletons $(LIBARENA_MAKE_ARGS) + $(Q)touch $@ + +$(LIBARENA_SKELS): libarena/.libarena_skels_stamp + @: + # Define test_progs test runner. TRUNNER_TESTS_DIR := prog_tests TRUNNER_BPF_PROGS_DIR := progs
@@ -774,7 +784,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c \ flow_dissector_load.h \ ip_check_defrag_frags.h \ bpftool_helpers.c \ - usdt_1.c usdt_2.c + usdt_1.c usdt_2.c \ + $(LIBARENA_SKELS) TRUNNER_LIB_SOURCES := find_bit.c TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \ $(OUTPUT)/liburandom_read.so \
@@ -946,3 +957,6 @@ endef test_libarena: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS) + +test_libarena_asan: $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) + +$(MAKE) -C libarena $@ $(LIBARENA_MAKE_ARGS)
tools/testing/selftests/bpf/libarena/Makefile +29 -2
@@ -8,6 +8,7 @@ LIBARENA=$(abspath .) LIBARENA_SOURCES = $(wildcard $(LIBARENA)/src/*.bpf.c) $(wildcard $(LIBARENA)/selftests/*.bpf.c) LIBARENA_OBJECTS = $(notdir $(LIBARENA_SOURCES:.bpf.c=.bpf.o)) +LIBARENA_OBJECTS_ASAN = $(notdir $(LIBARENA_SOURCES:.bpf.c=_asan.bpf.o)) INCLUDES = -I$(LIBARENA)/include -I$(LIBARENA)/.. ifneq ($(INCLUDE_DIR),)
@@ -17,6 +18,13 @@ ifneq ($(LIBBPF_INCLUDE),) INCLUDES += -I$(LIBBPF_INCLUDE) endif +ASAN_FLAGS = -fsanitize=kernel-address -fno-stack-protector -fno-builtin +ASAN_FLAGS += -mllvm -asan-instrument-address-spaces=1 -mllvm -asan-shadow-addr-space=1 +ASAN_FLAGS += -mllvm -asan-use-stack-safety=0 -mllvm -asan-stack=0 +ASAN_FLAGS += -mllvm -asan-kernel=1 +ASAN_FLAGS += -mllvm -asan-constructor-kind=none +ASAN_FLAGS += -mllvm -asan-destructor-kind=none + # ENABLE_ATOMICS_TESTS required because we use arena spinlocks override BPF_CFLAGS += -DENABLE_ATOMICS_TESTS override BPF_CFLAGS += -O2 -Wno-incompatible-pointer-types-discards-qualifiers
@@ -28,23 +36,42 @@ CFLAGS += $(INCLUDES) vpath %.bpf.c $(LIBARENA)/src $(LIBARENA)/selftests vpath %.c $(LIBARENA)/src $(LIBARENA)/selftests -all: test_libarena +all: test_libarena test_libarena_asan + +test_libarena_asan: selftest.c $(BPFOBJ) libarena_asan.skel.h + $(call msg,BINARY,libarena,$@) + $(Q)$(CLANG) $(LDLIBS) $(CFLAGS) -DBPF_ARENA_ASAN $< $(BPFOBJ) -o $@ test_libarena: selftest.c $(BPFOBJ) libarena.skel.h $(call msg,BINARY,libarena,$@) $(Q)$(CLANG) $(CFLAGS) $< $(LDLIBS) $(BPFOBJ) -o $@ +skeletons: libarena.skel.h libarena_asan.skel.h +.PHONY: skeletons + +libarena_asan.skel.h: main_asan.bpf.o + $(call msg,GEN-SKEL,libarena,$@) + $(Q)$(BPFTOOL) gen skeleton $< name "libarena_asan" > $@ + libarena.skel.h: main.bpf.o $(call msg,GEN-SKEL,libarena,$@) $(Q)$(BPFTOOL) gen skeleton $< name "libarena" > $@ +main_asan.bpf.o: $(LIBARENA_OBJECTS_ASAN) + $(call msg,GEN-OBJ,libarena,$@) + $(Q)$(BPFTOOL) gen object $@ $^ + main.bpf.o: $(LIBARENA_OBJECTS) $(call msg,GEN-OBJ,libarena,$@) $(Q)$(BPFTOOL) gen object $@ $^ +%_asan.bpf.o: %.bpf.c + $(call msg,CLNG-BPF,libarena,$@) + $(Q)$(CLANG) $(BPF_CFLAGS) $(ASAN_FLAGS) -DBPF_ARENA_ASAN $(BPF_TARGET_ENDIAN) -c $< -o $@ + %.bpf.o: %.bpf.c $(call msg,CLNG-BPF,libarena,$@) $(Q)$(CLANG) $(BPF_CFLAGS) $(BPF_TARGET_ENDIAN) -c $< -o $@ clean: - $(Q)rm -f *.skel.h *.bpf.o test_libarena + $(Q)rm -f *.skel.h *.bpf.o *.linked*.o test_libarena test_libarena_asan
tools/testing/selftests/bpf/libarena/include/selftest_helpers.h +28 -0
@@ -79,3 +79,31 @@ static inline int libarena_get_globals_pages(int arena_get_base_fd,free (vec);return 0 ; } + +static inline int libarena_asan_init (int arena_get_base_fd, +int asan_init_fd, +size_t arena_all_pages) +{ +LIBBPF_OPTS (bpf_test_run_opts, opts); +struct asan_init_args args; +u64 globals_pages; +int ret; + + ret =libarena_get_globals_pages (arena_get_base_fd, + arena_all_pages, &globals_pages); +if (ret) +return ret; + + args = (struct asan_init_args ){ + .arena_all_pages = arena_all_pages, + .arena_globals_pages = globals_pages, + }; + + opts.ctx_in = &args; + opts.ctx_size_in =sizeof (args); + + ret =bpf_prog_test_run_opts (asan_init_fd, &opts); +if (ret) +return ret; +return opts.retval; +}
tools/testing/selftests/bpf/libarena/include/userapi.h +1 -0
@@ -24,3 +24,4 @@ typedef int64_t s64;#define arena_spinlock_t u64#include "common.h" +#include "asan.h"
tools/testing/selftests/bpf/libarena/selftests/selftest.c +26 -1
@@ -18,6 +18,17 @@#include <userapi.h> #include <selftest_helpers.h> +#include "selftest.h" + +#ifdef BPF_ARENA_ASAN +#include "../libarena_asan.skel.h" +typedef struct libarena_asan selftest ; +#define selftest__open libarena_asan__open +#define selftest__open_and_load libarena_asan__open_and_load +#define selftest__load libarena_asan__load +#define selftest__attach libarena_asan__attach +#define selftest__destroy libarena_asan__destroy +#else #include "../libarena.skel.h" typedef struct libarena selftest ;#define selftest__open libarena__open
@@ -25,6 +36,7 @@ typedef struct libarena selftest;#define selftest__load libarena__load#define selftest__attach libarena__attach#define selftest__destroy libarena__destroy +#endif static bool verbose = false;static int testno =1 ;
@@ -69,6 +81,7 @@ static int libbpf_print_fn(enum libbpf_print_level level,int run_test (selftest *skel,const struct bpf_program *prog) { +size_t arena_pages = (1UL <<32 ) /sysconf (_SC_PAGESIZE);int prog_fd;int ret;
@@ -76,6 +89,15 @@ int run_test(selftest *skel, const struct bpf_program *prog)if (ret)return ret; +#ifdef BPF_ARENA_ASAN + ret =libarena_asan_init ( +bpf_program__fd (skel->progs.arena_get_base), +bpf_program__fd (skel->progs.asan_init), + arena_pages); +if (ret) +return ret; +#endif + prog_fd =bpf_program__fd (prog);if (prog_fd <0 )return -ENOENT ;
@@ -119,10 +141,13 @@ static void banner(const char *progpath) {char *name =basename (progpath); +bool is_asan; +/* Check if our BPF programs are ASAN-capable using strstr on the prog name. */ printf ("%s\n" , name); + is_asan =strstr (name,"_asan" ); -printf ("=== %s ===\n" ,"libarena selftests" ); +printf ("=== %s %s===\n" ,"libarena selftests" , is_asan ?"(asan) " :"" ); }int main (int argc,char *argv[])
tools/testing/selftests/bpf/libarena/selftests/st_asan_common.h new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v3 8/9 selftests/bpf: Add buddy allocator for libarena
In reply to Emil TsalapatisAdd a byte-oriented buddy allocator for libarena. The buddy allocator provides an alloc/free interface for small arena allocations (down to 16 bytes).
--- .../selftests/bpf/libarena/include/buddy.h | 64 ++ .../selftests/bpf/libarena/src/buddy.bpf.c | 783 ++++++++++++++++++ 2 files changed, 847 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/include/buddy.h create mode 100644 tools/testing/selftests/bpf/libarena/src/buddy.bpf.c
tools/testing/selftests/bpf/libarena/include/buddy.h new file +64 -0
@@ -0,0 +1,64 @@ +// SPDX-License-Identifier: LGPL-2.1 OR BSD-2-Clause +/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */ +#pragma once + +/* Buddy allocator-related structs. */ + +struct buddy_chunk ; +typedef struct buddy_chunk __arena buddy_chunk_t; + +struct buddy_header ; +typedef struct buddy_header __arena buddy_header_t; + +enum buddy_consts { +BUDDY_MIN_ALLOC_SHIFT =4 , +BUDDY_MIN_ALLOC_BYTES =1 <<BUDDY_MIN_ALLOC_SHIFT , +BUDDY_CHUNK_NUM_ORDERS =1 <<4 ,/* 4 bits per order */ +BUDDY_CHUNK_BYTES =BUDDY_MIN_ALLOC_BYTES <<BUDDY_CHUNK_NUM_ORDERS , +BUDDY_HEADER_OFF =8 ,/* header byte offset, see buddy.bpf.c for details */ +BUDDY_CHUNK_PAGES =BUDDY_CHUNK_BYTES / __PAGE_SIZE, +BUDDY_CHUNK_ITEMS =1 <<BUDDY_CHUNK_NUM_ORDERS , +BUDDY_CHUNK_OFFSET_MASK =BUDDY_CHUNK_BYTES -1 , +BUDDY_VADDR_OFFSET =BUDDY_CHUNK_BYTES ,/* Start aligning at chunk */ +BUDDY_VADDR_SIZE =BUDDY_CHUNK_BYTES <<10 /* 1024 chunks maximum */ +}; + +struct buddy_header { +u32 prev_index;/* "Pointer" to the previous available allocation of the same size. */ +u32 next_index;/* Same for the next allocation. */ +}; + +/* +* We bring memory into the allocator 1MiB at a time. +*/ +struct buddy_chunk { +/* The order of the current allocation for a item. 4 bits per order. */ +u8 orders[BUDDY_CHUNK_ITEMS /2 ]; +/* +* Bit to denote whether chunk is allocated. Size of the allocated/free +* chunk found from the orders array. +*/ +u8 allocated[BUDDY_CHUNK_ITEMS /8 ]; +/* Freelists for O(1) allocation. */ +u64 freelists[BUDDY_CHUNK_NUM_ORDERS ]; +buddy_chunk_t *prev; +buddy_chunk_t *next; +}; + +struct buddy { +buddy_chunk_t *first_chunk;/* Pointer to the chunk linked list. */ + arena_spinlock_t __arena *lock; /* Allocator lock */ +u64 vaddr;/* Allocation into reserved vaddr */ +}; + +#ifdef __BPF__ + +int buddy_init(struct buddy *buddy, arena_spinlock_t __arena *lock); +int buddy_destroy (struct buddy *buddy); +int buddy_free_internal (struct buddy *buddy,u64 free); +#define buddy_free (buddy, ptr) do { buddy_free_internal((buddy), (u64)(ptr)); } while (0) +u64 buddy_alloc_internal (struct buddy *buddy,size_t size); +#define buddy_alloc (alloc, size) ((void __arena *)buddy_alloc_internal((alloc), (size))) + + +#endif /* __BPF__ */
tools/testing/selftests/bpf/libarena/src/buddy.bpf.c new file +783 -0
783 lines not on this page · raw message
Signature · 1 line
2.53.0
Emil Tsalapatis
reviewerPATCH v3 9/9 selftests/bpf: Add selftests for libarena buddy allocator
In reply to Emil TsalapatisFrom: Emil Tsalapatis <[email protected]>
Introduce selftests for the buddy allocator with and without ASAN. Add the libarena selftests both to the libarena test runner and to test_progs, so that they are a) available when libarena is pulled as a standalone library, and b) exercised along with all other test programs in this directory.
--- .../bpf/libarena/selftests/selftest.c | 12 + .../libarena/selftests/st_asan_buddy.bpf.c | 251 ++++++++++++++++++ .../bpf/libarena/selftests/st_buddy.bpf.c | 229 ++++++++++++++++ .../selftests/bpf/prog_tests/libarena.c | 79 ++++++ 4 files changed, 571 insertions(+) create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c create mode 100644 tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c create mode 100644 tools/testing/selftests/bpf/prog_tests/libarena.c
tools/testing/selftests/bpf/libarena/selftests/selftest.c +12 -0
@@ -137,6 +137,12 @@ error_no_destroy: \return ret; \ } +TEST (test_buddy); + +#ifdef BPF_ARENA_ASAN +TEST (asan_test_buddy); +#endif +static void banner (const char *progpath) {
@@ -174,5 +180,11 @@ int main(int argc, char *argv[])libbpf_set_print (libbpf_print_fn); +run_test_buddy (); + +#ifdef BPF_ARENA_ASAN +run_asan_test_buddy (); +#endif +return 0 ; }
tools/testing/selftests/bpf/libarena/selftests/st_asan_buddy.bpf.c new file +251 -0
251 lines not on this page · raw message
tools/testing/selftests/bpf/libarena/selftests/st_buddy.bpf.c new file +229 -0
229 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/libarena.c new file +79 -0
79 lines not on this page · raw message
Signature · 1 line
2.53.0