bpf: Allow selected kfuncs under bpf_spin_lock
Subsystems (5)
v2 cover bpf: Allow selected kfuncs under bpf_spin_lock
The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.
This series adds a new KF_SPINLOCK_SAFE kfunc flag and teaches the verifier to use kfunc registration metadata when deciding whether a kfunc call is allowed while a bpf_spin_lock is held.
The built-in kfuncs that are currently accepted by the verifier's lock-held allowlist are annotated with the new flag. This preserves the existing behavior while removing the verifier-side category checks and uses the same mechanism for built-in and module kfuncs.
The selftest coverage marks one bpf_testmod kfunc as KF_SPINLOCK_SAFE and verifies that it can be called under a bpf_spin_lock. It also calls another registered but unmarked bpf_testmod kfunc under the lock and checks that the verifier rejects it.
Changes in v2:
- Rename KF_SPIN_LOCK to KF_SPINLOCK_SAFE. (Kumar Kartikeya Dwivedi, Leon Hwang)
- Deprecate the verifier's lock-held allowlist mechanism and annotate the relevant kfuncs uniformly with KF_SPINLOCK_SAFE (Kumar Kartikeya Dwivedi)
- Add selftests. (Leon Hwang)
Link to v1:
https://lore.kernel.org/bpf/[email protected]/
Kaitao Cheng (3):
bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock
bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE selftests/bpf: Test module kfunc calls under spin lock
include/linux/btf.h | 1 + kernel/bpf/arena.c | 6 +- kernel/bpf/helpers.c | 56 ++++++++++--------- kernel/bpf/rqspinlock.c | 8 +-- kernel/bpf/verifier.c | 38 +++---------- .../selftests/bpf/prog_tests/kfunc_call.c | 2 + .../selftests/bpf/progs/kfunc_call_fail.c | 12 ++++ .../selftests/bpf/progs/kfunc_call_test.c | 12 ++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 2 +- 9 files changed, 72 insertions(+), 65 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH v2 1/3 bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock
From: Kaitao Cheng <[email protected]>
Introduce the KF_SPINLOCK_SAFE kfunc metadata flag in BTF so kfuncs may be explicitly marked as safe to call while holding bpf_spin_lock.
Allow kfuncs defined in kernel modules to be marked with KF_SPINLOCK_SAFE.
Example: BTF_ID_FLAGS(func, $kfunc_name, KF_SPINLOCK_SAFE)
--- include/linux/btf.h | 1 + kernel/bpf/verifier.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-)
include/linux/btf.h +1 -0
@@ -79,6 +79,7 @@#define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ #define KF_IMPLICIT_ARGS (1 << 16)/* kfunc has implicit arguments supplied by the verifier */ +#define KF_SPINLOCK_SAFE (1 << 17)/* kfunc is allowed inside bpf_spin_lock-ed region */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +15 -5
@@ -11832,11 +11832,21 @@ static bool is_bpf_stream_kfunc(u32 btf_id) btf_id == special_kfunc_list[KF_bpf_stream_print_stack]; } -static bool kfunc_spin_allowed (u32 btf_id) +static bool kfunc_spin_allowed (struct bpf_verifier_env *env,s32 func_id,s16 offset) { -return is_bpf_graph_api_kfunc (btf_id) ||is_bpf_iter_num_api_kfunc (btf_id) || -is_bpf_res_spin_lock_kfunc (btf_id) ||is_bpf_arena_kfunc (btf_id) || -is_bpf_stream_kfunc (btf_id); +struct bpf_kfunc_meta kfunc; +int err; + +if (is_bpf_graph_api_kfunc (func_id) ||is_bpf_iter_num_api_kfunc (func_id) || +is_bpf_res_spin_lock_kfunc (func_id) ||is_bpf_arena_kfunc (func_id) || +is_bpf_stream_kfunc (func_id)) +return true; + + err =fetch_kfunc_meta (env, func_id, offset, &kfunc); +if (err || !kfunc.flags) +return false; + +return *kfunc.flags &KF_SPINLOCK_SAFE ; }static bool is_sync_callback_calling_kfunc (u32 btf_id)
@@ -17415,7 +17425,7 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state) insn->imm != BPF_FUNC_spin_unlock && insn->imm != BPF_FUNC_kptr_xchg) || (insn->src_reg == BPF_PSEUDO_KFUNC_CALL && - (insn->off != 0 || !kfunc_spin_allowed(insn->imm)))) { + !kfunc_spin_allowed(env, insn->imm, insn->off))) {verbose (env,"function calls are not allowed while holding a lock\n" );return -EINVAL ;
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH v2 2/3 bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE
In reply to Kaitao ChengFrom: Kaitao Cheng <[email protected]>
The verifier currently keeps a hard-coded list of kfuncs that may be called while holding a bpf_spin_lock. With KF_SPINLOCK_SAFE available, retaining this list creates two sources of truth and requires verifier changes whenever another lock-safe kfunc is added.
Mark every kfunc currently accepted by kfunc_spin_allowed() with KF_SPINLOCK_SAFE. This covers the graph, numeric iterator, resource spin lock, arena, and stream kfuncs.
Remove the obsolete category checks and make kfunc_spin_allowed() rely solely on the kfunc registration metadata. This preserves the behavior of existing kfuncs while using the same mechanism for built-in and module kfuncs.
--- kernel/bpf/arena.c | 6 ++--- kernel/bpf/helpers.c | 56 +++++++++++++++++++++-------------------- kernel/bpf/rqspinlock.c | 8 +++--- kernel/bpf/verifier.c | 32 ----------------------- 4 files changed, 36 insertions(+), 66 deletions(-)
kernel/bpf/arena.c +3 -3
@@ -1118,9 +1118,9 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c__bpf_kfunc_end_defs (); BTF_KFUNCS_START(arena_kfuncs) -BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2) -BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2) -BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2) +BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2 | KF_SPINLOCK_SAFE) +BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE) +BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE) BTF_KFUNCS_END(arena_kfuncs) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/helpers.c +29 -27
76 lines not on this page · raw message
kernel/bpf/rqspinlock.c +4 -4
@@ -744,10 +744,10 @@ __bpf_kfunc void bpf_res_spin_unlock_irqrestore(struct bpf_res_spin_lock *lock,__bpf_kfunc_end_defs (); BTF_KFUNCS_START(rqspinlock_kfunc_ids) -BTF_ID_FLAGS (func ,bpf_res_spin_lock ,KF_RET_NULL ) -BTF_ID_FLAGS (func, bpf_res_spin_unlock) -BTF_ID_FLAGS (func, bpf_res_spin_lock_irqsave,KF_RET_NULL ) -BTF_ID_FLAGS (func, bpf_res_spin_unlock_irqrestore) +BTF_ID_FLAGS(func, bpf_res_spin_lock, KF_RET_NULL | KF_SPINLOCK_SAFE) +BTF_ID_FLAGS(func, bpf_res_spin_unlock, KF_SPINLOCK_SAFE) +BTF_ID_FLAGS(func, bpf_res_spin_lock_irqsave, KF_RET_NULL | KF_SPINLOCK_SAFE) +BTF_ID_FLAGS(func, bpf_res_spin_unlock_irqrestore, KF_SPINLOCK_SAFE) BTF_KFUNCS_END(rqspinlock_kfunc_ids) static const struct btf_kfunc_id_set rqspinlock_kfunc_set = {
kernel/bpf/verifier.c +0 -32
@@ -11797,20 +11797,6 @@ static bool is_bpf_rbtree_api_kfunc(u32 btf_id) btf_id == special_kfunc_list[KF_bpf_rbtree_right]; } -static bool is_bpf_iter_num_api_kfunc (u32 btf_id) -{ -return btf_id == special_kfunc_list[KF_bpf_iter_num_new] || - btf_id == special_kfunc_list[KF_bpf_iter_num_next] || - btf_id == special_kfunc_list[KF_bpf_iter_num_destroy]; -} - -static bool is_bpf_graph_api_kfunc (u32 btf_id) -{ -return is_bpf_list_api_kfunc (btf_id) || -is_bpf_rbtree_api_kfunc (btf_id) || -is_bpf_refcount_acquire_kfunc (btf_id); -} -static bool is_bpf_res_spin_lock_kfunc (u32 btf_id) { return btf_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
@@ -11819,29 +11805,11 @@ static bool is_bpf_res_spin_lock_kfunc(u32 btf_id) btf_id == special_kfunc_list[KF_bpf_res_spin_unlock_irqrestore]; } -static bool is_bpf_arena_kfunc (u32 btf_id) -{ -return btf_id == special_kfunc_list[KF_bpf_arena_alloc_pages] || - btf_id == special_kfunc_list[KF_bpf_arena_free_pages] || - btf_id == special_kfunc_list[KF_bpf_arena_reserve_pages]; -} - -static bool is_bpf_stream_kfunc (u32 btf_id) -{ -return btf_id == special_kfunc_list[KF_bpf_stream_vprintk] || - btf_id == special_kfunc_list[KF_bpf_stream_print_stack]; -} -static bool kfunc_spin_allowed (struct bpf_verifier_env *env,s32 func_id,s16 offset) {struct bpf_kfunc_meta kfunc;int err; -if (is_bpf_graph_api_kfunc (func_id) ||is_bpf_iter_num_api_kfunc (func_id) || -is_bpf_res_spin_lock_kfunc (func_id) ||is_bpf_arena_kfunc (func_id) || -is_bpf_stream_kfunc (func_id)) -return true; - err =fetch_kfunc_meta (env, func_id, offset, &kfunc);if (err || !kfunc.flags)return false;
Signature · 1 line
2.50.1 (Apple Git-155)
PATCH v2 3/3 selftests/bpf: Test module kfunc calls under spin lock
In reply to Kaitao ChengFrom: Kaitao Cheng <[email protected]>
The verifier uses kfunc registration flags to decide whether a kfunc may be called while a BPF program holds a bpf_spin_lock.
Mark bpf_testmod_test_mod_kfunc() as KF_SPINLOCK_SAFE and verify that it can be called while holding a bpf_spin_lock. Also attempt to call the unmarked bpf_kfunc_trigger_ctx_check() under the lock and verify that the program is rejected.
--- tools/testing/selftests/bpf/prog_tests/kfunc_call.c | 2 ++ tools/testing/selftests/bpf/progs/kfunc_call_fail.c | 12 ++++++++++++ tools/testing/selftests/bpf/progs/kfunc_call_test.c | 12 ++++++++++++ tools/testing/selftests/bpf/test_kmods/bpf_testmod.c | 2 +- 4 files changed, 27 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +2 -0
@@ -71,8 +71,10 @@ static struct kfunc_test_params kfunc_tests[] = {TC_FAIL (kfunc_call_test_get_mem_fail_not_const,0 ,"is not a const" ),TC_FAIL (kfunc_call_test_mem_acquire_fail,0 ,"acquire kernel function does not return PTR_TO_BTF_ID" ),TC_FAIL (kfunc_call_test_pointer_arg_type_mismatch,0 ,"R1 expected pointer to ctx, but got scalar" ), +TC_FAIL (kfunc_call_test_spin_lock_unsafe,0 ,"function calls are not allowed while holding a lock" ),/* success cases */ +TC_TEST (kfunc_call_test_spin_lock_safe,0 ),TC_TEST (kfunc_call_test1,12 ),TC_TEST (kfunc_call_test2,3 ), TC_TEST(kfunc_call_test4, -1234),
tools/testing/selftests/bpf/progs/kfunc_call_fail.c +12 -0
@@ -4,6 +4,18 @@#include <bpf/bpf_helpers.h> #include "../test_kmods/bpf_testmod_kfunc.h" +static struct bpf_spin_lock kfunc_call_lock SEC(".data.A"); + +SEC("?tc") +int kfunc_call_test_spin_lock_unsafe (struct __sk_buff *skb) +{ +bpf_spin_lock (&kfunc_call_lock); +bpf_kfunc_trigger_ctx_check (); +bpf_spin_unlock (&kfunc_call_lock); + +return 0 ; +} +struct syscall_test_args {__u8 data[16 ];size_t size;
tools/testing/selftests/bpf/progs/kfunc_call_test.c +12 -0
@@ -5,6 +5,18 @@#include "bpf_misc.h" #include "../test_kmods/bpf_testmod_kfunc.h" +static struct bpf_spin_lock kfunc_call_lock SEC(".data.A"); + +SEC("tc") +int kfunc_call_test_spin_lock_safe (struct __sk_buff *skb) +{ +bpf_spin_lock (&kfunc_call_lock); +bpf_testmod_test_mod_kfunc (42 ); +bpf_spin_unlock (&kfunc_call_lock); + +return 0 ; +} + SEC("tc")int kfunc_call_test5 (struct __sk_buff *skb) {
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +1 -1
@@ -1384,7 +1384,7 @@ __bpf_kfunc void bpf_kfunc_trigger_ctx_check(void) } BTF_KFUNCS_START(bpf_testmod_check_kfunc_ids) -BTF_ID_FLAGS(func, bpf_testmod_test_mod_kfunc) +BTF_ID_FLAGS(func, bpf_testmod_test_mod_kfunc, KF_SPINLOCK_SAFE) BTF_ID_FLAGS(func, bpf_kfunc_call_test1) BTF_ID_FLAGS(func, bpf_kfunc_call_test2) BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
Signature · 1 line
2.50.1 (Apple Git-155)
Quoted 36 lines · Kaitao Cheng
The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a
bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.This series adds a new
KF_SPINLOCK_SAFEkfunc flag and teaches the verifier to use kfunc registration metadata when deciding whether a kfunc call is allowed while abpf_spin_lockis held.The built-in kfuncs that are currently accepted by the verifier's lock-held allowlist are annotated with the new flag. This preserves the existing behavior while removing the verifier-side category checks and uses the same mechanism for built-in and module kfuncs.
The selftest coverage marks one
bpf_testmodkfunc asKF_SPINLOCK_SAFEand verifies that it can be called under abpf_spin_lock. It also calls another registered but unmarkedbpf_testmodkfunc under the lock and checks that the verifier rejects it.Changes in v2:
- RenameKF_SPIN_LOCKtoKF_SPINLOCK_SAFE. (Kumar Kartikeya Dwivedi, Leon Hwang)
- Deprecate the verifier's lock-held allowlist mechanism and annotate the relevant kfuncs uniformly withKF_SPINLOCK_SAFE(Kumar Kartikeya Dwivedi)
- Add selftests. (Leon Hwang)Link to v1:
https://lore.kernel.org/bpf/[email protected]/Kaitao Cheng (3):
bpf: AddKF_SPINLOCK_SAFEflag for kfuncs underbpf_spin_lock
bpf: Mark existing lock-safe kfuncs withKF_SPINLOCK_SAFEselftests/bpf: Test module kfunc calls under spin lock
Quoted 11 lines
include/linux/btf.h | 1 + kernel/bpf/arena.c | 6 +- kernel/bpf/helpers.c | 56 ++++++++++--------- kernel/bpf/rqspinlock.c | 8 +-- kernel/bpf/verifier.c | 38 +++---------- .../selftests/bpf/prog_tests/kfunc_call.c | 2 + .../selftests/bpf/progs/kfunc_call_fail.c | 12 ++++ .../selftests/bpf/progs/kfunc_call_test.c | 12 ++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 2 +- 9 files changed, 72 insertions(+), 65 deletions(-)
Nice to drop those 'is_*_kfunc()' helpers in verifier.
BTW, some of those kfuncs can be dropped from the special_kfunc_list. That would be better.
Any way, this series looks good to me.
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 8 lines · On Wed, 5 Aug 2026 23:33:37 +0800 you wrote:
The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a
bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.[...]
Here is the summary with links:
- [bpf-next,v2,1/3] bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock
https://git.kernel.org/bpf/bpf-next/c/ed3b3093b624 - [bpf-next,v2,2/3] bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE https://git.kernel.org/bpf/bpf-next/c/7619a0ee9340 - [bpf-next,v2,3/3] selftests/bpf: Test module kfunc calls under spin lock https://git.kernel.org/bpf/bpf-next/c/bca83aa31f15
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html