Skip to contentSkip to conversation

bpf: Allow selected kfuncs under bpf_spin_lock

PATCH Applied Kaitao Cheng 3 participants 6 messages Aug 5 – Aug 6

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover bpf: Allow selected kfuncs under bpf_spin_lock

The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.

This series adds a new KF_SPINLOCK_SAFE kfunc flag and teaches the verifier to use kfunc registration metadata when deciding whether a kfunc call is allowed while a bpf_spin_lock is held.

The built-in kfuncs that are currently accepted by the verifier's lock-held allowlist are annotated with the new flag. This preserves the existing behavior while removing the verifier-side category checks and uses the same mechanism for built-in and module kfuncs.

The selftest coverage marks one bpf_testmod kfunc as KF_SPINLOCK_SAFE and verifies that it can be called under a bpf_spin_lock. It also calls another registered but unmarked bpf_testmod kfunc under the lock and checks that the verifier rejects it.

Changes in v2:
- Rename KF_SPIN_LOCK to KF_SPINLOCK_SAFE. (Kumar Kartikeya Dwivedi, Leon Hwang)
- Deprecate the verifier's lock-held allowlist mechanism and annotate the relevant kfuncs uniformly with KF_SPINLOCK_SAFE (Kumar Kartikeya Dwivedi)
- Add selftests. (Leon Hwang)

Link to v1:
https://lore.kernel.org/bpf/[email protected]/

Kaitao Cheng (3):
bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock
bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE selftests/bpf: Test module kfunc calls under spin lock

 include/linux/btf.h                           |  1 +
 kernel/bpf/arena.c                            |  6 +-
 kernel/bpf/helpers.c                          | 56 ++++++++++---------
 kernel/bpf/rqspinlock.c                       |  8 +--
 kernel/bpf/verifier.c                         | 38 +++----------
 .../selftests/bpf/prog_tests/kfunc_call.c     |  2 +
 .../selftests/bpf/progs/kfunc_call_fail.c     | 12 ++++
 .../selftests/bpf/progs/kfunc_call_test.c     | 12 ++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  2 +-
 9 files changed, 72 insertions(+), 65 deletions(-)
Signature · 1 line
2.50.1 (Apple Git-155)
#1 lore raw

PATCH v2 1/3 bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock

From: Kaitao Cheng <[email protected]>

Introduce the KF_SPINLOCK_SAFE kfunc metadata flag in BTF so kfuncs may be explicitly marked as safe to call while holding bpf_spin_lock.

Allow kfuncs defined in kernel modules to be marked with KF_SPINLOCK_SAFE.

Example: BTF_ID_FLAGS(func, $kfunc_name, KF_SPINLOCK_SAFE)

---
 include/linux/btf.h   |  1 +
 kernel/bpf/verifier.c | 20 +++++++++++++++-----
 2 files changed, 16 insertions(+), 5 deletions(-)
include/linux/btf.h +1 -0
index c09b7994de4e..3f5255d095a2 100644
@@ -79,6 +79,7 @@
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
 #define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplied by the verifier */
+#define KF_SPINLOCK_SAFE (1 << 17) /* kfunc is allowed inside bpf_spin_lock-ed region */
 
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +15 -5
index b274004fccfd..9c1f8d552655 100644
@@ -11832,11 +11832,21 @@ static bool is_bpf_stream_kfunc(u32 btf_id)
 	       btf_id == special_kfunc_list[KF_bpf_stream_print_stack];
 }
 
-static bool kfunc_spin_allowed(u32 btf_id)
+static bool kfunc_spin_allowed(struct bpf_verifier_env *env, s32 func_id, s16 offset)
 {
-	return is_bpf_graph_api_kfunc(btf_id) || is_bpf_iter_num_api_kfunc(btf_id) ||
-	       is_bpf_res_spin_lock_kfunc(btf_id) || is_bpf_arena_kfunc(btf_id) ||
-	       is_bpf_stream_kfunc(btf_id);
+	struct bpf_kfunc_meta kfunc;
+	int err;
+
+	if (is_bpf_graph_api_kfunc(func_id) || is_bpf_iter_num_api_kfunc(func_id) ||
+	    is_bpf_res_spin_lock_kfunc(func_id) || is_bpf_arena_kfunc(func_id) ||
+	    is_bpf_stream_kfunc(func_id))
+		return true;
+
+	err = fetch_kfunc_meta(env, func_id, offset, &kfunc);
+	if (err || !kfunc.flags)
+		return false;
+
+	return *kfunc.flags & KF_SPINLOCK_SAFE;
 }
 
 static bool is_sync_callback_calling_kfunc(u32 btf_id)
@@ -17415,7 +17425,7 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
 				     insn->imm != BPF_FUNC_spin_unlock &&
 				     insn->imm != BPF_FUNC_kptr_xchg) ||
 				    (insn->src_reg == BPF_PSEUDO_KFUNC_CALL &&
-				     (insn->off != 0 || !kfunc_spin_allowed(insn->imm)))) {
+				     !kfunc_spin_allowed(env, insn->imm, insn->off))) {
 					verbose(env,
 						"function calls are not allowed while holding a lock\n");
 					return -EINVAL;
Signature · 1 line
2.50.1 (Apple Git-155)
#2 lore raw

PATCH v2 2/3 bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE

In reply to Kaitao Cheng

From: Kaitao Cheng <[email protected]>

The verifier currently keeps a hard-coded list of kfuncs that may be called while holding a bpf_spin_lock. With KF_SPINLOCK_SAFE available, retaining this list creates two sources of truth and requires verifier changes whenever another lock-safe kfunc is added.

Mark every kfunc currently accepted by kfunc_spin_allowed() with KF_SPINLOCK_SAFE. This covers the graph, numeric iterator, resource spin lock, arena, and stream kfuncs.

Remove the obsolete category checks and make kfunc_spin_allowed() rely solely on the kfunc registration metadata. This preserves the behavior of existing kfuncs while using the same mechanism for built-in and module kfuncs.

---
 kernel/bpf/arena.c      |  6 ++---
 kernel/bpf/helpers.c    | 56 +++++++++++++++++++++--------------------
 kernel/bpf/rqspinlock.c |  8 +++---
 kernel/bpf/verifier.c   | 32 -----------------------
 4 files changed, 36 insertions(+), 66 deletions(-)
kernel/bpf/arena.c +3 -3
index 555ee2531ef9..7b6847200b43 100644
@@ -1118,9 +1118,9 @@ __bpf_kfunc int bpf_arena_reserve_pages(void *p__map, void *ptr__ign, u32 page_c
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(arena_kfuncs)
-BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2)
-BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2)
-BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2)
+BTF_ID_FLAGS(func, bpf_arena_alloc_pages, KF_ARENA_RET | KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_free_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_arena_reserve_pages, KF_ARENA_ARG2 | KF_SPINLOCK_SAFE)
 BTF_KFUNCS_END(arena_kfuncs)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/helpers.c +29 -27

76 lines not on this page · raw message

kernel/bpf/rqspinlock.c +4 -4
index e4e338cdb437..e527cb425cf4 100644
@@ -744,10 +744,10 @@ __bpf_kfunc void bpf_res_spin_unlock_irqrestore(struct bpf_res_spin_lock *lock,
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(rqspinlock_kfunc_ids)
-BTF_ID_FLAGS(func, bpf_res_spin_lock, KF_RET_NULL)
-BTF_ID_FLAGS(func, bpf_res_spin_unlock)
-BTF_ID_FLAGS(func, bpf_res_spin_lock_irqsave, KF_RET_NULL)
-BTF_ID_FLAGS(func, bpf_res_spin_unlock_irqrestore)
+BTF_ID_FLAGS(func, bpf_res_spin_lock, KF_RET_NULL | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_res_spin_unlock, KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_res_spin_lock_irqsave, KF_RET_NULL | KF_SPINLOCK_SAFE)
+BTF_ID_FLAGS(func, bpf_res_spin_unlock_irqrestore, KF_SPINLOCK_SAFE)
 BTF_KFUNCS_END(rqspinlock_kfunc_ids)
 
 static const struct btf_kfunc_id_set rqspinlock_kfunc_set = {
kernel/bpf/verifier.c +0 -32
index 9c1f8d552655..7c1edcae293a 100644
@@ -11797,20 +11797,6 @@ static bool is_bpf_rbtree_api_kfunc(u32 btf_id)
 	       btf_id == special_kfunc_list[KF_bpf_rbtree_right];
 }
 
-static bool is_bpf_iter_num_api_kfunc(u32 btf_id)
-{
-	return btf_id == special_kfunc_list[KF_bpf_iter_num_new] ||
-	       btf_id == special_kfunc_list[KF_bpf_iter_num_next] ||
-	       btf_id == special_kfunc_list[KF_bpf_iter_num_destroy];
-}
-
-static bool is_bpf_graph_api_kfunc(u32 btf_id)
-{
-	return is_bpf_list_api_kfunc(btf_id) ||
-	       is_bpf_rbtree_api_kfunc(btf_id) ||
-	       is_bpf_refcount_acquire_kfunc(btf_id);
-}
-
 static bool is_bpf_res_spin_lock_kfunc(u32 btf_id)
 {
 	return btf_id == special_kfunc_list[KF_bpf_res_spin_lock] ||
@@ -11819,29 +11805,11 @@ static bool is_bpf_res_spin_lock_kfunc(u32 btf_id)
 	       btf_id == special_kfunc_list[KF_bpf_res_spin_unlock_irqrestore];
 }
 
-static bool is_bpf_arena_kfunc(u32 btf_id)
-{
-	return btf_id == special_kfunc_list[KF_bpf_arena_alloc_pages] ||
-	       btf_id == special_kfunc_list[KF_bpf_arena_free_pages] ||
-	       btf_id == special_kfunc_list[KF_bpf_arena_reserve_pages];
-}
-
-static bool is_bpf_stream_kfunc(u32 btf_id)
-{
-	return btf_id == special_kfunc_list[KF_bpf_stream_vprintk] ||
-	       btf_id == special_kfunc_list[KF_bpf_stream_print_stack];
-}
-
 static bool kfunc_spin_allowed(struct bpf_verifier_env *env, s32 func_id, s16 offset)
 {
 	struct bpf_kfunc_meta kfunc;
 	int err;
 
-	if (is_bpf_graph_api_kfunc(func_id) || is_bpf_iter_num_api_kfunc(func_id) ||
-	    is_bpf_res_spin_lock_kfunc(func_id) || is_bpf_arena_kfunc(func_id) ||
-	    is_bpf_stream_kfunc(func_id))
-		return true;
-
 	err = fetch_kfunc_meta(env, func_id, offset, &kfunc);
 	if (err || !kfunc.flags)
 		return false;
Signature · 1 line
2.50.1 (Apple Git-155)
#3 lore raw

PATCH v2 3/3 selftests/bpf: Test module kfunc calls under spin lock

In reply to Kaitao Cheng

From: Kaitao Cheng <[email protected]>

The verifier uses kfunc registration flags to decide whether a kfunc may be called while a BPF program holds a bpf_spin_lock.

Mark bpf_testmod_test_mod_kfunc() as KF_SPINLOCK_SAFE and verify that it can be called while holding a bpf_spin_lock. Also attempt to call the unmarked bpf_kfunc_trigger_ctx_check() under the lock and verify that the program is rejected.

---
 tools/testing/selftests/bpf/prog_tests/kfunc_call.c  |  2 ++
 tools/testing/selftests/bpf/progs/kfunc_call_fail.c  | 12 ++++++++++++
 tools/testing/selftests/bpf/progs/kfunc_call_test.c  | 12 ++++++++++++
 tools/testing/selftests/bpf/test_kmods/bpf_testmod.c |  2 +-
 4 files changed, 27 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/kfunc_call.c +2 -0
index 7af5560f2a08..2b39cc1b09f9 100644
@@ -71,8 +71,10 @@ static struct kfunc_test_params kfunc_tests[] = {
 	TC_FAIL(kfunc_call_test_get_mem_fail_not_const, 0, "is not a const"),
 	TC_FAIL(kfunc_call_test_mem_acquire_fail, 0, "acquire kernel function does not return PTR_TO_BTF_ID"),
 	TC_FAIL(kfunc_call_test_pointer_arg_type_mismatch, 0, "R1 expected pointer to ctx, but got scalar"),
+	TC_FAIL(kfunc_call_test_spin_lock_unsafe, 0, "function calls are not allowed while holding a lock"),
 
 	/* success cases */
+	TC_TEST(kfunc_call_test_spin_lock_safe, 0),
 	TC_TEST(kfunc_call_test1, 12),
 	TC_TEST(kfunc_call_test2, 3),
 	TC_TEST(kfunc_call_test4, -1234),
tools/testing/selftests/bpf/progs/kfunc_call_fail.c +12 -0
index 64b6a0b0ab1c..7e93f7fb1329 100644
@@ -4,6 +4,18 @@
 #include <bpf/bpf_helpers.h>
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
+static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");
+
+SEC("?tc")
+int kfunc_call_test_spin_lock_unsafe(struct __sk_buff *skb)
+{
+	bpf_spin_lock(&kfunc_call_lock);
+	bpf_kfunc_trigger_ctx_check();
+	bpf_spin_unlock(&kfunc_call_lock);
+
+	return 0;
+}
+
 struct syscall_test_args {
 	__u8 data[16];
 	size_t size;
tools/testing/selftests/bpf/progs/kfunc_call_test.c +12 -0
index 5edc51564f71..8e6560c31e78 100644
@@ -5,6 +5,18 @@
 #include "bpf_misc.h"
 #include "../test_kmods/bpf_testmod_kfunc.h"
 
+static struct bpf_spin_lock kfunc_call_lock SEC(".data.A");
+
+SEC("tc")
+int kfunc_call_test_spin_lock_safe(struct __sk_buff *skb)
+{
+	bpf_spin_lock(&kfunc_call_lock);
+	bpf_testmod_test_mod_kfunc(42);
+	bpf_spin_unlock(&kfunc_call_lock);
+
+	return 0;
+}
+
 SEC("tc")
 int kfunc_call_test5(struct __sk_buff *skb)
 {
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +1 -1
index eb0f9b5e18d8..0585794606ed 100644
@@ -1384,7 +1384,7 @@ __bpf_kfunc void bpf_kfunc_trigger_ctx_check(void)
 }
 
 BTF_KFUNCS_START(bpf_testmod_check_kfunc_ids)
-BTF_ID_FLAGS(func, bpf_testmod_test_mod_kfunc)
+BTF_ID_FLAGS(func, bpf_testmod_test_mod_kfunc, KF_SPINLOCK_SAFE)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test1)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test2)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test3)
Signature · 1 line
2.50.1 (Apple Git-155)
Acked-by on the v2 series
#4 lore raw
In reply to Kaitao Cheng
Quoted 36 lines · Kaitao Cheng

The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.

This series adds a new KF_SPINLOCK_SAFE kfunc flag and teaches the verifier to use kfunc registration metadata when deciding whether a kfunc call is allowed while a bpf_spin_lock is held.

The built-in kfuncs that are currently accepted by the verifier's lock-held allowlist are annotated with the new flag. This preserves the existing behavior while removing the verifier-side category checks and uses the same mechanism for built-in and module kfuncs.

The selftest coverage marks one bpf_testmod kfunc as KF_SPINLOCK_SAFE and verifies that it can be called under a bpf_spin_lock. It also calls another registered but unmarked bpf_testmod kfunc under the lock and checks that the verifier rejects it.

Changes in v2:
- Rename KF_SPIN_LOCK to KF_SPINLOCK_SAFE. (Kumar Kartikeya Dwivedi, Leon Hwang)
- Deprecate the verifier's lock-held allowlist mechanism and annotate the relevant kfuncs uniformly with KF_SPINLOCK_SAFE (Kumar Kartikeya Dwivedi)
- Add selftests. (Leon Hwang)

Link to v1:
https://lore.kernel.org/bpf/[email protected]/

Kaitao Cheng (3):
bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock
bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE selftests/bpf: Test module kfunc calls under spin lock

Quoted 11 lines
 include/linux/btf.h                           |  1 +
 kernel/bpf/arena.c                            |  6 +-
 kernel/bpf/helpers.c                          | 56 ++++++++++---------
 kernel/bpf/rqspinlock.c                       |  8 +--
 kernel/bpf/verifier.c                         | 38 +++----------
 .../selftests/bpf/prog_tests/kfunc_call.c     |  2 +
 .../selftests/bpf/progs/kfunc_call_fail.c     | 12 ++++
 .../selftests/bpf/progs/kfunc_call_test.c     | 12 ++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  2 +-
 9 files changed, 72 insertions(+), 65 deletions(-)

Nice to drop those 'is_*_kfunc()' helpers in verifier.

BTW, some of those kfuncs can be dropped from the special_kfunc_list. That would be better.

Any way, this series looks good to me.

#5 lore raw
In reply to Kaitao Cheng
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 8 lines · On Wed, 5 Aug 2026 23:33:37 +0800 you wrote:

The verifier currently has a hard-coded allowlist of kfuncs that may be called while a BPF program holds a bpf_spin_lock. This works for the small set of built-in kfuncs known to the verifier, but it does not give kfunc providers a registration-time way to declare that a kfunc is safe in such a region. In particular, module kfuncs cannot be added to that allowlist without changing verifier code.

[...]

Here is the summary with links:
- [bpf-next,v2,1/3] bpf: Add KF_SPINLOCK_SAFE flag for kfuncs under bpf_spin_lock

    https://git.kernel.org/bpf/bpf-next/c/ed3b3093b624
  - [bpf-next,v2,2/3] bpf: Mark existing lock-safe kfuncs with KF_SPINLOCK_SAFE
    https://git.kernel.org/bpf/bpf-next/c/7619a0ee9340
  - [bpf-next,v2,3/3] selftests/bpf: Test module kfunc calls under spin lock
    https://git.kernel.org/bpf/bpf-next/c/bca83aa31f15

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html