bpf: Allow bpf_for/bpf_repeat while holding spin
Subsystems (4)
Emil Tsalapatis
reviewerv2 cover bpf: Allow bpf_for/bpf_repeat while holding spin
From: Emil Tsalapatis (Meta) <[email protected]>
In BPF programs, kfunc calls while holding a lock are not allowed because kfuncs may sleep by default. The exception to this rule are the functions in special_kfunc_list, which are guaranteed to not sleep. The bpf_iter_num_* functions used by the bpf_for and bpf_repeat macros make no function calls themselves, and as such are guaranteed to not sleep. Add them to special_kfunc_list to allow them within BPF spinlock critical sections.
Emil Tsalapatis (2):
bpf: Allow bpf_for/bpf_repeat calls while holding a spinlock selftests/bpf: test bpf_for within spin lock section
kernel/bpf/verifier.c | 20 +++++++++++++- .../selftests/bpf/progs/verifier_spin_lock.c | 26 +++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-)
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v2 1/2 bpf: Allow bpf_for/bpf_repeat calls while holding a spinlock
Add the bpf_iter_num_* kfuncs called by bpf_for in special_kfunc_list, and allow the calls even while holding a spin lock.
--- kernel/bpf/verifier.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +19 -1
@@ -11690,6 +11690,9 @@ enum special_kfunc_type { KF_bpf_get_kmem_cache, KF_bpf_local_irq_save, KF_bpf_local_irq_restore, + KF_bpf_iter_num_new, + KF_bpf_iter_num_next, + KF_bpf_iter_num_destroy, };BTF_SET_START (special_kfunc_set)
@@ -11765,6 +11768,9 @@ BTF_ID_UNUSED BTF_ID(func, bpf_get_kmem_cache) BTF_ID(func, bpf_local_irq_save) BTF_ID(func, bpf_local_irq_restore) +BTF_ID(func, bpf_iter_num_new) +BTF_ID(func, bpf_iter_num_next) +BTF_ID(func, bpf_iter_num_destroy)static bool is_kfunc_ret_null (struct bpf_kfunc_call_arg_meta *meta) {
@@ -12151,12 +12157,24 @@ static bool is_bpf_rbtree_api_kfunc(u32 btf_id) btf_id == special_kfunc_list[KF_bpf_rbtree_first]; } +static bool is_bpf_iter_num_api_kfunc (u32 btf_id) +{ +return btf_id == special_kfunc_list[KF_bpf_iter_num_new] || + btf_id == special_kfunc_list[KF_bpf_iter_num_next] || + btf_id == special_kfunc_list[KF_bpf_iter_num_destroy]; +} +static bool is_bpf_graph_api_kfunc (u32 btf_id) {return is_bpf_list_api_kfunc (btf_id) ||is_bpf_rbtree_api_kfunc (btf_id) || btf_id == special_kfunc_list[KF_bpf_refcount_acquire_impl]; } +static bool kfunc_spin_allowed (u32 btf_id) +{ +return is_bpf_graph_api_kfunc (btf_id) ||is_bpf_iter_num_api_kfunc (btf_id); +} +static bool is_sync_callback_calling_kfunc (u32 btf_id) {return btf_id == special_kfunc_list[KF_bpf_rbtree_add_impl];
@@ -19048,7 +19066,7 @@ static int do_check(struct bpf_verifier_env *env)if (env->cur_state->active_locks) {if ((insn->src_reg ==BPF_REG_0 && insn->imm != BPF_FUNC_spin_unlock) || (insn->src_reg ==BPF_PSEUDO_KFUNC_CALL && - (insn->off !=0 || !is_bpf_graph_api_kfunc (insn->imm)))) { + (insn->off !=0 || !kfunc_spin_allowed (insn->imm)))) {verbose (env,"function calls are not allowed while holding a lock\n" );return -EINVAL ; }
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v2 2/2 selftests/bpf: test bpf_for within spin lock section
In reply to Emil TsalapatisAdd a selftest to ensure BPF for loops within critical sections are accepted by the verifier.
--- .../selftests/bpf/progs/verifier_spin_lock.c | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+)
tools/testing/selftests/bpf/progs/verifier_spin_lock.c +26 -0
@@ -530,4 +530,30 @@ l1_%=: exit; \ : __clobber_all); } +SEC("tc") +__description("spin_lock: loop within a locked region") +__success __failure_unpriv __msg_unpriv("") +__retval(0) +int bpf_loop_inside_locked_region(void) +{ +const int zero =0 ; +struct val *val; +int i, j =0 ; + + val =bpf_map_lookup_elem (&map_spin_lock, &zero); +if (!val) +return -1 ; + +bpf_spin_lock (&val->l); + bpf_for(i, 0, 10) { + j++; +/* Silence "unused variable" warnings. */ +if (j ==10 ) +break ; + } +bpf_spin_unlock (&val->l); + +return 0 ; +} + char _license[] SEC("license") = "GPL";
Signature · 1 line
2.47.1
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 11 lines · On Sat, 4 Jan 2025 15:25:26 -0500 you wrote:
From: Emil Tsalapatis (Meta) <[email protected]>
In BPF programs, kfunc calls while holding a lock are not allowed because kfuncs may sleep by default. The exception to this rule are the functions in
special_kfunc_list, which are guaranteed to not sleep. The bpf_iter_num_* functions used by thebpf_forandbpf_repeatmacros make no function calls themselves, and as such are guaranteed to not sleep. Add them tospecial_kfunc_listto allow them within BPF spinlock critical sections.[...]
Here is the summary with links:
- [v2,1/2] bpf: Allow bpf_for/bpf_repeat calls while holding a spinlock
https://git.kernel.org/bpf/bpf-next/c/512816403ece - [v2,2/2] selftests/bpf: test bpf_for within spin lock section https://git.kernel.org/bpf/bpf-next/c/87091dd986db
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html