bpf: Consolidate sleepable context checks in verifier
Subsystems (4)
cover bpf: Consolidate sleepable context checks in verifier
The BPF verifier has multiple call-checking functions that independently validate whether sleepable operations are permitted in the current context. Each function open-codes its own checks against active_rcu_locks, active_preempt_locks, active_irq_id, and in_sleepable, duplicating the logic already provided by in_sleepable_context().
This series consolidates these scattered checks into calls to in_sleepable_context() across check_helper_call(), check_kfunc_call(), and check_func_call(), reducing code duplication and making the error reporting consistent. No functional change.
Puranjay Mohan (3):
bpf: Consolidate sleepable checks in check_helper_call()
bpf: Consolidate sleepable checks in check_kfunc_call()
bpf: Consolidate sleepable checks in check_func_call()
kernel/bpf/verifier.c | 91 +++++++------------ .../selftests/bpf/prog_tests/summarization.c | 2 +- tools/testing/selftests/bpf/progs/irq.c | 4 +- .../selftests/bpf/progs/preempt_lock.c | 6 +- .../bpf/progs/verifier_async_cb_context.c | 4 +- 5 files changed, 43 insertions(+), 64 deletions(-)
base-commit: 77378dabb50f593c756d393d8eacb0b91b758863
Signature · 1 line
2.52.0
PATCH 1/3 bpf: Consolidate sleepable checks in check_helper_call()
check_helper_call() prints the error message for every env->cur_state->active* element when calling a sleepable helper. Consolidate all of them into a single print statement.
The check for env->cur_state->active_locks was not part of the removed print statements and will not be triggered with the consolidated print as well because it is checked in do_check() before check_helper_call() is even reached.
--- kernel/bpf/verifier.c | 44 +++++++------------ .../selftests/bpf/prog_tests/summarization.c | 2 +- .../bpf/progs/verifier_async_cb_context.c | 4 +- 3 files changed, 20 insertions(+), 30 deletions(-)
kernel/bpf/verifier.c +17 -27
@@ -11678,6 +11678,19 @@ static inline bool in_sleepable_context(struct bpf_verifier_env *env)in_sleepable (env); } +static const char *non_sleepable_context_description (struct bpf_verifier_env *env) +{ +if (env->cur_state->active_rcu_locks) +return "rcu_read_lock region" ; +if (env->cur_state->active_preempt_locks) +return "non-preemptible region" ; +if (env->cur_state->active_irq_id) +return "IRQ-disabled region" ; +if (env->cur_state->active_locks) +return "lock region" ; +return "non-sleepable prog" ; +} +static int check_helper_call (struct bpf_verifier_env *env,struct bpf_insn *insn,int *insn_idx_p) {
@@ -11717,11 +11730,6 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insnreturn -EINVAL ; } -if (!in_sleepable (env) && fn->might_sleep) { -verbose (env,"helper call might sleep in a non-sleepable prog\n" ); -return -EINVAL ; - } -/* With LD_ABS/IND some JITs save/restore skb from r1. */ changes_data =bpf_helper_changes_pkt_data (func_id);if (changes_data && fn->arg1_type !=ARG_PTR_TO_CTX ) {
@@ -11738,28 +11746,10 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insnreturn err; } -if (env->cur_state->active_rcu_locks) { -if (fn->might_sleep) { -verbose (env,"sleepable helper %s#%d in rcu_read_lock region\n" , -func_id_name (func_id), func_id); -return -EINVAL ; - } - } - -if (env->cur_state->active_preempt_locks) { -if (fn->might_sleep) { -verbose (env,"sleepable helper %s#%d in non-preemptible region\n" , -func_id_name (func_id), func_id); -return -EINVAL ; - } - } - -if (env->cur_state->active_irq_id) { -if (fn->might_sleep) { -verbose (env,"sleepable helper %s#%d in IRQ-disabled region\n" , -func_id_name (func_id), func_id); -return -EINVAL ; - } +if (fn->might_sleep && !in_sleepable_context (env)) { +verbose (env,"sleepable helper %s#%d in %s\n" ,func_id_name (func_id), func_id, +non_sleepable_context_description (env)); +return -EINVAL ; }/* Track non-sleepable context for helpers. */
tools/testing/selftests/bpf/prog_tests/summarization.c +1 -1
@@ -58,7 +58,7 @@ static void test_aux(const char *main_prog_name,* this particular combination can be enabled. */ if (!strcmp ("might_sleep" , replacement) && err) { -ASSERT_HAS_SUBSTR (log,"helper call might sleep in a non-sleepable prog" ,"error log" ); +ASSERT_HAS_SUBSTR (log,"sleepable helper bpf_copy_from_user#" ,"error log" );ASSERT_EQ (err, -EINVAL ,"err" );test__skip (); goto out;
tools/testing/selftests/bpf/progs/verifier_async_cb_context.c +2 -2
@@ -31,7 +31,7 @@ static int timer_cb(void *map, int *key, struct bpf_timer *timer) } SEC("fentry/bpf_fentry_test1") -__failure __msg("helper call might sleep in a non-sleepable prog") +__failure __msg("sleepable helper bpf_copy_from_user#{{[0-9]+}} in non-sleepable prog") int timer_non_sleepable_prog(void *ctx) {struct timer_elem *val;
@@ -47,7 +47,7 @@ int timer_non_sleepable_prog(void *ctx) } SEC("lsm.s/file_open") -__failure __msg("helper call might sleep in a non-sleepable prog") +__failure __msg("sleepable helper bpf_copy_from_user#{{[0-9]+}} in non-sleepable prog") int timer_sleepable_prog(void *ctx) {struct timer_elem *val;
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 1/3] bpf: Consolidate sleepable checks in check_helper_call()
Quoted 10 lines · Puranjay Mohan
check_helper_call()prints the error message for every env->cur_state->active* element when calling a sleepable helper. Consolidate all of them into a single print statement.The check for
env->cur_state->active_lockswas not part of the removed print statements and will not be triggered with the consolidated print as well because it is checked indo_check()beforecheck_helper_call()is even reached.
Quoted 4 lines
Acked-by Mykyta Yatsenko <[email protected]>Acked-by Eduard Zingerman <[email protected]>Signed-off-by Puranjay Mohan <[email protected]>---
Quoted 1 line
[...]
PATCH 2/3 bpf: Consolidate sleepable checks in check_kfunc_call()
In reply to Puranjay Mohancheck_kfunc_call() has multiple scattered checks that reject sleepable kfuncs in various non-sleepable contexts (RCU, preempt-disabled, IRQ- disabled). These are the same conditions already checked by in_sleepable_context(), so replace them with a single consolidated check.
This also simplifies the preempt lock tracking by flattening the nested if/else structure into a linear chain: preempt_disable increments, preempt_enable checks for underflow and decrements. The sleepable check is kept as a separate block since it is logically distinct from the lock accounting.
No functional change since in_sleepable_context() checks all the same state (active_rcu_locks, active_preempt_locks, active_locks, active_irq_id, in_sleepable).
--- kernel/bpf/verifier.c | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-)
kernel/bpf/verifier.c +13 -23
@@ -14276,34 +14276,24 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, } })); } - }else if (sleepable && env->cur_state->active_rcu_locks) { -verbose (env,"kernel func %s is sleepable within rcu_read_lock region\n" , func_name); -return -EACCES ; - } - -if (in_rbtree_lock_required_cb (env) && (rcu_lock || rcu_unlock)) { -verbose (env,"Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n" ); -return -EACCES ; - } - -if (env->cur_state->active_preempt_locks) { -if (preempt_disable) { - env->cur_state->active_preempt_locks++; - }else if (preempt_enable) { - env->cur_state->active_preempt_locks--; - }else if (sleepable) { -verbose (env,"kernel func %s is sleepable within non-preemptible region\n" , func_name); -return -EACCES ; - } }else if (preempt_disable) { env->cur_state->active_preempt_locks++; }else if (preempt_enable) { -verbose (env,"unmatched attempt to enable preemption (kernel function %s)\n" , func_name); -return -EINVAL ; +if (env->cur_state->active_preempt_locks ==0 ) { +verbose (env,"unmatched attempt to enable preemption (kernel function %s)\n" , func_name); +return -EINVAL ; + } + env->cur_state->active_preempt_locks--; } -if (env->cur_state->active_irq_id && sleepable) { -verbose (env,"kernel func %s is sleepable within IRQ-disabled region\n" , func_name); +if (sleepable && !in_sleepable_context (env)) { +verbose (env,"kernel func %s is sleepable within %s\n" , + func_name,non_sleepable_context_description (env)); +return -EACCES ; + } + +if (in_rbtree_lock_required_cb (env) && (rcu_lock || rcu_unlock)) { +verbose (env,"Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n" );return -EACCES ; }
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 2/3] bpf: Consolidate sleepable checks in check_kfunc_call()
Quoted 13 lines · Puranjay Mohan
check_kfunc_call()has multiple scattered checks that reject sleepable kfuncs in various non-sleepable contexts (RCU, preempt-disabled, IRQ- disabled). These are the same conditions already checked byin_sleepable_context(), so replace them with a single consolidated check.This also simplifies the preempt lock tracking by flattening the nested if/else structure into a linear chain:
preempt_disableincrements,preempt_enablechecks for underflow and decrements. The sleepable check is kept as a separate block since it is logically distinct from the lock accounting.
Quoted 8 lines
No functional change since
in_sleepable_context()checks all the same state (active_rcu_locks,active_preempt_locks,active_locks,active_irq_id,in_sleepable).Acked-by Eduard Zingerman <[email protected]>Acked-by Mykyta Yatsenko <[email protected]>Signed-off-by Puranjay Mohan <[email protected]>---
Quoted 1 line
[...]
PATCH 3/3 bpf: Consolidate sleepable checks in check_func_call()
In reply to Puranjay MohanThe sleepable context check for global function calls in check_func_call() open-codes the same checks that in_sleepable_context() already performs. Replace the open-coded check with a call to in_sleepable_context() and use non_sleepable_context_description() for the error message, consistent with check_helper_call() and check_kfunc_call().
Note that in_sleepable_context() also checks active_locks, which overlaps with the existing active_locks check above it. However, the two checks serve different purposes: the active_locks check rejects all global function calls while holding a lock (not just sleepable ones), so it must remain as a separate guard.
Update the expected error messages in the irq and preempt_lock selftests to match.
--- kernel/bpf/verifier.c | 11 +++++------ tools/testing/selftests/bpf/progs/irq.c | 4 ++-- tools/testing/selftests/bpf/progs/preempt_lock.c | 6 +++--- 3 files changed, 10 insertions(+), 11 deletions(-)
kernel/bpf/verifier.c +5 -6
@@ -210,6 +210,8 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env);static int ref_set_non_owning (struct bpf_verifier_env *env,struct bpf_reg_state *reg);static bool is_trusted_reg (const struct bpf_reg_state *reg); +static inline bool in_sleepable_context (struct bpf_verifier_env *env); +static const char *non_sleepable_context_description (struct bpf_verifier_env *env);static bool bpf_map_ptr_poisoned (const struct bpf_insn_aux_data *aux) {
@@ -10948,12 +10950,9 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,return -EINVAL ; } -if (env->subprog_info[subprog].might_sleep && - (env->cur_state->active_rcu_locks || env->cur_state->active_preempt_locks || - env->cur_state->active_irq_id || !in_sleepable (env))) { -verbose (env,"global functions that may sleep are not allowed in non-sleepable context,\n" -"i.e., in a RCU/IRQ/preempt-disabled section, or in\n" -"a non-sleepable BPF program context\n" ); +if (env->subprog_info[subprog].might_sleep && !in_sleepable_context (env)) { +verbose (env,"sleepable global function %s() called in %s\n" , + sub_name,non_sleepable_context_description (env));return -EINVAL ; }
tools/testing/selftests/bpf/progs/irq.c +2 -2
@@ -490,7 +490,7 @@ int irq_non_sleepable_global_subprog(void *ctx) } SEC("?syscall") -__failure __msg("global functions that may sleep are not allowed in non-sleepable context") +__failure __msg("sleepable global function") int irq_sleepable_helper_global_subprog(void *ctx) {unsigned long flags;
@@ -502,7 +502,7 @@ int irq_sleepable_helper_global_subprog(void *ctx) } SEC("?syscall") -__failure __msg("global functions that may sleep are not allowed in non-sleepable context") +__failure __msg("sleepable global function") int irq_sleepable_global_subprog_indirect(void *ctx) {unsigned long flags;
tools/testing/selftests/bpf/progs/preempt_lock.c +3 -3
@@ -177,7 +177,7 @@ global_subprog_calling_sleepable_global(int i) } SEC("?syscall") -__failure __msg("global functions that may sleep are not allowed in non-sleepable context") +__failure __msg("sleepable global function") int preempt_global_sleepable_helper_subprog(struct __sk_buff *ctx) {preempt_disable ();
@@ -188,7 +188,7 @@ int preempt_global_sleepable_helper_subprog(struct __sk_buff *ctx) } SEC("?syscall") -__failure __msg("global functions that may sleep are not allowed in non-sleepable context") +__failure __msg("sleepable global function") int preempt_global_sleepable_kfunc_subprog(struct __sk_buff *ctx) {preempt_disable ();
@@ -199,7 +199,7 @@ int preempt_global_sleepable_kfunc_subprog(struct __sk_buff *ctx) } SEC("?syscall") -__failure __msg("global functions that may sleep are not allowed in non-sleepable context") +__failure __msg("sleepable global function") int preempt_global_sleepable_subprog_indirect(struct __sk_buff *ctx) {preempt_disable ();
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 3/3] bpf: Consolidate sleepable checks in check_func_call()
Quoted 14 lines · Puranjay Mohan
The sleepable context check for global function calls in
check_func_call()open-codes the same checks thatin_sleepable_context()already performs. Replace the open-coded check with a call toin_sleepable_context()and usenon_sleepable_context_description()for the error message, consistent withcheck_helper_call()andcheck_kfunc_call().Note that
in_sleepable_context()also checksactive_locks, which overlaps with the existingactive_lockscheck above it. However, the two checks serve different purposes: theactive_lockscheck rejects all global function calls while holding a lock (not just sleepable ones), so it must remain as a separate guard.
Quoted 6 lines
Update the expected error messages in the irq and
preempt_lockselftests to match.Acked-by Eduard Zingerman <[email protected]>Signed-off-by Puranjay Mohan <[email protected]>---
Quoted 1 line
[...]
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 6 lines · On Wed, 18 Mar 2026 10:43:23 -0700 you wrote:
The BPF verifier has multiple call-checking functions that independently validate whether sleepable operations are permitted in the current context. Each function open-codes its own checks against
active_rcu_locks,active_preempt_locks,active_irq_id, andin_sleepable, duplicating the logic already provided byin_sleepable_context().
Quoted 6 lines
This series consolidates these scattered checks into calls to
in_sleepable_context()acrosscheck_helper_call(),check_kfunc_call(), andcheck_func_call(), reducing code duplication and making the error reporting consistent. No functional change.[...]
Here is the summary with links:
- [bpf-next,1/3] bpf: Consolidate sleepable checks in check_helper_call()
https://git.kernel.org/bpf/bpf-next/c/a0d06cf102e4 - [bpf-next,2/3] bpf: Consolidate sleepable checks in check_kfunc_call() https://git.kernel.org/bpf/bpf-next/c/cd9840c413e3 - [bpf-next,3/3] bpf: Consolidate sleepable checks in check_func_call() https://git.kernel.org/bpf/bpf-next/c/a2542a91aafd
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html