Skip to contentSkip to conversation

bpf: Consolidate sleepable context checks in verifier

PATCH Applied Puranjay Mohan 3 participants 8 messages Mar 18 – Mar 21

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover bpf: Consolidate sleepable context checks in verifier

The BPF verifier has multiple call-checking functions that independently validate whether sleepable operations are permitted in the current context. Each function open-codes its own checks against active_rcu_locks, active_preempt_locks, active_irq_id, and in_sleepable, duplicating the logic already provided by in_sleepable_context().

This series consolidates these scattered checks into calls to in_sleepable_context() across check_helper_call(), check_kfunc_call(), and check_func_call(), reducing code duplication and making the error reporting consistent. No functional change.

Puranjay Mohan (3):
bpf: Consolidate sleepable checks in check_helper_call()
bpf: Consolidate sleepable checks in check_kfunc_call()
bpf: Consolidate sleepable checks in check_func_call()

 kernel/bpf/verifier.c                         | 91 +++++++------------
 .../selftests/bpf/prog_tests/summarization.c  |  2 +-
 tools/testing/selftests/bpf/progs/irq.c       |  4 +-
 .../selftests/bpf/progs/preempt_lock.c        |  6 +-
 .../bpf/progs/verifier_async_cb_context.c     |  4 +-
 5 files changed, 43 insertions(+), 64 deletions(-)

base-commit: 77378dabb50f593c756d393d8eacb0b91b758863

Signature · 1 line
2.52.0
#1 lore raw

PATCH 1/3 bpf: Consolidate sleepable checks in check_helper_call()

check_helper_call() prints the error message for every env->cur_state->active* element when calling a sleepable helper. Consolidate all of them into a single print statement.

The check for env->cur_state->active_locks was not part of the removed print statements and will not be triggered with the consolidated print as well because it is checked in do_check() before check_helper_call() is even reached.

---
 kernel/bpf/verifier.c                         | 44 +++++++------------
 .../selftests/bpf/prog_tests/summarization.c  |  2 +-
 .../bpf/progs/verifier_async_cb_context.c     |  4 +-
 3 files changed, 20 insertions(+), 30 deletions(-)
kernel/bpf/verifier.c +17 -27
index 01c18f4268de..4e0f6be5d2d5 100644
@@ -11678,6 +11678,19 @@ static inline bool in_sleepable_context(struct bpf_verifier_env *env)
 	       in_sleepable(env);
 }
 
+static const char *non_sleepable_context_description(struct bpf_verifier_env *env)
+{
+	if (env->cur_state->active_rcu_locks)
+		return "rcu_read_lock region";
+	if (env->cur_state->active_preempt_locks)
+		return "non-preemptible region";
+	if (env->cur_state->active_irq_id)
+		return "IRQ-disabled region";
+	if (env->cur_state->active_locks)
+		return "lock region";
+	return "non-sleepable prog";
+}
+
 static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			     int *insn_idx_p)
 {
@@ -11717,11 +11730,6 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 		return -EINVAL;
 	}
 
-	if (!in_sleepable(env) && fn->might_sleep) {
-		verbose(env, "helper call might sleep in a non-sleepable prog\n");
-		return -EINVAL;
-	}
-
 	/* With LD_ABS/IND some JITs save/restore skb from r1. */
 	changes_data = bpf_helper_changes_pkt_data(func_id);
 	if (changes_data && fn->arg1_type != ARG_PTR_TO_CTX) {
@@ -11738,28 +11746,10 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 		return err;
 	}
 
-	if (env->cur_state->active_rcu_locks) {
-		if (fn->might_sleep) {
-			verbose(env, "sleepable helper %s#%d in rcu_read_lock region\n",
-				func_id_name(func_id), func_id);
-			return -EINVAL;
-		}
-	}
-
-	if (env->cur_state->active_preempt_locks) {
-		if (fn->might_sleep) {
-			verbose(env, "sleepable helper %s#%d in non-preemptible region\n",
-				func_id_name(func_id), func_id);
-			return -EINVAL;
-		}
-	}
-
-	if (env->cur_state->active_irq_id) {
-		if (fn->might_sleep) {
-			verbose(env, "sleepable helper %s#%d in IRQ-disabled region\n",
-				func_id_name(func_id), func_id);
-			return -EINVAL;
-		}
+	if (fn->might_sleep && !in_sleepable_context(env)) {
+		verbose(env, "sleepable helper %s#%d in %s\n", func_id_name(func_id), func_id,
+			non_sleepable_context_description(env));
+		return -EINVAL;
 	}
 
 	/* Track non-sleepable context for helpers. */
tools/testing/selftests/bpf/prog_tests/summarization.c +1 -1
index 5dd6c120a838..6951786044ca 100644
@@ -58,7 +58,7 @@ static void test_aux(const char *main_prog_name,
 	 * this particular combination can be enabled.
 	 */
 	if (!strcmp("might_sleep", replacement) && err) {
-		ASSERT_HAS_SUBSTR(log, "helper call might sleep in a non-sleepable prog", "error log");
+		ASSERT_HAS_SUBSTR(log, "sleepable helper bpf_copy_from_user#", "error log");
 		ASSERT_EQ(err, -EINVAL, "err");
 		test__skip();
 		goto out;
tools/testing/selftests/bpf/progs/verifier_async_cb_context.c +2 -2
index 39aff82549c9..6bf95550a024 100644
@@ -31,7 +31,7 @@ static int timer_cb(void *map, int *key, struct bpf_timer *timer)
 }
 
 SEC("fentry/bpf_fentry_test1")
-__failure __msg("helper call might sleep in a non-sleepable prog")
+__failure __msg("sleepable helper bpf_copy_from_user#{{[0-9]+}} in non-sleepable prog")
 int timer_non_sleepable_prog(void *ctx)
 {
 	struct timer_elem *val;
@@ -47,7 +47,7 @@ int timer_non_sleepable_prog(void *ctx)
 }
 
 SEC("lsm.s/file_open")
-__failure __msg("helper call might sleep in a non-sleepable prog")
+__failure __msg("sleepable helper bpf_copy_from_user#{{[0-9]+}} in non-sleepable prog")
 int timer_sleepable_prog(void *ctx)
 {
 	struct timer_elem *val;
Signature · 1 line
2.52.0
#3 lore raw

PATCH 2/3 bpf: Consolidate sleepable checks in check_kfunc_call()

In reply to Puranjay Mohan

check_kfunc_call() has multiple scattered checks that reject sleepable kfuncs in various non-sleepable contexts (RCU, preempt-disabled, IRQ- disabled). These are the same conditions already checked by in_sleepable_context(), so replace them with a single consolidated check.

This also simplifies the preempt lock tracking by flattening the nested if/else structure into a linear chain: preempt_disable increments, preempt_enable checks for underflow and decrements. The sleepable check is kept as a separate block since it is logically distinct from the lock accounting.

No functional change since in_sleepable_context() checks all the same state (active_rcu_locks, active_preempt_locks, active_locks, active_irq_id, in_sleepable).

---
 kernel/bpf/verifier.c | 36 +++++++++++++-----------------------
 1 file changed, 13 insertions(+), 23 deletions(-)
kernel/bpf/verifier.c +13 -23
index 4e0f6be5d2d5..a9130c4888f7 100644
@@ -14276,34 +14276,24 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 				}
 			}));
 		}
-	} else if (sleepable && env->cur_state->active_rcu_locks) {
-		verbose(env, "kernel func %s is sleepable within rcu_read_lock region\n", func_name);
-		return -EACCES;
-	}
-
-	if (in_rbtree_lock_required_cb(env) && (rcu_lock || rcu_unlock)) {
-		verbose(env, "Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n");
-		return -EACCES;
-	}
-
-	if (env->cur_state->active_preempt_locks) {
-		if (preempt_disable) {
-			env->cur_state->active_preempt_locks++;
-		} else if (preempt_enable) {
-			env->cur_state->active_preempt_locks--;
-		} else if (sleepable) {
-			verbose(env, "kernel func %s is sleepable within non-preemptible region\n", func_name);
-			return -EACCES;
-		}
 	} else if (preempt_disable) {
 		env->cur_state->active_preempt_locks++;
 	} else if (preempt_enable) {
-		verbose(env, "unmatched attempt to enable preemption (kernel function %s)\n", func_name);
-		return -EINVAL;
+		if (env->cur_state->active_preempt_locks == 0) {
+			verbose(env, "unmatched attempt to enable preemption (kernel function %s)\n", func_name);
+			return -EINVAL;
+		}
+		env->cur_state->active_preempt_locks--;
 	}
 
-	if (env->cur_state->active_irq_id && sleepable) {
-		verbose(env, "kernel func %s is sleepable within IRQ-disabled region\n", func_name);
+	if (sleepable && !in_sleepable_context(env)) {
+		verbose(env, "kernel func %s is sleepable within %s\n",
+			func_name, non_sleepable_context_description(env));
+		return -EACCES;
+	}
+
+	if (in_rbtree_lock_required_cb(env) && (rcu_lock || rcu_unlock)) {
+		verbose(env, "Calling bpf_rcu_read_{lock,unlock} in unnecessary rbtree callback\n");
 		return -EACCES;
 	}
 
Signature · 1 line
2.52.0
#5 lore raw

PATCH 3/3 bpf: Consolidate sleepable checks in check_func_call()

In reply to Puranjay Mohan

The sleepable context check for global function calls in check_func_call() open-codes the same checks that in_sleepable_context() already performs. Replace the open-coded check with a call to in_sleepable_context() and use non_sleepable_context_description() for the error message, consistent with check_helper_call() and check_kfunc_call().

Note that in_sleepable_context() also checks active_locks, which overlaps with the existing active_locks check above it. However, the two checks serve different purposes: the active_locks check rejects all global function calls while holding a lock (not just sleepable ones), so it must remain as a separate guard.

Update the expected error messages in the irq and preempt_lock selftests to match.

---
 kernel/bpf/verifier.c                            | 11 +++++------
 tools/testing/selftests/bpf/progs/irq.c          |  4 ++--
 tools/testing/selftests/bpf/progs/preempt_lock.c |  6 +++---
 3 files changed, 10 insertions(+), 11 deletions(-)
kernel/bpf/verifier.c +5 -6
index a9130c4888f7..80a9eab79cac 100644
@@ -210,6 +210,8 @@ static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env);
 static int ref_set_non_owning(struct bpf_verifier_env *env,
 			      struct bpf_reg_state *reg);
 static bool is_trusted_reg(const struct bpf_reg_state *reg);
+static inline bool in_sleepable_context(struct bpf_verifier_env *env);
+static const char *non_sleepable_context_description(struct bpf_verifier_env *env);
 
 static bool bpf_map_ptr_poisoned(const struct bpf_insn_aux_data *aux)
 {
@@ -10948,12 +10950,9 @@ static int check_func_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			return -EINVAL;
 		}
 
-		if (env->subprog_info[subprog].might_sleep &&
-		    (env->cur_state->active_rcu_locks || env->cur_state->active_preempt_locks ||
-		     env->cur_state->active_irq_id || !in_sleepable(env))) {
-			verbose(env, "global functions that may sleep are not allowed in non-sleepable context,\n"
-				     "i.e., in a RCU/IRQ/preempt-disabled section, or in\n"
-				     "a non-sleepable BPF program context\n");
+		if (env->subprog_info[subprog].might_sleep && !in_sleepable_context(env)) {
+			verbose(env, "sleepable global function %s() called in %s\n",
+				sub_name, non_sleepable_context_description(env));
 			return -EINVAL;
 		}
 
tools/testing/selftests/bpf/progs/irq.c +2 -2
index 74d912b22de9..e11e82d98904 100644
@@ -490,7 +490,7 @@ int irq_non_sleepable_global_subprog(void *ctx)
 }
 
 SEC("?syscall")
-__failure __msg("global functions that may sleep are not allowed in non-sleepable context")
+__failure __msg("sleepable global function")
 int irq_sleepable_helper_global_subprog(void *ctx)
 {
 	unsigned long flags;
@@ -502,7 +502,7 @@ int irq_sleepable_helper_global_subprog(void *ctx)
 }
 
 SEC("?syscall")
-__failure __msg("global functions that may sleep are not allowed in non-sleepable context")
+__failure __msg("sleepable global function")
 int irq_sleepable_global_subprog_indirect(void *ctx)
 {
 	unsigned long flags;
tools/testing/selftests/bpf/progs/preempt_lock.c +3 -3
index 7d04254e61f1..6d5fce7e6ffc 100644
@@ -177,7 +177,7 @@ global_subprog_calling_sleepable_global(int i)
 }
 
 SEC("?syscall")
-__failure __msg("global functions that may sleep are not allowed in non-sleepable context")
+__failure __msg("sleepable global function")
 int preempt_global_sleepable_helper_subprog(struct __sk_buff *ctx)
 {
 	preempt_disable();
@@ -188,7 +188,7 @@ int preempt_global_sleepable_helper_subprog(struct __sk_buff *ctx)
 }
 
 SEC("?syscall")
-__failure __msg("global functions that may sleep are not allowed in non-sleepable context")
+__failure __msg("sleepable global function")
 int preempt_global_sleepable_kfunc_subprog(struct __sk_buff *ctx)
 {
 	preempt_disable();
@@ -199,7 +199,7 @@ int preempt_global_sleepable_kfunc_subprog(struct __sk_buff *ctx)
 }
 
 SEC("?syscall")
-__failure __msg("global functions that may sleep are not allowed in non-sleepable context")
+__failure __msg("sleepable global function")
 int preempt_global_sleepable_subprog_indirect(struct __sk_buff *ctx)
 {
 	preempt_disable();
Signature · 1 line
2.52.0
maintainer Acked-by on v1 1/3
#2 lore raw

Re: [PATCH bpf-next 1/3] bpf: Consolidate sleepable checks in check_helper_call()

In reply to Puranjay Mohan
Quoted 10 lines · Puranjay Mohan

check_helper_call() prints the error message for every env->cur_state->active* element when calling a sleepable helper. Consolidate all of them into a single print statement.

The check for env->cur_state->active_locks was not part of the removed print statements and will not be triggered with the consolidated print as well because it is checked in do_check() before check_helper_call() is even reached.

Quoted 4 lines

---

Quoted 1 line

[...]

maintainer Acked-by on v1 2/3
#4 lore raw

Re: [PATCH bpf-next 2/3] bpf: Consolidate sleepable checks in check_kfunc_call()

In reply to Puranjay Mohan
Quoted 13 lines · Puranjay Mohan

check_kfunc_call() has multiple scattered checks that reject sleepable kfuncs in various non-sleepable contexts (RCU, preempt-disabled, IRQ- disabled). These are the same conditions already checked by in_sleepable_context(), so replace them with a single consolidated check.

This also simplifies the preempt lock tracking by flattening the nested if/else structure into a linear chain: preempt_disable increments, preempt_enable checks for underflow and decrements. The sleepable check is kept as a separate block since it is logically distinct from the lock accounting.

Quoted 8 lines

No functional change since in_sleepable_context() checks all the same state (active_rcu_locks, active_preempt_locks, active_locks, active_irq_id, in_sleepable).

---

Quoted 1 line

[...]

maintainer Acked-by on v1 3/3
#6 lore raw

Re: [PATCH bpf-next 3/3] bpf: Consolidate sleepable checks in check_func_call()

In reply to Puranjay Mohan
Quoted 14 lines · Puranjay Mohan

The sleepable context check for global function calls in check_func_call() open-codes the same checks that in_sleepable_context() already performs. Replace the open-coded check with a call to in_sleepable_context() and use non_sleepable_context_description() for the error message, consistent with check_helper_call() and check_kfunc_call().

Note that in_sleepable_context() also checks active_locks, which overlaps with the existing active_locks check above it. However, the two checks serve different purposes: the active_locks check rejects all global function calls while holding a lock (not just sleepable ones), so it must remain as a separate guard.

Quoted 6 lines

Update the expected error messages in the irq and preempt_lock selftests to match.

---

Quoted 1 line

[...]

#7 lore raw
In reply to Puranjay Mohan
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 6 lines · On Wed, 18 Mar 2026 10:43:23 -0700 you wrote:

The BPF verifier has multiple call-checking functions that independently validate whether sleepable operations are permitted in the current context. Each function open-codes its own checks against active_rcu_locks, active_preempt_locks, active_irq_id, and in_sleepable, duplicating the logic already provided by in_sleepable_context().

Quoted 6 lines

This series consolidates these scattered checks into calls to in_sleepable_context() across check_helper_call(), check_kfunc_call(), and check_func_call(), reducing code duplication and making the error reporting consistent. No functional change.

[...]

Here is the summary with links:
- [bpf-next,1/3] bpf: Consolidate sleepable checks in check_helper_call()

    https://git.kernel.org/bpf/bpf-next/c/a0d06cf102e4
  - [bpf-next,2/3] bpf: Consolidate sleepable checks in check_kfunc_call()
    https://git.kernel.org/bpf/bpf-next/c/cd9840c413e3
  - [bpf-next,3/3] bpf: Consolidate sleepable checks in check_func_call()
    https://git.kernel.org/bpf/bpf-next/c/a2542a91aafd

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html