Skip to contentSkip to conversation

Fix sleepable context tracking for async callbacks

PATCH Superseded Kumar Kartikeya Dwivedi 2 participants 14 messages Oct 7, 2025

Subsystems (4)
Kselftest BPF BPF selftests BPF core

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

cover Fix sleepable context tracking for async callbacks

Currently, asynchronous execution primitives set up their callback execution simulation using push_async_cb, which will end up inheriting the sleepable or non-sleepable bit from the program triggering the simulation of the callback. This is incorrect, as the actual execution context of the asynchronous callback has nothing to do with the program arming its execution.

This set fixes this oversight, and supplies a few test cases ensuring the correct behavior is tested across different types of primitives (i.e. timer, wq, task_work).

While looking at this bug, it was noticed that the GFP flag setting logic for storage_get helpers is also broken, hence fix it while we are at it.

PSA: These fixes and unit tests were primarily produced by prompting an AI assistant (Claude), and then modified in minor ways, in an exercise to understand how useful it can be at general kernel development tasks.

Kumar Kartikeya Dwivedi (3):
bpf: Fix sleepable context for async callbacks
bpf: Fix GFP flags for non-sleepable async callbacks selftests/bpf: Add tests for async cb context

 kernel/bpf/verifier.c                         |  41 ++--
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_async_cb_context.c     | 181 ++++++++++++++++++
 3 files changed, 213 insertions(+), 11 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_async_cb_context.c

base-commit: de7342228b7343774d6a9981c2ddbfb5e201044b

Signature · 1 line
2.51.0
author maintainer
#1 lore raw

PATCH 1/3 bpf: Fix sleepable context for async callbacks

Fix the BPF verifier to correctly determine the sleepable context of async callbacks based on the async primitive type rather than the arming program's context.

The bug is in in_sleepable() which uses OR logic to check if the current execution context is sleepable. When a sleepable program arms a timer callback, the callback's state correctly has in_sleepable=false, but in_sleepable() would still return true due to env->prog->sleepable being true. This incorrectly allows sleepable helpers like bpf_copy_from_user() inside timer callbacks when armed from sleepable programs, even though timer callbacks always execute in non-sleepable context.

Fix in_sleepable() to rely solely on env->cur_state->in_sleepable, and initialize state->in_sleepable to env->prog->sleepable in do_check_common() for the main program entry. This ensures the sleepable context is properly tracked per verification state rather than being overridden by the program's sleepability.

The env->cur_state NULL check in in_sleepable() was only needed for do_misc_fixups() which runs after verification when env->cur_state is set to NULL. Update do_misc_fixups() to use env->prog->sleepable directly for the storage_get_function check, and remove the redundant NULL check from in_sleepable().

Introduce is_async_cb_sleepable() helper to explicitly determine async callback sleepability based on the primitive type:
- bpf_timer callbacks are never sleepable
- bpf_wq and bpf_task_work callbacks are always sleepable

Add verifier_bug() check to catch unhandled async callback types, ensuring future additions cannot be silently mishandled. Move the is_task_work_add_kfunc() forward declaration to the top alongside other callback-related helpers.

Finally, update push_async_cb() to adjust to the new changes.

Fixes 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 kernel/bpf/verifier.c | 27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
kernel/bpf/verifier.c +20 -7
index ff40e5e65c43..eff81ad182c8 100644
@@ -515,6 +515,7 @@ static bool is_callback_calling_kfunc(u32 btf_id);
 static bool is_bpf_throw_kfunc(struct bpf_insn *insn);
 
 static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id);
+static bool is_task_work_add_kfunc(u32 func_id);
 
 static bool is_sync_callback_calling_function(enum bpf_func_id func_id)
 {

@@ -547,6 +548,21 @@ static bool is_async_callback_calling_insn(struct bpf_insn *insn)
 	       (bpf_pseudo_kfunc_call(insn) && is_async_callback_calling_kfunc(insn->imm));
 }
 
+static bool is_async_cb_sleepable(struct bpf_verifier_env *env, struct bpf_insn *insn)
+{
+	/* bpf_timer callbacks are never sleepable. */
+	if (bpf_helper_call(insn) && insn->imm == BPF_FUNC_timer_set_callback)
+		return false;
+
+	/* bpf_wq and bpf_task_work callbacks are always sleepable. */
+	if (bpf_pseudo_kfunc_call(insn) && insn->off == 0 &&
+	    (is_bpf_wq_set_callback_impl_kfunc(insn->imm) || is_task_work_add_kfunc(insn->imm)))
+		return true;
+
+	verifier_bug(env, "unhandled async callback in is_async_cb_sleepable");
+	return false;
+}
+
 static bool is_may_goto_insn(struct bpf_insn *insn)
 {
 	return insn->code == (BPF_JMP | BPF_JCOND) && insn->src_reg == BPF_MAY_GOTO;

@@ -5826,8 +5842,7 @@ static int map_kptr_match_type(struct bpf_verifier_env *env,
 
 static bool in_sleepable(struct bpf_verifier_env *env)
 {
-	return env->prog->sleepable ||
-	       (env->cur_state && env->cur_state->in_sleepable);
+	return env->cur_state->in_sleepable;
 }
 
 /* The non-sleepable programs and sleepable programs with explicit bpf_rcu_read_lock()

@@ -10366,8 +10381,6 @@ typedef int (*set_callee_state_fn)(struct bpf_verifier_env *env,
 				   struct bpf_func_state *callee,
 				   int insn_idx);
 
-static bool is_task_work_add_kfunc(u32 func_id);
-
 static int set_callee_state(struct bpf_verifier_env *env,
 			    struct bpf_func_state *caller,
 			    struct bpf_func_state *callee, int insn_idx);

@@ -10586,8 +10599,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 		env->subprog_info[subprog].is_async_cb = true;
 		async_cb = push_async_cb(env, env->subprog_info[subprog].start,
 					 insn_idx, subprog,
-					 is_bpf_wq_set_callback_impl_kfunc(insn->imm) ||
-					 is_task_work_add_kfunc(insn->imm));
+					 is_async_cb_sleepable(env, insn));
 		if (!async_cb)
 			return -EFAULT;
 		callee = async_cb->frame[0];

@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
 		}
 
 		if (is_storage_get_function(insn->imm)) {
-			if (!in_sleepable(env) ||
+			if (!env->prog->sleepable ||
 			    env->insn_aux_data[i + delta].storage_get_func_atomic)
 				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
 			else

@@ -23154,6 +23166,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog)
 	state->curframe = 0;
 	state->speculative = false;
 	state->branches = 1;
+	state->in_sleepable = env->prog->sleepable;
 	state->frame[0] = kzalloc(sizeof(struct bpf_func_state), GFP_KERNEL_ACCOUNT);
 	if (!state->frame[0]) {
 		kfree(state);
Signature · 1 line
2.51.0
maintainer Acked-by on v1 1/3
#2 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix sleepable context for async callbacks

Quoted 35 lines · Kumar Kartikeya Dwivedi

Fix the BPF verifier to correctly determine the sleepable context of async callbacks based on the async primitive type rather than the arming program's context.

The bug is in in_sleepable() which uses OR logic to check if the current execution context is sleepable. When a sleepable program arms a timer callback, the callback's state correctly has in_sleepable=false, but in_sleepable() would still return true due to env->prog->sleepable being true. This incorrectly allows sleepable helpers like bpf_copy_from_user() inside timer callbacks when armed from sleepable programs, even though timer callbacks always execute in non-sleepable context.

Fix in_sleepable() to rely solely on env->cur_state->in_sleepable, and initialize state->in_sleepable to env->prog->sleepable in do_check_common() for the main program entry. This ensures the sleepable context is properly tracked per verification state rather than being overridden by the program's sleepability.

The env->cur_state NULL check in in_sleepable() was only needed for do_misc_fixups() which runs after verification when env->cur_state is set to NULL. Update do_misc_fixups() to use env->prog->sleepable directly for the storage_get_function check, and remove the redundant NULL check from in_sleepable().

Introduce is_async_cb_sleepable() helper to explicitly determine async callback sleepability based on the primitive type:
- bpf_timer callbacks are never sleepable
- bpf_wq and bpf_task_work callbacks are always sleepable

Add verifier_bug() check to catch unhandled async callback types, ensuring future additions cannot be silently mishandled. Move the is_task_work_add_kfunc() forward declaration to the top alongside other callback-related helpers.

Quoted 5 lines

Finally, update push_async_cb() to adjust to the new changes.

Fixes 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>

---

Acked-by Eduard Zingerman <[email protected]>

[...]

kernel/bpf/verifier.c static int do_misc_fixups(struct bpf_verifier_env *env) · line 22498

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
 		}
 
 		if (is_storage_get_function(insn->imm)) {
-			if (!in_sleepable(env) ||
+			if (!env->prog->sleepable ||

This is not exactly correct.
I think that this and the second patch need to be squashed.

Quoted 3 lines
kernel/bpf/verifier.c +0 -0
 			    env->insn_aux_data[i + delta].storage_get_func_atomic)
 				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
 			else

[...]

author maintainer
#3 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix sleepable context for async callbacks

Quoted 50 lines · Eduard Zingerman
Quoted 35 lines · Kumar Kartikeya Dwivedi

Fix the BPF verifier to correctly determine the sleepable context of async callbacks based on the async primitive type rather than the arming program's context.

The bug is in in_sleepable() which uses OR logic to check if the current execution context is sleepable. When a sleepable program arms a timer callback, the callback's state correctly has in_sleepable=false, but in_sleepable() would still return true due to env->prog->sleepable being true. This incorrectly allows sleepable helpers like bpf_copy_from_user() inside timer callbacks when armed from sleepable programs, even though timer callbacks always execute in non-sleepable context.

Fix in_sleepable() to rely solely on env->cur_state->in_sleepable, and initialize state->in_sleepable to env->prog->sleepable in do_check_common() for the main program entry. This ensures the sleepable context is properly tracked per verification state rather than being overridden by the program's sleepability.

The env->cur_state NULL check in in_sleepable() was only needed for do_misc_fixups() which runs after verification when env->cur_state is set to NULL. Update do_misc_fixups() to use env->prog->sleepable directly for the storage_get_function check, and remove the redundant NULL check from in_sleepable().

Introduce is_async_cb_sleepable() helper to explicitly determine async callback sleepability based on the primitive type:
- bpf_timer callbacks are never sleepable
- bpf_wq and bpf_task_work callbacks are always sleepable

Add verifier_bug() check to catch unhandled async callback types, ensuring future additions cannot be silently mishandled. Move the is_task_work_add_kfunc() forward declaration to the top alongside other callback-related helpers.

Quoted 5 lines

Finally, update push_async_cb() to adjust to the new changes.

Fixes 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>

---

Acked-by Eduard Zingerman <[email protected]>

[...]

Quoted 3 lines
(quoted diff) +0 -0
@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
              }
Quoted 6 lines

kernel/bpf/verifier.c static int do_misc_fixups(struct bpf_verifier_env *env) · line 22498

Quoted 3 lines
kernel/bpf/verifier.c +1 -1
              if (is_storage_get_function(insn->imm)) {
-                     if (!in_sleepable(env) ||
+                     if (!env->prog->sleepable ||

This is not exactly correct.
I think that this and the second patch need to be squashed.

I was mostly trying to reduce it to what it would evaluate to. env->cur_state is always false, so the only check that matters is this one. And we fix it separately in the next one. Unless I missed something.

Quoted 6 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
                          env->insn_aux_data[i + delta].storage_get_func_atomic)
                              insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
                      else

[...]

maintainer
#4 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix sleepable context for async callbacks

On Tue, 2025-10-07 at 21:14 +0200, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/verifier.c static int do_misc_fixups(struct bpf_verifier_env *env) · line 22498

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
              }
 
              if (is_storage_get_function(insn->imm)) {
-                     if (!in_sleepable(env) ||
+                     if (!env->prog->sleepable ||

This is not exactly correct.
I think that this and the second patch need to be squashed.

Quoted 3 lines

I was mostly trying to reduce it to what it would evaluate to. env->cur_state is always false, so the only check that matters is this one. And we fix it separately in the next one. Unless I missed something.

Well, yes, but that is not a complete fix, you need a second patch in any case, right?

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
                          env->insn_aux_data[i + delta].storage_get_func_atomic)
                              insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
                      else

[...]

author maintainer
#5 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix sleepable context for async callbacks

Quoted 15 lines · Eduard Zingerman

On Tue, 2025-10-07 at 21:14 +0200, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/verifier.c static int do_misc_fixups(struct bpf_verifier_env *env) · line 22498

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
              }
 
              if (is_storage_get_function(insn->imm)) {
-                     if (!in_sleepable(env) ||
+                     if (!env->prog->sleepable ||

This is not exactly correct.
I think that this and the second patch need to be squashed.

Quoted 7 lines
Quoted 3 lines

I was mostly trying to reduce it to what it would evaluate to. env->cur_state is always false, so the only check that matters is this one. And we fix it separately in the next one. Unless I missed something.

Well, yes, but that is not a complete fix, you need a second patch in any case, right?

Yeah, but I meant it's an orthogonal problem wrt GFP flags. I can actually squash both into the same patch and add two fixes tags then to the commit log, and expand the message a bit, I don't have a preference here.
Let me know if that sounds good.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
                          env->insn_aux_data[i + delta].storage_get_func_atomic)
                              insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
                      else

[...]

maintainer
#6 lore raw

Re: [PATCH bpf-next v1 1/3] bpf: Fix sleepable context for async callbacks

Quoted 24 lines · Kumar Kartikeya Dwivedi
Quoted 15 lines · Eduard Zingerman

On Tue, 2025-10-07 at 21:14 +0200, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/verifier.c static int do_misc_fixups(struct bpf_verifier_env *env) · line 22498

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -22483,7 +22495,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
              }
 
              if (is_storage_get_function(insn->imm)) {
-                     if (!in_sleepable(env) ||
+                     if (!env->prog->sleepable ||

This is not exactly correct.
I think that this and the second patch need to be squashed.

Quoted 7 lines
Quoted 3 lines

I was mostly trying to reduce it to what it would evaluate to. env->cur_state is always false, so the only check that matters is this one. And we fix it separately in the next one. Unless I missed something.

Well, yes, but that is not a complete fix, you need a second patch in any case, right?

Quoted 5 lines

Yeah, but I meant it's an orthogonal problem wrt GFP flags. I can actually squash both into the same patch and add two fixes tags then to the commit log, and expand the message a bit, I don't have a preference here.
Let me know if that sounds good.

I think these should be squashed, but no hard feelings if they are not.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
                          env->insn_aux_data[i + delta].storage_get_func_atomic)
                              insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
                      else

[...]

author maintainer
#7 lore raw

PATCH 2/3 bpf: Fix GFP flags for non-sleepable async callbacks

In reply to Kumar Kartikeya Dwivedi

Fix storage_get helpers to use GFP_ATOMIC when called from non-sleepable contexts within sleepable programs, such as bpf_timer callbacks. Currently, the check in do_misc_fixups assumes that env->prog->sleepable, previously in_sleepable(env) which only resolved to this check before last commit, holds across the program's execution, but that is not true. Instead, the func_atomic bit must be set whenever we see the function being called in an atomic context. Previously, this is being done when the helper is invoked in atomic contexts in sleepable programs, we can simply just set the value to true without doing an in_sleepable() check.

We must also do a standalone in_sleepable() check to handle cases where the async callback itself is armed from a sleepable program, but is itself non-sleepable (e.g., timer callback) and invokes such a helper, thus needing the func_atomic bit to be true for the said call.

Adjust do_misc_fixups() to drop any checks regarding sleepable nature of the program, and just depend on the func_atomic bit to decide which GFP flag to pass.

Fixes b00fa38a9c1c ("bpf: Enable non-atomic allocations in local storage")
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 kernel/bpf/verifier.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)
kernel/bpf/verifier.c +11 -5
index eff81ad182c8..32123c4b041a 100644
@@ -11438,7 +11438,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 			return -EINVAL;
 		}
 
-		if (in_sleepable(env) && is_storage_get_function(func_id))
+		if (is_storage_get_function(func_id))
 			env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
 	}
 

@@ -11449,7 +11449,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 			return -EINVAL;
 		}
 
-		if (in_sleepable(env) && is_storage_get_function(func_id))
+		if (is_storage_get_function(func_id))
 			env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
 	}
 

@@ -11460,10 +11460,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 			return -EINVAL;
 		}
 
-		if (in_sleepable(env) && is_storage_get_function(func_id))
+		if (is_storage_get_function(func_id))
 			env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
 	}
 
+	/*
+	 * Non-sleepable contexts in sleepable programs (e.g., timer callbacks)
+	 * are atomic and must use GFP_ATOMIC for storage_get helpers.
+	 */
+	if (!in_sleepable(env) && is_storage_get_function(func_id))
+		env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+
 	meta.func_id = func_id;
 	/* check args */
 	for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) {

@@ -22495,8 +22502,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)
 		}
 
 		if (is_storage_get_function(insn->imm)) {
-			if (!env->prog->sleepable ||
-			    env->insn_aux_data[i + delta].storage_get_func_atomic)
+			if (env->insn_aux_data[i + delta].storage_get_func_atomic)
 				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC);
 			else
 				insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL);
Signature · 1 line
2.51.0
maintainer
#8 lore raw

Re: [PATCH bpf-next v1 2/3] bpf: Fix GFP flags for non-sleepable async callbacks

On Tue, 2025-10-07 at 01:43 +0000, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -11460,10 +11460,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 			return -EINVAL;
 		}
 
-		if (in_sleepable(env) && is_storage_get_function(func_id))
+		if (is_storage_get_function(func_id))

kernel/bpf/verifier.c static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn · line 11473

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
 			env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
 	}
 
+	/*
+	 * Non-sleepable contexts in sleepable programs (e.g., timer callbacks)
+	 * are atomic and must use GFP_ATOMIC for storage_get helpers.
+	 */
+	if (!in_sleepable(env) && is_storage_get_function(func_id))
+		env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+

Note this discussion:
https://lore.kernel.org/bpf/[email protected]/

It appears there is already a need to have a flag in struct subprog_info, indicating whether subprogram might run in a sleepable context. Maybe add this flag and remove .storage_get_func_atomic altogether? (And check subprog_info in the do_misc_fixups()).

Quoted 3 lines
kernel/bpf/verifier.c +0 -0
 	meta.func_id = func_id;
 	/* check args */
 	for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) {

[...]

author maintainer
#9 lore raw

Re: [PATCH bpf-next v1 2/3] bpf: Fix GFP flags for non-sleepable async callbacks

Quoted 25 lines · Eduard Zingerman

On Tue, 2025-10-07 at 01:43 +0000, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -11460,10 +11460,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
                      return -EINVAL;
              }
 
-             if (in_sleepable(env) && is_storage_get_function(func_id))
+             if (is_storage_get_function(func_id))

kernel/bpf/verifier.c static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn · line 11473

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
                      env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
      }
 
+     /*
+      * Non-sleepable contexts in sleepable programs (e.g., timer callbacks)
+      * are atomic and must use GFP_ATOMIC for storage_get helpers.
+      */
+     if (!in_sleepable(env) && is_storage_get_function(func_id))
+             env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+

Note this discussion:
https://lore.kernel.org/bpf/[email protected]/

Quoted 4 lines

It appears there is already a need to have a flag in struct subprog_info, indicating whether subprogram might run in a sleepable context. Maybe add this flag and remove .storage_get_func_atomic altogether? (And check subprog_info in the do_misc_fixups()).

Ok, I can add a subprog field and check it that way.

Quoted 6 lines
Quoted 4 lines
kernel/bpf/verifier.c +0 -0
      meta.func_id = func_id;
      /* check args */
      for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) {

[...]

maintainer
#10 lore raw

Re: [PATCH bpf-next v1 2/3] bpf: Fix GFP flags for non-sleepable async callbacks

Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman

On Tue, 2025-10-07 at 01:43 +0000, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -11460,10 +11460,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
                      return -EINVAL;
              }
 
-             if (in_sleepable(env) && is_storage_get_function(func_id))
+             if (is_storage_get_function(func_id))

kernel/bpf/verifier.c static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn · line 11473

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
                      env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
      }
 
+     /*
+      * Non-sleepable contexts in sleepable programs (e.g., timer callbacks)
+      * are atomic and must use GFP_ATOMIC for storage_get helpers.
+      */
+     if (!in_sleepable(env) && is_storage_get_function(func_id))
+             env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+

Note this discussion:
https://lore.kernel.org/bpf/[email protected]/

Quoted 6 lines
Quoted 4 lines

It appears there is already a need to have a flag in struct subprog_info, indicating whether subprogram might run in a sleepable context. Maybe add this flag and remove .storage_get_func_atomic altogether? (And check subprog_info in the do_misc_fixups()).

Ok, I can add a subprog field and check it that way.

To be useful both here and in Mykyta's case I think the flag should be tri-state:
- never sleeps
- always sleeps [Mykyta's case]
- sometimes sleeps [this case]

Wdyt?

author maintainer
#11 lore raw

Re: [PATCH bpf-next v1 2/3] bpf: Fix GFP flags for non-sleepable async callbacks

Quoted 35 lines · Eduard Zingerman
Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman

On Tue, 2025-10-07 at 01:43 +0000, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 6 lines
kernel/bpf/verifier.c +1 -1
@@ -11460,10 +11460,17 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
                      return -EINVAL;
              }
 
-             if (in_sleepable(env) && is_storage_get_function(func_id))
+             if (is_storage_get_function(func_id))

kernel/bpf/verifier.c static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn · line 11473

Quoted 10 lines
kernel/bpf/verifier.c +7 -0
                      env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
      }
 
+     /*
+      * Non-sleepable contexts in sleepable programs (e.g., timer callbacks)
+      * are atomic and must use GFP_ATOMIC for storage_get helpers.
+      */
+     if (!in_sleepable(env) && is_storage_get_function(func_id))
+             env->insn_aux_data[insn_idx].storage_get_func_atomic = true;
+

Note this discussion:
https://lore.kernel.org/bpf/[email protected]/

Quoted 6 lines
Quoted 4 lines

It appears there is already a need to have a flag in struct subprog_info, indicating whether subprogram might run in a sleepable context. Maybe add this flag and remove .storage_get_func_atomic altogether? (And check subprog_info in the do_misc_fixups()).

Ok, I can add a subprog field and check it that way.

Quoted 7 lines

To be useful both here and in Mykyta's case I think the flag should be tri-state:
- never sleeps
- always sleeps [Mykyta's case]
- sometimes sleeps [this case]

Wdyt?

We discussed offline, in summary:
The current flag is per instruction, which means it can accurately reflect cases where a sleepable subprog may have delineated atomic sections where the helper is called, and the subprog_info flag applies to the whole subprog, so the information they provide is a bit different. The agreement is to replace func_atomic bit with a non_sleepable bit to record whether the helper is ever called in atomic context (if not set, it is always called in non-atomic / sleepable context, otherwise it may be called at least once in atomic / non-sleepable context).

author maintainer
#12 lore raw

PATCH 3/3 selftests/bpf: Add tests for async cb context

In reply to Kumar Kartikeya Dwivedi

Add tests to verify that async callback's sleepable attribute is correctly determined by the callback type, not the arming program's context, reflecting its true execution context.

Introduce verifier_async_cb_context.c with tests for all three async callback primitives: bpf_timer, bpf_wq, and bpf_task_work. Each primitive is tested when armed from both sleepable (lsm.s/file_open) and non-sleepable (fentry) programs.

Test coverage:
- bpf_timer callbacks: Verify they are never sleepable, even when armed from sleepable programs. Both tests should fail when attempting to use sleepable helper bpf_copy_from_user() in the callback.

- bpf_wq callbacks: Verify they are always sleepable, even when armed from non-sleepable programs. Both tests should succeed when using sleepable helpers in the callback.

- bpf_task_work callbacks: Verify they are always sleepable, even when armed from non-sleepable programs. Both tests should succeed when using sleepable helpers in the callback.

Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../bpf/progs/verifier_async_cb_context.c     | 181 ++++++++++++++++++
 2 files changed, 183 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_async_cb_context.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 28e81161e6fc..c0e8ffdaa484 100644
@@ -7,6 +7,7 @@
 #include "verifier_arena.skel.h"
 #include "verifier_arena_large.skel.h"
 #include "verifier_array_access.skel.h"
+#include "verifier_async_cb_context.skel.h"
 #include "verifier_basic_stack.skel.h"
 #include "verifier_bitfield_write.skel.h"
 #include "verifier_bounds.skel.h"

@@ -280,6 +281,7 @@ void test_verifier_array_access(void)
 		      verifier_array_access__elf_bytes,
 		      init_array_access_maps);
 }
+void test_verifier_async_cb_context(void)    { RUN(verifier_async_cb_context); }
 
 static int init_value_ptr_arith_maps(struct bpf_object *obj)
 {
tools/testing/selftests/bpf/progs/verifier_async_cb_context.c new file +181 -0

181 lines not on this page · raw message

Signature · 1 line
2.51.0
maintainer Acked-by on v1 3/3
#13 lore raw

Re: [PATCH bpf-next v1 3/3] selftests/bpf: Add tests for async cb context

Quoted 18 lines · Kumar Kartikeya Dwivedi

Add tests to verify that async callback's sleepable attribute is correctly determined by the callback type, not the arming program's context, reflecting its true execution context.

Introduce verifier_async_cb_context.c with tests for all three async callback primitives: bpf_timer, bpf_wq, and bpf_task_work. Each primitive is tested when armed from both sleepable (lsm.s/file_open) and non-sleepable (fentry) programs.

Test coverage:
- bpf_timer callbacks: Verify they are never sleepable, even when armed from sleepable programs. Both tests should fail when attempting to use sleepable helper bpf_copy_from_user() in the callback.

- bpf_wq callbacks: Verify they are always sleepable, even when armed from non-sleepable programs. Both tests should succeed when using sleepable helpers in the callback.

Quoted 6 lines

- bpf_task_work callbacks: Verify they are always sleepable, even when armed from non-sleepable programs. Both tests should succeed when using sleepable helpers in the callback.

Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>

---

Maybe also add a test case with a global subprogram, so that it is verified that subprogram inherits program sleepable status?

Acked-by Eduard Zingerman <[email protected]>

[...]