Skip to contentSkip to conversation

IRQ save/restore

PATCH Superseded Kumar Kartikeya Dwivedi 4 participants 30 messages Nov 21, 2024 – Nov 22, 2024

Subsystems (4)

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

cover IRQ save/restore

This set introduces support for managing IRQ state from BPF programs. Two new kfuncs, bpf_local_irq_save, and bpf_local_irq_restore are introduced to enable this functionality.

Intended use cases are writing IRQ safe data structures (e.g. memory allocator) in BPF programs natively, and use in new spin locking primitives intended to be introduced in the next few weeks.

The set begins with some refactoring patches before the actual functionality is introduced. First, we rename bpf_reference_state to bpf_resource_state, and stick to calling pointer resources as actual references, as the meaning is right now mixed (with lock state management functions calling acquire_reference_state and check_reference_leak not considering lock resoures, causing confusion).

The first patch also takes this opportunity to eliminate code duplication where possible. The second patch resolves an inconsistency in the same set of functions for managing lock state, in preparation for later patches that require passing bpf_func_state explicitly to find a lock state.

Since we now manage locks in bpf_func_state alongside other resources, this is a good opportunity to consolidate all other resources like RCU read sections and preempt-disabled section tracking into bpf_func_state as well, which is what patch 3 achieves.

After this, patch 4 refactors stack slot liveness marking logic to be shared between dynptr, and iterators, in preparation for introducing same logic for irq flag object on stack.

Finally, patch 5 and 7 introduce the new kfuncs and their selftests. For more details, please inspect the patch commit logs. Patch 6 expands coverage of existing preempt-disable selftest to cover sleepable kfuncs.

Kumar Kartikeya Dwivedi (7):
bpf: Refactor and rename resource management
bpf: Be consistent between {acquire,find,release}_lock_state
bpf: Consolidate RCU and preempt locks in bpf_func_state
bpf: Refactor mark_{dynptr,iter}_read
bpf: Introduce support for bpf_local_irq_{save,restore} selftests/bpf: Expand coverage of preempt tests to sleepable kfunc selftests/bpf: Add IRQ save/restore tests

 include/linux/bpf_verifier.h                  |  47 +-
 kernel/bpf/helpers.c                          |  24 +
 kernel/bpf/log.c                              |  11 +-
 kernel/bpf/verifier.c                         | 540 ++++++++++++++----
 tools/testing/selftests/bpf/prog_tests/irq.c  |   9 +
 tools/testing/selftests/bpf/progs/irq.c       | 393 +++++++++++++
 .../selftests/bpf/progs/preempt_lock.c        |  12 +
 7 files changed, 889 insertions(+), 147 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/irq.c
 create mode 100644 tools/testing/selftests/bpf/progs/irq.c

base-commit: 2c8b09ac2537299511c898bc71b1a5f2756c831c

Signature · 1 line
2.43.5
author maintainer
#1 lore raw

PATCH 1/7 bpf: Refactor and rename resource management

With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.

This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:

Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.

Next, factor out common code paths for managing addition and removal of resource state in acquire_resource_state and erase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.

---
 include/linux/bpf_verifier.h |  24 +++--
 kernel/bpf/log.c             |  10 +-
 kernel/bpf/verifier.c        | 173 +++++++++++++++++++----------------
 3 files changed, 108 insertions(+), 99 deletions(-)
include/linux/bpf_verifier.h +11 -13
index f4290c179bee..e5123b6804eb 100644
@@ -249,20 +249,18 @@ struct bpf_stack_state {
 	u8 slot_type[BPF_REG_SIZE];
 };
 
-struct bpf_reference_state {
-	/* Each reference object has a type. Ensure REF_TYPE_PTR is zero to
-	 * default to pointer reference on zero initialization of a state.
-	 */
-	enum ref_state_type {
-		REF_TYPE_PTR = 0,
-		REF_TYPE_LOCK,
+struct bpf_resource_state {
+	enum res_state_type {
+		RES_TYPE_INV = -1,
+		RES_TYPE_PTR = 0,
+		RES_TYPE_LOCK,
 	} type;
-	/* Track each reference created with a unique id, even if the same
-	 * instruction creates the reference multiple times (eg, via CALL).
+	/* Track each resource created with a unique id, even if the same
+	 * instruction creates the resource multiple times (eg, via CALL).
 	 */
 	int id;
-	/* Instruction where the allocation of this reference occurred. This
-	 * is used purely to inform the user of a reference leak.
+	/* Instruction where the allocation of this resource occurred. This
+	 * is used purely to inform the user of a resource leak.
 	 */
 	int insn_idx;
 	/* Use to keep track of the source object of a lock, to ensure
@@ -315,9 +313,9 @@ struct bpf_func_state {
 	u32 callback_depth;
 
 	/* The following fields should be last. See copy_func_state() */
-	int acquired_refs;
+	int acquired_res;
 	int active_locks;
-	struct bpf_reference_state *refs;
+	struct bpf_resource_state *res;
 	/* The state of the stack. Each element of the array describes BPF_REG_SIZE
 	 * (i.e. 8) bytes worth of stack memory.
 	 * stack[0] represents bytes [*(r10-8)..*(r10-1)]
kernel/bpf/log.c +5 -5
index 4a858fdb6476..0ad6f0737c57 100644
@@ -843,11 +843,11 @@ void print_verifier_state(struct bpf_verifier_env *env, const struct bpf_func_st
 			break;
 		}
 	}
-	if (state->acquired_refs && state->refs[0].id) {
-		verbose(env, " refs=%d", state->refs[0].id);
-		for (i = 1; i < state->acquired_refs; i++)
-			if (state->refs[i].id)
-				verbose(env, ",%d", state->refs[i].id);
+	if (state->acquired_res && state->res[0].id) {
+		verbose(env, " refs=%d", state->res[0].id);
+		for (i = 1; i < state->acquired_res; i++)
+			if (state->res[i].id)
+				verbose(env, ",%d", state->res[i].id);
 	}
 	if (state->in_callback_fn)
 		verbose(env, " cb");
kernel/bpf/verifier.c +92 -81

326 lines not on this page · raw message

Signature · 1 line
2.43.5
maintainer Reviewed-by on v1 1/7
#2 lore raw

Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management

Quoted 23 lines · Kumar Kartikeya Dwivedi

With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.

This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:

Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.

Next, factor out common code paths for managing addition and removal of resource state in acquire_resource_state and erase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.

Quoted 2 lines

---

Tbh, I like the old name a bit more.
The patch itself looks good.

[...]

kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1345

Quoted 5 lines
kernel/bpf/verifier.c +1 -0
@@ -1342,6 +1342,25 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state
 	return 0;
 }
 
+static struct bpf_resource_state *acquire_resource_state(struct bpf_verifier_env *env, int insn_idx, int *id)

Nit: there is no need to pass int *id, as it is available as (returned)->id.

Quoted 21 lines
kernel/bpf/verifier.c +18 -0
+{
+	struct bpf_func_state *state = cur_func(env);
+	int new_ofs = state->acquired_res;
+	struct bpf_resource_state *s;
+	int err;
+
+	err = resize_resource_state(state, state->acquired_res + 1);
+	if (err)
+		return NULL;
+	s = &state->res[new_ofs];
+	s->type = RES_TYPE_INV;
+	if (id)
+		*id = s->id = ++env->id_gen;
+	s->insn_idx = insn_idx;
+
+	return s;
+}
+
 /* Acquire a pointer id from the env and update the state->refs to include
  * this new pointer reference.
  * On success, returns a valid pointer id to associate with the register

[...]

Quoted 1 line
(quoted diff) +0 -0
@@ -1349,55 +1368,52 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state

[...]

kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1398

Quoted 2 lines
kernel/bpf/verifier.c +1 -1
-/* release function corresponding to acquire_reference_state(). Idempotent. */
+static void erase_resource_state(struct bpf_func_state *state, int res_idx)

Nit: why not "release_..." to be consistent with the rest of the functions?

Quoted 31 lines
kernel/bpf/verifier.c +14 -10
+{
+	int last_idx = state->acquired_res - 1;
+
+	if (last_idx && res_idx != last_idx)
+		memcpy(&state->res[res_idx], &state->res[last_idx], sizeof(*state->res));
+	memset(&state->res[last_idx], 0, sizeof(*state->res));
+	state->acquired_res--;
+}
+
 static int release_reference_state(struct bpf_func_state *state, int ptr_id)
 {
-	int i, last_idx;
+	int i;
 
-	last_idx = state->acquired_refs - 1;
-	for (i = 0; i < state->acquired_refs; i++) {
-		if (state->refs[i].type != REF_TYPE_PTR)
+	for (i = 0; i < state->acquired_res; i++) {
+		if (state->res[i].type != RES_TYPE_PTR)
 			continue;
-		if (state->refs[i].id == ptr_id) {
-			if (last_idx && i != last_idx)
-				memcpy(&state->refs[i], &state->refs[last_idx],
-				       sizeof(*state->refs));
-			memset(&state->refs[last_idx], 0, sizeof(*state->refs));
-			state->acquired_refs--;
+		if (state->res[i].id == ptr_id) {
+			erase_resource_state(state, i);
 			return 0;
 		}
 	}

[...]

author maintainer
#3 lore raw

Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management

Quoted 25 lines · Eduard Zingerman
Quoted 23 lines · Kumar Kartikeya Dwivedi

With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.

This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:

Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.

Next, factor out common code paths for managing addition and removal of resource state in acquire_resource_state and erase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.

Quoted 6 lines
Quoted 2 lines

---

Tbh, I like the old name a bit more.
The patch itself looks good.

I am happy for suggestions on better naming, but it would be better to make a distinction somehow.

Quoted 4 lines

[...]

Quoted 8 lines

kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1345

Quoted 5 lines
kernel/bpf/verifier.c +1 -0
@@ -1342,6 +1342,25 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state
      return 0;
 }
 
+static struct bpf_resource_state *acquire_resource_state(struct bpf_verifier_env *env, int insn_idx, int *id)

Nit: there is no need to pass int *id, as it is available as (returned)->id.

Replaced with a bool alloc_id to decide whether it generates a new id or not, and fixed.

Quoted 24 lines
Quoted 21 lines
kernel/bpf/verifier.c +18 -0
+{
+     struct bpf_func_state *state = cur_func(env);
+     int new_ofs = state->acquired_res;
+     struct bpf_resource_state *s;
+     int err;
+
+     err = resize_resource_state(state, state->acquired_res + 1);
+     if (err)
+             return NULL;
+     s = &state->res[new_ofs];
+     s->type = RES_TYPE_INV;
+     if (id)
+             *id = s->id = ++env->id_gen;
+     s->insn_idx = insn_idx;
+
+     return s;
+}
+
 /* Acquire a pointer id from the env and update the state->refs to include
  * this new pointer reference.
  * On success, returns a valid pointer id to associate with the register

[...]

Quoted 9 lines
Quoted 1 line
(quoted diff) +0 -0
@@ -1349,55 +1368,52 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state

[...]

kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1398

Quoted 2 lines
kernel/bpf/verifier.c +1 -1
-/* release function corresponding to acquire_reference_state(). Idempotent. */
+static void erase_resource_state(struct bpf_func_state *state, int res_idx)

Nit: why not "release_..." to be consistent with the rest of the functions?

This was a subset of what "release_resource_state" would have done, since it erases a res_idx,
but on second thought, it's probably better to rename, so fixed as well.

Thanks for the review.

Quoted 34 lines
Quoted 31 lines
kernel/bpf/verifier.c +14 -10
+{
+     int last_idx = state->acquired_res - 1;
+
+     if (last_idx && res_idx != last_idx)
+             memcpy(&state->res[res_idx], &state->res[last_idx], sizeof(*state->res));
+     memset(&state->res[last_idx], 0, sizeof(*state->res));
+     state->acquired_res--;
+}
+
 static int release_reference_state(struct bpf_func_state *state, int ptr_id)
 {
-     int i, last_idx;
+     int i;
 
-     last_idx = state->acquired_refs - 1;
-     for (i = 0; i < state->acquired_refs; i++) {
-             if (state->refs[i].type != REF_TYPE_PTR)
+     for (i = 0; i < state->acquired_res; i++) {
+             if (state->res[i].type != RES_TYPE_PTR)
                      continue;
-             if (state->refs[i].id == ptr_id) {
-                     if (last_idx && i != last_idx)
-                             memcpy(&state->refs[i], &state->refs[last_idx],
-                                    sizeof(*state->refs));
-                     memset(&state->refs[last_idx], 0, sizeof(*state->refs));
-                     state->acquired_refs--;
+             if (state->res[i].id == ptr_id) {
+                     erase_resource_state(state, i);
                      return 0;
              }
      }

[...]

#4 lore raw

Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management

In reply to Kumar Kartikeya Dwivedi
Quoted 12 lines · Kumar Kartikeya Dwivedi

With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.

This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:

Quoted 4 lines

Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.

Sorry I don't like this renaming.
reference state is already understood as a set of resources that were acquired.
Whether it's an object allocated by bpf_obj_new or any other resource.
I think this patch has a net negative effect.
People familiar with the verifier already understand what refsafe() or acquired_refs are for.
Calling them slightly different names adds confusion, not clarity.

pw-bot: cr

author maintainer
#5 lore raw

Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management

Quoted 31 lines · Alexei Starovoitov
Quoted 12 lines · Kumar Kartikeya Dwivedi

With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.

This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:

Quoted 4 lines

Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.

Sorry I don't like this renaming.
reference state is already understood as a set of resources that were acquired.
Whether it's an object allocated by bpf_obj_new or any other resource.
I think this patch has a net negative effect.
People familiar with the verifier already understand what refsafe() or acquired_refs are for.
Calling them slightly different names adds confusion, not clarity.

Quoted 1 line

pw-bot: cr

Ok

author maintainer
#6 lore raw

PATCH 2/7 bpf: Be consistent between {acquire,find,release}_lock_state

In reply to Kumar Kartikeya Dwivedi

Both acquire_lock_state and release_lock_state take the bpf_func_state as a parameter, while find_lock_state does not. Future patches will end up requiring operating on non-cur_func(env) bpf_func_state (for resilient locks), hence just make the prototype consistent and take bpf_func_state directly.

---
 kernel/bpf/verifier.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +2 -3
index c106720d0c62..0ff436c06c13 100644
@@ -1436,10 +1436,9 @@ static int release_lock_state(struct bpf_func_state *state, int type, int id, vo
 	return -EINVAL;
 }
 
-static struct bpf_resource_state *find_lock_state(struct bpf_verifier_env *env, enum res_state_type type,
+static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state, enum res_state_type type,
 						   int id, void *ptr)
 {
-	struct bpf_func_state *state = cur_func(env);
 	int i;
 
 	for (i = 0; i < state->acquired_res; i++) {
@@ -11873,7 +11872,7 @@ static int check_reg_allocation_locked(struct bpf_verifier_env *env, struct bpf_
 
 	if (!cur_func(env)->active_locks)
 		return -EINVAL;
-	s = find_lock_state(env, RES_TYPE_LOCK, id, ptr);
+	s = find_lock_state(cur_func(env), RES_TYPE_LOCK, id, ptr);
 	if (!s) {
 		verbose(env, "held lock and object are not in the same allocation\n");
 		return -EINVAL;
Signature · 1 line
2.43.5
maintainer Acked-by on v1 2/7
#7 lore raw

Re: [PATCH bpf-next v1 2/7] bpf: Be consistent between {acquire,find,release}_lock_state

Quoted 8 lines · Kumar Kartikeya Dwivedi

Both acquire_lock_state and release_lock_state take the bpf_func_state as a parameter, while find_lock_state does not. Future patches will end up requiring operating on non-cur_func(env) bpf_func_state (for resilient locks), hence just make the prototype consistent and take bpf_func_state directly.

---

author maintainer
#8 lore raw

PATCH 3/7 bpf: Consolidate RCU and preempt locks in bpf_func_state

In reply to Kumar Kartikeya Dwivedi

To ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).

For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.

OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.

---
 include/linux/bpf_verifier.h |  4 ++--
 kernel/bpf/verifier.c        | 46 ++++++++++++++++++++----------------
 2 files changed, 27 insertions(+), 23 deletions(-)
include/linux/bpf_verifier.h +2 -2
index e5123b6804eb..fa09538a35bc 100644
@@ -315,6 +315,8 @@ struct bpf_func_state {
 	/* The following fields should be last. See copy_func_state() */
 	int acquired_res;
 	int active_locks;
+	int active_preempt_locks;
+	bool active_rcu_lock;
 	struct bpf_resource_state *res;
 	/* The state of the stack. Each element of the array describes BPF_REG_SIZE
 	 * (i.e. 8) bytes worth of stack memory.
@@ -418,8 +420,6 @@ struct bpf_verifier_state {
 	u32 curframe;
 
 	bool speculative;
-	bool active_rcu_lock;
-	u32 active_preempt_lock;
 	/* If this state was ever pointed-to by other state's loop_entry field
 	 * this flag would be set to true. Used to avoid freeing such states
 	 * while they are still in use.
kernel/bpf/verifier.c +25 -21

134 lines not on this page · raw message

Signature · 1 line
2.43.5
maintainer
#9 lore raw

Re: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state

Quoted 8 lines · Kumar Kartikeya Dwivedi

To ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).

For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.

Quoted 7 lines

OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.

---

This change is a bit confusing to me.
The following is done currently:
- in setup_func_entry() called from check_func_call():
copy_resource_state(callee, caller);
- in prepare_func_exit():
copy_resource_state(caller, callee);

So it seems that it is logical to track resources in the bpf_verifier_state and avoid copying.
There is probably something I don't understand.

[...]

author maintainer
#10 lore raw

Re: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state

Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi

To ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).

For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.

Quoted 7 lines

OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.

---

This change is a bit confusing to me.
The following is done currently:
- in setup_func_entry() called from check_func_call():
copy_resource_state(callee, caller);
- in prepare_func_exit():
copy_resource_state(caller, callee);

Quoted 4 lines

So it seems that it is logical to track resources in the bpf_verifier_state and avoid copying.
There is probably something I don't understand.

This is what we were doing all along, and you're right, it is sort of a global entity.
But we've moved active_locks to bpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be in
cur_func, or in bpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.

Alexei's preference has been stashing this in bpf_func_state instead in [0]. Let me know what you think.

[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com

Quoted 2 lines

[...]

maintainer
#11 lore raw

Re: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state

Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi

To ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).

For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.

Quoted 7 lines

OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.

---

This change is a bit confusing to me.
The following is done currently:
- in setup_func_entry() called from check_func_call():
copy_resource_state(callee, caller);
- in prepare_func_exit():
copy_resource_state(caller, callee);

Quoted 7 lines
Quoted 4 lines

So it seems that it is logical to track resources in the bpf_verifier_state and avoid copying.
There is probably something I don't understand.

This is what we were doing all along, and you're right, it is sort of a global entity.

Right, but since this patch-set does a refactoring, might be a good time to change.

Quoted 11 lines

But we've moved active_locks to bpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be in
cur_func, or in bpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.

Alexei's preference has been stashing this in bpf_func_state instead in [0]. Let me know what you think.

[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com

As far as I understand check_func_call(), function calls to static functions are allowed while holding each kind of resources currently tracked. So it seems odd to track it as a part of function state. The way I understand Alexei in the thread [0] the idea is more to track all counters in one place.

Let's wait what Alexei has to say.

author maintainer
#12 lore raw

Re: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state

Quoted 51 lines · Eduard Zingerman
Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi

To ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).

For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.

Quoted 7 lines

OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.

---

This change is a bit confusing to me.
The following is done currently:
- in setup_func_entry() called from check_func_call():
copy_resource_state(callee, caller);
- in prepare_func_exit():
copy_resource_state(caller, callee);

Quoted 7 lines
Quoted 4 lines

So it seems that it is logical to track resources in the bpf_verifier_state and avoid copying.
There is probably something I don't understand.

This is what we were doing all along, and you're right, it is sort of a global entity.

Right, but since this patch-set does a refactoring, might be a good time to change.

Quoted 11 lines

But we've moved active_locks to bpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be in
cur_func, or in bpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.

Alexei's preference has been stashing this in bpf_func_state instead in [0]. Let me know what you think.

[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com

Quoted 8 lines

As far as I understand check_func_call(), function calls to static functions are allowed while holding each kind of resources currently tracked. So it seems odd to track it as a part of function state. The way I understand Alexei in the thread [0] the idea is more to track all counters in one place.

Let's wait what Alexei has to say.

Discussed with Alexei (who discussed with you I presume) that we're doing this in bpf_verifier_state, will fix.

author maintainer
#13 lore raw

PATCH 4/7 bpf: Refactor mark_{dynptr,iter}_read

In reply to Kumar Kartikeya Dwivedi

There is possibility of sharing code between mark_dynptr_read and mark_iter_read for updating liveness information of their stack slots. Consolidate common logic into mark_stack_slot_obj_read function in preparation for the next patch which needs the same logic for its own stack slots.

---
 kernel/bpf/verifier.c | 43 +++++++++++++++++++++----------------------
 1 file changed, 21 insertions(+), 22 deletions(-)
kernel/bpf/verifier.c +21 -22
index 25c44b68f16a..6cd2bbed4583 100644
@@ -3213,10 +3213,27 @@ static int mark_reg_read(struct bpf_verifier_env *env,
 	return 0;
 }
 
-static int mark_dynptr_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg)
+static int mark_stack_slot_obj_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
+				    int spi, int nr_slots)
 {
 	struct bpf_func_state *state = func(env, reg);
-	int spi, ret;
+	int err, i;
+
+	for (i = 0; i < nr_slots; i++) {
+		struct bpf_reg_state *st = &state->stack[spi - i].spilled_ptr;
+
+		err = mark_reg_read(env, st, st->parent, REG_LIVE_READ64);
+		if (err)
+			return err;
+
+		mark_stack_slot_scratched(env, spi - i);
+	}
+	return 0;
+}
+
+static int mark_dynptr_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg)
+{
+	int spi;
 
 	/* For CONST_PTR_TO_DYNPTR, it must have already been done by
 	 * check_reg_arg in check_helper_call and mark_btf_func_reg_size in
@@ -3231,31 +3248,13 @@ static int mark_dynptr_read(struct bpf_verifier_env *env, struct bpf_reg_state *
 	 * bounds and spi is the first dynptr slot. Simply mark stack slot as
 	 * read.
 	 */
-	ret = mark_reg_read(env, &state->stack[spi].spilled_ptr,
-			    state->stack[spi].spilled_ptr.parent, REG_LIVE_READ64);
-	if (ret)
-		return ret;
-	return mark_reg_read(env, &state->stack[spi - 1].spilled_ptr,
-			     state->stack[spi - 1].spilled_ptr.parent, REG_LIVE_READ64);
+	return mark_stack_slot_obj_read(env, reg, spi, BPF_DYNPTR_NR_SLOTS);
 }
 
 static int mark_iter_read(struct bpf_verifier_env *env, struct bpf_reg_state *reg,
 			  int spi, int nr_slots)
 {
-	struct bpf_func_state *state = func(env, reg);
-	int err, i;
-
-	for (i = 0; i < nr_slots; i++) {
-		struct bpf_reg_state *st = &state->stack[spi - i].spilled_ptr;
-
-		err = mark_reg_read(env, st, st->parent, REG_LIVE_READ64);
-		if (err)
-			return err;
-
-		mark_stack_slot_scratched(env, spi - i);
-	}
-
-	return 0;
+	return mark_stack_slot_obj_read(env, reg, spi, nr_slots);
 }
 
 /* This function is supposed to be used by the following 32-bit optimization
Signature · 1 line
2.43.5
maintainer Acked-by on v1 4/7
#14 lore raw

Re: [PATCH bpf-next v1 4/7] bpf: Refactor mark_{dynptr,iter}_read

Quoted 8 lines · Kumar Kartikeya Dwivedi

There is possibility of sharing code between mark_dynptr_read and mark_iter_read for updating liveness information of their stack slots. Consolidate common logic into mark_stack_slot_obj_read function in preparation for the next patch which needs the same logic for its own stack slots.

---

author maintainer
#15 lore raw

PATCH 5/7 bpf: Introduce support for bpf_local_irq_{save,restore}

In reply to Kumar Kartikeya Dwivedi

Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.

For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.

To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.

One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.

The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.

The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.

---
 include/linux/bpf_verifier.h |  19 ++-
 kernel/bpf/helpers.c         |  24 +++
 kernel/bpf/log.c             |   1 +
 kernel/bpf/verifier.c        | 283 ++++++++++++++++++++++++++++++++++-
 4 files changed, 322 insertions(+), 5 deletions(-)
include/linux/bpf_verifier.h +15 -4
index fa09538a35bc..f44961dccbac 100644
@@ -233,6 +233,7 @@ enum bpf_stack_slot_type {
 	 */
 	STACK_DYNPTR,
 	STACK_ITER,
+	STACK_IRQ_FLAG,
 };
 
 #define BPF_REG_SIZE 8	/* size of eBPF register in bytes */
@@ -253,6 +254,9 @@ struct bpf_resource_state {
 	enum res_state_type {
 		RES_TYPE_INV = -1,
 		RES_TYPE_PTR = 0,
+		RES_TYPE_IRQ,
+
+		__RES_TYPE_LOCK_BEGIN,
 		RES_TYPE_LOCK,
 	} type;
 	/* Track each resource created with a unique id, even if the same
@@ -263,10 +267,16 @@ struct bpf_resource_state {
 	 * is used purely to inform the user of a resource leak.
 	 */
 	int insn_idx;
-	/* Use to keep track of the source object of a lock, to ensure
-	 * it matches on unlock.
-	 */
-	void *ptr;
+	union {
+		/* Use to keep track of the source object of a lock, to ensure
+		 * it matches on unlock.
+		 */
+		void *ptr;
+		/* Track the reference id preceding the IRQ entry in acquisition
+		 * order, to enforce an ordering on the release.
+		 */
+		int prev_id;
+	};
 };
 
 struct bpf_retval_range {
@@ -317,6 +327,7 @@ struct bpf_func_state {
 	int active_locks;
 	int active_preempt_locks;
 	bool active_rcu_lock;
+	int active_irq_id;
 	struct bpf_resource_state *res;
 	/* The state of the stack. Each element of the array describes BPF_REG_SIZE
 	 * (i.e. 8) bytes worth of stack memory.
kernel/bpf/helpers.c +24 -0
index 751c150f9e1c..302f0d5976be 100644
@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user
 	return ret + 1;
 }
 
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)
+{
+	u64 *ptr = (u64 *)flags__irq_flag;
+	unsigned long flags;
+
+	local_irq_save(flags);
+	*ptr = flags;
+}
+
+__bpf_kfunc void bpf_local_irq_restore(unsigned long *flags__irq_flag)
+{
+	u64 *ptr = (u64 *)flags__irq_flag;
+	unsigned long flags = *ptr;
+
+	local_irq_restore(flags);
+}
+
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(generic_btf_ids)
@@ -3149,6 +3171,8 @@ BTF_ID_FLAGS(func, bpf_get_kmem_cache)
 BTF_ID_FLAGS(func, bpf_iter_kmem_cache_new, KF_ITER_NEW | KF_SLEEPABLE)
 BTF_ID_FLAGS(func, bpf_iter_kmem_cache_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE)
 BTF_ID_FLAGS(func, bpf_iter_kmem_cache_destroy, KF_ITER_DESTROY | KF_SLEEPABLE)
+BTF_ID_FLAGS(func, bpf_local_irq_save)
+BTF_ID_FLAGS(func, bpf_local_irq_restore)
 BTF_KFUNCS_END(common_btf_ids)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/log.c +1 -0
index 0ad6f0737c57..fc5520782e5d 100644
@@ -537,6 +537,7 @@ static char slot_type_char[] = {
 	[STACK_ZERO]	= '0',
 	[STACK_DYNPTR]	= 'd',
 	[STACK_ITER]	= 'i',
+	[STACK_IRQ_FLAG] = 'f'
 };
 
 static void print_liveness(struct bpf_verifier_env *env,
kernel/bpf/verifier.c +282 -1

425 lines not on this page · raw message

Signature · 1 line
2.43.5
maintainer Acked-by on v1 5/7
#16 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

Quoted 30 lines · Kumar Kartikeya Dwivedi

Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.

For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.

To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.

One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.

The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.

Quoted 5 lines

The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.

---

I think this matches what is done for iterators and dynptrs.

[...]

Quoted 8 lines
include/linux/bpf_verifier.h +0 -4
@@ -263,10 +267,16 @@ struct bpf_resource_state {
 	 * is used purely to inform the user of a resource leak.
 	 */
 	int insn_idx;
-	/* Use to keep track of the source object of a lock, to ensure
-	 * it matches on unlock.
-	 */
-	void *ptr;

include/linux/bpf_verifier.h struct bpf_resource_state { · line 279

Quoted 10 lines
include/linux/bpf_verifier.h +10 -0
+	union {
+		/* Use to keep track of the source object of a lock, to ensure
+		 * it matches on unlock.
+		 */
+		void *ptr;
+		/* Track the reference id preceding the IRQ entry in acquisition
+		 * order, to enforce an ordering on the release.
+		 */
+		int prev_id;
+	};

Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative to prev_id would be to organize bpf_func_state->res as a stack (by changing erase_resource_state() implementation).

[...]

Quoted 5 lines
kernel/bpf/helpers.c +0 -0
index 751c150f9e1c..302f0d5976be 100644

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065

Quoted 9 lines
kernel/bpf/helpers.c +6 -0
@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user
 	return ret + 1;
 }
 
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)

Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?

Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+{
+	u64 *ptr = (u64 *)flags__irq_flag;
+	unsigned long flags;
+
+	local_irq_save(flags);
+	*ptr = flags;
+}

[...]

kernel/bpf/helpers.c static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state, · line 1610

Quoted 6 lines
kernel/bpf/helpers.c +1 -1
@@ -1447,7 +1607,7 @@ static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state,
 	for (i = 0; i < state->acquired_res; i++) {
 		struct bpf_resource_state *s = &state->res[i];
 
-		if (s->type == RES_TYPE_PTR || s->type != type)
+		if (s->type < __RES_TYPE_LOCK_BEGIN || s->type != type)

Nit: I think this would be easier to read if there was a bitmask
associated with lock types.

Quoted 3 lines
kernel/bpf/helpers.c +0 -0
 			continue;
 
 		if (s->id == id && s->ptr == ptr)

[...]

author maintainer
#17 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

Quoted 58 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi

Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.

For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.

To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.

One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.

The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.

Quoted 5 lines

The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.

---

I think this matches what is done for iterators and dynptrs.

[...]

Quoted 14 lines
include/linux/bpf_verifier.h +6 -4
@@ -263,10 +267,16 @@ struct bpf_resource_state {
       * is used purely to inform the user of a resource leak.
       */
      int insn_idx;
-     /* Use to keep track of the source object of a lock, to ensure
-      * it matches on unlock.
-      */
-     void *ptr;
+     union {
+             /* Use to keep track of the source object of a lock, to ensure
+              * it matches on unlock.
+              */
+             void *ptr;
+             /* Track the reference id preceding the IRQ entry in acquisition
Quoted 8 lines

include/linux/bpf_verifier.h struct bpf_resource_state { · line 279

Quoted 4 lines
include/linux/bpf_verifier.h +4 -0
+              * order, to enforce an ordering on the release.
+              */
+             int prev_id;
+     };

Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative to prev_id would be to organize bpf_func_state->res as a stack (by changing erase_resource_state() implementation).

I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.

Quoted 11 lines

[...]

Quoted 8 lines
kernel/bpf/helpers.c +0 -0
index 751c150f9e1c..302f0d5976be 100644
@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user
      return ret + 1;
 }
 
Quoted 9 lines

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065

Quoted 6 lines
kernel/bpf/helpers.c +6 -0
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)

Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?

It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted to vmlinux.h expects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.

Quoted 15 lines
Quoted 8 lines
kernel/bpf/helpers.c +7 -0
+{
+     u64 *ptr = (u64 *)flags__irq_flag;
+     unsigned long flags;
+
+     local_irq_save(flags);
+     *ptr = flags;
+}

[...]

Quoted 4 lines
kernel/bpf/helpers.c +0 -0
@@ -1447,7 +1607,7 @@ static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state,
      for (i = 0; i < state->acquired_res; i++) {
              struct bpf_resource_state *s = &state->res[i];
 
Quoted 5 lines

kernel/bpf/helpers.c static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state, · line 1610

Quoted 2 lines
kernel/bpf/helpers.c +1 -1
-             if (s->type == RES_TYPE_PTR || s->type != type)
+             if (s->type < __RES_TYPE_LOCK_BEGIN || s->type != type)

Nit: I think this would be easier to read if there was a bitmask
associated with lock types.

Ack, will fix.

Quoted 7 lines
Quoted 4 lines
kernel/bpf/helpers.c +0 -0
                      continue;
 
              if (s->id == id && s->ptr == ptr)

[...]

maintainer
#18 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065

Quoted 6 lines
kernel/bpf/helpers.c +6 -0
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)

Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?

Quoted 9 lines

It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted to vmlinux.h expects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.

So, the goal is to enable the following pattern:

  unsigned long flags;
  bpf_local_irq_save(&flags);

Right?

For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:

unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.

  bpf_local_irq_save(&flags);

I think that in such case '*ptr = flags;' would overwrite foo.

[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+{
+     u64 *ptr = (u64 *)flags__irq_flag;
+     unsigned long flags;
+
+     local_irq_save(flags);
+     *ptr = flags;
+}
author maintainer
#19 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

Quoted 35 lines · Eduard Zingerman

On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065

Quoted 6 lines
kernel/bpf/helpers.c +6 -0
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)

Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?

Quoted 9 lines

It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted to vmlinux.h expects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.

So, the goal is to enable the following pattern:

  unsigned long flags;
  bpf_local_irq_save(&flags);

Right?

For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:

Quoted 6 lines

unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.

  bpf_local_irq_save(&flags);

I think that in such case '*ptr = flags;' would overwrite foo.

In the kernel or userspace, yes, but I'm assuming unsigned long will always be 64-bit for target=BPF.
Would that be incorrect? This pattern will only happen within BPF programs.

Quoted 11 lines

[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
kernel/bpf/helpers.c +7 -0
+{
+     u64 *ptr = (u64 *)flags__irq_flag;
+     unsigned long flags;
+
+     local_irq_save(flags);
+     *ptr = flags;
+}
maintainer
#20 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 38 lines · Eduard Zingerman

On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 9 lines

kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065

Quoted 6 lines
kernel/bpf/helpers.c +6 -0
+/* Keep unsinged long in prototype so that kfunc is usable when emitted to
+ * vmlinux.h in BPF programs directly, but since unsigned long may potentially
+ * be 4 byte, always cast to u64 when reading/writing from this pointer as it
+ * always points to an 8-byte memory region in BPF stack.
+ */
+__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)

Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?

Quoted 9 lines

It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted to vmlinux.h expects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.

So, the goal is to enable the following pattern:

  unsigned long flags;
  bpf_local_irq_save(&flags);

Right?

For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:

unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.

Quoted 7 lines
Quoted 3 lines
  bpf_local_irq_save(&flags);

I think that in such case '*ptr = flags;' would overwrite foo.

In the kernel or userspace, yes, but I'm assuming unsigned long will always be 64-bit for target=BPF.
Would that be incorrect? This pattern will only happen within BPF programs.

Discussed off-list.
Kumar is right, and there is no problem, as on BPF side 'unsigned long' is always 8 bytes.

[...]

#21 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

In reply to Kumar Kartikeya Dwivedi
Quoted 69 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi

Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.

For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.

To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.

One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.

The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.

Quoted 5 lines

The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.

---

I think this matches what is done for iterators and dynptrs.

[...]

Quoted 14 lines
include/linux/bpf_verifier.h +6 -4
@@ -263,10 +267,16 @@ struct bpf_resource_state {
       * is used purely to inform the user of a resource leak.
       */
      int insn_idx;
-     /* Use to keep track of the source object of a lock, to ensure
-      * it matches on unlock.
-      */
-     void *ptr;
+     union {
+             /* Use to keep track of the source object of a lock, to ensure
+              * it matches on unlock.
+              */
+             void *ptr;
+             /* Track the reference id preceding the IRQ entry in acquisition
Quoted 8 lines

include/linux/bpf_verifier.h struct bpf_resource_state { · line 279

Quoted 4 lines
include/linux/bpf_verifier.h +4 -0
+              * order, to enforce an ordering on the release.
+              */
+             int prev_id;
+     };

Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative to prev_id would be to organize bpf_func_state->res as a stack (by changing erase_resource_state() implementation).

Quoted 2 lines

I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.

Acquire_refs is already a stack.
Manual push/pop via prev_id looks unnecessary. Just search the top of acquired_refs for id. If it doesn't match error.

I don't like this bit either:
+ if (id != state->active_irq_id)
+ return -EPROTO;

Why invent new error codes for such conditions? It's EACESS or EINVAL like everywhere else in the verifier.

author maintainer
#22 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

Quoted 72 lines · Alexei Starovoitov
Quoted 69 lines · Kumar Kartikeya Dwivedi
Quoted 66 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi

Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.

For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.

To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.

One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.

The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.

Quoted 5 lines

The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.

---

I think this matches what is done for iterators and dynptrs.

[...]

Quoted 8 lines
include/linux/bpf_verifier.h +0 -4
@@ -263,10 +267,16 @@ struct bpf_resource_state {
       * is used purely to inform the user of a resource leak.
       */
      int insn_idx;
-     /* Use to keep track of the source object of a lock, to ensure
-      * it matches on unlock.
-      */
-     void *ptr;

include/linux/bpf_verifier.h struct bpf_resource_state { · line 279

Quoted 10 lines
include/linux/bpf_verifier.h +10 -0
+     union {
+             /* Use to keep track of the source object of a lock, to ensure
+              * it matches on unlock.
+              */
+             void *ptr;
+             /* Track the reference id preceding the IRQ entry in acquisition
+              * order, to enforce an ordering on the release.
+              */
+             int prev_id;
+     };

Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative to prev_id would be to organize bpf_func_state->res as a stack (by changing erase_resource_state() implementation).

Quoted 7 lines
Quoted 2 lines

I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.

Acquire_refs is already a stack.
Manual push/pop via prev_id looks unnecessary. Just search the top of acquired_refs for id. If it doesn't match error.

Ok.

Quoted 7 lines

I don't like this bit either:
+ if (id != state->active_irq_id)
+ return -EPROTO;

Why invent new error codes for such conditions? It's EACESS or EINVAL like everywhere else in the verifier.

Ok, though we do use EPROTO in a bunch of places. I just needed something to distinguish the two errors.
I'll switch to these.

#23 lore raw

Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}

In reply to Kumar Kartikeya Dwivedi
kernel test robot noticed the following build warnings:

Hi Kumar,

kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf-next/for-next]
[also build test WARNING on bpf-next/master linus/master next-20241121] [cannot apply to bpf/master v6.12]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Kumar-Kartikeya-Dwivedi/bpf-Refactor-and-rename-resource-management/20241121-140722
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git for-next
patch link:    https://lore.kernel.org/r/20241121005329.408873-6-memxor%40gmail.com
patch subject: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
config: arc-randconfig-001-20241122 (https://download.01.org/0day-ci/archive/20241122/[email protected]/config)
compiler: arc-elf-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241122/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

kernel/bpf/verifier.c: In function 'process_irq_flag':

>> kernel/bpf/verifier.c:11964:32: warning: variable 'irq_restore' set but not used [-Wunused-but-set-variable]
   11964 |         bool irq_save = false, irq_restore = false;
         |                                ^~~~~~~~~~~

vim +/irq_restore +11964 kernel/bpf/verifier.c

11959
11960 static int process_irq_flag(struct bpf_verifier_env *env, int regno,
11961 struct bpf_kfunc_call_arg_meta *meta)
11962 {
11963 struct bpf_reg_state *regs = cur_regs(env), *reg = &regs[regno];
11964 bool irq_save = false, irq_restore = false;
11965 int err;
11966
11967 if (meta->func_id == special_kfunc_list[KF_bpf_local_irq_save]) {
11968 irq_save = true;
11969 } else if (meta->func_id == special_kfunc_list[KF_bpf_local_irq_restore]) {
11970 irq_restore = true;
11971 } else {
11972 verbose(env, "verifier internal error: unknown irq flags kfunc\n");
11973 return -EFAULT;
11974 }
11975
11976 if (irq_save) {
11977 if (!is_irq_flag_reg_valid_uninit(env, reg)) {
11978 verbose(env, "expected uninitialized irq flag as arg#%d\n", regno);
11979 return -EINVAL;
11980 }
11981
11982 err = check_mem_access(env, env->insn_idx, regno, 0, BPF_DW, BPF_WRITE, -1, false, false);
11983 if (err)
11984 return err;
11985
11986 err = mark_stack_slot_irq_flag(env, meta, reg, env->insn_idx);
11987 if (err)
11988 return err;
11989 } else {
11990 err = is_irq_flag_reg_valid_init(env, reg);
11991 if (err) {
11992 verbose(env, "expected an initialized irq flag as arg#%d\n", regno);
11993 return err;
11994 }
11995
11996 err = mark_irq_flag_read(env, reg);
11997 if (err)
11998 return err;
11999
12000 err = unmark_stack_slot_irq_flag(env, reg);
12001 if (err)
12002 return err;
12003 }
12004 return 0;
12005 }
12006

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#24 lore raw

PATCH 6/7 selftests/bpf: Expand coverage of preempt tests to sleepable kfunc

In reply to Kumar Kartikeya Dwivedi

For preemption-related kfuncs, we don't test their interaction with sleepable kfuncs (we do test helpers) even though the verifier has code to protect against such a pattern. Expand coverage of the selftest to include this case.

---
 tools/testing/selftests/bpf/progs/preempt_lock.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
tools/testing/selftests/bpf/progs/preempt_lock.c +12 -0
index 885377e83607..d24314c394c7 100644
@@ -113,6 +113,18 @@ int preempt_sleepable_helper(void *ctx)
 	return 0;
 }
 
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+__failure __msg("kernel func bpf_copy_from_user_str is sleepable within non-preemptible region")
+int preempt_sleepable_kfunc(void *ctx)
+{
+	u32 data;
+
+	bpf_preempt_disable();
+	bpf_copy_from_user_str(&data, sizeof(data), NULL, 0);
+	bpf_preempt_enable();
+	return 0;
+}
+
 int __noinline preempt_global_subprog(void)
 {
 	preempt_balance_subprog();
Signature · 1 line
2.43.5
maintainer Acked-by on v1 6/7
#25 lore raw

Re: [PATCH bpf-next v1 6/7] selftests/bpf: Expand coverage of preempt tests to sleepable kfunc

Quoted 6 lines · Kumar Kartikeya Dwivedi

For preemption-related kfuncs, we don't test their interaction with sleepable kfuncs (we do test helpers) even though the verifier has code to protect against such a pattern. Expand coverage of the selftest to include this case.

[...]

author maintainer
#26 lore raw

PATCH 7/7 selftests/bpf: Add IRQ save/restore tests

In reply to Kumar Kartikeya Dwivedi

Include tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.

#123/1   irq/irq_restore_missing_1:OK
#123/2   irq/irq_restore_missing_2:OK
#123/3   irq/irq_restore_missing_3:OK
#123/4   irq/irq_restore_missing_3_minus_2:OK
#123/5   irq/irq_restore_missing_1_subprog:OK
#123/6   irq/irq_restore_missing_2_subprog:OK
#123/7   irq/irq_restore_missing_3_subprog:OK
#123/8   irq/irq_restore_missing_3_minus_2_subprog:OK
#123/9   irq/irq_balance:OK
#123/10  irq/irq_balance_n:OK
#123/11  irq/irq_balance_subprog:OK
#123/12  irq/irq_balance_n_subprog:OK
#123/13  irq/irq_global_subprog:OK
#123/14  irq/irq_restore_ooo:OK
#123/15  irq/irq_restore_ooo_3:OK
#123/16  irq/irq_restore_3_subprog:OK
#123/17  irq/irq_restore_4_subprog:OK
#123/18  irq/irq_restore_ooo_3_subprog:OK
#123/19  irq/irq_restore_invalid:OK
#123/20  irq/irq_save_invalid:OK
#123/21  irq/irq_restore_iter:OK
#123/22  irq/irq_save_iter:OK
#123/23  irq/irq_flag_overwrite:OK
#123/24  irq/irq_flag_overwrite_partial:OK
#123/25  irq/irq_sleepable_helper:OK
#123/26  irq/irq_sleepable_kfunc:OK
#123     irq:OK

Summary: 1/26 PASSED, 0 SKIPPED, 0 FAILED

---
 tools/testing/selftests/bpf/prog_tests/irq.c |   9 +
 tools/testing/selftests/bpf/progs/irq.c      | 393 +++++++++++++++++++
 2 files changed, 402 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/irq.c
 create mode 100644 tools/testing/selftests/bpf/progs/irq.c
tools/testing/selftests/bpf/prog_tests/irq.c new file +9 -0
new file mode 100644
index 000000000000..496f4826ac37
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <irq.skel.h>
+
+void test_irq(void)
+{
+	RUN_TESTS(irq);
+}
tools/testing/selftests/bpf/progs/irq.c new file +393 -0

393 lines not on this page · raw message

Signature · 1 line
2.43.5
maintainer Acked-by on v1 7/7
#27 lore raw

Re: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests

Quoted 34 lines · Kumar Kartikeya Dwivedi

Include tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.

#123/1   irq/irq_restore_missing_1:OK
#123/2   irq/irq_restore_missing_2:OK
#123/3   irq/irq_restore_missing_3:OK
#123/4   irq/irq_restore_missing_3_minus_2:OK
#123/5   irq/irq_restore_missing_1_subprog:OK
#123/6   irq/irq_restore_missing_2_subprog:OK
#123/7   irq/irq_restore_missing_3_subprog:OK
#123/8   irq/irq_restore_missing_3_minus_2_subprog:OK
#123/9   irq/irq_balance:OK
#123/10  irq/irq_balance_n:OK
#123/11  irq/irq_balance_subprog:OK
#123/12  irq/irq_balance_n_subprog:OK
#123/13  irq/irq_global_subprog:OK
#123/14  irq/irq_restore_ooo:OK
#123/15  irq/irq_restore_ooo_3:OK
#123/16  irq/irq_restore_3_subprog:OK
#123/17  irq/irq_restore_4_subprog:OK
#123/18  irq/irq_restore_ooo_3_subprog:OK
#123/19  irq/irq_restore_invalid:OK
#123/20  irq/irq_save_invalid:OK
#123/21  irq/irq_restore_iter:OK
#123/22  irq/irq_save_iter:OK
#123/23  irq/irq_flag_overwrite:OK
#123/24  irq/irq_flag_overwrite_partial:OK
#123/25  irq/irq_sleepable_helper:OK
#123/26  irq/irq_sleepable_kfunc:OK
#123     irq:OK
Quoted 4 lines

Summary: 1/26 PASSED, 0 SKIPPED, 0 FAILED

---

The following error condition is not tested:
"arg#%d doesn't point to an irq flag on stack".
Also, I think a few tests are excessive.
Otherwise looks good.

[...]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +0 -0
new file mode 100644
index 000000000000..496f4826ac37

tools/testing/selftests/bpf/prog_tests/irq.c line 9

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +9 -0
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <irq.skel.h>
+
+void test_irq(void)
+{
+	RUN_TESTS(irq);
+}

Nit: tools/testing/selftests/bpf/prog_tests/verifier.c
could be used instead of a separate file.

Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c new file +0 -0
new file mode 100644
index 000000000000..5301b66fc752

tools/testing/selftests/bpf/progs/irq.c line 8

Quoted 8 lines
tools/testing/selftests/bpf/progs/irq.c new file +8 -0
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("?tc")
+__failure __msg("BPF_EXIT instruction cannot be used inside bpf_local_irq_save-ed region")

Nit: I know this is not a fault of this series, but the error message
is sort of confusing. BPF_EXIT is allowed for irq saved region, just it has to be an exit from a sub-program, not a whole program.

tools/testing/selftests/bpf/progs/irq.c line 15

Quoted 7 lines
tools/testing/selftests/bpf/progs/irq.c new file +7 -0
+int irq_restore_missing_1(struct __sk_buff *ctx)
+{
+	unsigned long flags;
+
+	bpf_local_irq_save(&flags);
+	return 0;
+}

[...]

Nit: don't think this test adds much compared to irq_restore_missing_2.

Quoted 10 lines
tools/testing/selftests/bpf/progs/irq.c new file +10 -0
+{
+	unsigned long flags1;
+	unsigned long flags2;
+	unsigned long flags3;
+
+	bpf_local_irq_save(&flags1);
+	bpf_local_irq_save(&flags2);
+	bpf_local_irq_save(&flags3);
+	return 0;
+}

[...]

tools/testing/selftests/bpf/progs/irq.c line 181

Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3 -0
+SEC("?tc")
+__success
+int irq_balance_n_subprog(struct __sk_buff *ctx)

Nit: don't think this test adds much given irq_balance_n()
and irq_balance_subprog().

Quoted 4 lines
tools/testing/selftests/bpf/progs/irq.c new file +4 -0
+{
+	local_irq_balance_n();
+	return 0;
+}

[...]

author maintainer
#28 lore raw

Re: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests

Quoted 38 lines · Eduard Zingerman
Quoted 36 lines · Kumar Kartikeya Dwivedi

Include tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.

#123/1   irq/irq_restore_missing_1:OK
#123/2   irq/irq_restore_missing_2:OK
#123/3   irq/irq_restore_missing_3:OK
#123/4   irq/irq_restore_missing_3_minus_2:OK
#123/5   irq/irq_restore_missing_1_subprog:OK
#123/6   irq/irq_restore_missing_2_subprog:OK
#123/7   irq/irq_restore_missing_3_subprog:OK
#123/8   irq/irq_restore_missing_3_minus_2_subprog:OK
#123/9   irq/irq_balance:OK
#123/10  irq/irq_balance_n:OK
#123/11  irq/irq_balance_subprog:OK
#123/12  irq/irq_balance_n_subprog:OK
#123/13  irq/irq_global_subprog:OK
#123/14  irq/irq_restore_ooo:OK
#123/15  irq/irq_restore_ooo_3:OK
#123/16  irq/irq_restore_3_subprog:OK
#123/17  irq/irq_restore_4_subprog:OK
#123/18  irq/irq_restore_ooo_3_subprog:OK
#123/19  irq/irq_restore_invalid:OK
#123/20  irq/irq_save_invalid:OK
#123/21  irq/irq_restore_iter:OK
#123/22  irq/irq_save_iter:OK
#123/23  irq/irq_flag_overwrite:OK
#123/24  irq/irq_flag_overwrite_partial:OK
#123/25  irq/irq_sleepable_helper:OK
#123/26  irq/irq_sleepable_kfunc:OK
#123     irq:OK

Summary: 1/26 PASSED, 0 SKIPPED, 0 FAILED

Quoted 8 lines
Quoted 2 lines

---

The following error condition is not tested:
"arg#%d doesn't point to an irq flag on stack".
Also, I think a few tests are excessive.
Otherwise looks good.

Will add.

Quoted 67 lines

[...]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +0 -0
new file mode 100644
index 000000000000..496f4826ac37

tools/testing/selftests/bpf/prog_tests/irq.c line 9

Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +9 -0
@@ -0,0 +1,9 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <irq.skel.h>
+
+void test_irq(void)
+{
+     RUN_TESTS(irq);
+}

Nit: tools/testing/selftests/bpf/prog_tests/verifier.c
could be used instead of a separate file.

Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c new file +0 -0
new file mode 100644
index 000000000000..5301b66fc752

tools/testing/selftests/bpf/progs/irq.c line 8

Quoted 8 lines
tools/testing/selftests/bpf/progs/irq.c new file +8 -0
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("?tc")
+__failure __msg("BPF_EXIT instruction cannot be used inside bpf_local_irq_save-ed region")

Nit: I know this is not a fault of this series, but the error message
is sort of confusing. BPF_EXIT is allowed for irq saved region, just it has to be an exit from a sub-program, not a whole program.

tools/testing/selftests/bpf/progs/irq.c line 15

Quoted 7 lines
tools/testing/selftests/bpf/progs/irq.c new file +7 -0
+int irq_restore_missing_1(struct __sk_buff *ctx)
+{
+     unsigned long flags;
+
+     bpf_local_irq_save(&flags);
+     return 0;
+}

[...]

Nit: don't think this test adds much compared to irq_restore_missing_2.

Quoted 10 lines
tools/testing/selftests/bpf/progs/irq.c new file +10 -0
+{
+     unsigned long flags1;
+     unsigned long flags2;
+     unsigned long flags3;
+
+     bpf_local_irq_save(&flags1);
+     bpf_local_irq_save(&flags2);
+     bpf_local_irq_save(&flags3);
+     return 0;
+}

[...]

Quoted 6 lines

tools/testing/selftests/bpf/progs/irq.c line 181

Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3 -0
+SEC("?tc")
+__success
+int irq_balance_n_subprog(struct __sk_buff *ctx)

Nit: don't think this test adds much given irq_balance_n()
and irq_balance_subprog().

My idea with both of these was to ensure when the state is copied in and out on calls and when we're doing one or more than one save/restore (which links prev_id into active_irq_id etc.) we don't have problems, so they were definitely testing different scenarios. But with the move into bpf_verifier_state they will indeed become redundant, so I'm going to drop them in v2.

Quoted 8 lines
Quoted 5 lines
tools/testing/selftests/bpf/progs/irq.c new file +4 -0
+{
+     local_irq_balance_n();
+     return 0;
+}

[...]

maintainer
#29 lore raw

Re: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests

On Thu, 2024-11-21 at 23:07 +0100, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 7 lines
Quoted 6 lines

tools/testing/selftests/bpf/progs/irq.c line 181

Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3 -0
+SEC("?tc")
+__success
+int irq_balance_n_subprog(struct __sk_buff *ctx)

Nit: don't think this test adds much given irq_balance_n()
and irq_balance_subprog().

Quoted 6 lines

My idea with both of these was to ensure when the state is copied in and out on calls and when we're doing one or more than one save/restore (which links prev_id into active_irq_id etc.) we don't have problems, so they were definitely testing different scenarios. But with the move into bpf_verifier_state they will indeed become redundant, so I'm going to drop them in v2.

Understood, thank you for explaining.