IRQ save/restore
Subsystems (4)
Kumar Kartikeya Dwivedi
maintainercover IRQ save/restore
This set introduces support for managing IRQ state from BPF programs. Two new kfuncs, bpf_local_irq_save, and bpf_local_irq_restore are introduced to enable this functionality.
Intended use cases are writing IRQ safe data structures (e.g. memory allocator) in BPF programs natively, and use in new spin locking primitives intended to be introduced in the next few weeks.
The set begins with some refactoring patches before the actual functionality is introduced. First, we rename bpf_reference_state to bpf_resource_state, and stick to calling pointer resources as actual references, as the meaning is right now mixed (with lock state management functions calling acquire_reference_state and check_reference_leak not considering lock resoures, causing confusion).
The first patch also takes this opportunity to eliminate code duplication where possible. The second patch resolves an inconsistency in the same set of functions for managing lock state, in preparation for later patches that require passing bpf_func_state explicitly to find a lock state.
Since we now manage locks in bpf_func_state alongside other resources, this is a good opportunity to consolidate all other resources like RCU read sections and preempt-disabled section tracking into bpf_func_state as well, which is what patch 3 achieves.
After this, patch 4 refactors stack slot liveness marking logic to be shared between dynptr, and iterators, in preparation for introducing same logic for irq flag object on stack.
Finally, patch 5 and 7 introduce the new kfuncs and their selftests. For more details, please inspect the patch commit logs. Patch 6 expands coverage of existing preempt-disable selftest to cover sleepable kfuncs.
Kumar Kartikeya Dwivedi (7):
bpf: Refactor and rename resource management
bpf: Be consistent between {acquire,find,release}_lock_state
bpf: Consolidate RCU and preempt locks in bpf_func_state
bpf: Refactor mark_{dynptr,iter}_read
bpf: Introduce support for bpf_local_irq_{save,restore} selftests/bpf: Expand coverage of preempt tests to sleepable kfunc selftests/bpf: Add IRQ save/restore tests
include/linux/bpf_verifier.h | 47 +- kernel/bpf/helpers.c | 24 + kernel/bpf/log.c | 11 +- kernel/bpf/verifier.c | 540 ++++++++++++++---- tools/testing/selftests/bpf/prog_tests/irq.c | 9 + tools/testing/selftests/bpf/progs/irq.c | 393 +++++++++++++ .../selftests/bpf/progs/preempt_lock.c | 12 + 7 files changed, 889 insertions(+), 147 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/irq.c create mode 100644 tools/testing/selftests/bpf/progs/irq.c
base-commit: 2c8b09ac2537299511c898bc71b1a5f2756c831c
Signature · 1 line
2.43.5
Kumar Kartikeya Dwivedi
maintainerPATCH 1/7 bpf: Refactor and rename resource management
With the commit f6b9a69a9e56 ("bpf: Refactor active lock management"), we have begun using the acquired_refs array to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.
This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:
Rename bpf_reference_state to bpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use in check_resource_leak. Next, "reference" now only means RES_TYPE_PTR, and the usage and meaning is updated accordingly.
Next, factor out common code paths for managing addition and removal of resource state in acquire_resource_state and erase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.
--- include/linux/bpf_verifier.h | 24 +++-- kernel/bpf/log.c | 10 +- kernel/bpf/verifier.c | 173 +++++++++++++++++++---------------- 3 files changed, 108 insertions(+), 99 deletions(-)
include/linux/bpf_verifier.h +11 -13
@@ -249,20 +249,18 @@ struct bpf_stack_state {u8 slot_type[BPF_REG_SIZE ]; }; -struct bpf_reference_state { -/* Each reference object has a type. Ensure REF_TYPE_PTR is zero to -* default to pointer reference on zero initialization of a state. -*/ -enum ref_state_type { -REF_TYPE_PTR =0 , -REF_TYPE_LOCK , +struct bpf_resource_state { +enum res_state_type { +RES_TYPE_INV =-1 , +RES_TYPE_PTR =0 , +RES_TYPE_LOCK , } type; -/* Track each reference created with a unique id, even if the same -* instruction creates the reference multiple times (eg, via CALL). +/* Track each resource created with a unique id, even if the same +* instruction creates the resource multiple times (eg, via CALL). */ int id; -/* Instruction where the allocation of this reference occurred. This -* is used purely to inform the user of a reference leak. +/* Instruction where the allocation of this resource occurred. This +* is used purely to inform the user of a resource leak. */ int insn_idx;/* Use to keep track of the source object of a lock, to ensure
@@ -315,9 +313,9 @@ struct bpf_func_state {u32 callback_depth;/* The following fields should be last. See copy_func_state() */ -int acquired_refs; +int acquired_res;int active_locks; -struct bpf_reference_state *refs; +struct bpf_resource_state *res;/* The state of the stack. Each element of the array describes BPF_REG_SIZE * (i.e. 8) bytes worth of stack memory. * stack[0] represents bytes [*(r10-8)..*(r10-1)]
kernel/bpf/log.c +5 -5
@@ -843,11 +843,11 @@ void print_verifier_state(struct bpf_verifier_env *env, const struct bpf_func_stbreak ; } } -if (state->acquired_refs && state->refs[0 ].id) { -verbose (env," refs=%d" , state->refs[0 ].id); -for (i =1 ; i < state->acquired_refs; i++) -if (state->refs[i].id) -verbose (env,",%d" , state->refs[i].id); +if (state->acquired_res && state->res[0 ].id) { +verbose (env," refs=%d" , state->res[0 ].id); +for (i =1 ; i < state->acquired_res; i++) +if (state->res[i].id) +verbose (env,",%d" , state->res[i].id); }if (state->in_callback_fn)verbose (env," cb" );
kernel/bpf/verifier.c +92 -81
326 lines not on this page · raw message
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management
Quoted 23 lines · Kumar Kartikeya Dwivedi
With the commit
f6b9a69a9e56("bpf: Refactor active lock management"), we have begun using theacquired_refsarray to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:
Rename
bpf_reference_statetobpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use incheck_resource_leak. Next, "reference" now only meansRES_TYPE_PTR, and the usage and meaning is updated accordingly.Next, factor out common code paths for managing addition and removal of resource state in
acquire_resource_stateanderase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.
Quoted 2 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
Tbh, I like the old name a bit more.
The patch itself looks good.
[...]
kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1345
Quoted 5 lines
kernel/bpf/verifier.c +1
-0@@ -1342,6 +1342,25 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_statereturn 0 ; } +static struct bpf_resource_state *acquire_resource_state (struct bpf_verifier_env *env,int insn_idx,int *id)
Nit: there is no need to pass int *id, as it is available as (returned)->id.
Quoted 21 lines
kernel/bpf/verifier.c +18
-0+{ +struct bpf_func_state *state =cur_func (env); +int new_ofs = state->acquired_res; +struct bpf_resource_state *s; +int err; + + err =resize_resource_state (state, state->acquired_res +1 ); +if (err) +return NULL ; + s = &state->res[new_ofs]; + s->type =RES_TYPE_INV ; +if (id) + *id = s->id = ++env->id_gen; + s->insn_idx = insn_idx; + +return s; +} +/* Acquire a pointer id from the env and update the state->refs to include * this new pointer reference. * On success, returns a valid pointer id to associate with the register
[...]
Quoted 1 line
(quoted diff) +0
-0@@ -1349,55 +1368,52 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state
[...]
kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1398
Quoted 2 lines
kernel/bpf/verifier.c +1
-1-/* release function corresponding to acquire_reference_state(). Idempotent. */ +static void erase_resource_state (struct bpf_func_state *state,int res_idx)
Nit: why not "release_..." to be consistent with the rest of the functions?
Quoted 31 lines
kernel/bpf/verifier.c +14
-10+{ +int last_idx = state->acquired_res -1 ; + +if (last_idx && res_idx != last_idx) +memcpy (&state->res[res_idx], &state->res[last_idx],sizeof (*state->res)); +memset (&state->res[last_idx],0 ,sizeof (*state->res)); + state->acquired_res--; +} +static int release_reference_state (struct bpf_func_state *state,int ptr_id) { -int i, last_idx; +int i; - last_idx = state->acquired_refs -1 ; -for (i =0 ; i < state->acquired_refs; i++) { -if (state->refs[i].type !=REF_TYPE_PTR ) +for (i =0 ; i < state->acquired_res; i++) { +if (state->res[i].type !=RES_TYPE_PTR )continue ; -if (state->refs[i].id == ptr_id) { -if (last_idx && i != last_idx) -memcpy (&state->refs[i], &state->refs[last_idx], -sizeof (*state->refs)); -memset (&state->refs[last_idx],0 ,sizeof (*state->refs)); - state->acquired_refs--; +if (state->res[i].id == ptr_id) { +erase_resource_state (state, i);return 0 ; } }
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management
Quoted 25 lines · Eduard Zingerman
Quoted 23 lines · Kumar Kartikeya Dwivedi
With the commit
f6b9a69a9e56("bpf: Refactor active lock management"), we have begun using theacquired_refsarray to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:
Rename
bpf_reference_statetobpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use incheck_resource_leak. Next, "reference" now only meansRES_TYPE_PTR, and the usage and meaning is updated accordingly.Next, factor out common code paths for managing addition and removal of resource state in
acquire_resource_stateanderase_resource_state, and then implement type specific resource handling on top of these common functions. Overall, this patch improves upon the confusion and minimizes code duplication, as we prepare to introduce new resource types in subsequent patches.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
Tbh, I like the old name a bit more.
The patch itself looks good.
I am happy for suggestions on better naming, but it would be better to make a distinction somehow.
Quoted 4 lines
Reviewed-by Eduard Zingerman <[email protected]>[...]
Quoted 8 lines
kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1345
Quoted 5 lines
kernel/bpf/verifier.c +1
-0@@ -1342,6 +1342,25 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_statereturn 0 ; } +static struct bpf_resource_state *acquire_resource_state (struct bpf_verifier_env *env,int insn_idx,int *id)Nit: there is no need to pass
int *id, as it is available as (returned)->id.
Replaced with a bool alloc_id to decide whether it generates a new id or not, and fixed.
Quoted 24 lines
Quoted 21 lines
kernel/bpf/verifier.c +18
-0+{ +struct bpf_func_state *state =cur_func (env); +int new_ofs = state->acquired_res; +struct bpf_resource_state *s; +int err; + + err =resize_resource_state (state, state->acquired_res +1 ); +if (err) +return NULL ; + s = &state->res[new_ofs]; + s->type =RES_TYPE_INV ; +if (id) + *id = s->id = ++env->id_gen; + s->insn_idx = insn_idx; + +return s; +} +/* Acquire a pointer id from the env and update the state->refs to include * this new pointer reference. * On success, returns a valid pointer id to associate with the register [...]
Quoted 9 lines
Quoted 1 line
(quoted diff) +0
-0@@ -1349,55 +1368,52 @@ static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state[...]
kernel/bpf/verifier.c static int grow_stack_state(struct bpf_verifier_env *env, struct bpf_func_state · line 1398
Quoted 2 lines
kernel/bpf/verifier.c +1
-1-/* release function corresponding to acquire_reference_state(). Idempotent. */ +static void erase_resource_state (struct bpf_func_state *state,int res_idx)Nit: why not "
release_..." to be consistent with the rest of the functions?
This was a subset of what "release_resource_state" would have done, since it erases a res_idx,
but on second thought, it's probably better to rename, so fixed as well.
Thanks for the review.
Quoted 34 lines
Quoted 31 lines
kernel/bpf/verifier.c +14
-10+{ +int last_idx = state->acquired_res -1 ; + +if (last_idx && res_idx != last_idx) +memcpy (&state->res[res_idx], &state->res[last_idx],sizeof (*state->res)); +memset (&state->res[last_idx],0 ,sizeof (*state->res)); + state->acquired_res--; +} +static int release_reference_state (struct bpf_func_state *state,int ptr_id) { -int i, last_idx; +int i; - last_idx = state->acquired_refs -1 ; -for (i =0 ; i < state->acquired_refs; i++) { -if (state->refs[i].type !=REF_TYPE_PTR ) +for (i =0 ; i < state->acquired_res; i++) { +if (state->res[i].type !=RES_TYPE_PTR )continue ; -if (state->refs[i].id == ptr_id) { -if (last_idx && i != last_idx) -memcpy (&state->refs[i], &state->refs[last_idx], -sizeof (*state->refs)); -memset (&state->refs[last_idx],0 ,sizeof (*state->refs)); - state->acquired_refs--; +if (state->res[i].id == ptr_id) { +erase_resource_state (state, i);return 0 ; } }[...]
Re: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management
In reply to Kumar Kartikeya DwivediQuoted 12 lines · Kumar Kartikeya Dwivedi
With the commit
f6b9a69a9e56("bpf: Refactor active lock management"), we have begun using theacquired_refsarray to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:
Quoted 4 lines
Rename
bpf_reference_statetobpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use incheck_resource_leak. Next, "reference" now only meansRES_TYPE_PTR, and the usage and meaning is updated accordingly.
Sorry I don't like this renaming.
reference state is already understood as a set of resources that were acquired.
Whether it's an object allocated by bpf_obj_new or any other resource.
I think this patch has a net negative effect.
People familiar with the verifier already understand what refsafe() or acquired_refs are for.
Calling them slightly different names adds confusion, not clarity.
pw-bot: cr
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 1/7] bpf: Refactor and rename resource management
Quoted 31 lines · Alexei Starovoitov
Quoted 12 lines · Kumar Kartikeya Dwivedi
With the commit
f6b9a69a9e56("bpf: Refactor active lock management"), we have begun using theacquired_refsarray to also store active lock metadata, as a way to consolidate and manage all kernel resources that the program may acquire.This is beginning to cause some confusion and duplication in existing code, where the terms references now both mean lock reference state and the references for acquired kernel object pointers. To clarify and improve the current state of affairs, as well as reduce code duplication, make the following changes:
Quoted 4 lines
Rename
bpf_reference_statetobpf_resource_state, and begin using resource as the umbrella term. This terminology matches what we use incheck_resource_leak. Next, "reference" now only meansRES_TYPE_PTR, and the usage and meaning is updated accordingly.Sorry I don't like this renaming.
reference state is already understood as a set of resources that were acquired.
Whether it's an object allocated bybpf_obj_newor any other resource.
I think this patch has a net negative effect.
People familiar with the verifier already understand whatrefsafe()oracquired_refsare for.
Calling them slightly different names adds confusion, not clarity.
Quoted 1 line
pw-bot: cr
Ok
Kumar Kartikeya Dwivedi
maintainerPATCH 2/7 bpf: Be consistent between {acquire,find,release}_lock_state
In reply to Kumar Kartikeya DwivediBoth acquire_lock_state and release_lock_state take the bpf_func_state as a parameter, while find_lock_state does not. Future patches will end up requiring operating on non-cur_func(env) bpf_func_state (for resilient locks), hence just make the prototype consistent and take bpf_func_state directly.
--- kernel/bpf/verifier.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +2 -3
@@ -1436,10 +1436,9 @@ static int release_lock_state(struct bpf_func_state *state, int type, int id, voreturn -EINVAL ; } -static struct bpf_resource_state *find_lock_state (struct bpf_verifier_env *env,enum res_state_type type, +static struct bpf_resource_state *find_lock_state (struct bpf_func_state *state,enum res_state_type type,int id,void *ptr) { -struct bpf_func_state *state =cur_func (env);int i;for (i =0 ; i < state->acquired_res; i++) {
@@ -11873,7 +11872,7 @@ static int check_reg_allocation_locked(struct bpf_verifier_env *env, struct bpf_if (!cur_func (env)->active_locks)return -EINVAL ; - s =find_lock_state (env,RES_TYPE_LOCK , id, ptr); + s =find_lock_state (cur_func (env),RES_TYPE_LOCK , id, ptr);if (!s) {verbose (env,"held lock and object are not in the same allocation\n" );return -EINVAL ;
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 2/7] bpf: Be consistent between {acquire,find,release}_lock_state
Quoted 8 lines · Kumar Kartikeya Dwivedi
Both
acquire_lock_stateandrelease_lock_statetake thebpf_func_stateas a parameter, whilefind_lock_statedoes not. Future patches will end up requiring operating on non-cur_func(env)bpf_func_state(for resilient locks), hence just make the prototype consistent and takebpf_func_statedirectly.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
Kumar Kartikeya Dwivedi
maintainerPATCH 3/7 bpf: Consolidate RCU and preempt locks in bpf_func_state
In reply to Kumar Kartikeya DwivediTo ensure consistency in resource handling, move RCU and preemption state counters to bpf_func_state, and convert all users to access them through cur_func(env).
For the sake of consistency, also compare active_locks in ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.
OTOH, the comparison of active_preempt_locks and active_rcu_lock is needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in the acquired_res array.
--- include/linux/bpf_verifier.h | 4 ++-- kernel/bpf/verifier.c | 46 ++++++++++++++++++++---------------- 2 files changed, 27 insertions(+), 23 deletions(-)
include/linux/bpf_verifier.h +2 -2
@@ -315,6 +315,8 @@ struct bpf_func_state {/* The following fields should be last. See copy_func_state() */ int acquired_res;int active_locks; +int active_preempt_locks; +bool active_rcu_lock;struct bpf_resource_state *res;/* The state of the stack. Each element of the array describes BPF_REG_SIZE * (i.e. 8) bytes worth of stack memory.
@@ -418,8 +420,6 @@ struct bpf_verifier_state {u32 curframe;bool speculative; -bool active_rcu_lock; -u32 active_preempt_lock;/* If this state was ever pointed-to by other state's loop_entry field * this flag would be set to true. Used to avoid freeing such states * while they are still in use.
kernel/bpf/verifier.c +25 -21
134 lines not on this page · raw message
Signature · 1 line
2.43.5
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state
Quoted 8 lines · Kumar Kartikeya Dwivedi
To ensure consistency in resource handling, move RCU and preemption state counters to
bpf_func_state, and convert all users to access them throughcur_func(env).For the sake of consistency, also compare
active_locksin ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.
Quoted 7 lines
OTOH, the comparison of
active_preempt_locksandactive_rcu_lockis needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in theacquired_resarray.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
This change is a bit confusing to me.
The following is done currently:
- in setup_func_entry() called from check_func_call():
copy_resource_state(callee, caller);
- in prepare_func_exit():
copy_resource_state(caller, callee);
So it seems that it is logical to track resources in the bpf_verifier_state and avoid copying.
There is probably something I don't understand.
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state
Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi
To ensure consistency in resource handling, move RCU and preemption state counters to
bpf_func_state, and convert all users to access them throughcur_func(env).For the sake of consistency, also compare
active_locksin ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.Quoted 7 lines
OTOH, the comparison of
active_preempt_locksandactive_rcu_lockis needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in theacquired_resarray.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
This change is a bit confusing to me.
The following is done currently:
- insetup_func_entry()called fromcheck_func_call():
copy_resource_state(callee, caller);
- inprepare_func_exit():
copy_resource_state(caller, callee);
Quoted 4 lines
So it seems that it is logical to track resources in the
bpf_verifier_stateand avoid copying.
There is probably something I don't understand.
This is what we were doing all along, and you're right, it is sort of a global entity.
But we've moved active_locks to bpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be incur_func, or in bpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.
Alexei's preference has been stashing this in bpf_func_state instead in [0]. Let me know what you think.
[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com
Quoted 2 lines
[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state
Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi
To ensure consistency in resource handling, move RCU and preemption state counters to
bpf_func_state, and convert all users to access them throughcur_func(env).For the sake of consistency, also compare
active_locksin ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.Quoted 7 lines
OTOH, the comparison of
active_preempt_locksandactive_rcu_lockis needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in theacquired_resarray.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
This change is a bit confusing to me.
The following is done currently:
- insetup_func_entry()called fromcheck_func_call():
copy_resource_state(callee, caller);
- inprepare_func_exit():
copy_resource_state(caller, callee);
Quoted 7 lines
Quoted 4 lines
So it seems that it is logical to track resources in the
bpf_verifier_stateand avoid copying.
There is probably something I don't understand.This is what we were doing all along, and you're right, it is sort of a global entity.
Right, but since this patch-set does a refactoring, might be a good time to change.
Quoted 11 lines
But we've moved
active_lockstobpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be incur_func, or inbpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.Alexei's preference has been stashing this in
bpf_func_stateinstead in [0]. Let me know what you think.[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com
As far as I understand check_func_call(), function calls to static functions are allowed while holding each kind of resources currently tracked. So it seems odd to track it as a part of function state. The way I understand Alexei in the thread [0] the idea is more to track all counters in one place.
Let's wait what Alexei has to say.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 3/7] bpf: Consolidate RCU and preempt locks in bpf_func_state
Quoted 51 lines · Eduard Zingerman
Quoted 26 lines · Kumar Kartikeya Dwivedi
Quoted 25 lines · Eduard Zingerman
Quoted 8 lines · Kumar Kartikeya Dwivedi
To ensure consistency in resource handling, move RCU and preemption state counters to
bpf_func_state, and convert all users to access them throughcur_func(env).For the sake of consistency, also compare
active_locksin ressafe as a quick way to eliminate iteration and entry matching if the number of locks are not the same.Quoted 7 lines
OTOH, the comparison of
active_preempt_locksandactive_rcu_lockis needed for correctness, as state exploration cannot be avoided if these counters do not match, and not comparing them will lead to problems since they lack an actual entry in theacquired_resarray.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
This change is a bit confusing to me.
The following is done currently:
- insetup_func_entry()called fromcheck_func_call():
copy_resource_state(callee, caller);
- inprepare_func_exit():
copy_resource_state(caller, callee);Quoted 7 lines
Quoted 4 lines
So it seems that it is logical to track resources in the
bpf_verifier_stateand avoid copying.
There is probably something I don't understand.This is what we were doing all along, and you're right, it is sort of a global entity.
Right, but since this patch-set does a refactoring, might be a good time to change.
Quoted 11 lines
But we've moved
active_lockstobpf_func_state, where references reside, while RCU and preempt lock state stays in verifier state. Either everything should be incur_func, or inbpf_verifier_state. I am fine with either of them, because it would
materially does not matter too much.Alexei's preference has been stashing this in
bpf_func_stateinstead in [0]. Let me know what you think.[0] https://lore.kernel.org/bpf/CAADnVQKxgE7=WhjNckvMDTZ5GZujPuT3Dqd+sY=pW8CWoaF9FA@mail.gmail.com
Quoted 8 lines
As far as I understand
check_func_call(), function calls to static functions are allowed while holding each kind of resources currently tracked. So it seems odd to track it as a part of function state. The way I understand Alexei in the thread [0] the idea is more to track all counters in one place.Let's wait what Alexei has to say.
Discussed with Alexei (who discussed with you I presume) that we're doing this in bpf_verifier_state, will fix.
Kumar Kartikeya Dwivedi
maintainerPATCH 4/7 bpf: Refactor mark_{dynptr,iter}_read
In reply to Kumar Kartikeya DwivediThere is possibility of sharing code between mark_dynptr_read and mark_iter_read for updating liveness information of their stack slots. Consolidate common logic into mark_stack_slot_obj_read function in preparation for the next patch which needs the same logic for its own stack slots.
--- kernel/bpf/verifier.c | 43 +++++++++++++++++++++---------------------- 1 file changed, 21 insertions(+), 22 deletions(-)
kernel/bpf/verifier.c +21 -22
@@ -3213,10 +3213,27 @@ static int mark_reg_read(struct bpf_verifier_env *env,return 0 ; } -static int mark_dynptr_read (struct bpf_verifier_env *env,struct bpf_reg_state *reg) +static int mark_stack_slot_obj_read (struct bpf_verifier_env *env,struct bpf_reg_state *reg, +int spi,int nr_slots) {struct bpf_func_state *state =func (env, reg); -int spi, ret; +int err, i; + +for (i =0 ; i < nr_slots; i++) { +struct bpf_reg_state *st = &state->stack[spi - i].spilled_ptr; + + err =mark_reg_read (env, st, st->parent,REG_LIVE_READ64 ); +if (err) +return err; + +mark_stack_slot_scratched (env, spi - i); + } +return 0 ; +} + +static int mark_dynptr_read (struct bpf_verifier_env *env,struct bpf_reg_state *reg) +{ +int spi;/* For CONST_PTR_TO_DYNPTR, it must have already been done by * check_reg_arg in check_helper_call and mark_btf_func_reg_size in
@@ -3231,31 +3248,13 @@ static int mark_dynptr_read(struct bpf_verifier_env *env, struct bpf_reg_state ** bounds and spi is the first dynptr slot. Simply mark stack slot as * read. */ - ret =mark_reg_read (env, &state->stack[spi].spilled_ptr, - state->stack[spi].spilled_ptr.parent,REG_LIVE_READ64 ); -if (ret) -return ret; -return mark_reg_read (env, &state->stack[spi -1 ].spilled_ptr, - state->stack[spi -1 ].spilled_ptr.parent,REG_LIVE_READ64 ); +return mark_stack_slot_obj_read (env, reg, spi,BPF_DYNPTR_NR_SLOTS ); }static int mark_iter_read (struct bpf_verifier_env *env,struct bpf_reg_state *reg,int spi,int nr_slots) { -struct bpf_func_state *state =func (env, reg); -int err, i; - -for (i =0 ; i < nr_slots; i++) { -struct bpf_reg_state *st = &state->stack[spi - i].spilled_ptr; - - err =mark_reg_read (env, st, st->parent,REG_LIVE_READ64 ); -if (err) -return err; - -mark_stack_slot_scratched (env, spi - i); - } - -return 0 ; +return mark_stack_slot_obj_read (env, reg, spi, nr_slots); }/* This function is supposed to be used by the following 32-bit optimization
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 4/7] bpf: Refactor mark_{dynptr,iter}_read
Quoted 8 lines · Kumar Kartikeya Dwivedi
There is possibility of sharing code between
mark_dynptr_readandmark_iter_readfor updating liveness information of their stack slots. Consolidate common logic intomark_stack_slot_obj_readfunction in preparation for the next patch which needs the same logic for its own stack slots.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
Kumar Kartikeya Dwivedi
maintainerPATCH 5/7 bpf: Introduce support for bpf_local_irq_{save,restore}
In reply to Kumar Kartikeya DwivediTeach the verifier about IRQ-disabled sections through the introduction of two new kfuncs, bpf_local_irq_save, to save IRQ state and disable them, and bpf_local_irq_restore, to restore IRQ state and enable them back again.
For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.
To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type RES_TYPE_IRQ is introduced, which its state management functions: acquire_irq_state and release_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.
One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member 'prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state's active_irq_id.
The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform check_ids based satisfiability check, and additionally match prev_id for RES_TYPE_IRQ entries in the resource array.
The kfuncs themselves are plain wrappers over local_irq_save and local_irq_restore macros.
--- include/linux/bpf_verifier.h | 19 ++- kernel/bpf/helpers.c | 24 +++ kernel/bpf/log.c | 1 + kernel/bpf/verifier.c | 283 ++++++++++++++++++++++++++++++++++- 4 files changed, 322 insertions(+), 5 deletions(-)
include/linux/bpf_verifier.h +15 -4
@@ -233,6 +233,7 @@ enum bpf_stack_slot_type {*/ STACK_DYNPTR ,STACK_ITER , + STACK_IRQ_FLAG, };#define BPF_REG_SIZE 8/* size of eBPF register in bytes */
@@ -253,6 +254,9 @@ struct bpf_resource_state {enum res_state_type {RES_TYPE_INV =-1 ,RES_TYPE_PTR =0 , +RES_TYPE_IRQ , + + __RES_TYPE_LOCK_BEGIN,RES_TYPE_LOCK , } type;/* Track each resource created with a unique id, even if the same
@@ -263,10 +267,16 @@ struct bpf_resource_state {* is used purely to inform the user of a resource leak. */ int insn_idx; -/* Use to keep track of the source object of a lock, to ensure -* it matches on unlock. -*/ -void *ptr; +union { +/* Use to keep track of the source object of a lock, to ensure +* it matches on unlock. +*/ +void *ptr; +/* Track the reference id preceding the IRQ entry in acquisition +* order, to enforce an ordering on the release. +*/ +int prev_id; + }; };struct bpf_retval_range {
@@ -317,6 +327,7 @@ struct bpf_func_state {int active_locks;int active_preempt_locks;bool active_rcu_lock; +int active_irq_id;struct bpf_resource_state *res;/* The state of the stack. Each element of the array describes BPF_REG_SIZE * (i.e. 8) bytes worth of stack memory.
kernel/bpf/helpers.c +24 -0
@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __userreturn ret +1 ; } +/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag) +{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags; + +local_irq_save (flags); + *ptr = flags; +} + +__bpf_kfunc void bpf_local_irq_restore(unsigned long *flags__irq_flag) +{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags = *ptr; + +local_irq_restore (flags); +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (generic_btf_ids)
@@ -3149,6 +3171,8 @@ BTF_ID_FLAGS(func, bpf_get_kmem_cache) BTF_ID_FLAGS(func, bpf_iter_kmem_cache_new, KF_ITER_NEW | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_kmem_cache_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_kmem_cache_destroy, KF_ITER_DESTROY | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_local_irq_save) +BTF_ID_FLAGS(func, bpf_local_irq_restore)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/log.c +1 -0
@@ -537,6 +537,7 @@ static char slot_type_char[] = { [STACK_ZERO] = '0', [STACK_DYNPTR] = 'd', [STACK_ITER] = 'i', + [STACK_IRQ_FLAG ] ='f' }; static void print_liveness(struct bpf_verifier_env *env,
kernel/bpf/verifier.c +282 -1
425 lines not on this page · raw message
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
Quoted 30 lines · Kumar Kartikeya Dwivedi
Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs,
bpf_local_irq_save, to save IRQ state and disable them, andbpf_local_irq_restore, to restore IRQ state and enable them back again.For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type
STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type
RES_TYPE_IRQis introduced, which its state management functions:acquire_irq_stateandrelease_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member '
prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state'sactive_irq_id.The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform
check_idsbased satisfiability check, and additionally matchprev_idforRES_TYPE_IRQentries in the resource array.
Quoted 5 lines
The kfuncs themselves are plain wrappers over
local_irq_saveandlocal_irq_restoremacros.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
I think this matches what is done for iterators and dynptrs.
[...]
Quoted 8 lines
include/linux/bpf_verifier.h +0
-4@@ -263,10 +267,16 @@ struct bpf_resource_state {* is used purely to inform the user of a resource leak. */ int insn_idx; -/* Use to keep track of the source object of a lock, to ensure -* it matches on unlock. -*/ -void *ptr;
include/linux/bpf_verifier.h struct bpf_resource_state { · line 279
Quoted 10 lines
include/linux/bpf_verifier.h +10
-0+union { +/* Use to keep track of the source object of a lock, to ensure +* it matches on unlock. +*/ +void *ptr; +/* Track the reference id preceding the IRQ entry in acquisition +* order, to enforce an ordering on the release. +*/ +int prev_id; + };
Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative to prev_id would be to organize bpf_func_state->res as a stack (by changing erase_resource_state() implementation).
[...]
Quoted 5 lines
kernel/bpf/helpers.c +0
-0
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065
Quoted 9 lines
kernel/bpf/helpers.c +6
-0@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __userreturn ret +1 ; } +/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)
Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?
Quoted 7 lines
kernel/bpf/helpers.c +7
-0+{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags; + +local_irq_save (flags); + *ptr = flags; +}
[...]
kernel/bpf/helpers.c static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state, · line 1610
Quoted 6 lines
kernel/bpf/helpers.c +1
-1@@ -1447,7 +1607,7 @@ static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state,for (i =0 ; i < state->acquired_res; i++) {struct bpf_resource_state *s = &state->res[i]; -if (s->type ==RES_TYPE_PTR || s->type != type) +if (s->type < __RES_TYPE_LOCK_BEGIN || s->type != type)
Nit: I think this would be easier to read if there was a bitmask
associated with lock types.
Quoted 3 lines
kernel/bpf/helpers.c +0
-0continue ;if (s->id == id && s->ptr == ptr)
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
Quoted 58 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi
Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs,
bpf_local_irq_save, to save IRQ state and disable them, andbpf_local_irq_restore, to restore IRQ state and enable them back again.For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type
STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type
RES_TYPE_IRQis introduced, which its state management functions:acquire_irq_stateandrelease_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member '
prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state'sactive_irq_id.The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform
check_idsbased satisfiability check, and additionally matchprev_idforRES_TYPE_IRQentries in the resource array.Quoted 5 lines
The kfuncs themselves are plain wrappers over
local_irq_saveandlocal_irq_restoremacros.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
I think this matches what is done for iterators and dynptrs.
Acked-by Eduard Zingerman <[email protected]>[...]
Quoted 14 lines
include/linux/bpf_verifier.h +6
-4@@ -263,10 +267,16 @@ struct bpf_resource_state {* is used purely to inform the user of a resource leak. */ int insn_idx; -/* Use to keep track of the source object of a lock, to ensure -* it matches on unlock. -*/ -void *ptr; +union { +/* Use to keep track of the source object of a lock, to ensure +* it matches on unlock. +*/ +void *ptr; +/* Track the reference id preceding the IRQ entry in acquisition
Quoted 8 lines
include/linux/bpf_verifier.h struct bpf_resource_state { · line 279
Quoted 4 lines
include/linux/bpf_verifier.h +4
-0+* order, to enforce an ordering on the release. +*/ +int prev_id; + };Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative toprev_idwould be to organizebpf_func_state->resas a stack (by changingerase_resource_state()implementation).
I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.
Quoted 11 lines
[...]
Quoted 8 lines
kernel/bpf/helpers.c +0
-0@@ -3057,6 +3057,28 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __userreturn ret +1 ; }
Quoted 9 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065
Quoted 6 lines
kernel/bpf/helpers.c +6
-0+/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?
It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted to vmlinux.h expects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.
Quoted 15 lines
Quoted 8 lines
kernel/bpf/helpers.c +7
-0+{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags; + +local_irq_save (flags); + *ptr = flags; +}[...]
Quoted 4 lines
kernel/bpf/helpers.c +0
-0@@ -1447,7 +1607,7 @@ static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state,for (i =0 ; i < state->acquired_res; i++) {struct bpf_resource_state *s = &state->res[i];
Quoted 5 lines
kernel/bpf/helpers.c static struct bpf_resource_state *find_lock_state(struct bpf_func_state *state, · line 1610
Quoted 2 lines
kernel/bpf/helpers.c +1
-1-if (s->type ==RES_TYPE_PTR || s->type != type) +if (s->type < __RES_TYPE_LOCK_BEGIN || s->type != type)Nit: I think this would be easier to read if there was a bitmask
associated with lock types.
Ack, will fix.
Quoted 7 lines
Quoted 4 lines
kernel/bpf/helpers.c +0
-0continue ;if (s->id == id && s->ptr == ptr)[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:
[...]
Quoted 10 lines
Quoted 9 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065
Quoted 6 lines
kernel/bpf/helpers.c +6
-0+/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?
Quoted 9 lines
It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted tovmlinux.hexpects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.
So, the goal is to enable the following pattern:
unsigned long flags; bpf_local_irq_save(&flags);
Right?
For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:
unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.
bpf_local_irq_save(&flags);
I think that in such case '*ptr = flags;' would overwrite foo.
[...]
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
kernel/bpf/helpers.c +7
-0+{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags; + +local_irq_save (flags); + *ptr = flags; +}
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
Quoted 35 lines · Eduard Zingerman
On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:
[...]
Quoted 10 lines
Quoted 9 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065
Quoted 6 lines
kernel/bpf/helpers.c +6
-0+/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?Quoted 9 lines
It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted tovmlinux.hexpects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.So, the goal is to enable the following pattern:
unsigned long flags; bpf_local_irq_save(&flags);Right?
For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:
Quoted 6 lines
unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.bpf_local_irq_save(&flags);I think that in such case '*ptr = flags;' would overwrite foo.
In the kernel or userspace, yes, but I'm assuming unsigned long will always be 64-bit for target=BPF.
Would that be incorrect? This pattern will only happen within BPF programs.
Quoted 11 lines
[...]
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
kernel/bpf/helpers.c +7
-0+{ +u64 *ptr = (u64 *)flags__irq_flag; +unsigned long flags; + +local_irq_save (flags); + *ptr = flags; +}
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 38 lines · Eduard Zingerman
On Thu, 2024-11-21 at 23:06 +0100, Kumar Kartikeya Dwivedi wrote:
[...]
Quoted 10 lines
Quoted 9 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user · line 3065
Quoted 6 lines
kernel/bpf/helpers.c +6
-0+/* Keep unsinged long in prototype so that kfunc is usable when emitted to +* vmlinux.h in BPF programs directly, but since unsigned long may potentially +* be 4 byte, always cast to u64 when reading/writing from this pointer as it +* always points to an 8-byte memory region in BPF stack. +*/ +__bpf_kfunc void bpf_local_irq_save(unsigned long *flags__irq_flag)Nit: 'unsigned long long' is guaranteed to be at-least 64 bit.
What would go wrong if 'u64' is used here?Quoted 9 lines
It goes like this:
If I make this unsigned long long * or u64 *, the kfunc emitted tovmlinux.hexpects a pointer of that type.
Typically, kernel code is always passing unsigned long flags to these functions, and that's what people are used to.
Given for --target=bpf unsigned long * is always a 8-byte value, I just did this, so that in kernels that are 32-bit,
we don't end up relying on unsigned long still being 8 when fetching/storing flags on BPF stack.So, the goal is to enable the following pattern:
unsigned long flags; bpf_local_irq_save(&flags);Right?
For a 32-bit system 'flags' would be 4 bytes long. Consider the following example:
unsigned long flags; // assume 'flags' and 'foo'
int foo; // are allocated sequentially.
Quoted 7 lines
Quoted 3 lines
bpf_local_irq_save(&flags);I think that in such case '*ptr = flags;' would overwrite foo.
In the kernel or userspace, yes, but I'm assuming unsigned long will always be 64-bit for target=BPF.
Would that be incorrect? This pattern will only happen within BPF programs.
Discussed off-list.
Kumar is right, and there is no problem, as on BPF side 'unsigned long' is always 8 bytes.
[...]
Re: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
In reply to Kumar Kartikeya DwivediQuoted 69 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi
Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs,
bpf_local_irq_save, to save IRQ state and disable them, andbpf_local_irq_restore, to restore IRQ state and enable them back again.For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type
STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type
RES_TYPE_IRQis introduced, which its state management functions:acquire_irq_stateandrelease_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member '
prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state'sactive_irq_id.The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform
check_idsbased satisfiability check, and additionally matchprev_idforRES_TYPE_IRQentries in the resource array.Quoted 5 lines
The kfuncs themselves are plain wrappers over
local_irq_saveandlocal_irq_restoremacros.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
I think this matches what is done for iterators and dynptrs.
Acked-by Eduard Zingerman <[email protected]>[...]
Quoted 14 lines
include/linux/bpf_verifier.h +6
-4@@ -263,10 +267,16 @@ struct bpf_resource_state {* is used purely to inform the user of a resource leak. */ int insn_idx; -/* Use to keep track of the source object of a lock, to ensure -* it matches on unlock. -*/ -void *ptr; +union { +/* Use to keep track of the source object of a lock, to ensure +* it matches on unlock. +*/ +void *ptr; +/* Track the reference id preceding the IRQ entry in acquisition Quoted 8 lines
include/linux/bpf_verifier.h struct bpf_resource_state { · line 279
Quoted 4 lines
include/linux/bpf_verifier.h +4
-0+* order, to enforce an ordering on the release. +*/ +int prev_id; + };Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative toprev_idwould be to organizebpf_func_state->resas a stack (by changingerase_resource_state()implementation).
Quoted 2 lines
I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.
Acquire_refs is already a stack.
Manual push/pop via prev_id looks unnecessary. Just search the top of acquired_refs for id. If it doesn't match error.
I don't like this bit either:
+ if (id != state->active_irq_id)
+ return -EPROTO;
Why invent new error codes for such conditions? It's EACESS or EINVAL like everywhere else in the verifier.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
Quoted 72 lines · Alexei Starovoitov
Quoted 69 lines · Kumar Kartikeya Dwivedi
Quoted 66 lines · Eduard Zingerman
Quoted 30 lines · Kumar Kartikeya Dwivedi
Teach the verifier about IRQ-disabled sections through the introduction of two new kfuncs,
bpf_local_irq_save, to save IRQ state and disable them, andbpf_local_irq_restore, to restore IRQ state and enable them back again.For the purposes of tracking the saved IRQ state, the verifier is taught about a new special object on the stack of type
STACK_IRQ_FLAG. This is a 8 byte value which saves the IRQ flags which are to be passed back to the IRQ restore kfunc.To track a dynamic number of IRQ-disabled regions and their associated saved states, a new resource type
RES_TYPE_IRQis introduced, which its state management functions:acquire_irq_stateandrelease_irq_state, taking advantage of the refactoring and clean ups made in earlier commits.One notable requirement of the kernel's IRQ save and restore API is that they cannot happen out of order. For this purpose, resource state is extended with a new type-specific member '
prev_id'. This is used to remember the ordering of acquisitions of IRQ saved states, so that we maintain a logical stack in acquisition order of resource identities, and can enforce LIFO ordering when restoring IRQ state. The top of the stack is maintained using bpf_func_state'sactive_irq_id.The logic to detect initialized and unitialized irq flag slots, marking and unmarking is similar to how it's done for iterators. We do need to update ressafe to perform
check_idsbased satisfiability check, and additionally matchprev_idforRES_TYPE_IRQentries in the resource array.Quoted 5 lines
The kfuncs themselves are plain wrappers over
local_irq_saveandlocal_irq_restoremacros.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
I think this matches what is done for iterators and dynptrs.
Acked-by Eduard Zingerman <[email protected]>[...]
Quoted 8 lines
include/linux/bpf_verifier.h +0
-4@@ -263,10 +267,16 @@ struct bpf_resource_state {* is used purely to inform the user of a resource leak. */ int insn_idx; -/* Use to keep track of the source object of a lock, to ensure -* it matches on unlock. -*/ -void *ptr;include/linux/bpf_verifier.h struct bpf_resource_state { · line 279
Quoted 10 lines
include/linux/bpf_verifier.h +10
-0+union { +/* Use to keep track of the source object of a lock, to ensure +* it matches on unlock. +*/ +void *ptr; +/* Track the reference id preceding the IRQ entry in acquisition +* order, to enforce an ordering on the release. +*/ +int prev_id; + };Nit: Do we anticipate any other resource kinds that would need LIFO acquire/release?
If we do, an alternative toprev_idwould be to organizebpf_func_state->resas a stack (by changingerase_resource_state()implementation).
Quoted 7 lines
Quoted 2 lines
I don't think so, this was the weird case requiring such an ordering, so I tried to find the least intrusive way.
Acquire_refsis already a stack.
Manual push/pop viaprev_idlooks unnecessary. Just search the top ofacquired_refsfor id. If it doesn't match error.
Ok.
Quoted 7 lines
I don't like this bit either:
+ if (id !=state->active_irq_id)
+ return-EPROTO;Why invent new error codes for such conditions? It's EACESS or EINVAL like everywhere else in the verifier.
Ok, though we do use EPROTO in a bunch of places. I just needed something to distinguish the two errors.
I'll switch to these.
kernel test robot
build botRe: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore}
In reply to Kumar Kartikeya Dwivedikernel test robot noticed the following build warnings:
Hi Kumar,
kernel test robot noticed the following build warnings:
[auto build test WARNING on bpf-next/for-next]
[also build test WARNING on bpf-next/master linus/master next-20241121] [cannot apply to bpf/master v6.12]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Kumar-Kartikeya-Dwivedi/bpf-Refactor-and-rename-resource-management/20241121-140722 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git for-next patch link: https://lore.kernel.org/r/20241121005329.408873-6-memxor%40gmail.com patch subject: [PATCH bpf-next v1 5/7] bpf: Introduce support for bpf_local_irq_{save,restore} config: arc-randconfig-001-20241122 (https://download.01.org/0day-ci/archive/20241122/[email protected]/config) compiler: arc-elf-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241122/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
kernel/bpf/verifier.c: In function 'process_irq_flag':
>> kernel/bpf/verifier.c:11964:32: warning: variable 'irq_restore' set but not used [-Wunused-but-set-variable]
11964 | bool irq_save = false, irq_restore = false;
| ^~~~~~~~~~~vim +/irq_restore +11964 kernel/bpf/verifier.c
11959
11960 static int process_irq_flag(struct bpf_verifier_env *env, int regno,
11961 struct bpf_kfunc_call_arg_meta *meta)
11962 {
11963 struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno];
11964 bool irq_save = false, irq_restore = false;
11965 int err;
11966
11967 if (meta->func_id == special_kfunc_list[KF_bpf_local_irq_save]) {
11968 irq_save = true;
11969 } else if (meta->func_id == special_kfunc_list[KF_bpf_local_irq_restore]) {
11970 irq_restore = true;
11971 } else {
11972 verbose(env, "verifier internal error: unknown irq flags kfunc\n");
11973 return -EFAULT;
11974 }
11975
11976 if (irq_save) {
11977 if (!is_irq_flag_reg_valid_uninit(env, reg)) {
11978 verbose(env, "expected uninitialized irq flag as arg#%d\n", regno);
11979 return -EINVAL;
11980 }
11981
11982 err = check_mem_access(env, env->insn_idx, regno, 0, BPF_DW, BPF_WRITE, -1, false, false);
11983 if (err)
11984 return err;
11985
11986 err = mark_stack_slot_irq_flag(env, meta, reg, env->insn_idx);
11987 if (err)
11988 return err;
11989 } else {
11990 err = is_irq_flag_reg_valid_init(env, reg);
11991 if (err) {
11992 verbose(env, "expected an initialized irq flag as arg#%d\n", regno);
11993 return err;
11994 }
11995
11996 err = mark_irq_flag_read(env, reg);
11997 if (err)
11998 return err;
11999
12000 err = unmark_stack_slot_irq_flag(env, reg);
12001 if (err)
12002 return err;
12003 }
12004 return 0;
12005 }
12006
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Kumar Kartikeya Dwivedi
maintainerPATCH 6/7 selftests/bpf: Expand coverage of preempt tests to sleepable kfunc
In reply to Kumar Kartikeya DwivediFor preemption-related kfuncs, we don't test their interaction with sleepable kfuncs (we do test helpers) even though the verifier has code to protect against such a pattern. Expand coverage of the selftest to include this case.
--- tools/testing/selftests/bpf/progs/preempt_lock.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
tools/testing/selftests/bpf/progs/preempt_lock.c +12 -0
@@ -113,6 +113,18 @@ int preempt_sleepable_helper(void *ctx)return 0 ; } +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +__failure __msg("kernel func bpf_copy_from_user_str is sleepable within non-preemptible region") +int preempt_sleepable_kfunc(void *ctx) +{ +u32 data; + +bpf_preempt_disable (); +bpf_copy_from_user_str (&data,sizeof (data),NULL ,0 ); +bpf_preempt_enable (); +return 0 ; +} + int __noinline preempt_global_subprog(void) {preempt_balance_subprog ();
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 6/7] selftests/bpf: Expand coverage of preempt tests to sleepable kfunc
Quoted 6 lines · Kumar Kartikeya Dwivedi
For preemption-related kfuncs, we don't test their interaction with sleepable kfuncs (we do test helpers) even though the verifier has code to protect against such a pattern. Expand coverage of the selftest to include this case.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
[...]
Kumar Kartikeya Dwivedi
maintainerPATCH 7/7 selftests/bpf: Add IRQ save/restore tests
In reply to Kumar Kartikeya DwivediInclude tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.
#123/1 irq/irq_restore_missing_1:OK #123/2 irq/irq_restore_missing_2:OK #123/3 irq/irq_restore_missing_3:OK #123/4 irq/irq_restore_missing_3_minus_2:OK #123/5 irq/irq_restore_missing_1_subprog:OK #123/6 irq/irq_restore_missing_2_subprog:OK #123/7 irq/irq_restore_missing_3_subprog:OK #123/8 irq/irq_restore_missing_3_minus_2_subprog:OK #123/9 irq/irq_balance:OK #123/10 irq/irq_balance_n:OK #123/11 irq/irq_balance_subprog:OK #123/12 irq/irq_balance_n_subprog:OK #123/13 irq/irq_global_subprog:OK #123/14 irq/irq_restore_ooo:OK #123/15 irq/irq_restore_ooo_3:OK #123/16 irq/irq_restore_3_subprog:OK #123/17 irq/irq_restore_4_subprog:OK #123/18 irq/irq_restore_ooo_3_subprog:OK #123/19 irq/irq_restore_invalid:OK #123/20 irq/irq_save_invalid:OK #123/21 irq/irq_restore_iter:OK #123/22 irq/irq_save_iter:OK #123/23 irq/irq_flag_overwrite:OK #123/24 irq/irq_flag_overwrite_partial:OK #123/25 irq/irq_sleepable_helper:OK #123/26 irq/irq_sleepable_kfunc:OK #123 irq:OK
Summary: 1/26 PASSED, 0 SKIPPED, 0 FAILED
--- tools/testing/selftests/bpf/prog_tests/irq.c | 9 + tools/testing/selftests/bpf/progs/irq.c | 393 +++++++++++++++++++ 2 files changed, 402 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/irq.c create mode 100644 tools/testing/selftests/bpf/progs/irq.c
tools/testing/selftests/bpf/prog_tests/irq.c new file +9 -0
@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include <test_progs.h> +#include <irq.skel.h> + +void test_irq (void ) +{ +RUN_TESTS (irq); +}
tools/testing/selftests/bpf/progs/irq.c new file +393 -0
393 lines not on this page · raw message
Signature · 1 line
2.43.5
Re: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests
Quoted 34 lines · Kumar Kartikeya Dwivedi
Include tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.
#123/1 irq/irq_restore_missing_1:OK #123/2 irq/irq_restore_missing_2:OK #123/3 irq/irq_restore_missing_3:OK #123/4 irq/irq_restore_missing_3_minus_2:OK #123/5 irq/irq_restore_missing_1_subprog:OK #123/6 irq/irq_restore_missing_2_subprog:OK #123/7 irq/irq_restore_missing_3_subprog:OK #123/8 irq/irq_restore_missing_3_minus_2_subprog:OK #123/9 irq/irq_balance:OK #123/10 irq/irq_balance_n:OK #123/11 irq/irq_balance_subprog:OK #123/12 irq/irq_balance_n_subprog:OK #123/13 irq/irq_global_subprog:OK #123/14 irq/irq_restore_ooo:OK #123/15 irq/irq_restore_ooo_3:OK #123/16 irq/irq_restore_3_subprog:OK #123/17 irq/irq_restore_4_subprog:OK #123/18 irq/irq_restore_ooo_3_subprog:OK #123/19 irq/irq_restore_invalid:OK #123/20 irq/irq_save_invalid:OK #123/21 irq/irq_restore_iter:OK #123/22 irq/irq_save_iter:OK #123/23 irq/irq_flag_overwrite:OK #123/24 irq/irq_flag_overwrite_partial:OK #123/25 irq/irq_sleepable_helper:OK #123/26 irq/irq_sleepable_kfunc:OK #123 irq:OK
Quoted 4 lines
Summary: 1/26 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
The following error condition is not tested:
"arg#%d doesn't point to an irq flag on stack".
Also, I think a few tests are excessive.
Otherwise looks good.
[...]
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +0
-0
tools/testing/selftests/bpf/prog_tests/irq.c line 9
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +9
-0@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include <test_progs.h> +#include <irq.skel.h> + +void test_irq (void ) +{ +RUN_TESTS (irq); +}
Nit: tools/testing/selftests/bpf/prog_tests/verifier.c
could be used instead of a separate file.
Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c new file +0
-0
tools/testing/selftests/bpf/progs/irq.c line 8
Quoted 8 lines
tools/testing/selftests/bpf/progs/irq.c new file +8
-0@@ -0,0 +1,393 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +SEC("?tc") +__failure __msg("BPF_EXIT instruction cannot be used inside bpf_local_irq_save-ed region")
Nit: I know this is not a fault of this series, but the error message
is sort of confusing. BPF_EXIT is allowed for irq saved region, just it has to be an exit from a sub-program, not a whole program.
tools/testing/selftests/bpf/progs/irq.c line 15
Quoted 7 lines
tools/testing/selftests/bpf/progs/irq.c new file +7
-0+int irq_restore_missing_1 (struct __sk_buff *ctx) +{ +unsigned long flags; + +bpf_local_irq_save (&flags); +return 0 ; +}
[...]
Nit: don't think this test adds much compared to irq_restore_missing_2.
Quoted 10 lines
tools/testing/selftests/bpf/progs/irq.c new file +10
-0+{ +unsigned long flags1; +unsigned long flags2; +unsigned long flags3; + +bpf_local_irq_save (&flags1); +bpf_local_irq_save (&flags2); +bpf_local_irq_save (&flags3); +return 0 ; +}
[...]
tools/testing/selftests/bpf/progs/irq.c line 181
Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3
-0+SEC("?tc") +__success +int irq_balance_n_subprog (struct __sk_buff *ctx)
Nit: don't think this test adds much given irq_balance_n()
and irq_balance_subprog().
Quoted 4 lines
tools/testing/selftests/bpf/progs/irq.c new file +4
-0+{ +local_irq_balance_n (); +return 0 ; +}
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests
Quoted 38 lines · Eduard Zingerman
Quoted 36 lines · Kumar Kartikeya Dwivedi
Include tests that check for rejection in erroneous cases, like unbalanced IRQ-disabled counts, within and across subprogs, invalid IRQ flag state or input to kfuncs, behavior upon overwriting IRQ saved state on stack, interaction with sleepable kfuncs/helpers, global functions, and out of order restore. Include some success scenarios as well to demonstrate usage.
#123/1 irq/irq_restore_missing_1:OK #123/2 irq/irq_restore_missing_2:OK #123/3 irq/irq_restore_missing_3:OK #123/4 irq/irq_restore_missing_3_minus_2:OK #123/5 irq/irq_restore_missing_1_subprog:OK #123/6 irq/irq_restore_missing_2_subprog:OK #123/7 irq/irq_restore_missing_3_subprog:OK #123/8 irq/irq_restore_missing_3_minus_2_subprog:OK #123/9 irq/irq_balance:OK #123/10 irq/irq_balance_n:OK #123/11 irq/irq_balance_subprog:OK #123/12 irq/irq_balance_n_subprog:OK #123/13 irq/irq_global_subprog:OK #123/14 irq/irq_restore_ooo:OK #123/15 irq/irq_restore_ooo_3:OK #123/16 irq/irq_restore_3_subprog:OK #123/17 irq/irq_restore_4_subprog:OK #123/18 irq/irq_restore_ooo_3_subprog:OK #123/19 irq/irq_restore_invalid:OK #123/20 irq/irq_save_invalid:OK #123/21 irq/irq_restore_iter:OK #123/22 irq/irq_save_iter:OK #123/23 irq/irq_flag_overwrite:OK #123/24 irq/irq_flag_overwrite_partial:OK #123/25 irq/irq_sleepable_helper:OK #123/26 irq/irq_sleepable_kfunc:OK #123 irq:OKSummary: 1/26 PASSED, 0 SKIPPED, 0 FAILED
Quoted 8 lines
Quoted 2 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>---
The following error condition is not tested:
"arg#%d doesn't point to an irq flag on stack".
Also, I think a few tests are excessive.
Otherwise looks good.
Will add.
Quoted 67 lines
Acked-by Eduard Zingerman <[email protected]>[...]
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +0
-0tools/testing/selftests/bpf/prog_tests/irq.c line 9
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/irq.c new file +9
-0@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include <test_progs.h> +#include <irq.skel.h> + +void test_irq (void ) +{ +RUN_TESTS (irq); +}Nit:
tools/testing/selftests/bpf/prog_tests/verifier.c
could be used instead of a separate file.Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c new file +0
-0tools/testing/selftests/bpf/progs/irq.c line 8
Quoted 8 lines
tools/testing/selftests/bpf/progs/irq.c new file +8
-0@@ -0,0 +1,393 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ +#include <vmlinux.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +SEC("?tc") +__failure __msg("BPF_EXIT instruction cannot be used inside bpf_local_irq_save-ed region")Nit: I know this is not a fault of this series, but the error message
is sort of confusing.BPF_EXITis allowed for irq saved region, just it has to be an exit from a sub-program, not a whole program.tools/testing/selftests/bpf/progs/irq.c line 15
Quoted 7 lines
tools/testing/selftests/bpf/progs/irq.c new file +7
-0+int irq_restore_missing_1 (struct __sk_buff *ctx) +{ +unsigned long flags; + +bpf_local_irq_save (&flags); +return 0 ; +}[...]
Nit: don't think this test adds much compared to
irq_restore_missing_2.Quoted 10 lines
tools/testing/selftests/bpf/progs/irq.c new file +10
-0+{ +unsigned long flags1; +unsigned long flags2; +unsigned long flags3; + +bpf_local_irq_save (&flags1); +bpf_local_irq_save (&flags2); +bpf_local_irq_save (&flags3); +return 0 ; +}[...]
Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c line 181
Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3
-0+SEC("?tc") +__success +int irq_balance_n_subprog (struct __sk_buff *ctx)Nit: don't think this test adds much given
irq_balance_n()
andirq_balance_subprog().
My idea with both of these was to ensure when the state is copied in and out on calls and when we're doing one or more than one save/restore (which links prev_id into active_irq_id etc.) we don't have problems, so they were definitely testing different scenarios. But with the move into bpf_verifier_state they will indeed become redundant, so I'm going to drop them in v2.
Quoted 8 lines
Quoted 5 lines
tools/testing/selftests/bpf/progs/irq.c new file +4
-0+{ +local_irq_balance_n (); +return 0 ; +}[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 7/7] selftests/bpf: Add IRQ save/restore tests
On Thu, 2024-11-21 at 23:07 +0100, Kumar Kartikeya Dwivedi wrote:
[...]
Quoted 7 lines
Quoted 6 lines
tools/testing/selftests/bpf/progs/irq.c line 181
Quoted 3 lines
tools/testing/selftests/bpf/progs/irq.c new file +3
-0+SEC("?tc") +__success +int irq_balance_n_subprog (struct __sk_buff *ctx)Nit: don't think this test adds much given
irq_balance_n()
andirq_balance_subprog().
Quoted 6 lines
My idea with both of these was to ensure when the state is copied in and out on calls and when we're doing one or more than one save/restore (which links
prev_idintoactive_irq_idetc.) we don't have problems, so they were definitely testing different scenarios. But with the move intobpf_verifier_statethey will indeed become redundant, so I'm going to drop them in v2.
Understood, thank you for explaining.