bpf: implicit bpf_prog_aux argument for kfuncs
Subsystems (6)
Ihor Solodrai
reviewercover bpf: implicit bpf_prog_aux argument for kfuncs
This series implements KF_IMPLICIT_PROG_AUX_ARG kfunc flag. The flag indicates that the last argument of a BPF kfunc is a pointer to struct bpf_prog_aux implicitly set by the verifier.
There is already a mechanism for passing bpf_prog_aux to kfuncs:__prog kfunc parameter annotation [1]. While it works technically, it requires explicit declaration of the argument in kfunc prototype in BPF (for example, in vmlinux.h) and a dummy actual parameter on call sites. Which then prompts adding macros like the following:
#define bpf_wq_set_callback(timer, cb, flags) \
bpf_wq_set_callback_impl(timer, cb, flags, NULL)
This is cumbersome in light of potentially many more kfuncs requiring access to bpf_prog_aux data [2].
An alternative approach is implemented in this series in combination with a change in pahole's BTF encoding [3].
When a kfunc needs access to bpf_prog_aux, it's declaration on the kernel side must have struct bpf_prog_aux pointer as it's *last* argument, and the kfunc must have KF_IMPLICIT_PROG_AUX_ARG flag set in its BTF_ID_FLAGS entry.
When generating BTF pahole recognizes KF_IMPLICIT_PROG_AUX_ARG flag and does not emit the last argument of the kfunc to BTF.
During BPF patching in the verifier KF_IMPLICIT_PROG_AUX_ARG is checked for to set the actual pointer, exactly as it happens for __prog annotation currently.
For backwards compatibility __prog annotation handling is preserved in the verifier. Existing kfuncs that use __prog annotation are updated to KF_IMPLICIT_PROG_AUX_ARG in this series.
Successful BPF CI run with modified pahole:
https://github.com/kernel-patches/bpf/actions/runs/17987713438
[1] https://docs.kernel.org/bpf/kfuncs.html#prog-annotation
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://github.com/theihor/dwarves/tree/prog-aux.v1
Ihor Solodrai (6):
bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag bpf,docs: Add documentation for KF_IMPLICIT_PROG_AUX_ARG selftests/bpf: update bpf_wq_set_callback macro
bpf: implement bpf_wq_set_callback kfunc with implicit prog_aux
bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG
Documentation/bpf/kfuncs.rst | 39 ++++++++++++- include/linux/btf.h | 3 + kernel/bpf/helpers.c | 36 +++++++----- kernel/bpf/stream.c | 3 +- kernel/bpf/verifier.c | 58 ++++++++++++++----- tools/lib/bpf/bpf_helpers.h | 4 +- .../testing/selftests/bpf/bpf_experimental.h | 7 ++- .../testing/selftests/bpf/progs/stream_fail.c | 6 +- tools/testing/selftests/bpf/progs/task_work.c | 6 +- .../selftests/bpf/progs/task_work_fail.c | 8 +-- .../selftests/bpf/progs/task_work_stress.c | 2 +- .../testing/selftests/bpf/progs/wq_failures.c | 4 +- 12 files changed, 130 insertions(+), 46 deletions(-)
Signature · 1 line
2.51.0
Ihor Solodrai
reviewerPATCH 1/6 bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Define KF_IMPLICIT_PROG_AUX_ARG and handle it in the BPF verifier.
The mechanism of patching is exactly the same as for __prog parameter
annotation: in check_kfunc_args() detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.
Then the (unchanged in this patch) fixup_kfunc_call() adds a mov instruction to set the actual pointer to prog_aux.
The caveat for KF_IMPLICIT_PROG_AUX_ARG is in implicitness. We have to separately check that the number of arguments is under MAX_BPF_FUNC_REG_ARGS.
--- include/linux/btf.h | 3 +++ kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 39 insertions(+), 7 deletions(-)
include/linux/btf.h +3 -0
@@ -79,6 +79,9 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +/* kfunc takes a pointer to struct bpf_prog_aux as the last argument, +* passed implicitly in BPF */ +#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)/* * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +36 -7
68 lines not on this page · raw message
Signature · 1 line
2.51.0
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 25 lines · Ihor Solodrai
Define
KF_IMPLICIT_PROG_AUX_ARGand handle it in the BPF verifier.The mechanism of patching is exactly the same as for
__progparameter
annotation: incheck_kfunc_args()detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.Then the (unchanged in this patch)
fixup_kfunc_call()adds a mov instruction to set the actual pointer toprog_aux.The caveat for
KF_IMPLICIT_PROG_AUX_ARGis in implicitness. We have to separately check that the number of arguments is underMAX_BPF_FUNC_REG_ARGS.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf.h | 3 +++ kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 39 insertions(+), 7 deletions(-)include/linux/btf.h +0
-0
include/linux/btf.h line 83
Quoted 5 lines
include/linux/btf.h +2
-0@@ -79,6 +79,9 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +/* kfunc takes a pointer to struct bpf_prog_aux as the last argument, +* passed implicitly in BPF */
This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.
include/linux/btf.h line 84
Quoted 1 line
include/linux/btf.h +1
-0+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)
The name is too verbose imo.
How aboutKF_HIDDEN_PROG_ARG
orKF_PROG_LAST_ARG
"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is KF_PROG_LAST_ARG.
"aux" part is also an implementation detail.
Quoted 38 lines
include/linux/btf.h +0
-0/* * Tag marking a kernel function as a kfunc. This is meant to minimize the kernel/bpf/verifier.c +17
-0@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)return meta->kfunc_flags &KF_RCU_PROTECTED ; } +static bool is_kfunc_with_implicit_prog_aux_arg (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_IMPLICIT_PROG_AUX_ARG ; +} +static bool is_kfunc_arg_mem_size (const struct btf *btf,const struct btf_param *arg,const struct bpf_reg_state *reg)@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *argreturn btf_param_match_suffix (btf, arg,"__prog" ); } +static int set_kfunc_arg_prog_regno (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta,u32 regno) +{ +if (meta->arg_prog) { +verifier_bug (env,"Only 1 prog->aux argument supported per-kfunc" ); +return -EFAULT ; + } + meta->arg_prog = true; +cur_aux (env)->arg_prog = regno; + +return 0 ; +} +static bool is_kfunc_arg_scalar_with_name (const struct btf *btf,const struct btf_param *arg,const char *name)
kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072
Quoted 6 lines
kernel/bpf/verifier.c +3
-0@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_return -EINVAL ; } +/* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF, +* so we have to set_kfunc_arg_prog_regno() outside the arg check loop. +*/
Use kernel comment style.
kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+if (is_kfunc_with_implicit_prog_aux_arg (meta)) { +if (nargs +1 >MAX_BPF_FUNC_REG_ARGS ) { +verifier_bug (env,"A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args" , + nargs +1 ,MAX_BPF_FUNC_REG_ARGS ); +return -EFAULT ; + } +u32 regno = nargs +1 ;
Variable declaration should be first in the block followed by a blank line.
Also I would remove this double "> MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
and actual_nargs = nargs + 1;
if (actual_nargs > MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry that verbose() isn't too specific.
If it prints nargs and actual_nargs whoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add more KF_FOO_LAST_ARG flags to cleanup other *_impl() kfuncs that have a special last argument, like bpf_rbtree_add_impl. If all of them copy paste "> MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.
pw-bot: cr
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 109 lines · Alexei Starovoitov
Quoted 25 lines · Ihor Solodrai
Define
KF_IMPLICIT_PROG_AUX_ARGand handle it in the BPF verifier.The mechanism of patching is exactly the same as for
__progparameter
annotation: incheck_kfunc_args()detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.Then the (unchanged in this patch)
fixup_kfunc_call()adds a mov instruction to set the actual pointer toprog_aux.The caveat for
KF_IMPLICIT_PROG_AUX_ARGis in implicitness. We have to separately check that the number of arguments is underMAX_BPF_FUNC_REG_ARGS.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf.h | 3 +++ kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 39 insertions(+), 7 deletions(-)include/linux/btf.h +0
-0include/linux/btf.h line 83
Quoted 5 lines
include/linux/btf.h +2
-0@@ -79,6 +79,9 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +/* kfunc takes a pointer to struct bpf_prog_aux as the last argument, +* passed implicitly in BPF */ This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.
include/linux/btf.h line 84
Quoted 1 line
include/linux/btf.h +1
-0+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)The name is too verbose imo.
How aboutKF_HIDDEN_PROG_ARG
orKF_PROG_LAST_ARG"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is
KF_PROG_LAST_ARG."aux" part is also an implementation detail.
Quoted 38 lines
include/linux/btf.h +0
-0/* * Tag marking a kernel function as a kfunc. This is meant to minimize the kernel/bpf/verifier.c +17
-0@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)return meta->kfunc_flags &KF_RCU_PROTECTED ; } +static bool is_kfunc_with_implicit_prog_aux_arg (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_IMPLICIT_PROG_AUX_ARG ; +} +static bool is_kfunc_arg_mem_size (const struct btf *btf,const struct btf_param *arg,const struct bpf_reg_state *reg)@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *argreturn btf_param_match_suffix (btf, arg,"__prog" ); } +static int set_kfunc_arg_prog_regno (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta,u32 regno) +{ +if (meta->arg_prog) { +verifier_bug (env,"Only 1 prog->aux argument supported per-kfunc" ); +return -EFAULT ; + } + meta->arg_prog = true; +cur_aux (env)->arg_prog = regno; + +return 0 ; +} +static bool is_kfunc_arg_scalar_with_name (const struct btf *btf,const struct btf_param *arg,const char *name)kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072
Quoted 6 lines
kernel/bpf/verifier.c +3
-0@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_return -EINVAL ; } +/* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF, +* so we have to set_kfunc_arg_prog_regno() outside the arg check loop. +*/ Use kernel comment style.
kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+if (is_kfunc_with_implicit_prog_aux_arg (meta)) { +if (nargs +1 >MAX_BPF_FUNC_REG_ARGS ) { +verifier_bug (env,"A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args" , + nargs +1 ,MAX_BPF_FUNC_REG_ARGS ); +return -EFAULT ; + } +u32 regno = nargs +1 ;Variable declaration should be first in the block followed by a blank line.
Quoted 14 lines
Also I would remove this double ">
MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
andactual_nargs= nargs + 1;
if (actual_nargs>MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry thatverbose()isn't too specific.
If it prints nargs andactual_nargswhoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add moreKF_FOO_LAST_ARGflags to cleanup other*_impl()kfuncs that have a special last argument, likebpf_rbtree_add_impl. If all of them copy paste ">MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.
Hi Alexei, thank you for the review.
Sorry for the styling mistakes, forgot to run patches through checkpatch.pl
In the other thread Eduard proposes a different approach to the implementation [1]. Basically, leave BTF unmodified and move argument hiding logic to bpftool's vmlinux.h generation.
IMO modifying BTF is more straightforward, but if the main goal is to have a nice BPF C interface, maybe Eduard is onto something.
Curious to hear yours and Andrii's opinion on that.
Thanks.
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 2 lines
pw-bot: cr
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 48 lines · Ihor Solodrai
Quoted 44 lines · Alexei Starovoitov
Quoted 25 lines · Ihor Solodrai
Define
KF_IMPLICIT_PROG_AUX_ARGand handle it in the BPF verifier.The mechanism of patching is exactly the same as for
__progparameter
annotation: incheck_kfunc_args()detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.Then the (unchanged in this patch)
fixup_kfunc_call()adds a mov instruction to set the actual pointer toprog_aux.The caveat for
KF_IMPLICIT_PROG_AUX_ARGis in implicitness. We have to separately check that the number of arguments is underMAX_BPF_FUNC_REG_ARGS.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf.h | 3 +++ kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 39 insertions(+), 7 deletions(-)include/linux/btf.h +0
-0include/linux/btf.h line 83
Quoted 5 lines
include/linux/btf.h +2
-0@@ -79,6 +79,9 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +/* kfunc takes a pointer to struct bpf_prog_aux as the last argument, +* passed implicitly in BPF */ This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.
include/linux/btf.h line 84
Quoted 1 line
include/linux/btf.h +1
-0+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)The name is too verbose imo.
How aboutKF_HIDDEN_PROG_ARG
orKF_PROG_LAST_ARG
Quoted 6 lines
Quoted 6 lines
"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is
KF_PROG_LAST_ARG."aux" part is also an implementation detail.
+1, I'd consider even just naming it KF_PROG_ARG, where "last" is implied, but that's minor.
Quoted 79 lines
Quoted 59 lines
Quoted 39 lines
include/linux/btf.h +0
-0/* * Tag marking a kernel function as a kfunc. This is meant to minimize the kernel/bpf/verifier.c +17
-0@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)return meta->kfunc_flags &KF_RCU_PROTECTED ; } +static bool is_kfunc_with_implicit_prog_aux_arg (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_IMPLICIT_PROG_AUX_ARG ; +} +static bool is_kfunc_arg_mem_size (const struct btf *btf,const struct btf_param *arg,const struct bpf_reg_state *reg)@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *argreturn btf_param_match_suffix (btf, arg,"__prog" ); } +static int set_kfunc_arg_prog_regno (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta,u32 regno) +{ +if (meta->arg_prog) { +verifier_bug (env,"Only 1 prog->aux argument supported per-kfunc" ); +return -EFAULT ; + } + meta->arg_prog = true; +cur_aux (env)->arg_prog = regno; + +return 0 ; +} +static bool is_kfunc_arg_scalar_with_name (const struct btf *btf,const struct btf_param *arg,const char *name)kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072
Quoted 6 lines
kernel/bpf/verifier.c +3
-0@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_return -EINVAL ; } +/* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF, +* so we have to set_kfunc_arg_prog_regno() outside the arg check loop. +*/ Use kernel comment style.
kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+if (is_kfunc_with_implicit_prog_aux_arg (meta)) { +if (nargs +1 >MAX_BPF_FUNC_REG_ARGS ) { +verifier_bug (env,"A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args" , + nargs +1 ,MAX_BPF_FUNC_REG_ARGS ); +return -EFAULT ; + } +u32 regno = nargs +1 ;Variable declaration should be first in the block followed by a blank line.
Quoted 14 lines
Also I would remove this double ">
MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
andactual_nargs= nargs + 1;
if (actual_nargs>MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry thatverbose()isn't too specific.
If it prints nargs andactual_nargswhoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add moreKF_FOO_LAST_ARGflags to cleanup other*_impl()kfuncs that have a special last argument, likebpf_rbtree_add_impl. If all of them copy paste ">MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.Hi Alexei, thank you for the review.
Sorry for the styling mistakes, forgot to run patches through checkpatch.pl
Quoted 9 lines
In the other thread Eduard proposes a different approach to the implementation [1]. Basically, leave BTF unmodified and move argument hiding logic to bpftool's
vmlinux.hgeneration.IMO modifying BTF is more straightforward, but if the main goal is to have a nice BPF C interface, maybe Eduard is onto something.
Curious to hear yours and Andrii's opinion on that.
I explicitly really-really don't want to modify BPF CO-RE type matching rules for these special kfuncs. So that's why I think it's better overall to hide those special arguments from the BTF information altogether.
I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?
Thoughts?
Quoted 7 lines
Thanks.
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 2 lines
pw-bot: cr
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?
Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl() definition doesn't change. Only:
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)
and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.
The runtime performance will be slightly better too, since no need for wrappers like:
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}
It's just one jmpl insn, but still.
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs:
kfunc_foo(args),kfunc_foo_impl(args, void *aux)into vmlinux BTF.
bpftool will emit both invmlinux.hand bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl()definition doesn't change. Only:(quoted diff) +1
-1-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)and the verifier can check that the last arg is
aux__progwhenKF_PROG_ARGis specified.The runtime performance will be slightly better too, since no need for wrappers like:
(quoted diff) +7
-0+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + void *aux__prog) +{ + return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}
Quoted 1 line
It's just one jmpl insn, but still.
So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.
Two notes:
a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type
b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 54 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs:
kfunc_foo(args),kfunc_foo_impl(args, void *aux)into vmlinux BTF.
bpftool will emit both invmlinux.hand bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl()definition doesn't change. Only:(quoted diff) +1
-1-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)and the verifier can check that the last arg is
aux__progwhenKF_PROG_ARGis specified.The runtime performance will be slightly better too, since no need for wrappers like:
(quoted diff) +7
-0+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + void *aux__prog) +{ + return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}Quoted 1 line
It's just one jmpl insn, but still.
So basically
xxx_impl()will be a phantom function that verifier will recognize and it will need to have correspondingxxx()kfunc with correspondingKF_PROG_ARGfor everything to work. Makes sense.Two notes:
Quoted 7 lines
a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for
bpf_obj_new_impl, for example), which will be distinguished based on their BTF typeb)
bpf_stream_vprintk()throws a bit of a wrench into all this because it doesn't follow_implnaming convention. Any suggestions on how to deal with that?
We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.
And then double down on this convention going forward.
Quoted 1 line
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 61 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs:
kfunc_foo(args),kfunc_foo_impl(args, void *aux)into vmlinux BTF.
bpftool will emit both invmlinux.hand bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl()definition doesn't change. Only:(quoted diff) +1
-1-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)and the verifier can check that the last arg is
aux__progwhenKF_PROG_ARGis specified.The runtime performance will be slightly better too, since no need for wrappers like:
(quoted diff) +7
-0+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + void *aux__prog) +{ + return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}Quoted 1 line
It's just one jmpl insn, but still.
So basically
xxx_impl()will be a phantom function that verifier will recognize and it will need to have correspondingxxx()kfunc with correspondingKF_PROG_ARGfor everything to work. Makes sense.Two notes:
a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for
bpf_obj_new_impl, for example), which will be distinguished based on their BTF type
Quoted 7 lines
Quoted 3 lines
b)
bpf_stream_vprintk()throws a bit of a wrench into all this because it doesn't follow_implnaming convention. Any suggestions on how to deal with that?We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.
Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".
Quoted 3 lines
And then double down on this convention going forward.
Quoted 1 line
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 58 lines · Andrii Nakryiko
Quoted 55 lines · Kumar Kartikeya Dwivedi
Quoted 52 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs:
kfunc_foo(args),kfunc_foo_impl(args, void *aux)into vmlinux BTF.
bpftool will emit both invmlinux.hand bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl()definition doesn't change. Only:(quoted diff) +1
-1-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)and the verifier can check that the last arg is
aux__progwhenKF_PROG_ARGis specified.The runtime performance will be slightly better too, since no need for wrappers like:
(quoted diff) +7
-0+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + void *aux__prog) +{ + return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}Quoted 1 line
It's just one jmpl insn, but still.
So basically
xxx_impl()will be a phantom function that verifier will recognize and it will need to have correspondingxxx()kfunc with correspondingKF_PROG_ARGfor everything to work. Makes sense.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Two notes:
a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for
bpf_obj_new_impl, for example), which will be distinguished based on their BTF type
agree
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
b)
bpf_stream_vprintk()throws a bit of a wrench into all this because it doesn't follow_implnaming convention. Any suggestions on how to deal with that?
Quoted 8 lines
Quoted 3 lines
We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.
Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".
I think we can still do it even after 6.17 is released and backport it as a fix.
Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag
Quoted 78 lines · Alexei Starovoitov
Quoted 58 lines · Andrii Nakryiko
Quoted 55 lines · Kumar Kartikeya Dwivedi
Quoted 52 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko
I do see the benefit of having the generic "
KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understandKF_MAGIC_ARGSanyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc hasBPF_MAGIC_ARGSkflag we just look up "my_fancy_kfunc.magic" FUNC definition?Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs:
kfunc_foo(args),kfunc_foo_impl(args, void *aux)into vmlinux BTF.
bpftool will emit both invmlinux.hand bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.bpf_wq_set_callback_impl()definition doesn't change. Only:(quoted diff) +1
-1-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)and the verifier can check that the last arg is
aux__progwhenKF_PROG_ARGis specified.The runtime performance will be slightly better too, since no need for wrappers like:
(quoted diff) +7
-0+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + void *aux__prog) +{ + return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}Quoted 1 line
It's just one jmpl insn, but still.
So basically
xxx_impl()will be a phantom function that verifier will recognize and it will need to have correspondingxxx()kfunc with correspondingKF_PROG_ARGfor everything to work. Makes sense.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Two notes:
a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for
bpf_obj_new_impl, for example), which will be distinguished based on their BTF typeagree
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
b)
bpf_stream_vprintk()throws a bit of a wrench into all this because it doesn't follow_implnaming convention. Any suggestions on how to deal with that?We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.
Quoted 7 lines
Quoted 4 lines
Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".
I think we can still do it even after 6.17 is released and backport it as a fix.
ok then, sounds like a plan
Ihor Solodrai
reviewerPATCH 2/6 bpf,docs: Add documentation for KF_IMPLICIT_PROG_AUX_ARG
In reply to Ihor SolodraiAdd a section explaining KF_IMPLICIT_PROG_AUX_ARG kfunc flag. Mark __prog annotation as deprecated.
--- Documentation/bpf/kfuncs.rst | 39 +++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)
Documentation/bpf/kfuncs.rst +38 -1
@@ -160,7 +160,7 @@ Or:: ... } -2.2.6 __prog Annotation +2.2.6 __prog Annotation (deprecated, use KF_IMPLICIT_PROG_AUX_ARG instead) --------------------------- This annotation is used to indicate that the argument needs to be fixed up to the bpf_prog_aux of the caller BPF program. Any value passed into this argument
@@ -374,6 +374,43 @@ encouraged to make their use-cases known as early as possible, and participate in upstream discussions regarding whether to keep, change, deprecate, or remove those kfuncs if and when such discussions occur. +2.4.10 KF_IMPLICIT_PROG_AUX_ARG flag +------------------------------------ + +The KF_IMPLICIT_PROG_AUX_ARG flag is used to indicate that the last +argument of a BPF kfunc is a pointer to struct bpf_prog_aux of the +caller BPF program, implicitly set by the verifier. + +If a kfunc is marked with this flag, the function declaration in the +kernel must have ``struct bpf_prog_aux *`` as the last argument. +However in the kernel BTF (produced by pahole) this argument will be +omitted, making it invisible to BPF programs calling the kfunc. + +Note that the implicit argument is an actual BPF argument passed +through a register, reducing the number of the available function +arguments. + +Example declaration: + +.. code-block:: c + + __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw, + void *map__map, bpf_task_work_callback_t callback, + struct bpf_prog_aux *aux) + { + return bpf_task_work_schedule(task, tw, map__map, callback, aux, TWA_RESUME); + } + + BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG) + +Example usage: + +.. code-block:: c + + /* note the last argument is ommitted */ + bpf_task_work_schedule_resume(task, &tw, &hmap, process_work); + + 2.5 Registering the kfuncs --------------------------
Signature · 1 line
2.51.0
Ihor Solodrai
reviewerPATCH 3/6 selftests/bpf: update bpf_wq_set_callback macro
In reply to Ihor SolodraiSubsequent patch introduces bpf_wq_set_callback kfunc with an implicit bpf_prog_aux argument.
To ensure backward compatibility add a weak declaration and make bpf_wq_set_callback macro to check for the new kfunc first.
--- tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_experimental.h +6 -1
@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, int (callback_fn)(void *map, int *key, void *value), unsigned int flags__k, void *aux__ign) __ksym; +extern int bpf_wq_set_callback(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags) __weak __ksym;#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL) + (bpf_wq_set_callback ? \ + bpf_wq_set_callback(timer, cb, flags) : \ + bpf_wq_set_callback_impl(timer, cb, flags, NULL))struct bpf_iter_kmem_cache ; extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym;
Signature · 1 line
2.51.0
Re: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro
Quoted 18 lines · Ihor Solodrai
Subsequent patch introduces
bpf_wq_set_callbackkfunc with an implicitbpf_prog_auxargument.To ensure backward compatibility add a weak declaration and make
bpf_wq_set_callbackmacro to check for the new kfunc first.Signed-off-by Ihor Solodrai <[email protected]>--- tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/bpf_experimental.h +0
-0@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,
tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 592
Quoted 10 lines
tools/testing/selftests/bpf/bpf_experimental.h +6
-1int (callback_fn)(void *map,int *key,void *value), unsigned int flags__k, void *aux__ign) __ksym; +extern int bpf_wq_set_callback(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags) __weak __ksym;#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL) + (bpf_wq_set_callback ? \ + bpf_wq_set_callback(timer, cb, flags) : \ + bpf_wq_set_callback_impl(timer, cb, flags, NULL))
There is also drivers/hid/bpf/progs/hid_bpf_helpers.h Pls double check that hid-bpf still compiles and works.
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro
Quoted 24 lines · Alexei Starovoitov
Quoted 23 lines · Ihor Solodrai
Subsequent patch introduces
bpf_wq_set_callbackkfunc with an implicitbpf_prog_auxargument.To ensure backward compatibility add a weak declaration and make
bpf_wq_set_callbackmacro to check for the new kfunc first.Signed-off-by Ihor Solodrai <[email protected]>--- tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/bpf_experimental.h +3
-0@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, int (callback_fn)(void *map, int *key, void *value), unsigned int flags__k, void *aux__ign) __ksym; +extern int bpf_wq_set_callback(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags) __weak __ksym;
Quoted 8 lines
tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 592
Quoted 5 lines
tools/testing/selftests/bpf/bpf_experimental.h +3
-1#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL) + (bpf_wq_set_callback ? \ + bpf_wq_set_callback(timer, cb, flags) : \ + bpf_wq_set_callback_impl(timer, cb, flags, NULL))There is also
drivers/hid/bpf/progs/hid_bpf_helpers.hPls double check that hid-bpf still compiles and works.
Yes, I noticed the usage.
I plan to send separate patches to hid when this series gets closer to landing.
Re: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro
In reply to Ihor SolodraiQuoted 17 lines · Ihor Solodrai
Subsequent patch introduces
bpf_wq_set_callbackkfunc with an implicitbpf_prog_auxargument.To ensure backward compatibility add a weak declaration and make
bpf_wq_set_callbackmacro to check for the new kfunc first.Signed-off-by Ihor Solodrai <[email protected]>--- tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/bpf_experimental.h +0
-0
tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 590
Quoted 9 lines
tools/testing/selftests/bpf/bpf_experimental.h +4
-1@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, int (callback_fn)(void *map, int *key, void *value), unsigned int flags__k, void *aux__ign) __ksym; +extern int bpf_wq_set_callback(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags) __weak __ksym;#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL) + (bpf_wq_set_callback ? \
use bpf_ksym_exists(), it has additional "fool-proofing" checks
Quoted 8 lines
tools/testing/selftests/bpf/bpf_experimental.h +2
-1+ bpf_wq_set_callback(timer, cb, flags) : \ + bpf_wq_set_callback_impl(timer, cb, flags, NULL))struct bpf_iter_kmem_cache ; extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym; --2.51.0
Ihor Solodrai
reviewerPATCH 4/6 bpf: implement bpf_wq_set_callback kfunc with implicit prog_aux
In reply to Ihor SolodraiAdd bpf_wq_set_callback BPF kfunc with KF_IMPLICIT_PROG_AUX_ARG, corresponding to bpf_wq_set_callback_impl kfunc. Teach the verifier about it.
To be handled correctly, BTF for this kfunc must be generated with pahole version that supports KF_IMPLICIT_PROG_AUX_ARG.
--- kernel/bpf/helpers.c | 18 +++++++++++++----- kernel/bpf/verifier.c | 15 +++++++++------ .../testing/selftests/bpf/progs/wq_failures.c | 4 ++-- 3 files changed, 24 insertions(+), 13 deletions(-)
kernel/bpf/helpers.c +13 -5
@@ -3065,12 +3065,11 @@ __bpf_kfunc int bpf_wq_start(struct bpf_wq *wq, unsigned int flags)return 0 ; } -__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, -int (callback_fn )(void *map,int *key,void *value), -unsigned int flags, -void *aux__prog) +__bpf_kfunc int bpf_wq_set_callback(struct bpf_wq *wq, +int (callback_fn )(void *map,int *key,void *value), +unsigned int flags, +struct bpf_prog_aux *aux) { -struct bpf_prog_aux *aux = (struct bpf_prog_aux *)aux__prog;struct bpf_async_kern *async = (struct bpf_async_kern *)wq;if (flags)
@@ -3079,6 +3078,14 @@ __bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,return __bpf_async_set_callback (async, callback_fn, aux, flags,BPF_ASYNC_TYPE_WQ ); } +__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, +int (callback_fn )(void *map,int *key,void *value), +unsigned int flags, +void *aux__prog) +{ +return bpf_wq_set_callback (wq, callback_fn, flags, aux__prog); +} + __bpf_kfunc void bpf_preempt_disable(void) {preempt_disable ();
@@ -4374,6 +4381,7 @@ BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU) BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_IMPLICIT_PROG_AUX_ARG)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +9 -6
@@ -514,7 +514,7 @@ static bool is_async_callback_calling_kfunc(u32 btf_id);static bool is_callback_calling_kfunc (u32 btf_id);static bool is_bpf_throw_kfunc (struct bpf_insn *insn); -static bool is_bpf_wq_set_callback_impl_kfunc (u32 btf_id); +static bool is_bpf_wq_set_callback_kfunc (u32 btf_id);static bool is_sync_callback_calling_function (enum bpf_func_id func_id) {
@@ -10586,7 +10586,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins env->subprog_info[subprog].is_async_cb = true; async_cb =push_async_cb (env, env->subprog_info[subprog].start, insn_idx, subprog, -is_bpf_wq_set_callback_impl_kfunc (insn->imm) || +is_bpf_wq_set_callback_kfunc (insn->imm) ||is_task_work_add_kfunc (insn->imm));if (!async_cb)return -EFAULT ;
@@ -12278,6 +12278,7 @@ enum special_kfunc_type { KF___bpf_trap, KF_bpf_task_work_schedule_signal, KF_bpf_task_work_schedule_resume, + KF_bpf_wq_set_callback, };BTF_ID_LIST (special_kfunc_list)
@@ -12350,6 +12351,7 @@ BTF_ID(func, bpf_res_spin_unlock_irqrestore) BTF_ID(func, __bpf_trap) BTF_ID(func, bpf_task_work_schedule_signal) BTF_ID(func, bpf_task_work_schedule_resume) +BTF_ID(func, bpf_wq_set_callback)static bool is_task_work_add_kfunc (u32 func_id) {
@@ -12797,7 +12799,7 @@ static bool is_sync_callback_calling_kfunc(u32 btf_id)static bool is_async_callback_calling_kfunc (u32 btf_id) { -return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] || +return is_bpf_wq_set_callback_kfunc (btf_id) ||is_task_work_add_kfunc (btf_id); }
@@ -12807,9 +12809,10 @@ static bool is_bpf_throw_kfunc(struct bpf_insn *insn) insn->imm == special_kfunc_list[KF_bpf_throw]; } -static bool is_bpf_wq_set_callback_impl_kfunc (u32 btf_id) +static bool is_bpf_wq_set_callback_kfunc (u32 btf_id) { -return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl]; +return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] || + btf_id == special_kfunc_list[KF_bpf_wq_set_callback]; }static bool is_callback_calling_kfunc (u32 btf_id)
@@ -13910,7 +13913,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, meta.r0_rdonly = false; } -if (is_bpf_wq_set_callback_impl_kfunc (meta.func_id)) { +if (is_bpf_wq_set_callback_kfunc (meta.func_id)) { err =push_callback_call (env, insn, insn_idx, meta.subprogno, set_timer_callback_state);if (err) {
tools/testing/selftests/bpf/progs/wq_failures.c +2 -2
@@ -97,7 +97,7 @@ __failure/* check that the first argument of bpf_wq_set_callback() * is a correct bpf_wq pointer. */ -__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */ +__msg(": (85) call bpf_wq_set_callback#") /* anchor message */ __msg("arg#0 doesn't point to a map value")long test_wrong_wq_pointer (void *ctx) {
@@ -123,7 +123,7 @@ __failure/* check that the first argument of bpf_wq_set_callback() * is a correct bpf_wq pointer. */ -__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */ +__msg(": (85) call bpf_wq_set_callback#") /* anchor message */ __msg("off 1 doesn't point to 'struct bpf_wq' that is at 0")long test_wrong_wq_pointer_offset (void *ctx) {
Signature · 1 line
2.51.0
Ihor Solodrai
reviewerPATCH 5/6 bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
In reply to Ihor SolodraiUpdate bpf_stream_vprink macro in libbpf and fix call sites in the relevant selftests.
--- kernel/bpf/helpers.c | 2 +- kernel/bpf/stream.c | 3 +-- tools/lib/bpf/bpf_helpers.h | 4 ++-- tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++--- 4 files changed, 7 insertions(+), 8 deletions(-)
kernel/bpf/helpers.c +1 -1
@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);#if defined(CONFIG_BPF_LSM ) && defined(CONFIG_CGROUPS ) BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif -BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG) BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS)BTF_ID_FLAGS (func, bpf_wq_set_callback,KF_IMPLICIT_PROG_AUX_ARG )
kernel/bpf/stream.c +1 -2
@@ -355,13 +355,12 @@ __bpf_kfunc_start_defs();* Avoid using enum bpf_stream_id so that kfunc users don't have to pull in the * enum in headers. */ -__bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, u32 len__sz, void *aux__prog) +__bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, u32 len__sz, struct bpf_prog_aux *aux) {struct bpf_bprintf_data data = { .get_bin_args = true, .get_buf = true, }; -struct bpf_prog_aux *aux = aux__prog;u32 fmt_size =strlen (fmt__str) +1 ;struct bpf_stream *stream;u32 data_len = len__sz;
tools/lib/bpf/bpf_helpers.h +2 -2
@@ -316,7 +316,7 @@ enum libbpf_tristate { }) extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, - __u32 len__sz, void *aux__prog) __weak __ksym; + __u32 len__sz) __weak __ksym; #define bpf_stream_printk(stream_id, fmt, args...) \ ({ \
@@ -328,7 +328,7 @@ extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *a___bpf_fill (___param, args); \ _Pragma("GCC diagnostic pop") \ \ -bpf_stream_vprintk (stream_id, ___fmt, ___param,sizeof (___param),NULL );\ +bpf_stream_vprintk (stream_id, ___fmt, ___param,sizeof (___param));\ })/* Use __bpf_printk when bpf_printk call has 3 or fewer fmt args
tools/testing/selftests/bpf/progs/stream_fail.c +3 -3
@@ -10,7 +10,7 @@ SEC("syscall") __failure __msg("Possibly NULL pointer passed")int stream_vprintk_null_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT ,"" ,NULL ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT ,"" ,NULL ,0 );return 0 ; }
@@ -18,7 +18,7 @@ SEC("syscall") __failure __msg("R3 type=scalar expected=")int stream_vprintk_scalar_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT ,"" , (void *)46 ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT ,"" , (void *)46 ,0 );return 0 ; }
@@ -26,7 +26,7 @@ SEC("syscall") __failure __msg("arg#1 doesn't point to a const string")int stream_vprintk_string_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT , ctx,NULL ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT , ctx,NULL ,0 );return 0 ; }
Signature · 1 line
2.51.0
Re: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
Quoted 2 lines · Ihor Solodrai
Update
bpf_stream_vprinkmacro in libbpf and fix call sites in
't' is missing in bpf_stream_vprintk().
Quoted 15 lines
the relevant selftests.
Signed-off-by Ihor Solodrai <[email protected]>--- kernel/bpf/helpers.c | 2 +- kernel/bpf/stream.c | 3 +-- tools/lib/bpf/bpf_helpers.h | 4 ++-- tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++--- 4 files changed, 7 insertions(+), 8 deletions(-)kernel/bpf/helpers.c +0
-0
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381
Quoted 5 lines
kernel/bpf/helpers.c +1
-1@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);#if defined(CONFIG_BPF_LSM ) && defined(CONFIG_CGROUPS ) BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif -BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS ) +BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS |KF_IMPLICIT_PROG_AUX_ARG )
This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.
tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319
Quoted 8 lines
tools/lib/bpf/bpf_helpers.h +1
-1@@ -316,7 +316,7 @@ enum libbpf_tristate { }) extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, - __u32 len__sz, void *aux__prog) __weak __ksym; + __u32 len__sz) __weak __ksym;
CI is complaining of conflicting types for 'bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
Quoted 16 lines · Alexei Starovoitov
Quoted 2 lines · Ihor Solodrai
Update
bpf_stream_vprinkmacro in libbpf and fix call sites in't' is missing in
bpf_stream_vprintk().Quoted 10 lines
the relevant selftests.
Signed-off-by Ihor Solodrai <[email protected]>--- kernel/bpf/helpers.c | 2 +- kernel/bpf/stream.c | 3 +-- tools/lib/bpf/bpf_helpers.h | 4 ++-- tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++--- 4 files changed, 7 insertions(+), 8 deletions(-)
Quoted 14 lines
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381
Quoted 10 lines
kernel/bpf/helpers.c +1
-1@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);#if defined(CONFIG_BPF_LSM ) && defined(CONFIG_CGROUPS ) BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif -BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS ) +BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS |KF_IMPLICIT_PROG_AUX_ARG )This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.
Acked.
Quoted 6 lines
Quoted 6 lines
tools/lib/bpf/bpf_helpers.h +0
-0@@ -316,7 +316,7 @@ enum libbpf_tristate { })
Quoted 10 lines
tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319
Quoted 3 lines
tools/lib/bpf/bpf_helpers.h +1
-1extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, - __u32 len__sz, void *aux__prog) __weak __ksym; + __u32 len__sz) __weak __ksym;CI is complaining of conflicting types for '
bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.
Right.
One way to do it is to conditionalize new kfunc defitions under a config flag similar to how we have CONFIG_PAHOLE_HAS_SPLIT_BTF and co.
But this seems like too much trouble for just a couple of kfuncs, especially since we do not guarantee stable interface for kfuncs.
We could ask people to update pahole version if they run into issues. And they shouldn't unless they are actually using the kfuncs, or more specifically: have their own declarations of said kfuncs.
bpf_stream_vprintk() is a special case, because it is already in libbpf.
Re: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
In reply to Alexei StarovoitovQuoted 17 lines · Alexei Starovoitov
Quoted 2 lines · Ihor Solodrai
Update
bpf_stream_vprinkmacro in libbpf and fix call sites in't' is missing in
bpf_stream_vprintk().Quoted 10 lines
the relevant selftests.
Signed-off-by Ihor Solodrai <[email protected]>--- kernel/bpf/helpers.c | 2 +- kernel/bpf/stream.c | 3 +-- tools/lib/bpf/bpf_helpers.h | 4 ++-- tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++--- 4 files changed, 7 insertions(+), 8 deletions(-)
Quoted 14 lines
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381
Quoted 10 lines
kernel/bpf/helpers.c +1
-1@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);#if defined(CONFIG_BPF_LSM ) && defined(CONFIG_CGROUPS ) BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif -BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS ) +BTF_ID_FLAGS (func, bpf_stream_vprintk,KF_TRUSTED_ARGS |KF_IMPLICIT_PROG_AUX_ARG )This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.
Can we still rename current "explicit prog_aux" version to bpf_stream_vprintk_impl() still?
That would solve backwards compat problems and would be in line with bpf_wq_set_callback_impl() approach.
If we can't, then we'd have to name the new function something different?
Don't know if there is some other BPF CO-RE and ELF symbol aliasing magic that can be used, we'll need to play with this to see what works.
Quoted 16 lines
tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319
Quoted 9 lines
tools/lib/bpf/bpf_helpers.h +1
-1@@ -316,7 +316,7 @@ enum libbpf_tristate { }) extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, - __u32 len__sz, void *aux__prog) __weak __ksym; + __u32 len__sz) __weak __ksym;CI is complaining of conflicting types for '
bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.
Ihor Solodrai
reviewerPATCH 6/6 bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG
In reply to Ihor SolodraiTwo kfuncs that use aux__prog argument were recently added [1]:
* bpf_task_work_schedule_resume
* bpf_task_work_schedule_signal
Update them to use the new kfunc flag and fix usages in the selftests.
[1] https://lore.kernel.org/bpf/[email protected]/
--- kernel/bpf/helpers.c | 16 ++++++++-------- tools/testing/selftests/bpf/progs/task_work.c | 6 +++--- .../testing/selftests/bpf/progs/task_work_fail.c | 8 ++++---- .../selftests/bpf/progs/task_work_stress.c | 2 +- 4 files changed, 16 insertions(+), 16 deletions(-)
kernel/bpf/helpers.c +8 -8
@@ -4178,15 +4178,15 @@ static int bpf_task_work_schedule(struct task_struct *task, struct bpf_task_work* @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping * @map__map: bpf_map that embeds struct bpf_task_work in the values * @callback: pointer to BPF subprogram to call -* @aux__prog: user should pass NULL +* @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier * * Return: 0 if task work has been scheduled successfully, negative error code otherwise */ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct bpf_task_work *tw,void *map__map,bpf_task_work_callback_t callback, -void *aux__prog) +struct bpf_prog_aux *aux) { -return bpf_task_work_schedule (task, tw, map__map, callback, aux__prog,TWA_SIGNAL ); +return bpf_task_work_schedule (task, tw, map__map, callback, aux,TWA_SIGNAL ); }/**
@@ -4195,15 +4195,15 @@ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct b* @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping * @map__map: bpf_map that embeds struct bpf_task_work in the values * @callback: pointer to BPF subprogram to call -* @aux__prog: user should pass NULL +* @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier * * Return: 0 if task work has been scheduled successfully, negative error code otherwise */ __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw,void *map__map,bpf_task_work_callback_t callback, -void *aux__prog) +struct bpf_prog_aux *aux) { -return bpf_task_work_schedule (task, tw, map__map, callback, aux__prog,TWA_RESUME ); +return bpf_task_work_schedule (task, tw, map__map, callback, aux,TWA_RESUME ); }__bpf_kfunc_end_defs ();
@@ -4379,8 +4379,8 @@ BTF_ID_FLAGS(func, bpf_strnstr); BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG) -BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS) -BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG) +BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG) BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_IMPLICIT_PROG_AUX_ARG)BTF_KFUNCS_END (common_btf_ids)
tools/testing/selftests/bpf/progs/task_work.c +3 -3
@@ -66,7 +66,7 @@ int oncpu_hash_map(struct pt_regs *args)if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work);return 0 ; }
@@ -80,7 +80,7 @@ int oncpu_array_map(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_signal (task, &work->tw, &arrmap, process_work,NULL ); +bpf_task_work_schedule_signal (task, &work->tw, &arrmap, process_work);return 0 ; }
@@ -102,6 +102,6 @@ int oncpu_lru_map(struct pt_regs *args) work =bpf_map_lookup_elem (&lrumap, &key);if (!work || work->data[0 ])return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &lrumap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &lrumap, process_work);return 0 ; }
tools/testing/selftests/bpf/progs/task_work_fail.c +4 -4
@@ -53,7 +53,7 @@ int mismatch_map(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work);return 0 ; }
@@ -65,7 +65,7 @@ int no_map_task_work(struct pt_regs *args)struct bpf_task_work tw; task =bpf_get_current_task_btf (); -bpf_task_work_schedule_resume (task, &tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &tw, &hmap, process_work);return 0 ; }
@@ -76,7 +76,7 @@ int task_work_null(struct pt_regs *args)struct task_struct *task; task =bpf_get_current_task_btf (); -bpf_task_work_schedule_resume (task,NULL , &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task,NULL , &hmap, process_work);return 0 ; }
@@ -91,6 +91,6 @@ int map_null(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw,NULL , process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw,NULL , process_work);return 0 ; }
tools/testing/selftests/bpf/progs/task_work_stress.c +1 -1
@@ -52,7 +52,7 @@ int schedule_task_work(void *ctx)return 0 ; } err =bpf_task_work_schedule_signal (bpf_get_current_task_btf (), &work->tw, &hmap, - process_work,NULL ); + process_work);if (err)__sync_fetch_and_add (&schedule_error,1 );else
Signature · 1 line
2.51.0
Re: [PATCH bpf-next v1 6/6] bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG
Quoted 8 lines · Ihor Solodrai
Two kfuncs that use
aux__progargument were recently added [1]:
*bpf_task_work_schedule_resume
*bpf_task_work_schedule_signalUpdate them to use the new kfunc flag and fix usages in the selftests.
Quoted 7 lines
Signed-off-by Ihor Solodrai <[email protected]>--- kernel/bpf/helpers.c | 16 ++++++++-------- tools/testing/selftests/bpf/progs/task_work.c | 6 +++--- .../testing/selftests/bpf/progs/task_work_fail.c | 8 ++++---- .../selftests/bpf/progs/task_work_stress.c | 2 +- 4 files changed, 16 insertions(+), 16 deletions(-)