Skip to contentSkip to conversation

bpf: implicit bpf_prog_aux argument for kfuncs

PATCH In review Ihor Solodrai 5 participants 23 messages Sep 24, 2025 – Sep 26, 2025

Subsystems (6)
lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

cover bpf: implicit bpf_prog_aux argument for kfuncs

This series implements KF_IMPLICIT_PROG_AUX_ARG kfunc flag. The flag indicates that the last argument of a BPF kfunc is a pointer to struct bpf_prog_aux implicitly set by the verifier.

There is already a mechanism for passing bpf_prog_aux to kfuncs:
__prog kfunc parameter annotation [1]. While it works technically, it requires explicit declaration of the argument in kfunc prototype in BPF (for example, in vmlinux.h) and a dummy actual parameter on call sites. Which then prompts adding macros like the following:

#define bpf_wq_set_callback(timer, cb, flags) \
bpf_wq_set_callback_impl(timer, cb, flags, NULL)

This is cumbersome in light of potentially many more kfuncs requiring access to bpf_prog_aux data [2].

An alternative approach is implemented in this series in combination with a change in pahole's BTF encoding [3].

When a kfunc needs access to bpf_prog_aux, it's declaration on the kernel side must have struct bpf_prog_aux pointer as it's *last* argument, and the kfunc must have KF_IMPLICIT_PROG_AUX_ARG flag set in its BTF_ID_FLAGS entry.

When generating BTF pahole recognizes KF_IMPLICIT_PROG_AUX_ARG flag and does not emit the last argument of the kfunc to BTF.

During BPF patching in the verifier KF_IMPLICIT_PROG_AUX_ARG is checked for to set the actual pointer, exactly as it happens for __prog annotation currently.

For backwards compatibility __prog annotation handling is preserved in the verifier. Existing kfuncs that use __prog annotation are updated to KF_IMPLICIT_PROG_AUX_ARG in this series.

Successful BPF CI run with modified pahole:
https://github.com/kernel-patches/bpf/actions/runs/17987713438

[1] https://docs.kernel.org/bpf/kfuncs.html#prog-annotation
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://github.com/theihor/dwarves/tree/prog-aux.v1

Ihor Solodrai (6):
bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag bpf,docs: Add documentation for KF_IMPLICIT_PROG_AUX_ARG selftests/bpf: update bpf_wq_set_callback macro
bpf: implement bpf_wq_set_callback kfunc with implicit prog_aux
bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG
bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG

 Documentation/bpf/kfuncs.rst                  | 39 ++++++++++++-
 include/linux/btf.h                           |  3 +
 kernel/bpf/helpers.c                          | 36 +++++++-----
 kernel/bpf/stream.c                           |  3 +-
 kernel/bpf/verifier.c                         | 58 ++++++++++++++-----
 tools/lib/bpf/bpf_helpers.h                   |  4 +-
 .../testing/selftests/bpf/bpf_experimental.h  |  7 ++-
 .../testing/selftests/bpf/progs/stream_fail.c |  6 +-
 tools/testing/selftests/bpf/progs/task_work.c |  6 +-
 .../selftests/bpf/progs/task_work_fail.c      |  8 +--
 .../selftests/bpf/progs/task_work_stress.c    |  2 +-
 .../testing/selftests/bpf/progs/wq_failures.c |  4 +-
 12 files changed, 130 insertions(+), 46 deletions(-)
Signature · 1 line
2.51.0
author reviewer
#1 lore raw

PATCH 1/6 bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Define KF_IMPLICIT_PROG_AUX_ARG and handle it in the BPF verifier.

The mechanism of patching is exactly the same as for __prog parameter
annotation: in check_kfunc_args() detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.

Then the (unchanged in this patch) fixup_kfunc_call() adds a mov instruction to set the actual pointer to prog_aux.

The caveat for KF_IMPLICIT_PROG_AUX_ARG is in implicitness. We have to separately check that the number of arguments is under MAX_BPF_FUNC_REG_ARGS.

---
 include/linux/btf.h   |  3 +++
 kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++-------
 2 files changed, 39 insertions(+), 7 deletions(-)
include/linux/btf.h +3 -0
index f06976ffb63f..479ee96c2c97 100644
@@ -79,6 +79,9 @@
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
+/* kfunc takes a pointer to struct bpf_prog_aux as the last argument,
+ * passed implicitly in BPF */
+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)
 
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +36 -7

68 lines not on this page · raw message

Signature · 1 line
2.51.0
#2 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 25 lines · Ihor Solodrai

Define KF_IMPLICIT_PROG_AUX_ARG and handle it in the BPF verifier.

The mechanism of patching is exactly the same as for __prog parameter
annotation: in check_kfunc_args() detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.

Then the (unchanged in this patch) fixup_kfunc_call() adds a mov instruction to set the actual pointer to prog_aux.

The caveat for KF_IMPLICIT_PROG_AUX_ARG is in implicitness. We have to separately check that the number of arguments is under MAX_BPF_FUNC_REG_ARGS.

---
 include/linux/btf.h   |  3 +++
 kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++-------
 2 files changed, 39 insertions(+), 7 deletions(-)
include/linux/btf.h +0 -0
index f06976ffb63f..479ee96c2c97 100644

include/linux/btf.h line 83

Quoted 5 lines
include/linux/btf.h +2 -0
@@ -79,6 +79,9 @@
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
+/* kfunc takes a pointer to struct bpf_prog_aux as the last argument,
+ * passed implicitly in BPF */

This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.

include/linux/btf.h line 84

Quoted 1 line
include/linux/btf.h +1 -0
+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)

The name is too verbose imo.
How about
KF_HIDDEN_PROG_ARG
or
KF_PROG_LAST_ARG

"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is KF_PROG_LAST_ARG.

"aux" part is also an implementation detail.

Quoted 38 lines
include/linux/btf.h +0 -0
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +17 -0
index e892df386eed..f1f9ea21f99b 100644
@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)
        return meta->kfunc_flags & KF_RCU_PROTECTED;
 }
 
+static bool is_kfunc_with_implicit_prog_aux_arg(struct bpf_kfunc_call_arg_meta *meta)
+{
+       return meta->kfunc_flags & KF_IMPLICIT_PROG_AUX_ARG;
+}
+
 static bool is_kfunc_arg_mem_size(const struct btf *btf,
                                  const struct btf_param *arg,
                                  const struct bpf_reg_state *reg)
@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *arg
        return btf_param_match_suffix(btf, arg, "__prog");
 }
 
+static int set_kfunc_arg_prog_regno(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, u32 regno)
+{
+       if (meta->arg_prog) {
+               verifier_bug(env, "Only 1 prog->aux argument supported per-kfunc");
+               return -EFAULT;
+       }
+       meta->arg_prog = true;
+       cur_aux(env)->arg_prog = regno;
+
+       return 0;
+}
+
 static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
                                          const struct btf_param *arg,
                                          const char *name)

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072

Quoted 6 lines
kernel/bpf/verifier.c +3 -0
@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
                return -EINVAL;
        }
 
+       /* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF,
+        * so we have to set_kfunc_arg_prog_regno() outside the arg check loop.
+        */

Use kernel comment style.

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+       if (is_kfunc_with_implicit_prog_aux_arg(meta)) {
+               if (nargs + 1 > MAX_BPF_FUNC_REG_ARGS) {
+                       verifier_bug(env, "A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args",
+                                    nargs + 1, MAX_BPF_FUNC_REG_ARGS);
+                       return -EFAULT;
+               }
+               u32 regno = nargs + 1;

Variable declaration should be first in the block followed by a blank line.

Also I would remove this double "> MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
and actual_nargs = nargs + 1;
if (actual_nargs > MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry that verbose() isn't too specific.
If it prints nargs and actual_nargs whoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add more KF_FOO_LAST_ARG flags to cleanup other *_impl() kfuncs that have a special last argument, like bpf_rbtree_add_impl. If all of them copy paste "> MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.

pw-bot: cr

author reviewer
#3 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 109 lines · Alexei Starovoitov
Quoted 25 lines · Ihor Solodrai

Define KF_IMPLICIT_PROG_AUX_ARG and handle it in the BPF verifier.

The mechanism of patching is exactly the same as for __prog parameter
annotation: in check_kfunc_args() detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.

Then the (unchanged in this patch) fixup_kfunc_call() adds a mov instruction to set the actual pointer to prog_aux.

The caveat for KF_IMPLICIT_PROG_AUX_ARG is in implicitness. We have to separately check that the number of arguments is under MAX_BPF_FUNC_REG_ARGS.

---
 include/linux/btf.h   |  3 +++
 kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++-------
 2 files changed, 39 insertions(+), 7 deletions(-)
include/linux/btf.h +0 -0
index f06976ffb63f..479ee96c2c97 100644

include/linux/btf.h line 83

Quoted 5 lines
include/linux/btf.h +2 -0
@@ -79,6 +79,9 @@
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
+/* kfunc takes a pointer to struct bpf_prog_aux as the last argument,
+ * passed implicitly in BPF */

This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.

include/linux/btf.h line 84

Quoted 1 line
include/linux/btf.h +1 -0
+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)

The name is too verbose imo.
How about
KF_HIDDEN_PROG_ARG
or
KF_PROG_LAST_ARG

"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is KF_PROG_LAST_ARG.

"aux" part is also an implementation detail.

Quoted 38 lines
include/linux/btf.h +0 -0
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +17 -0
index e892df386eed..f1f9ea21f99b 100644
@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)
        return meta->kfunc_flags & KF_RCU_PROTECTED;
 }
 
+static bool is_kfunc_with_implicit_prog_aux_arg(struct bpf_kfunc_call_arg_meta *meta)
+{
+       return meta->kfunc_flags & KF_IMPLICIT_PROG_AUX_ARG;
+}
+
 static bool is_kfunc_arg_mem_size(const struct btf *btf,
                                  const struct btf_param *arg,
                                  const struct bpf_reg_state *reg)
@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *arg
        return btf_param_match_suffix(btf, arg, "__prog");
 }
 
+static int set_kfunc_arg_prog_regno(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, u32 regno)
+{
+       if (meta->arg_prog) {
+               verifier_bug(env, "Only 1 prog->aux argument supported per-kfunc");
+               return -EFAULT;
+       }
+       meta->arg_prog = true;
+       cur_aux(env)->arg_prog = regno;
+
+       return 0;
+}
+
 static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
                                          const struct btf_param *arg,
                                          const char *name)

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072

Quoted 6 lines
kernel/bpf/verifier.c +3 -0
@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
                return -EINVAL;
        }
 
+       /* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF,
+        * so we have to set_kfunc_arg_prog_regno() outside the arg check loop.
+        */

Use kernel comment style.

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+       if (is_kfunc_with_implicit_prog_aux_arg(meta)) {
+               if (nargs + 1 > MAX_BPF_FUNC_REG_ARGS) {
+                       verifier_bug(env, "A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args",
+                                    nargs + 1, MAX_BPF_FUNC_REG_ARGS);
+                       return -EFAULT;
+               }
+               u32 regno = nargs + 1;

Variable declaration should be first in the block followed by a blank line.

Quoted 14 lines

Also I would remove this double "> MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
and actual_nargs = nargs + 1;
if (actual_nargs > MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry that verbose() isn't too specific.
If it prints nargs and actual_nargs whoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add more KF_FOO_LAST_ARG flags to cleanup other *_impl() kfuncs that have a special last argument, like bpf_rbtree_add_impl. If all of them copy paste "> MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.

Hi Alexei, thank you for the review.

Sorry for the styling mistakes, forgot to run patches through checkpatch.pl

In the other thread Eduard proposes a different approach to the implementation [1]. Basically, leave BTF unmodified and move argument hiding logic to bpftool's vmlinux.h generation.

IMO modifying BTF is more straightforward, but if the main goal is to have a nice BPF C interface, maybe Eduard is onto something.

Curious to hear yours and Andrii's opinion on that.

Thanks.

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 2 lines

pw-bot: cr

#4 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 48 lines · Ihor Solodrai
Quoted 44 lines · Alexei Starovoitov
Quoted 25 lines · Ihor Solodrai

Define KF_IMPLICIT_PROG_AUX_ARG and handle it in the BPF verifier.

The mechanism of patching is exactly the same as for __prog parameter
annotation: in check_kfunc_args() detect the relevant parameter and remember regno in cur_aux(env)->arg_prog.

Then the (unchanged in this patch) fixup_kfunc_call() adds a mov instruction to set the actual pointer to prog_aux.

The caveat for KF_IMPLICIT_PROG_AUX_ARG is in implicitness. We have to separately check that the number of arguments is under MAX_BPF_FUNC_REG_ARGS.

---
 include/linux/btf.h   |  3 +++
 kernel/bpf/verifier.c | 43 ++++++++++++++++++++++++++++++++++++-------
 2 files changed, 39 insertions(+), 7 deletions(-)
include/linux/btf.h +0 -0
index f06976ffb63f..479ee96c2c97 100644

include/linux/btf.h line 83

Quoted 5 lines
include/linux/btf.h +2 -0
@@ -79,6 +79,9 @@
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
+/* kfunc takes a pointer to struct bpf_prog_aux as the last argument,
+ * passed implicitly in BPF */

This is neither networking nor kernel comment style. Pls use proper kernel comment style in a new code, and reformat old net/bpf style when adjusting old comments.

include/linux/btf.h line 84

Quoted 1 line
include/linux/btf.h +1 -0
+#define KF_IMPLICIT_PROG_AUX_ARG (1 << 16)

The name is too verbose imo.
How about
KF_HIDDEN_PROG_ARG
or
KF_PROG_LAST_ARG

Quoted 6 lines
Quoted 6 lines

"Implicit" is not 100% correct, since it's very explicit in kfunc definition in C, but removed from BTF. "Hidden" is also not an exact fit for the same reasons. Hence my preference is KF_PROG_LAST_ARG.

"aux" part is also an implementation detail.

+1, I'd consider even just naming it KF_PROG_ARG, where "last" is implied, but that's minor.

Quoted 79 lines
Quoted 59 lines
Quoted 39 lines
include/linux/btf.h +0 -0
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +17 -0
index e892df386eed..f1f9ea21f99b 100644
@@ -11948,6 +11948,11 @@ static bool is_kfunc_rcu_protected(struct bpf_kfunc_call_arg_meta *meta)
        return meta->kfunc_flags & KF_RCU_PROTECTED;
 }
 
+static bool is_kfunc_with_implicit_prog_aux_arg(struct bpf_kfunc_call_arg_meta *meta)
+{
+       return meta->kfunc_flags & KF_IMPLICIT_PROG_AUX_ARG;
+}
+
 static bool is_kfunc_arg_mem_size(const struct btf *btf,
                                  const struct btf_param *arg,
                                  const struct bpf_reg_state *reg)
@@ -12029,6 +12034,18 @@ static bool is_kfunc_arg_prog(const struct btf *btf, const struct btf_param *arg
        return btf_param_match_suffix(btf, arg, "__prog");
 }
 
+static int set_kfunc_arg_prog_regno(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, u32 regno)
+{
+       if (meta->arg_prog) {
+               verifier_bug(env, "Only 1 prog->aux argument supported per-kfunc");
+               return -EFAULT;
+       }
+       meta->arg_prog = true;
+       cur_aux(env)->arg_prog = regno;
+
+       return 0;
+}
+
 static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
                                          const struct btf_param *arg,
                                          const char *name)

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13072

Quoted 6 lines
kernel/bpf/verifier.c +3 -0
@@ -13050,6 +13067,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
                return -EINVAL;
        }
 
+       /* KF_IMPLICIT_PROG_AUX_ARG means that the kfunc has one less argument in BTF,
+        * so we have to set_kfunc_arg_prog_regno() outside the arg check loop.
+        */

Use kernel comment style.

kernel/bpf/verifier.c static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ · line 13079

Quoted 7 lines
kernel/bpf/verifier.c +7 -0
+       if (is_kfunc_with_implicit_prog_aux_arg(meta)) {
+               if (nargs + 1 > MAX_BPF_FUNC_REG_ARGS) {
+                       verifier_bug(env, "A kfunc with KF_IMPLICIT_PROG_AUX_ARG flag has %d > %d args",
+                                    nargs + 1, MAX_BPF_FUNC_REG_ARGS);
+                       return -EFAULT;
+               }
+               u32 regno = nargs + 1;

Variable declaration should be first in the block followed by a blank line.

Quoted 14 lines

Also I would remove this double "> MAX_BPF_FUNC_REG_ARGS" check. Move if (is_kfunc_with_prog_last_arg(meta))
couple lines above before the check,
and actual_nargs = nargs + 1;
if (actual_nargs > MAX_BPF_FUNC_REG_ARGS)
to cover both cases.
I wouldn't worry that verbose() isn't too specific.
If it prints nargs and actual_nargs whoever develops a kfunc can get an idea.
Also in the future there is a good chance we will add more KF_FOO_LAST_ARG flags to cleanup other *_impl() kfuncs that have a special last argument, like bpf_rbtree_add_impl. If all of them copy paste "> MAX_BPF_FUNC_REG_ARGS" check it will be too verbose. Hence one nargs check for them all.

Hi Alexei, thank you for the review.

Sorry for the styling mistakes, forgot to run patches through checkpatch.pl

Quoted 9 lines

In the other thread Eduard proposes a different approach to the implementation [1]. Basically, leave BTF unmodified and move argument hiding logic to bpftool's vmlinux.h generation.

IMO modifying BTF is more straightforward, but if the main goal is to have a nice BPF C interface, maybe Eduard is onto something.

Curious to hear yours and Andrii's opinion on that.

I explicitly really-really don't want to modify BPF CO-RE type matching rules for these special kfuncs. So that's why I think it's better overall to hide those special arguments from the BTF information altogether.

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Thoughts?

Quoted 7 lines

Thanks.

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 2 lines

pw-bot: cr

#5 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog); +}

It's just one jmpl insn, but still.

#6 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:

(quoted diff) +1 -1
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl)
+BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

(quoted diff) +7 -0
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
Quoted 1 line

It's just one jmpl insn, but still.

So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.

Two notes:

a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type

b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?

#7 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 54 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:

(quoted diff) +1 -1
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl)
+BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

(quoted diff) +7 -0
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
Quoted 1 line

It's just one jmpl insn, but still.

So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.

Two notes:

Quoted 7 lines

a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type

b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?

We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.
And then double down on this convention going forward.

Quoted 1 line
#8 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 61 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:

(quoted diff) +1 -1
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl)
+BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

(quoted diff) +7 -0
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
Quoted 1 line

It's just one jmpl insn, but still.

So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.

Two notes:

a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type

Quoted 7 lines
Quoted 3 lines

b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?

We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.

Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".

Quoted 3 lines

And then double down on this convention going forward.

Quoted 1 line
#9 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 58 lines · Andrii Nakryiko
Quoted 55 lines · Kumar Kartikeya Dwivedi
Quoted 52 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:

(quoted diff) +1 -1
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl)
+BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

(quoted diff) +7 -0
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
Quoted 1 line

It's just one jmpl insn, but still.

So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Two notes:

a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type

agree

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?

Quoted 8 lines
Quoted 3 lines

We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.

Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".

I think we can still do it even after 6.17 is released and backport it as a fix.

#10 lore raw

Re: [PATCH bpf-next v1 1/6] bpf: implement KF_IMPLICIT_PROG_AUX_ARG flag

Quoted 78 lines · Alexei Starovoitov
Quoted 58 lines · Andrii Nakryiko
Quoted 55 lines · Kumar Kartikeya Dwivedi
Quoted 52 lines · Andrii Nakryiko
Quoted 43 lines · Alexei Starovoitov
Quoted 13 lines · Andrii Nakryiko

I do see the benefit of having the generic "KF_MAGIC_ARG(s)" flag on the kernel side of things and having access to full BTF information for parameters to let verifier know what specific kind of magic argument that kfunc has, though. So as an alternative, maybe we can create both a kfunc definition *meant for BPF programs* (i.e., without magic argument(s)), and then have a full original definition (produced by pahole, it will need to understand KF_MAGIC_ARGS anyways) with full type information *for internal BPF verifier needs*. I don't know what's the best way to do that, maybe just a special ".magic" suffix, just to let the verifier easily find that? On the kernel side, if kfunc has BPF_MAGIC_ARGS kflag we just look up "my_fancy_kfunc.magic" FUNC definition?

Interesting idea. Maybe to simplify backward compat the pahole can emit two BTFs: kfunc_foo(args), kfunc_foo_impl(args, void *aux) into vmlinux BTF.
bpftool will emit both in vmlinux.h and bpf side doesn't need to change. libbpf doesn't need to change either.
The verifier would need a special check to resolve two kfunc BTFs name into one kallsym name, since both kfuncs is one actual function on the kernel.
bpf_wq_set_callback_impl() definition doesn't change. Only:

(quoted diff) +1 -1
-BTF_ID_FLAGS(func, bpf_wq_set_callback_impl)
+BTF_ID_FLAGS(func, bpf_wq_set_callback_impl, KF_PROG_ARG)

and the verifier can check that the last arg is aux__prog when KF_PROG_ARG is specified.

The runtime performance will be slightly better too, since no need for wrappers like:

(quoted diff) +7 -0
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+ int (callback_fn)(void *map, int *key, void *value),
+ unsigned int flags,
+ void *aux__prog)
+{
+ return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
Quoted 1 line

It's just one jmpl insn, but still.

So basically xxx_impl() will be a phantom function that verifier will recognize and it will need to have corresponding xxx() kfunc with corresponding KF_PROG_ARG for everything to work. Makes sense.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Two notes:

a) KF flag would need to be more generically named, because we'll have other implicit arguments (like those for bpf_obj_new_impl, for example), which will be distinguished based on their BTF type

agree

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

b) bpf_stream_vprintk() throws a bit of a wrench into all this because it doesn't follow _impl naming convention. Any suggestions on how to deal with that?

We can probably do a compat break for this kfunc alone for now, it's not been a long time since it's been out (1 release) not much adoption yet.

Quoted 7 lines
Quoted 4 lines

Ideally there would be 0 releases with that name :) it's just tantalizing that we can s/bpf_stream_vprintk/bpf_stream_vprintk_impl/ in like 8 places and avoid this altogether, but it is so late in the release that I suspect no one will want to do this last minute "fix".

I think we can still do it even after 6.17 is released and backport it as a fix.

ok then, sounds like a plan

author reviewer
#11 lore raw

PATCH 2/6 bpf,docs: Add documentation for KF_IMPLICIT_PROG_AUX_ARG

In reply to Ihor Solodrai

Add a section explaining KF_IMPLICIT_PROG_AUX_ARG kfunc flag. Mark __prog annotation as deprecated.

---
 Documentation/bpf/kfuncs.rst | 39 +++++++++++++++++++++++++++++++++++-
 1 file changed, 38 insertions(+), 1 deletion(-)
Documentation/bpf/kfuncs.rst +38 -1
index e38941370b90..cc04ffd9a667 100644
@@ -160,7 +160,7 @@ Or::
                 ...
         }
 
-2.2.6 __prog Annotation
+2.2.6 __prog Annotation (deprecated, use KF_IMPLICIT_PROG_AUX_ARG instead)
 ---------------------------
 This annotation is used to indicate that the argument needs to be fixed up to
 the bpf_prog_aux of the caller BPF program. Any value passed into this argument
@@ -374,6 +374,43 @@ encouraged to make their use-cases known as early as possible, and participate
 in upstream discussions regarding whether to keep, change, deprecate, or remove
 those kfuncs if and when such discussions occur.
 
+2.4.10 KF_IMPLICIT_PROG_AUX_ARG flag
+------------------------------------
+
+The KF_IMPLICIT_PROG_AUX_ARG flag is used to indicate that the last
+argument of a BPF kfunc is a pointer to struct bpf_prog_aux of the
+caller BPF program, implicitly set by the verifier.
+
+If a kfunc is marked with this flag, the function declaration in the
+kernel must have ``struct bpf_prog_aux *`` as the last argument.
+However in the kernel BTF (produced by pahole) this argument will be
+omitted, making it invisible to BPF programs calling the kfunc.
+
+Note that the implicit argument is an actual BPF argument passed
+through a register, reducing the number of the available function
+arguments.
+
+Example declaration:
+
+.. code-block:: c
+
+	__bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw,
+						void *map__map, bpf_task_work_callback_t callback,
+						struct bpf_prog_aux *aux)
+	{
+		return bpf_task_work_schedule(task, tw, map__map, callback, aux, TWA_RESUME);
+	}
+
+	BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)
+
+Example usage:
+
+.. code-block:: c
+
+	/* note the last argument is ommitted */
+	bpf_task_work_schedule_resume(task, &tw, &hmap, process_work);
+
+
 2.5 Registering the kfuncs
 --------------------------
 
Signature · 1 line
2.51.0
author reviewer
#12 lore raw

PATCH 3/6 selftests/bpf: update bpf_wq_set_callback macro

In reply to Ihor Solodrai

Subsequent patch introduces bpf_wq_set_callback kfunc with an implicit bpf_prog_aux argument.

To ensure backward compatibility add a weak declaration and make bpf_wq_set_callback macro to check for the new kfunc first.

---
 tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_experimental.h +6 -1
index d89eda3fd8a3..341408d017ea 100644
@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym;
 extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,
 		int (callback_fn)(void *map, int *key, void *value),
 		unsigned int flags__k, void *aux__ign) __ksym;
+extern int bpf_wq_set_callback(struct bpf_wq *wq,
+		int (callback_fn)(void *map, int *key, void *value),
+		unsigned int flags) __weak __ksym;
 #define bpf_wq_set_callback(timer, cb, flags) \
-	bpf_wq_set_callback_impl(timer, cb, flags, NULL)
+	(bpf_wq_set_callback ? \
+		bpf_wq_set_callback(timer, cb, flags) : \
+		bpf_wq_set_callback_impl(timer, cb, flags, NULL))
 
 struct bpf_iter_kmem_cache;
 extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym;
Signature · 1 line
2.51.0
#13 lore raw

Re: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro

Quoted 18 lines · Ihor Solodrai

Subsequent patch introduces bpf_wq_set_callback kfunc with an implicit bpf_prog_aux argument.

To ensure backward compatibility add a weak declaration and make bpf_wq_set_callback macro to check for the new kfunc first.

---
 tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_experimental.h +0 -0
index d89eda3fd8a3..341408d017ea 100644
@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym;
 extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,

tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 592

Quoted 10 lines
tools/testing/selftests/bpf/bpf_experimental.h +6 -1
                int (callback_fn)(void *map, int *key, void *value),
                unsigned int flags__k, void *aux__ign) __ksym;
+extern int bpf_wq_set_callback(struct bpf_wq *wq,
+               int (callback_fn)(void *map, int *key, void *value),
+               unsigned int flags) __weak __ksym;
 #define bpf_wq_set_callback(timer, cb, flags) \
-       bpf_wq_set_callback_impl(timer, cb, flags, NULL)
+       (bpf_wq_set_callback ? \
+               bpf_wq_set_callback(timer, cb, flags) : \
+               bpf_wq_set_callback_impl(timer, cb, flags, NULL))

There is also drivers/hid/bpf/progs/hid_bpf_helpers.h Pls double check that hid-bpf still compiles and works.

author reviewer
#14 lore raw

Re: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro

Quoted 24 lines · Alexei Starovoitov
Quoted 23 lines · Ihor Solodrai

Subsequent patch introduces bpf_wq_set_callback kfunc with an implicit bpf_prog_aux argument.

To ensure backward compatibility add a weak declaration and make bpf_wq_set_callback macro to check for the new kfunc first.

---
 tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_experimental.h +3 -0
index d89eda3fd8a3..341408d017ea 100644
@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym;
 extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,
                int (callback_fn)(void *map, int *key, void *value),
                unsigned int flags__k, void *aux__ign) __ksym;
+extern int bpf_wq_set_callback(struct bpf_wq *wq,
+               int (callback_fn)(void *map, int *key, void *value),
+               unsigned int flags) __weak __ksym;
Quoted 8 lines

tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 592

Quoted 5 lines
tools/testing/selftests/bpf/bpf_experimental.h +3 -1
 #define bpf_wq_set_callback(timer, cb, flags) \
-       bpf_wq_set_callback_impl(timer, cb, flags, NULL)
+       (bpf_wq_set_callback ? \
+               bpf_wq_set_callback(timer, cb, flags) : \
+               bpf_wq_set_callback_impl(timer, cb, flags, NULL))

There is also drivers/hid/bpf/progs/hid_bpf_helpers.h Pls double check that hid-bpf still compiles and works.

Yes, I noticed the usage.

I plan to send separate patches to hid when this series gets closer to landing.

#15 lore raw

Re: [PATCH bpf-next v1 3/6] selftests/bpf: update bpf_wq_set_callback macro

In reply to Ihor Solodrai
Quoted 17 lines · Ihor Solodrai

Subsequent patch introduces bpf_wq_set_callback kfunc with an implicit bpf_prog_aux argument.

To ensure backward compatibility add a weak declaration and make bpf_wq_set_callback macro to check for the new kfunc first.

---
 tools/testing/selftests/bpf/bpf_experimental.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_experimental.h +0 -0
index d89eda3fd8a3..341408d017ea 100644

tools/testing/selftests/bpf/bpf_experimental.h extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; · line 590

Quoted 9 lines
tools/testing/selftests/bpf/bpf_experimental.h +4 -1
@@ -583,8 +583,13 @@ extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym;
 extern int bpf_wq_set_callback_impl(struct bpf_wq *wq,
                int (callback_fn)(void *map, int *key, void *value),
                unsigned int flags__k, void *aux__ign) __ksym;
+extern int bpf_wq_set_callback(struct bpf_wq *wq,
+               int (callback_fn)(void *map, int *key, void *value),
+               unsigned int flags) __weak __ksym;
 #define bpf_wq_set_callback(timer, cb, flags) \
-       bpf_wq_set_callback_impl(timer, cb, flags, NULL)
+       (bpf_wq_set_callback ? \

use bpf_ksym_exists(), it has additional "fool-proofing" checks

Quoted 8 lines
tools/testing/selftests/bpf/bpf_experimental.h +2 -1
+               bpf_wq_set_callback(timer, cb, flags) : \
+               bpf_wq_set_callback_impl(timer, cb, flags, NULL))
 
 struct bpf_iter_kmem_cache;
 extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym;
--

2.51.0

author reviewer
#16 lore raw

PATCH 4/6 bpf: implement bpf_wq_set_callback kfunc with implicit prog_aux

In reply to Ihor Solodrai

Add bpf_wq_set_callback BPF kfunc with KF_IMPLICIT_PROG_AUX_ARG, corresponding to bpf_wq_set_callback_impl kfunc. Teach the verifier about it.

To be handled correctly, BTF for this kfunc must be generated with pahole version that supports KF_IMPLICIT_PROG_AUX_ARG.

---
 kernel/bpf/helpers.c                           | 18 +++++++++++++-----
 kernel/bpf/verifier.c                          | 15 +++++++++------
 .../testing/selftests/bpf/progs/wq_failures.c  |  4 ++--
 3 files changed, 24 insertions(+), 13 deletions(-)
kernel/bpf/helpers.c +13 -5
index c9fab9a356df..6b46acfec790 100644
@@ -3065,12 +3065,11 @@ __bpf_kfunc int bpf_wq_start(struct bpf_wq *wq, unsigned int flags)
 	return 0;
 }
 
-__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
-					 int (callback_fn)(void *map, int *key, void *value),
-					 unsigned int flags,
-					 void *aux__prog)
+__bpf_kfunc int bpf_wq_set_callback(struct bpf_wq *wq,
+				    int (callback_fn)(void *map, int *key, void *value),
+				    unsigned int flags,
+				    struct bpf_prog_aux *aux)
 {
-	struct bpf_prog_aux *aux = (struct bpf_prog_aux *)aux__prog;
 	struct bpf_async_kern *async = (struct bpf_async_kern *)wq;
 
 	if (flags)
@@ -3079,6 +3078,14 @@ __bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
 	return __bpf_async_set_callback(async, callback_fn, aux, flags, BPF_ASYNC_TYPE_WQ);
 }
 
+__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq,
+					 int (callback_fn)(void *map, int *key, void *value),
+					 unsigned int flags,
+					 void *aux__prog)
+{
+	return bpf_wq_set_callback(wq, callback_fn, flags, aux__prog);
+}
+
 __bpf_kfunc void bpf_preempt_disable(void)
 {
 	preempt_disable();
@@ -4374,6 +4381,7 @@ BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_IMPLICIT_PROG_AUX_ARG)
 BTF_KFUNCS_END(common_btf_ids)
 
 static const struct btf_kfunc_id_set common_kfunc_set = {
kernel/bpf/verifier.c +9 -6
index f1f9ea21f99b..a29d3b85aed2 100644
@@ -514,7 +514,7 @@ static bool is_async_callback_calling_kfunc(u32 btf_id);
 static bool is_callback_calling_kfunc(u32 btf_id);
 static bool is_bpf_throw_kfunc(struct bpf_insn *insn);
 
-static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id);
+static bool is_bpf_wq_set_callback_kfunc(u32 btf_id);
 
 static bool is_sync_callback_calling_function(enum bpf_func_id func_id)
 {
@@ -10586,7 +10586,7 @@ static int push_callback_call(struct bpf_verifier_env *env, struct bpf_insn *ins
 		env->subprog_info[subprog].is_async_cb = true;
 		async_cb = push_async_cb(env, env->subprog_info[subprog].start,
 					 insn_idx, subprog,
-					 is_bpf_wq_set_callback_impl_kfunc(insn->imm) ||
+					 is_bpf_wq_set_callback_kfunc(insn->imm) ||
 					 is_task_work_add_kfunc(insn->imm));
 		if (!async_cb)
 			return -EFAULT;
@@ -12278,6 +12278,7 @@ enum special_kfunc_type {
 	KF___bpf_trap,
 	KF_bpf_task_work_schedule_signal,
 	KF_bpf_task_work_schedule_resume,
+	KF_bpf_wq_set_callback,
 };
 
 BTF_ID_LIST(special_kfunc_list)
@@ -12350,6 +12351,7 @@ BTF_ID(func, bpf_res_spin_unlock_irqrestore)
 BTF_ID(func, __bpf_trap)
 BTF_ID(func, bpf_task_work_schedule_signal)
 BTF_ID(func, bpf_task_work_schedule_resume)
+BTF_ID(func, bpf_wq_set_callback)
 
 static bool is_task_work_add_kfunc(u32 func_id)
 {
@@ -12797,7 +12799,7 @@ static bool is_sync_callback_calling_kfunc(u32 btf_id)
 
 static bool is_async_callback_calling_kfunc(u32 btf_id)
 {
-	return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] ||
+	return is_bpf_wq_set_callback_kfunc(btf_id) ||
 	       is_task_work_add_kfunc(btf_id);
 }
 
@@ -12807,9 +12809,10 @@ static bool is_bpf_throw_kfunc(struct bpf_insn *insn)
 	       insn->imm == special_kfunc_list[KF_bpf_throw];
 }
 
-static bool is_bpf_wq_set_callback_impl_kfunc(u32 btf_id)
+static bool is_bpf_wq_set_callback_kfunc(u32 btf_id)
 {
-	return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl];
+	return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] ||
+	       btf_id == special_kfunc_list[KF_bpf_wq_set_callback];
 }
 
 static bool is_callback_calling_kfunc(u32 btf_id)
@@ -13910,7 +13913,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		meta.r0_rdonly = false;
 	}
 
-	if (is_bpf_wq_set_callback_impl_kfunc(meta.func_id)) {
+	if (is_bpf_wq_set_callback_kfunc(meta.func_id)) {
 		err = push_callback_call(env, insn, insn_idx, meta.subprogno,
 					 set_timer_callback_state);
 		if (err) {
tools/testing/selftests/bpf/progs/wq_failures.c +2 -2
index 4240211a1900..9295b45ef27c 100644
@@ -97,7 +97,7 @@ __failure
 /* check that the first argument of bpf_wq_set_callback()
  * is a correct bpf_wq pointer.
  */
-__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */
+__msg(": (85) call bpf_wq_set_callback#") /* anchor message */
 __msg("arg#0 doesn't point to a map value")
 long test_wrong_wq_pointer(void *ctx)
 {
@@ -123,7 +123,7 @@ __failure
 /* check that the first argument of bpf_wq_set_callback()
  * is a correct bpf_wq pointer.
  */
-__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */
+__msg(": (85) call bpf_wq_set_callback#") /* anchor message */
 __msg("off 1 doesn't point to 'struct bpf_wq' that is at 0")
 long test_wrong_wq_pointer_offset(void *ctx)
 {
Signature · 1 line
2.51.0
author reviewer
#17 lore raw

PATCH 5/6 bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG

In reply to Ihor Solodrai

Update bpf_stream_vprink macro in libbpf and fix call sites in the relevant selftests.

---
 kernel/bpf/helpers.c                            | 2 +-
 kernel/bpf/stream.c                             | 3 +--
 tools/lib/bpf/bpf_helpers.h                     | 4 ++--
 tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++---
 4 files changed, 7 insertions(+), 8 deletions(-)
kernel/bpf/helpers.c +1 -1
index 6b46acfec790..875195a0ea72 100644
@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);
 #if defined(CONFIG_BPF_LSM) && defined(CONFIG_CGROUPS)
 BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 #endif
-BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)
 BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_IMPLICIT_PROG_AUX_ARG)
kernel/bpf/stream.c +1 -2
index ab592db4a4bf..0c75d0a8cea2 100644
@@ -355,13 +355,12 @@ __bpf_kfunc_start_defs();
  * Avoid using enum bpf_stream_id so that kfunc users don't have to pull in the
  * enum in headers.
  */
-__bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, u32 len__sz, void *aux__prog)
+__bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, u32 len__sz, struct bpf_prog_aux *aux)
 {
 	struct bpf_bprintf_data data = {
 		.get_bin_args	= true,
 		.get_buf	= true,
 	};
-	struct bpf_prog_aux *aux = aux__prog;
 	u32 fmt_size = strlen(fmt__str) + 1;
 	struct bpf_stream *stream;
 	u32 data_len = len__sz;
tools/lib/bpf/bpf_helpers.h +2 -2
index 80c028540656..9a6d719315a3 100644
@@ -316,7 +316,7 @@ enum libbpf_tristate {
 })
 
 extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args,
-			      __u32 len__sz, void *aux__prog) __weak __ksym;
+			      __u32 len__sz) __weak __ksym;
 
 #define bpf_stream_printk(stream_id, fmt, args...)				\
 ({										\
@@ -328,7 +328,7 @@ extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *a
 	___bpf_fill(___param, args);						\
 	_Pragma("GCC diagnostic pop")						\
 										\
-	bpf_stream_vprintk(stream_id, ___fmt, ___param, sizeof(___param), NULL);\
+	bpf_stream_vprintk(stream_id, ___fmt, ___param, sizeof(___param));\
 })
 
 /* Use __bpf_printk when bpf_printk call has 3 or fewer fmt args
tools/testing/selftests/bpf/progs/stream_fail.c +3 -3
index b4a0d0cc8ec8..8e8249f3521c 100644
@@ -10,7 +10,7 @@ SEC("syscall")
 __failure __msg("Possibly NULL pointer passed")
 int stream_vprintk_null_arg(void *ctx)
 {
-	bpf_stream_vprintk(BPF_STDOUT, "", NULL, 0, NULL);
+	bpf_stream_vprintk(BPF_STDOUT, "", NULL, 0);
 	return 0;
 }
 
@@ -18,7 +18,7 @@ SEC("syscall")
 __failure __msg("R3 type=scalar expected=")
 int stream_vprintk_scalar_arg(void *ctx)
 {
-	bpf_stream_vprintk(BPF_STDOUT, "", (void *)46, 0, NULL);
+	bpf_stream_vprintk(BPF_STDOUT, "", (void *)46, 0);
 	return 0;
 }
 
@@ -26,7 +26,7 @@ SEC("syscall")
 __failure __msg("arg#1 doesn't point to a const string")
 int stream_vprintk_string_arg(void *ctx)
 {
-	bpf_stream_vprintk(BPF_STDOUT, ctx, NULL, 0, NULL);
+	bpf_stream_vprintk(BPF_STDOUT, ctx, NULL, 0);
 	return 0;
 }
 
Signature · 1 line
2.51.0
#18 lore raw

Re: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG

Quoted 2 lines · Ihor Solodrai

Update bpf_stream_vprink macro in libbpf and fix call sites in

't' is missing in bpf_stream_vprintk().

Quoted 15 lines

the relevant selftests.

---
 kernel/bpf/helpers.c                            | 2 +-
 kernel/bpf/stream.c                             | 3 +--
 tools/lib/bpf/bpf_helpers.h                     | 4 ++--
 tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++---
 4 files changed, 7 insertions(+), 8 deletions(-)
kernel/bpf/helpers.c +0 -0
index 6b46acfec790..875195a0ea72 100644

kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381

Quoted 5 lines
kernel/bpf/helpers.c +1 -1
@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);
 #if defined(CONFIG_BPF_LSM) && defined(CONFIG_CGROUPS)
 BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 #endif
-BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)

This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.

tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319

Quoted 8 lines
tools/lib/bpf/bpf_helpers.h +1 -1
@@ -316,7 +316,7 @@ enum libbpf_tristate {
 })
 
 extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args,
-                             __u32 len__sz, void *aux__prog) __weak __ksym;
+                             __u32 len__sz) __weak __ksym;

CI is complaining of conflicting types for 'bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.

author reviewer
#19 lore raw

Re: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG

Quoted 16 lines · Alexei Starovoitov
Quoted 2 lines · Ihor Solodrai

Update bpf_stream_vprink macro in libbpf and fix call sites in

't' is missing in bpf_stream_vprintk().

Quoted 10 lines

the relevant selftests.

---
 kernel/bpf/helpers.c                            | 2 +-
 kernel/bpf/stream.c                             | 3 +--
 tools/lib/bpf/bpf_helpers.h                     | 4 ++--
 tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++---
 4 files changed, 7 insertions(+), 8 deletions(-)
Quoted 14 lines

kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381

Quoted 10 lines
kernel/bpf/helpers.c +1 -1
index 6b46acfec790..875195a0ea72 100644
@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);
 #if defined(CONFIG_BPF_LSM) && defined(CONFIG_CGROUPS)
 BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 #endif
-BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)

This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.

Acked.

Quoted 6 lines
Quoted 6 lines
tools/lib/bpf/bpf_helpers.h +0 -0
@@ -316,7 +316,7 @@ enum libbpf_tristate {
 })
 
Quoted 10 lines

tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319

Quoted 3 lines
tools/lib/bpf/bpf_helpers.h +1 -1
 extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args,
-                             __u32 len__sz, void *aux__prog) __weak __ksym;
+                             __u32 len__sz) __weak __ksym;

CI is complaining of conflicting types for 'bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.

Right.

One way to do it is to conditionalize new kfunc defitions under a config flag similar to how we have CONFIG_PAHOLE_HAS_SPLIT_BTF and co.

But this seems like too much trouble for just a couple of kfuncs, especially since we do not guarantee stable interface for kfuncs.

We could ask people to update pahole version if they run into issues. And they shouldn't unless they are actually using the kfuncs, or more specifically: have their own declarations of said kfuncs.

bpf_stream_vprintk() is a special case, because it is already in libbpf.

#20 lore raw

Re: [PATCH bpf-next v1 5/6] bpf: mark bpf_stream_vprink kfunc with KF_IMPLICIT_PROG_AUX_ARG

In reply to Alexei Starovoitov
Quoted 17 lines · Alexei Starovoitov
Quoted 2 lines · Ihor Solodrai

Update bpf_stream_vprink macro in libbpf and fix call sites in

't' is missing in bpf_stream_vprintk().

Quoted 10 lines

the relevant selftests.

---
 kernel/bpf/helpers.c                            | 2 +-
 kernel/bpf/stream.c                             | 3 +--
 tools/lib/bpf/bpf_helpers.h                     | 4 ++--
 tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++---
 4 files changed, 7 insertions(+), 8 deletions(-)
Quoted 14 lines

kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_strnstr); · line 4381

Quoted 10 lines
kernel/bpf/helpers.c +1 -1
index 6b46acfec790..875195a0ea72 100644
@@ -4378,7 +4378,7 @@ BTF_ID_FLAGS(func, bpf_strnstr);
 #if defined(CONFIG_BPF_LSM) && defined(CONFIG_CGROUPS)
 BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 #endif
-BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)

This kfunc will be in part of 6.17 release in a couple days, so backward compat work is necessary.
I don't think we can just remove the arg.

Can we still rename current "explicit prog_aux" version to bpf_stream_vprintk_impl() still?

That would solve backwards compat problems and would be in line with bpf_wq_set_callback_impl() approach.

If we can't, then we'd have to name the new function something different?

Don't know if there is some other BPF CO-RE and ELF symbol aliasing magic that can be used, we'll need to play with this to see what works.

Quoted 16 lines

tools/lib/bpf/bpf_helpers.h enum libbpf_tristate { · line 319

Quoted 9 lines
tools/lib/bpf/bpf_helpers.h +1 -1
@@ -316,7 +316,7 @@ enum libbpf_tristate {
 })
 
 extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args,
-                             __u32 len__sz, void *aux__prog) __weak __ksym;
+                             __u32 len__sz) __weak __ksym;

CI is complaining of conflicting types for 'bpf_stream_vprintk', since it's using pahole master.
It will stop complaining once pahole changes are merged, but this issue will affect all developers until they update pahole.
Not sure how to keep backward compat.

author reviewer
#21 lore raw

PATCH 6/6 bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG

In reply to Ihor Solodrai

Two kfuncs that use aux__prog argument were recently added [1]:
* bpf_task_work_schedule_resume
* bpf_task_work_schedule_signal

Update them to use the new kfunc flag and fix usages in the selftests.

[1] https://lore.kernel.org/bpf/[email protected]/

---
 kernel/bpf/helpers.c                             | 16 ++++++++--------
 tools/testing/selftests/bpf/progs/task_work.c    |  6 +++---
 .../testing/selftests/bpf/progs/task_work_fail.c |  8 ++++----
 .../selftests/bpf/progs/task_work_stress.c       |  2 +-
 4 files changed, 16 insertions(+), 16 deletions(-)
kernel/bpf/helpers.c +8 -8
index 875195a0ea72..ccdc204a66cf 100644
@@ -4178,15 +4178,15 @@ static int bpf_task_work_schedule(struct task_struct *task, struct bpf_task_work
  * @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping
  * @map__map: bpf_map that embeds struct bpf_task_work in the values
  * @callback: pointer to BPF subprogram to call
- * @aux__prog: user should pass NULL
+ * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier
  *
  * Return: 0 if task work has been scheduled successfully, negative error code otherwise
  */
 __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct bpf_task_work *tw,
 					      void *map__map, bpf_task_work_callback_t callback,
-					      void *aux__prog)
+					      struct bpf_prog_aux *aux)
 {
-	return bpf_task_work_schedule(task, tw, map__map, callback, aux__prog, TWA_SIGNAL);
+	return bpf_task_work_schedule(task, tw, map__map, callback, aux, TWA_SIGNAL);
 }
 
 /**
@@ -4195,15 +4195,15 @@ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct b
  * @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping
  * @map__map: bpf_map that embeds struct bpf_task_work in the values
  * @callback: pointer to BPF subprogram to call
- * @aux__prog: user should pass NULL
+ * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier
  *
  * Return: 0 if task work has been scheduled successfully, negative error code otherwise
  */
 __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw,
 					      void *map__map, bpf_task_work_callback_t callback,
-					      void *aux__prog)
+					      struct bpf_prog_aux *aux)
 {
-	return bpf_task_work_schedule(task, tw, map__map, callback, aux__prog, TWA_RESUME);
+	return bpf_task_work_schedule(task, tw, map__map, callback, aux, TWA_RESUME);
 }
 
 __bpf_kfunc_end_defs();
@@ -4379,8 +4379,8 @@ BTF_ID_FLAGS(func, bpf_strnstr);
 BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)
 #endif
 BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)
-BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS)
-BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)
+BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS | KF_IMPLICIT_PROG_AUX_ARG)
 BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_IMPLICIT_PROG_AUX_ARG)
 BTF_KFUNCS_END(common_btf_ids)
 
tools/testing/selftests/bpf/progs/task_work.c +3 -3
index 23217f06a3ec..eedd5c3dabb4 100644
@@ -66,7 +66,7 @@ int oncpu_hash_map(struct pt_regs *args)
 	if (!work)
 		return 0;
 
-	bpf_task_work_schedule_resume(task, &work->tw, &hmap, process_work, NULL);
+	bpf_task_work_schedule_resume(task, &work->tw, &hmap, process_work);
 	return 0;
 }
 
@@ -80,7 +80,7 @@ int oncpu_array_map(struct pt_regs *args)
 	work = bpf_map_lookup_elem(&arrmap, &key);
 	if (!work)
 		return 0;
-	bpf_task_work_schedule_signal(task, &work->tw, &arrmap, process_work, NULL);
+	bpf_task_work_schedule_signal(task, &work->tw, &arrmap, process_work);
 	return 0;
 }
 
@@ -102,6 +102,6 @@ int oncpu_lru_map(struct pt_regs *args)
 	work = bpf_map_lookup_elem(&lrumap, &key);
 	if (!work || work->data[0])
 		return 0;
-	bpf_task_work_schedule_resume(task, &work->tw, &lrumap, process_work, NULL);
+	bpf_task_work_schedule_resume(task, &work->tw, &lrumap, process_work);
 	return 0;
 }
tools/testing/selftests/bpf/progs/task_work_fail.c +4 -4
index 77fe8f28facd..82e4b8913333 100644
@@ -53,7 +53,7 @@ int mismatch_map(struct pt_regs *args)
 	work = bpf_map_lookup_elem(&arrmap, &key);
 	if (!work)
 		return 0;
-	bpf_task_work_schedule_resume(task, &work->tw, &hmap, process_work, NULL);
+	bpf_task_work_schedule_resume(task, &work->tw, &hmap, process_work);
 	return 0;
 }
 
@@ -65,7 +65,7 @@ int no_map_task_work(struct pt_regs *args)
 	struct bpf_task_work tw;
 
 	task = bpf_get_current_task_btf();
-	bpf_task_work_schedule_resume(task, &tw, &hmap, process_work, NULL);
+	bpf_task_work_schedule_resume(task, &tw, &hmap, process_work);
 	return 0;
 }
 
@@ -76,7 +76,7 @@ int task_work_null(struct pt_regs *args)
 	struct task_struct *task;
 
 	task = bpf_get_current_task_btf();
-	bpf_task_work_schedule_resume(task, NULL, &hmap, process_work, NULL);
+	bpf_task_work_schedule_resume(task, NULL, &hmap, process_work);
 	return 0;
 }
 
@@ -91,6 +91,6 @@ int map_null(struct pt_regs *args)
 	work = bpf_map_lookup_elem(&arrmap, &key);
 	if (!work)
 		return 0;
-	bpf_task_work_schedule_resume(task, &work->tw, NULL, process_work, NULL);
+	bpf_task_work_schedule_resume(task, &work->tw, NULL, process_work);
 	return 0;
 }
tools/testing/selftests/bpf/progs/task_work_stress.c +1 -1
index 90fca06fff56..1d4378f351ef 100644
@@ -52,7 +52,7 @@ int schedule_task_work(void *ctx)
 			return 0;
 	}
 	err = bpf_task_work_schedule_signal(bpf_get_current_task_btf(), &work->tw, &hmap,
-					    process_work, NULL);
+					    process_work);
 	if (err)
 		__sync_fetch_and_add(&schedule_error, 1);
 	else
Signature · 1 line
2.51.0
Acked-by on v1 6/6
#22 lore raw

Re: [PATCH bpf-next v1 6/6] bpf: mark bpf_task_work_* kfuncs with KF_IMPLICIT_PROG_AUX_ARG

Quoted 8 lines · Ihor Solodrai

Two kfuncs that use aux__prog argument were recently added [1]:
* bpf_task_work_schedule_resume
* bpf_task_work_schedule_signal

Update them to use the new kfunc flag and fix usages in the selftests.

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 7 lines
---
  kernel/bpf/helpers.c                             | 16 ++++++++--------
  tools/testing/selftests/bpf/progs/task_work.c    |  6 +++---
  .../testing/selftests/bpf/progs/task_work_fail.c |  8 ++++----
  .../selftests/bpf/progs/task_work_stress.c       |  2 +-
  4 files changed, 16 insertions(+), 16 deletions(-)