bpf: magic kernel functions
Subsystems (6)
Ihor Solodrai
reviewercover bpf: magic kernel functions
This series develops the idea of implicit prog_aux argument for kfuncs
[1] into a generic "magic kfuncs" feature.
A mechanism is created for kfuncs to have arguments that are not visible to the BPF programs, and are provided to the kernel function implementation by the verifier.
This mechanism is then used in kfuncs that have an argument with __prog annotation [2], which is the current way of passing struct bpf_prog_aux pointer to kfuncs.
==== "Magic" ???
The usage of term "magic" is up for debate of course, I am open to suggestions. I used it as a placeholder first and now it weirdly makes sense. After all, "bpf" by itself doesn't mean anything either.
The feature effectively produces two variants of a kfunc signature/prototype: one with the full argument list, and another with particular arguments omitted.
There are many terms that in other contexts are used to describe similar properties: implicit, optional, virtual, overloaded, polymorphic, interface etc. None of them is quite right for this use case in BPF, and may trigger incorrect intuition if used.
An accurate term could be something like "verifier provided arguments" and "kfuncs with verifier provided arguments", but that's too long for usage in the identifiers. "Magic" on the other hand is a short and ambiguous adjective, which hopefully will prompt people to check the documentation.
==== Implementation
pahole's BTF encoding is changed [3] to detect magic kfuncs and emit two BTF functions from a single kfunc declaration:
* kfunc_impl() with the arguments matching kernel declaration
* kfunc() with __magic arguments omitted
BPF programs then can use both variants of the kfunc (to preserve backwards compatibility for BPF programs that include vmlinux.h), although non-_impl variant would be a preferred API.
To achieve this a few pieces of information must be possible to decode in pahole from kernel side kfunc declaration:
* which kfuncs are magic
* what arguments should be omitted
A simple way to mark magic kfuncs is with a KF_MAGIC_ARGS flag.
As for the arguments, I considered a couple of options:
* look at argument types and hardcode a list of types in pahole
* have multiple KF_MAGIC_ARGS_<N> flags, which would indicate the number of arguments omitted
* use a special arg name suffix (annotation), i.e. "__magic"
Of the three, the last one seems to be the simplest and most flexible.
It is also necessary for pahole to use conventional names for the emitted kfunc pair for the verifier to be able to recognize them.
These changes in BTF create discrepancies in the verifier:
* kfunc() now has an incorrect BTF function prototype
* kfunc_impl() doesn't have a corresponding ksym and BTF flags
In order to handle them correctly, it's necessary to be able to lookup kfunc() <-> kfunc_impl() pairs efficiently. Naive string lookup in BTF is possible, but it is slow, which may negatively impact verification performance for programs using relevant kfuncs.
Since the magic kfuncs are constant within a kernel, and their names in BTF are conventional, we can define a constant table of magic kfuncs using existing BTF_ID_LIST mechanism. A magic_kfuncs BTF ids table is therefore defined and used for efficient lookups.
An inconvenience of the implementation described above is that the writers of a magic kfunc have to do a couple of things:
* mark the kfunc with KF_MAGIC_ARGS
* mark the args with __magic annotation
* add kfunc to magic_kfuncs table
Another one is that for special kfuncs the relevant checks in the verifier must test for both original and _impl funcs. See changes to bpf_wq_set_callback_impl for an example.
==== Testing
A number of selftests are already using aux__prog -> magic kfuncs.
Successful BPF CI run with modified pahole:
https://github.com/kernel-patches/bpf/actions/runs/18918350607
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://docs.kernel.org/bpf/kfuncs.html#prog-annotation
[3] https://github.com/theihor/dwarves/tree/magic-args.draft
Ihor Solodrai (8):
bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
bpf: Refactor btf_kfunc_id_set_contains
bpf: Support for kfuncs with KF_MAGIC_ARGS
bpf: Support __magic prog_aux arguments for kfuncs
bpf: Re-define bpf_wq_set_callback as magic kfunc bpf,docs: Document KF_MAGIC_ARGS flag and __magic annotation
bpf: Re-define bpf_task_work_schedule_* kfuncs as magic
bpf: Re-define bpf_stream_vprintk as a magic kfunc
Documentation/bpf/kfuncs.rst | 49 ++++- include/linux/btf.h | 7 +- include/linux/btf_ids.h | 10 ++ kernel/bpf/btf.c | 70 ++++++-- kernel/bpf/helpers.c | 31 ++-- kernel/bpf/stream.c | 9 +- kernel/bpf/verifier.c | 167 ++++++++++++++++-- tools/lib/bpf/bpf_helpers.h | 7 +- .../testing/selftests/bpf/bpf_experimental.h | 5 - .../testing/selftests/bpf/progs/file_reader.c | 2 +- .../testing/selftests/bpf/progs/stream_fail.c | 6 +- tools/testing/selftests/bpf/progs/task_work.c | 6 +- .../selftests/bpf/progs/task_work_fail.c | 8 +- .../selftests/bpf/progs/task_work_stress.c | 2 +- .../bpf/progs/verifier_async_cb_context.c | 4 +- tools/testing/selftests/bpf/progs/wq.c | 2 +- .../testing/selftests/bpf/progs/wq_failures.c | 4 +- 17 files changed, 312 insertions(+), 77 deletions(-)
Signature · 1 line
2.51.1
Ihor Solodrai
reviewerPATCH 1/8 bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
Implement macros in btf_ids.h to enable a calculation of BTF_ID_LIST size. This is done by declaring an additional __end symbol which can then be used as an indicator of the end of an array.
--- include/linux/btf_ids.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
include/linux/btf_ids.h +10 -0
@@ -97,6 +97,16 @@ asm( \ __BTF_ID_LIST(name, local) \extern u32 name[]; +/* +* The BTF_ID_LIST_END macro may be used to denote an end +* of a BTF_ID_LIST. This enables calculation of the list +* size with BTF_ID_LIST_SIZE. +*/ +#define BTF_ID_LIST_END (name) \ +BTF_ID_LIST(name##__end) +#define BTF_ID_LIST_SIZE (name) \ +(name##__end - name) +#define BTF_ID_LIST_GLOBAL (name, n) \ __BTF_ID_LIST(name, globl)
Signature · 1 line
2.51.1
[email protected]
AI reviewRe: [PATCH bpf-next v1 1/8] bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
``` commit d616f53ce936875dd9f44cc77f9bed1198596463 Author: Ihor Solodrai <[email protected]> bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros This commit adds two new preprocessor macros to btf_ids.h for calculating the size of BTF_ID_LIST arrays using pointer arithmetic between a list and its __end marker. > diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h > index 139bdecec..27a4724d5 100644 > --- a/include/linux/btf_ids.h > +++ b/include/linux/btf_ids.h > @@ -97,6 +97,16 @@ asm( \ > __BTF_ID_LIST(name, local) \ > extern u32 name[]; > > +/* > + * The BTF_ID_LIST_END macro may be used to denote an end > + * of a BTF_ID_LIST. This enables calculation of the list > + * size with BTF_ID_LIST_SIZE. > + */ > +#define BTF_ID_LIST_END(name) \ > +BTF_ID_LIST(name##__end) > +#define BTF_ID_LIST_SIZE(name) \ > +(name##__end - name) > + Are the fallback definitions missing for CONFIG_DEBUG_INFO_BTF=n? Looking at the #else branch starting at line 228, all existing macros like BTF_ID_LIST, BTF_ID, BTF_SET_START, etc. have fallback definitions. These new macros BTF_ID_LIST_END and BTF_ID_LIST_SIZE only exist in the CONFIG_DEBUG_INFO_BTF=y path. When CONFIG_DEBUG_INFO_BTF is disabled, code using these macros will fail to compile. While there's no current usage that would break, the API is incomplete compared to the established pattern in this header where every macro works in both config states. > #define BTF_ID_LIST_GLOBAL(name, n) \ > __BTF_ID_LIST(name, globl) ```
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
In-Reply-To-Subject: bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros CI run summary: https://github.com/kernel-patches/bpf/actions/runs/18919699520
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 1/8] bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
Quoted 19 lines · On 10/29/25 12:41 PM, [email protected] wrote:
```
commitd616f53ce936875dd9f44cc77f9bed1198596463
Author: Ihor Solodrai <[email protected]>bpf: Add
BTF_ID_LIST_ENDandBTF_ID_LIST_SIZEmacrosThis commit adds two new preprocessor macros to
btf_ids.hfor calculating the size ofBTF_ID_LISTarrays using pointer arithmetic between a list and its__endmarker.Quoted 8 lines
include/linux/btf_ids.h +0
-0@@ -97,6 +97,16 @@ asm( \ __BTF_ID_LIST(name, local) \extern u32 name[];
Quoted 12 lines
include/linux/btf_ids.h asm( \ · line 109
Quoted 10 lines
include/linux/btf_ids.h +10
-0+/* +* The BTF_ID_LIST_END macro may be used to denote an end +* of a BTF_ID_LIST. This enables calculation of the list +* size with BTF_ID_LIST_SIZE. +*/ +#define BTF_ID_LIST_END (name) \ +BTF_ID_LIST(name##__end) +#define BTF_ID_LIST_SIZE (name) \ +(name##__end - name) +Are the fallback definitions missing for
CONFIG_DEBUG_INFO_BTF=n?
Indeed. Kernel build fails with CONFIG_DEBUG_INFO_BTF=n
Good bot.
Quoted 23 lines
Looking at the #else branch starting at line 228, all existing macros like
BTF_ID_LIST,BTF_ID,BTF_SET_START, etc. have fallback definitions. These new macrosBTF_ID_LIST_ENDandBTF_ID_LIST_SIZEonly exist in theCONFIG_DEBUG_INFO_BTF=ypath. WhenCONFIG_DEBUG_INFO_BTFis disabled, code using these macros will fail to compile.While there's no current usage that would break, the API is incomplete compared to the established pattern in this header where every macro works in both config states.
include/linux/btf_ids.h asm( \ · line 111
Quoted 2 lines
include/linux/btf_ids.h +0
-0#define BTF_ID_LIST_GLOBAL (name, n) \ __BTF_ID_LIST(name, globl)```
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdIn-Reply-To-Subject:
bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macrosCI run summary: https://github.com/kernel-patches/bpf/actions/runs/18919699520
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 1/8] bpf: Add BTF_ID_LIST_END and BTF_ID_LIST_SIZE macros
In reply to Ihor SolodraiQuoted 17 lines · Ihor Solodrai
Implement macros in
btf_ids.hto enable a calculation ofBTF_ID_LISTsize. This is done by declaring an additional__endsymbol which can then be used as an indicator of the end of an array.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf_ids.h | 10 ++++++++++ 1 file changed, 10 insertions(+)include/linux/btf_ids.h +0
-0@@ -97,6 +97,16 @@ asm( \ __BTF_ID_LIST(name, local) \extern u32 name[];
include/linux/btf_ids.h asm( \ · line 109
Quoted 10 lines
include/linux/btf_ids.h +10
-0+/* +* The BTF_ID_LIST_END macro may be used to denote an end +* of a BTF_ID_LIST. This enables calculation of the list +* size with BTF_ID_LIST_SIZE. +*/ +#define BTF_ID_LIST_END (name) \ +BTF_ID_LIST(name##__end) +#define BTF_ID_LIST_SIZE (name) \ +(name##__end - name) +
Nit: no need for line break?
#define BTF_ID_LIST_SIZE(name) (name##__end - name)
Quoted 3 lines
include/linux/btf_ids.h +0
-0#define BTF_ID_LIST_GLOBAL (name, n) \ __BTF_ID_LIST(name, globl)
Ihor Solodrai
reviewerPATCH 2/8 bpf: Refactor btf_kfunc_id_set_contains
In reply to Ihor Solodraibtf_kfunc_id_set_contains() is called by fetch_kfunc_meta() in the BPF verifier to get the kfunc flags stored in the .BTF_ids ELF section. If it returns NULL instead of a valid pointer, it's interpreted by the verifier as an illegal kfunc usage which fails the verification.
Conceptually, there are two potential reasons forbtf_kfunc_id_set_contains() to return NULL:
1. Provided kfunc BTF id is not present in relevant kfunc id sets.
2. The kfunc is not allowed, as determined by the program type specific filter [1].
The filter functions accept a pointer to struct bpf_prog, so they might implicitly depend on earlier stages of verification, when bpf_prog members are set.
For example, bpf_qdisc_kfunc_filter() in linux/net/sched/bpf_qdisc.c inspects prog->aux->st_ops [2], which is initialized in:
check_attach_btf_id() -> check_struct_ops_btf_id()
So far this hasn't been an issue, because fetch_kfunc_meta() is the only place where lookup + filter logic is applied to a kfunc id.
However in subsequent patches of this series it is necessary to inspect kfunc flags earlier in BPF verifier, in the add_kfunc_call().
To resolve this, refactor btf_kfunc_id_set_contains() into two interface functions: btf_kfunc_flags() that does not apply the filters, and btf_kfunc_flags_if_allowed() that does.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
--- include/linux/btf.h | 6 ++-- kernel/bpf/btf.c | 70 ++++++++++++++++++++++++++++++++----------- kernel/bpf/verifier.c | 2 +- 3 files changed, 58 insertions(+), 20 deletions(-)
include/linux/btf.h +4 -2
@@ -575,8 +575,10 @@ const char *btf_name_by_offset(const struct btf *btf, u32 offset);const char *btf_str_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );struct btf *bpf_prog_get_target_btf (const struct bpf_prog *prog); -u32 *btf_kfunc_id_set_contains (const struct btf *btf,u32 kfunc_btf_id, -const struct bpf_prog *prog); +u32 *btf_kfunc_flags (const struct btf *btf,u32 kfunc_btf_id,const struct bpf_prog *prog); +u32 *btf_kfunc_flags_if_allowed (const struct btf *btf, +u32 kfunc_btf_id, +const struct bpf_prog *prog);u32 *btf_kfunc_is_modify_return (const struct btf *btf,u32 kfunc_btf_id,const struct bpf_prog *prog); int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,
kernel/bpf/btf.c +53 -17
109 lines not on this page · raw message
kernel/bpf/verifier.c +1 -1
@@ -13631,7 +13631,7 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env, *kfunc_name = func_name; func_proto =btf_type_by_id (desc_btf, func->type); - kfunc_flags =btf_kfunc_id_set_contains (desc_btf, func_id, env->prog); + kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, func_id, env->prog);if (!kfunc_flags) {return -EACCES ; }
Signature · 1 line
2.51.1
Re: [PATCH bpf-next v1 2/8] bpf: Refactor btf_kfunc_id_set_contains
Quoted 31 lines · Ihor Solodrai
btf_kfunc_id_set_contains()is called byfetch_kfunc_meta()in the BPF verifier to get the kfunc flags stored in the .BTF_ids ELF section. If it returns NULL instead of a valid pointer, it's interpreted by the verifier as an illegal kfunc usage which fails the verification.Conceptually, there are two potential reasons for
btf_kfunc_id_set_contains()to return NULL:1. Provided kfunc BTF id is not present in relevant kfunc id sets.
2. The kfunc is not allowed, as determined by the program type specific filter [1].The filter functions accept a pointer to
struct bpf_prog, so they might implicitly depend on earlier stages of verification, whenbpf_progmembers are set.For example,
bpf_qdisc_kfunc_filter()inlinux/net/sched/bpf_qdisc.cinspectsprog->aux->st_ops[2], which is initialized in:check_attach_btf_id() -> check_struct_ops_btf_id()So far this hasn't been an issue, because
fetch_kfunc_meta()is the only place where lookup + filter logic is applied to a kfunc id.However in subsequent patches of this series it is necessary to inspect kfunc flags earlier in BPF verifier, in the
add_kfunc_call().To resolve this, refactor
btf_kfunc_id_set_contains()into two interface functions:btf_kfunc_flags()that does not apply the filters, andbtf_kfunc_flags_if_allowed()that does.
Quoted 5 lines
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>---
This preserves original semantics, as far as I can tell, although a more logical split for API functions seem to be:
- btf_kfunc_is_allowed()
- btf_kfunc_flags()
[...]
Ihor Solodrai
reviewerPATCH 3/8 bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Ihor SolodraiA kernel function bpf_foo with KF_MAGIC_ARGS flag is expected to have two types in BTF:
* bpf_foo with a function prototype that omits __magic arguments
* bpf_foo_impl with a function prototype that matches kernel declaration, but doesn't have a ksym associated with its name
In order to support magic kfuncs the verifier has to know how to resolve calls both of bpf_foo and bpf_foo_impl to the correct BTF function prototype and address.
In add_kfunc_call() kfunc flags are inspected to detect a magic kfunc or its _impl, and then the address and func_proto are adjusted for the kfunc descriptor.
In fetch_kfunc_meta() similar logic is used to fixup the contents of struct bpf_kfunc_call_arg_meta.
In check_kfunc_call() reset the subreg_def of registers holding magic arguments to correctly track zero extensions.
--- include/linux/btf.h | 1 + kernel/bpf/verifier.c | 123 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 120 insertions(+), 4 deletions(-)
include/linux/btf.h +1 -0
@@ -79,6 +79,7 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +#define KF_MAGIC_ARGS (1 << 16)/* kfunc signature is different from its BPF signature */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +119 -4
172 lines not on this page · raw message
Signature · 1 line
2.51.1
[email protected]
AI reviewRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
``` commit 974828216758f5fff2d9783439b8daa20188ce8c Author: Ihor Solodrai <[email protected]> bpf: Support for kfuncs with KF_MAGIC_ARGS This patch adds infrastructure for magic kfuncs, which have two BTF representations: one for BPF programs and one for kernel implementation. The verifier resolves these through a lookup table. > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index cb1b483be..fcf0872b9 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -3263,17 +3263,68 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) > return btf_vmlinux ?: ERR_PTR(-ENOENT); > } > > +/* > + * magic_kfuncs is used as a list of (foo, foo_impl) pairs > + */ > +BTF_ID_LIST(magic_kfuncs) > +BTF_ID_UNUSED > +BTF_ID_LIST_END(magic_kfuncs) > + > +static s32 magic_kfunc_by_impl(s32 impl_func_id) > +{ > + int i; > + > + for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) { > + if (magic_kfuncs[i] == impl_func_id) > + return magic_kfuncs[i - 1]; > + } > + return -ENOENT; > +} > + > +static s32 impl_by_magic_kfunc(s32 func_id) > +{ > + int i; > + > + for (i = 0; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) { > + if (magic_kfuncs[i] == func_id) > + return magic_kfuncs[i + 1]; ^^^^^^^^^^^^^^^^^ Can impl_by_magic_kfunc() overflow magic_kfuncs[]? With the current initialization using BTF_ID_UNUSED, BTF_ID_LIST_SIZE(magic_kfuncs) equals 1. The loop condition checks i < 1, so when i=0 it executes and accesses magic_kfuncs[i+1], which is magic_kfuncs[1]. This is outside the array bounds. The same issue applies generally: the loop checks i < size but accesses i+1, so it should check i+1 < size instead. > + } > + return -ENOENT; > +} [ ... ] ```
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
In-Reply-To-Subject: bpf: Support for kfuncs with KF_MAGIC_ARGS CI run summary: https://github.com/kernel-patches/bpf/actions/runs/18919699520
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
Quoted 46 lines · On 10/29/25 12:41 PM, [email protected] wrote:
```
commit974828216758f5fff2d9783439b8daa20188ce8c
Author: Ihor Solodrai <[email protected]>bpf: Support for kfuncs with
KF_MAGIC_ARGSThis patch adds infrastructure for magic kfuncs, which have two BTF
representations: one for BPF programs and one for kernel implementation. The verifier resolves these through a lookup table.Quoted 23 lines
kernel/bpf/verifier.c +15
-0@@ -3263,17 +3263,68 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset)return btf_vmlinux ?:ERR_PTR (-ENOENT ); } +/* +* magic_kfuncs is used as a list of (foo, foo_impl) pairs +*/ +BTF_ID_LIST (magic_kfuncs) +BTF_ID_UNUSED +BTF_ID_LIST_END (magic_kfuncs) + +static s32 magic_kfunc_by_impl(s32 impl_func_id) +{ + int i; + + for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) { +if (magic_kfuncs[i] == impl_func_id) +return magic_kfuncs[i -1 ]; + }kernel/bpf/verifier.c static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) · line 3290
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+return -ENOENT ; +} + +static s32 impl_by_magic_kfunc (s32 func_id) +{ +int i; + +for (i =0 ; i <BTF_ID_LIST_SIZE (magic_kfuncs); i +=2 ) { +if (magic_kfuncs[i] == func_id) +return magic_kfuncs[i +1 ];^^^^^^^^^^^^^^^^^
Quoted 5 lines
Can
impl_by_magic_kfunc()overflow magic_kfuncs[]? With the current initialization usingBTF_ID_UNUSED,BTF_ID_LIST_SIZE(magic_kfuncs)equals 1. The loop condition checks i < 1, so when i=0 it executes and accesses magic_kfuncs[i+1], which is magic_kfuncs[1]. This is outside the array bounds.
Hmm... Given we do i += 2, this can't happen if magic_kfuncs table is defined correctly. Also if BTF_ID_UNUSED is passed in here, we have bigger problems.
I guess changing the loop condition to size-1 wouldn't hurt.
Quoted 19 lines
The same issue applies generally: the loop checks i < size but accesses i+1, so it should check i+1 < size instead.
kernel/bpf/verifier.c static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) · line 3282
Quoted 3 lines
kernel/bpf/verifier.c +3
-0+ } +return -ENOENT ; +}[ ... ]
```
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdIn-Reply-To-Subject:
bpf: Support for kfuncs with KF_MAGIC_ARGSCI run summary: https://github.com/kernel-patches/bpf/actions/runs/18919699520
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
On Wed, 2025-10-29 at 13:49 -0700, Ihor Solodrai wrote:
[...]
Quoted 15 lines
Quoted 9 lines
kernel/bpf/verifier.c static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) · line 3290
Quoted 7 lines
kernel/bpf/verifier.c +7
-0+static s32 impl_by_magic_kfunc (s32 func_id) +{ +int i; + +for (i =0 ; i <BTF_ID_LIST_SIZE (magic_kfuncs); i +=2 ) { +if (magic_kfuncs[i] == func_id) +return magic_kfuncs[i +1 ];^^^^^^^^^^^^^^^^^Quoted 5 lines
Can
impl_by_magic_kfunc()overflow magic_kfuncs[]? With the current initialization usingBTF_ID_UNUSED,BTF_ID_LIST_SIZE(magic_kfuncs)equals 1. The loop condition checks i < 1, so when i=0 it executes and accesses magic_kfuncs[i+1], which is magic_kfuncs[1]. This is outside the array bounds.
Quoted 5 lines
Hmm... Given we do i += 2, this can't happen if
magic_kfuncstable is defined correctly. Also ifBTF_ID_UNUSEDis passed in here, we have bigger problems.I guess changing the loop condition to size-1 wouldn't hurt.
The code is fine and there is no need to bow to the AI overlord. That time will come, but it hasn't come yet.
[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Ihor SolodraiQuoted 1 line · Ihor Solodrai
A kernel function
bpf_foowithKF_MAGIC_ARGSflag is expected to have
^^^^^^^^^^^^^
I don't like this name very much.
It bears very little context.
Imo, KF_IMPLICIT_ARGS fits the use case much better.
Quoted 4 lines
two types in BTF:
*bpf_foowith a function prototype that omits__magicarguments
*bpf_foo_implwith a function prototype that matches kernel declaration, but doesn't have a ksym associated with its name
Could you please start with an example here?
Stating how bpf_foo needs to be declared in kernel, and what are the options to invoke it from bpf. Then proceed with BTF details, etc.
Quoted 43 lines
In order to support magic kfuncs the verifier has to know how to resolve calls both of
bpf_fooandbpf_foo_implto the correct BTF function prototype and address.In
add_kfunc_call()kfunc flags are inspected to detect a magic kfunc or its_impl, and then the address andfunc_protoare adjusted for the kfunc descriptor.In
fetch_kfunc_meta()similar logic is used to fixup the contents of structbpf_kfunc_call_arg_meta.In
check_kfunc_call()reset thesubreg_defof registers holding magic arguments to correctly track zero extensions.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf.h | 1 + kernel/bpf/verifier.c | 123 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 120 insertions(+), 4 deletions(-)include/linux/btf.h +1
-0@@ -79,6 +79,7 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +#define KF_MAGIC_ARGS (1 << 16)/* kfunc signature is different from its BPF signature */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the kernel/bpf/verifier.c +3
-0@@ -3263,17 +3263,68 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset)return btf_vmlinux ?:ERR_PTR (-ENOENT ); } +/* +* magic_kfuncs is used as a list of (foo, foo_impl) pairs +*/
kernel/bpf/verifier.c static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) · line 3278
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+BTF_ID_LIST (magic_kfuncs) +BTF_ID_UNUSED +BTF_ID_LIST_END (magic_kfuncs) + +static s32 magic_kfunc_by_impl(s32 impl_func_id) +{ + int i; + + for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) { +if (magic_kfuncs[i] == impl_func_id)
^^^^^
Nit: similarly, I'd rename this to something like "implicit_func_id"
or "fake_func_id. "impl" is confusing because this id has nothing to do with implementation.
Quoted 55 lines
kernel/bpf/verifier.c +40
-2+return magic_kfuncs[i -1 ]; + } +return -ENOENT ; +} + +static s32 impl_by_magic_kfunc (s32 func_id) +{ +int i; + +for (i =0 ; i <BTF_ID_LIST_SIZE (magic_kfuncs); i +=2 ) { +if (magic_kfuncs[i] == func_id) +return magic_kfuncs[i +1 ]; + } +return -ENOENT ; +} + +static const struct btf_type *find_magic_kfunc_proto (struct btf *desc_btf,s32 func_id) +{ +const struct btf_type *impl_func, *func_proto; +u32 impl_func_id; + + impl_func_id =impl_by_magic_kfunc (func_id); +if (impl_func_id <0 ) +return NULL ; + + impl_func =btf_type_by_id (desc_btf, impl_func_id); +if (!impl_func || !btf_type_is_func (impl_func)) +return NULL ; + + func_proto =btf_type_by_id (desc_btf, impl_func->type); +if (!func_proto || !btf_type_is_func_proto (func_proto)) +return NULL ; + +return func_proto; +} +static int add_kfunc_call (struct bpf_verifier_env *env,u32 func_id,s16 offset) { -const struct btf_type *func, *func_proto; +const struct btf_type *func, *func_proto, *tmp_func;struct bpf_kfunc_btf_tab *btf_tab; +const char *func_name, *tmp_name;struct btf_func_model func_model;struct bpf_kfunc_desc_tab *tab;struct bpf_prog_aux *prog_aux;struct bpf_kfunc_desc *desc; -const char *func_name;struct btf *desc_btf;unsigned long addr; +u32 *kfunc_flags; +s32 tmp_func_id;int err; prog_aux = env->prog->aux;
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3409
Quoted 10 lines
kernel/bpf/verifier.c +5
-0@@ -3349,8 +3400,37 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset)return -EINVAL ; } + kfunc_flags =btf_kfunc_flags (desc_btf, func_id, env->prog); func_name =btf_name_by_offset (desc_btf, func->name_off); addr =kallsyms_lookup_name (func_name); + +/* This may be an _impl kfunc with KF_MAGIC_ARGS counterpart */ +if (unlikely (!addr && !kfunc_flags)) { + tmp_func_id =magic_kfunc_by_impl (func_id);
I think there is no need to hide magic_kfunc_by_impl() call behind the above condition. It can be moved before kfunc_flags assignment. Then it wont be necessary to textually repeat btf_name_by_offset() and kallsyms_lookup_name() calls.
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3411
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+if (tmp_func_id <0 ) +return -EACCES ;
Nit: this skips proper error reporting: "cannot find address for kernel function %s\n".
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3413
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+ tmp_func =btf_type_by_id (desc_btf, tmp_func_id); +if (!tmp_func || !btf_type_is_func (tmp_func))
Nit: this condition indicates a verifier bug, should it be reported as such?
Quoted 44 lines
kernel/bpf/verifier.c +26
-0+return -EACCES ; + tmp_name =btf_name_by_offset (desc_btf, tmp_func->name_off); + addr =kallsyms_lookup_name (tmp_name); + } + +/* +* Note that kfunc_flags may be NULL at this point, which means that we couldn't find +* func_id in any relevant kfunc_id_set. This most likely indicates an invalid kfunc call. +* However we don't want to fail the verification here, because invalid calls may be +* eliminated as dead code later. +*/ +if (unlikely (kfunc_flags &&KF_MAGIC_ARGS & *kfunc_flags)) { + func_proto =find_magic_kfunc_proto (desc_btf, func_id); +if (!func_proto) { +verbose (env,"cannot find _impl proto for kernel function %s\n" , + func_name); +return -EINVAL ; + } + } +if (!addr) {verbose (env,"cannot find address for kernel function %s\n" , func_name);@@ -12051,6 +12131,11 @@ static bool is_kfunc_arg_irq_flag(const struct btf *btf, const struct btf_paramreturn btf_param_match_suffix (btf, arg,"__irq_flag" ); } +static bool is_kfunc_arg_magic (const struct btf *btf,const struct btf_param *arg) +{ +return btf_param_match_suffix (btf, arg,"__magic" ); +} +static bool is_kfunc_arg_prog (const struct btf *btf,const struct btf_param *arg) {return btf_param_match_suffix (btf, arg,"__prog" );@@ -13613,6 +13698,7 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env,u32 func_id, *kfunc_flags;const char *func_name;struct btf *desc_btf; +s32 tmp_func_id;if (kfunc_name) *kfunc_name =NULL ;
kernel/bpf/verifier.c static int fetch_kfunc_meta(struct bpf_verifier_env *env, · line 13721
Quoted 6 lines
kernel/bpf/verifier.c +1
-2@@ -13632,10 +13718,28 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env, func_proto =btf_type_by_id (desc_btf, func->type); kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, func_id, env->prog); -if (!kfunc_flags) { -return -EACCES ; +if (unlikely (!kfunc_flags)) {
What if we patch insn->imm to use the "fake" function id in add_kfunc_call()? Then modifications to fetch_kfunc_meta() wont be necessary.
Quoted 42 lines
kernel/bpf/verifier.c +30
-0+/* +* An _impl kfunc with KF_MAGIC_ARGS counterpart +* does not have its own kfunc flags. +*/ + tmp_func_id =magic_kfunc_by_impl (func_id); +if (tmp_func_id <0 ) +return -EACCES ; + kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, tmp_func_id, env->prog); +if (!kfunc_flags) +return -EACCES ; + }else if (unlikely (KF_MAGIC_ARGS & *kfunc_flags)) { +/* +* An actual func_proto of a kfunc with KF_MAGIC_ARGS flag +* can be found through the corresponding _impl kfunc. +*/ + func_proto =find_magic_kfunc_proto (desc_btf, func_id); } +if (!func_proto) +return -EACCES ; +memset (meta,0 ,sizeof (*meta)); meta->btf = desc_btf; meta->func_id = func_id;@@ -14189,6 +14293,17 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,for (i =0 ; i < nargs; i++) {u32 regno = i +1 ; +/* +* Magic arguments are set after main verification pass. +* For correct tracking of zero-extensions we have to reset subreg_def for such +* args. Otherwise mark_btf_func_reg_size() will be inspecting subreg_def of regs +* from an earlier (irrelevant) point in the program, which may lead to an error +* in opt_subreg_zext_lo32_rnd_hi32(). +*/ +if (unlikely (KF_MAGIC_ARGS & meta.kfunc_flags + &&is_kfunc_arg_magic (desc_btf, &args[i]))) + regs[regno].subreg_def =DEF_NOT_SUBREG ; + t =btf_type_skip_modifiers (desc_btf, args[i].type,NULL );if (btf_type_is_ptr (t))mark_btf_func_reg_size (env, regno,sizeof (void *));
Re: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
Quoted 7 lines · Eduard Zingerman
Quoted 1 line · Ihor Solodrai
A kernel function
bpf_foowithKF_MAGIC_ARGSflag is expected to have^^^^^^^^^^^^^
I don't like this name very much.
It bears very little context.
Imo,KF_IMPLICIT_ARGSfits the use case much better.
+1
I hate the name as well.KF_IMPLICIT_ARGS sounds much better and can be abbreviated to "__impl" in arg names.
That way it will accidentally match our existing _impl() kfuncs though there _impl meant "implementation".
I think this double meaning of "impl" as "implementation specific" and "implicit" actually fits.
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Eduard ZingermanHi Eduard, thank you for a quick review.
Quoted 6 lines · Eduard Zingerman
Quoted 1 line · Ihor Solodrai
A kernel function
bpf_foowithKF_MAGIC_ARGSflag is expected to have^^^^^^^^^^^^^
I don't like this name very much.
It bears very little context.
Imo,KF_IMPLICIT_ARGSfits the use case much better.
I know, naming is hard...
The issue is that it's not only the flag, across the code we need descriptive names for every "magic" thing:
* a flagged function
* how do we call it? kfunc_with_impl_args?
* a function that exists only in BTF (_impl)
* it's not an "implicit" function
* it's not exactly an "implementation" function
* "fake" is even worse than "magic" IMO, because it's not fake, but you could argue it's magical :D
* btf_only_kfunc?
* describing arguments is simpler: "implicit" seems ok, although as Alexei pointed out in previous iteration they are very much explicit in the kernel [1]
For me, "(BPF) interface" and "(kernel) implementation" pair of terms makes sense, but then I think it would be logical to have both declarations in the kernel.
The advantage of "magic" in this context is that it doesn't have loaded meaning. But I agree this is a stretch, so can't insist.
[1] https://lore.kernel.org/bpf/CAADnVQLvuubey0A0Fk=bzN-=JG2UUQHRqBijZpuvqMQ+xy4W4g@mail.gmail.com/
Quoted 9 lines
Quoted 5 lines
two types in BTF:
*bpf_foowith a function prototype that omits__magicarguments
*bpf_foo_implwith a function prototype that matches kernel declaration, but doesn't have a ksym associated with its nameCould you please start with an example here?
Stating howbpf_fooneeds to be declared in kernel, and what are the options to invoke it from bpf. Then proceed with BTF details, etc.
Ok. I think I can reshuffle explanations between the cover letter and commit message, it's a bit redundant already.
Quoted 117 lines
Quoted 44 lines
In order to support magic kfuncs the verifier has to know how to resolve calls both of
bpf_fooandbpf_foo_implto the correct BTF function prototype and address.In
add_kfunc_call()kfunc flags are inspected to detect a magic kfunc or its_impl, and then the address andfunc_protoare adjusted for the kfunc descriptor.In
fetch_kfunc_meta()similar logic is used to fixup the contents of structbpf_kfunc_call_arg_meta.In
check_kfunc_call()reset thesubreg_defof registers holding magic arguments to correctly track zero extensions.Signed-off-by Ihor Solodrai <[email protected]>--- include/linux/btf.h | 1 + kernel/bpf/verifier.c | 123 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 120 insertions(+), 4 deletions(-)include/linux/btf.h +1
-0@@ -79,6 +79,7 @@#define KF_ARENA_RET (1 << 13)/* kfunc returns an arena pointer */ #define KF_ARENA_ARG1 (1 << 14)/* kfunc takes an arena pointer as its first argument */ #define KF_ARENA_ARG2 (1 << 15)/* kfunc takes an arena pointer as its second argument */ +#define KF_MAGIC_ARGS (1 << 16)/* kfunc signature is different from its BPF signature */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the kernel/bpf/verifier.c +3
-0@@ -3263,17 +3263,68 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset)return btf_vmlinux ?:ERR_PTR (-ENOENT ); } +/* +* magic_kfuncs is used as a list of (foo, foo_impl) pairs +*/ kernel/bpf/verifier.c static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) · line 3278
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+BTF_ID_LIST (magic_kfuncs) +BTF_ID_UNUSED +BTF_ID_LIST_END (magic_kfuncs) + +static s32 magic_kfunc_by_impl(s32 impl_func_id) +{ + int i; + + for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) { +if (magic_kfuncs[i] == impl_func_id)^^^^^Nit: similarly, I'd rename this to something like "
implicit_func_id"
or "fake_func_id. "impl" is confusing because this id has nothing to do with implementation.Quoted 58 lines
kernel/bpf/verifier.c +40
-2+return magic_kfuncs[i -1 ]; + } +return -ENOENT ; +} + +static s32 impl_by_magic_kfunc (s32 func_id) +{ +int i; + +for (i =0 ; i <BTF_ID_LIST_SIZE (magic_kfuncs); i +=2 ) { +if (magic_kfuncs[i] == func_id) +return magic_kfuncs[i +1 ]; + } +return -ENOENT ; +} + +static const struct btf_type *find_magic_kfunc_proto (struct btf *desc_btf,s32 func_id) +{ +const struct btf_type *impl_func, *func_proto; +u32 impl_func_id; + + impl_func_id =impl_by_magic_kfunc (func_id); +if (impl_func_id <0 ) +return NULL ; + + impl_func =btf_type_by_id (desc_btf, impl_func_id); +if (!impl_func || !btf_type_is_func (impl_func)) +return NULL ; + + func_proto =btf_type_by_id (desc_btf, impl_func->type); +if (!func_proto || !btf_type_is_func_proto (func_proto)) +return NULL ; + +return func_proto; +} +static int add_kfunc_call (struct bpf_verifier_env *env,u32 func_id,s16 offset) { -const struct btf_type *func, *func_proto; +const struct btf_type *func, *func_proto, *tmp_func;struct bpf_kfunc_btf_tab *btf_tab; +const char *func_name, *tmp_name;struct btf_func_model func_model;struct bpf_kfunc_desc_tab *tab;struct bpf_prog_aux *prog_aux;struct bpf_kfunc_desc *desc; -const char *func_name;struct btf *desc_btf;unsigned long addr; +u32 *kfunc_flags; +s32 tmp_func_id;int err; prog_aux = env->prog->aux;@@ -3349,8 +3400,37 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset)return -EINVAL ; }
Quoted 12 lines
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3409
Quoted 7 lines
kernel/bpf/verifier.c +5
-0+ kfunc_flags =btf_kfunc_flags (desc_btf, func_id, env->prog); func_name =btf_name_by_offset (desc_btf, func->name_off); addr =kallsyms_lookup_name (func_name); + +/* This may be an _impl kfunc with KF_MAGIC_ARGS counterpart */ +if (unlikely (!addr && !kfunc_flags)) { + tmp_func_id =magic_kfunc_by_impl (func_id);I think there is no need to hide
magic_kfunc_by_impl()call behind the above condition. It can be moved beforekfunc_flagsassignment. Then it wont be necessary to textually repeatbtf_name_by_offset()andkallsyms_lookup_name()calls.
Not sure I follow...
Yes, !addr is enough to detect potential _impl function, but there is no way around name lookup in BTF and then another address lookup.
The _impl function doesn't have an address, so after failed
kallsyms_lookup_name("kfunc_impl");
we must do
kallsyms_lookup_name("kfunc");
to find the correct address.
Or do you suggest doing something like:
tmp_func_id = magic_kfunc_by_impl(func_id);
if (tmp_func_id > 0)
func_id = tmp_func_id;at the beginning of add_kfunc_call()?
Quoted 57 lines
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3411
Quoted 3 lines
kernel/bpf/verifier.c +2
-0+if (tmp_func_id <0 ) +return -EACCES ;Nit: this skips proper error reporting: "cannot find address for kernel function %s\n".
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3413
Quoted 2 lines
kernel/bpf/verifier.c +2
-0+ tmp_func =btf_type_by_id (desc_btf, tmp_func_id); +if (!tmp_func || !btf_type_is_func (tmp_func))Nit: this condition indicates a verifier bug, should it be reported as such?
Quoted 46 lines
kernel/bpf/verifier.c +26
-0+return -EACCES ; + tmp_name =btf_name_by_offset (desc_btf, tmp_func->name_off); + addr =kallsyms_lookup_name (tmp_name); + } + +/* +* Note that kfunc_flags may be NULL at this point, which means that we couldn't find +* func_id in any relevant kfunc_id_set. This most likely indicates an invalid kfunc call. +* However we don't want to fail the verification here, because invalid calls may be +* eliminated as dead code later. +*/ +if (unlikely (kfunc_flags &&KF_MAGIC_ARGS & *kfunc_flags)) { + func_proto =find_magic_kfunc_proto (desc_btf, func_id); +if (!func_proto) { +verbose (env,"cannot find _impl proto for kernel function %s\n" , + func_name); +return -EINVAL ; + } + } +if (!addr) {verbose (env,"cannot find address for kernel function %s\n" , func_name);@@ -12051,6 +12131,11 @@ static bool is_kfunc_arg_irq_flag(const struct btf *btf, const struct btf_paramreturn btf_param_match_suffix (btf, arg,"__irq_flag" ); } +static bool is_kfunc_arg_magic (const struct btf *btf,const struct btf_param *arg) +{ +return btf_param_match_suffix (btf, arg,"__magic" ); +} +static bool is_kfunc_arg_prog (const struct btf *btf,const struct btf_param *arg) {return btf_param_match_suffix (btf, arg,"__prog" );@@ -13613,6 +13698,7 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env,u32 func_id, *kfunc_flags;const char *func_name;struct btf *desc_btf; +s32 tmp_func_id;if (kfunc_name) *kfunc_name =NULL ;@@ -13632,10 +13718,28 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env, func_proto =btf_type_by_id (desc_btf, func->type);
Quoted 7 lines
kernel/bpf/verifier.c static int fetch_kfunc_meta(struct bpf_verifier_env *env, · line 13721
Quoted 4 lines
kernel/bpf/verifier.c +1
-2kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, func_id, env->prog); -if (!kfunc_flags) { -return -EACCES ; +if (unlikely (!kfunc_flags)) {What if we patch
insn->immto use the "fake" function id inadd_kfunc_call()? Then modifications tofetch_kfunc_meta()wont be necessary.
I considered this. I wasn't sure it's safe to patch insn->imm at this stage of verification. Also I thought it may be harder to debug the verifier if we do btf id replacement in the calls pre-verification (because we lose the original btf id).
Maybe I was too causious.
Alexei, Andrii, what do you think?
Quoted 43 lines
Quoted 43 lines
kernel/bpf/verifier.c +30
-0+/* +* An _impl kfunc with KF_MAGIC_ARGS counterpart +* does not have its own kfunc flags. +*/ + tmp_func_id =magic_kfunc_by_impl (func_id); +if (tmp_func_id <0 ) +return -EACCES ; + kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, tmp_func_id, env->prog); +if (!kfunc_flags) +return -EACCES ; + }else if (unlikely (KF_MAGIC_ARGS & *kfunc_flags)) { +/* +* An actual func_proto of a kfunc with KF_MAGIC_ARGS flag +* can be found through the corresponding _impl kfunc. +*/ + func_proto =find_magic_kfunc_proto (desc_btf, func_id); } +if (!func_proto) +return -EACCES ; +memset (meta,0 ,sizeof (*meta)); meta->btf = desc_btf; meta->func_id = func_id;@@ -14189,6 +14293,17 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,for (i =0 ; i < nargs; i++) {u32 regno = i +1 ; +/* +* Magic arguments are set after main verification pass. +* For correct tracking of zero-extensions we have to reset subreg_def for such +* args. Otherwise mark_btf_func_reg_size() will be inspecting subreg_def of regs +* from an earlier (irrelevant) point in the program, which may lead to an error +* in opt_subreg_zext_lo32_rnd_hi32(). +*/ +if (unlikely (KF_MAGIC_ARGS & meta.kfunc_flags + &&is_kfunc_arg_magic (desc_btf, &args[i]))) + regs[regno].subreg_def =DEF_NOT_SUBREG ; + t =btf_type_skip_modifiers (desc_btf, args[i].type,NULL );if (btf_type_is_ptr (t))mark_btf_func_reg_size (env, regno,sizeof (void *));
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
Quoted 26 lines · Ihor Solodrai
Hi Eduard, thank you for a quick review.
Quoted 6 lines · Eduard Zingerman
Quoted 1 line · Ihor Solodrai
A kernel function
bpf_foowithKF_MAGIC_ARGSflag is expected to have^^^^^^^^^^^^^
I don't like this name very much.
It bears very little context.
Imo,KF_IMPLICIT_ARGSfits the use case much better.I know, naming is hard...
The issue is that it's not only the flag, across the code we need descriptive names for every "magic" thing:
* a flagged function
* how do we call it?kfunc_with_impl_args?
* a function that exists only in BTF (_impl)
* it's not an "implicit" function
* it's not exactly an "implementation" function
* "fake" is even worse than "magic" IMO, because it's not fake, but you could argue it's magical :D
*btf_only_kfunc?
* describing arguments is simpler: "implicit" seems ok, although as Alexei pointed out in previous iteration they are very much explicit in the kernel [1]
Quoted 8 lines
For me, "(BPF) interface" and "(kernel) implementation" pair of terms makes sense, but then I think it would be logical to have both declarations in the kernel.
The advantage of "magic" in this context is that it doesn't have loaded meaning. But I agree this is a stretch, so can't insist.
[1] https://lore.kernel.org/bpf/CAADnVQLvuubey0A0Fk=bzN-=JG2UUQHRqBijZpuvqMQ+xy4W4g@mail.gmail.com/
- KF_IMPLICIT_ARGS
- explicit_args_id -- for prototype with full set of args
- implicit_args_id -- for prototype with missing args
[...]
Quoted 28 lines
Quoted 4 lines
Quoted 4 lines
kernel/bpf/verifier.c +0
-0@@ -3349,8 +3400,37 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset)return -EINVAL ; }Quoted 12 lines
kernel/bpf/verifier.c static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) · line 3409
Quoted 7 lines
kernel/bpf/verifier.c +5
-0+ kfunc_flags =btf_kfunc_flags (desc_btf, func_id, env->prog); func_name =btf_name_by_offset (desc_btf, func->name_off); addr =kallsyms_lookup_name (func_name); + +/* This may be an _impl kfunc with KF_MAGIC_ARGS counterpart */ +if (unlikely (!addr && !kfunc_flags)) { + tmp_func_id =magic_kfunc_by_impl (func_id);I think there is no need to hide
magic_kfunc_by_impl()call behind the above condition. It can be moved beforekfunc_flagsassignment. Then it wont be necessary to textually repeatbtf_name_by_offset()andkallsyms_lookup_name()calls.Not sure I follow...
Yes, !addr is enough to detect potential
_implfunction, but there is no way around name lookup in BTF and then another address lookup.The
_implfunction doesn't have an address, so after failed
kallsyms_lookup_name("kfunc_impl");
we must do
kallsyms_lookup_name("kfunc");
to find the correct address.
Quoted 7 lines
Or do you suggest doing something like:
tmp_func_id = magic_kfunc_by_impl(func_id); if (tmp_func_id > 0) func_id = tmp_func_id;at the beginning of
add_kfunc_call()?
This, just check for magic_kfunc_by_impl() and replace func_id.
[...]
Quoted 12 lines
Quoted 10 lines
kernel/bpf/verifier.c static int fetch_kfunc_meta(struct bpf_verifier_env *env, · line 13721
Quoted 7 lines
kernel/bpf/verifier.c +1
-2@@ -13632,10 +13718,28 @@ static int fetch_kfunc_meta(struct bpf_verifier_env *env, func_proto =btf_type_by_id (desc_btf, func->type); kfunc_flags =btf_kfunc_flags_if_allowed (desc_btf, func_id, env->prog); -if (!kfunc_flags) { -return -EACCES ; +if (unlikely (!kfunc_flags)) {What if we patch
insn->immto use the "fake" function id inadd_kfunc_call()? Then modifications tofetch_kfunc_meta()wont be necessary.
Quoted 4 lines
I considered this. I wasn't sure it's safe to patch
insn->immat this stage of verification. Also I thought it may be harder to debug the verifier if we do btf id replacement in the calls pre-verification (because we lose the original btf id).
See no issues with that.
Quoted 3 lines
Maybe I was too causious.
Alexei, Andrii, what do you think?
[...]
kernel test robot
build botRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Ihor Solodraikernel test robot noticed the following build errors:
Hi Ihor,
kernel test robot noticed the following build errors:
[auto build test ERROR on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/bpf-Add-BTF_ID_LIST_END-and-BTF_ID_LIST_SIZE-macros/20251030-030608 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20251029190113.3323406-4-ihor.solodrai%40linux.dev patch subject: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS config: x86_64-buildonly-randconfig-003-20251030 (https://download.01.org/0day-ci/archive/20251030/[email protected]/config) compiler: gcc-13 (Debian 13.3.0-16) 13.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251030/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
| ^
include/linux/compiler.h:166:29: note: in expansion of macro '__PASTE'
166 | #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
| ^~~~~~~
include/linux/compiler_types.h:84:22: note: in expansion of macro '___PASTE'
84 | #define __PASTE(a,b) ___PASTE(a,b)
| ^~~~~~~~
include/linux/compiler.h:166:37: note: in expansion of macro '__PASTE'
166 | #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__)
| ^~~~~~~
include/linux/compiler.h:286:9: note: in expansion of macro '__UNIQUE_ID'
286 | __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
| ^~~~~~~~~~~
include/linux/compiler.h:289:9: note: in expansion of macro '___ADDRESSABLE'
289 | ___ADDRESSABLE(sym, __section(".discard.addressable"))
| ^~~~~~~~~~~~~~
include/linux/init.h:250:9: note: in expansion of macro '__ADDRESSABLE'
250 | __ADDRESSABLE(fn)
| ^~~~~~~~~~~~~
include/linux/init.h:255:9: note: in expansion of macro '__define_initcall_stub'
255 | __define_initcall_stub(__stub, fn) \
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/init.h:268:9: note: in expansion of macro '____define_initcall'
268 | ____define_initcall(fn, \
| ^~~~~~~~~~~~~~~~~~~
include/linux/init.h:274:9: note: in expansion of macro '__unique_initcall'
274 | __unique_initcall(fn, id, __sec, __initcall_id(fn))
| ^~~~~~~~~~~~~~~~~
include/linux/init.h:276:35: note: in expansion of macro '___define_initcall'
276 | #define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
| ^~~~~~~~~~~~~~~~~~
include/linux/init.h:307:41: note: in expansion of macro '__define_initcall'
307 | #define late_initcall(fn) __define_initcall(fn, 7)
| ^~~~~~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:1: note: in expansion of macro 'late_initcall'
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~
In file included from include/uapi/linux/filter.h:9,
from include/linux/bpf.h:8:
kernel/bpf/verifier.c:18983:15: error: 'unbound_reg_init' undeclared (first use in this function); did you mean 'unbound_reg'?
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~~~~
include/linux/compiler.h:286:72: note: in definition of macro '___ADDRESSABLE'
286 | __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
| ^~~
include/linux/init.h:250:9: note: in expansion of macro '__ADDRESSABLE'
250 | __ADDRESSABLE(fn)
| ^~~~~~~~~~~~~
include/linux/init.h:255:9: note: in expansion of macro '__define_initcall_stub'
255 | __define_initcall_stub(__stub, fn) \
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/init.h:268:9: note: in expansion of macro '____define_initcall'
268 | ____define_initcall(fn, \
| ^~~~~~~~~~~~~~~~~~~
include/linux/init.h:274:9: note: in expansion of macro '__unique_initcall'
274 | __unique_initcall(fn, id, __sec, __initcall_id(fn))
| ^~~~~~~~~~~~~~~~~
include/linux/init.h:276:35: note: in expansion of macro '___define_initcall'
276 | #define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
| ^~~~~~~~~~~~~~~~~~
include/linux/init.h:307:41: note: in expansion of macro '__define_initcall'
307 | #define late_initcall(fn) __define_initcall(fn, 7)
| ^~~~~~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:1: note: in expansion of macro 'late_initcall'
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:15: note: each undeclared identifier is reported only once for each function it appears in
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~~~~
include/linux/compiler.h:286:72: note: in definition of macro '___ADDRESSABLE'
286 | __UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
| ^~~
include/linux/init.h:250:9: note: in expansion of macro '__ADDRESSABLE'
250 | __ADDRESSABLE(fn)
| ^~~~~~~~~~~~~
include/linux/init.h:255:9: note: in expansion of macro '__define_initcall_stub'
255 | __define_initcall_stub(__stub, fn) \
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/init.h:268:9: note: in expansion of macro '____define_initcall'
268 | ____define_initcall(fn, \
| ^~~~~~~~~~~~~~~~~~~
include/linux/init.h:274:9: note: in expansion of macro '__unique_initcall'
274 | __unique_initcall(fn, id, __sec, __initcall_id(fn))
| ^~~~~~~~~~~~~~~~~
include/linux/init.h:276:35: note: in expansion of macro '___define_initcall'
276 | #define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
| ^~~~~~~~~~~~~~~~~~
include/linux/init.h:307:41: note: in expansion of macro '__define_initcall'
307 | #define late_initcall(fn) __define_initcall(fn, 7)
| ^~~~~~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:1: note: in expansion of macro 'late_initcall'
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~
In file included from include/linux/printk.h:6,
from include/asm-generic/bug.h:22,
from arch/x86/include/asm/bug.h:108,
from include/linux/bug.h:5,
from include/linux/alloc_tag.h:8,
from include/linux/workqueue.h:9,
from include/linux/bpf.h:11:
>> include/linux/init.h:256:9: error: expected declaration specifiers before 'asm'
256 | asm(".section \"" __sec "\", \"a\" \n" \
| ^~~
include/linux/init.h:268:9: note: in expansion of macro '____define_initcall'
268 | ____define_initcall(fn, \
| ^~~~~~~~~~~~~~~~~~~
include/linux/init.h:274:9: note: in expansion of macro '__unique_initcall'
274 | __unique_initcall(fn, id, __sec, __initcall_id(fn))
| ^~~~~~~~~~~~~~~~~
include/linux/init.h:276:35: note: in expansion of macro '___define_initcall'
276 | #define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
| ^~~~~~~~~~~~~~~~~~
include/linux/init.h:307:41: note: in expansion of macro '__define_initcall'
307 | #define late_initcall(fn) __define_initcall(fn, 7)
| ^~~~~~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:1: note: in expansion of macro 'late_initcall'
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~
In file included from include/linux/init.h:5:
include/linux/build_bug.h:78:41: error: expected declaration specifiers before '_Static_assert'
78 | #define __static_assert(expr, msg, ...) _Static_assert(expr, msg)
| ^~~~~~~~~~~~~~
include/linux/build_bug.h:77:34: note: in expansion of macro '__static_assert'
77 | #define static_assert(expr, ...) __static_assert(expr, ##__VA_ARGS__, #expr)
| ^~~~~~~~~~~~~~~
include/linux/init.h:260:9: note: in expansion of macro 'static_assert'
260 | static_assert(__same_type(initcall_t, &fn));
| ^~~~~~~~~~~~~
include/linux/init.h:268:9: note: in expansion of macro '____define_initcall'
268 | ____define_initcall(fn, \
| ^~~~~~~~~~~~~~~~~~~
include/linux/init.h:274:9: note: in expansion of macro '__unique_initcall'
274 | __unique_initcall(fn, id, __sec, __initcall_id(fn))
| ^~~~~~~~~~~~~~~~~
include/linux/init.h:276:35: note: in expansion of macro '___define_initcall'
276 | #define __define_initcall(fn, id) ___define_initcall(fn, id, .initcall##id)
| ^~~~~~~~~~~~~~~~~~
include/linux/init.h:307:41: note: in expansion of macro '__define_initcall'
307 | #define late_initcall(fn) __define_initcall(fn, 7)
| ^~~~~~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:1: note: in expansion of macro 'late_initcall'
18983 | late_initcall(unbound_reg_init);
| ^~~~~~~~~~~~~
kernel/bpf/verifier.c:18983:32: error: expected declaration specifiers before ';' token
18983 | late_initcall(unbound_reg_init);
| ^
kernel/bpf/verifier.c:18987:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
18987 | {
| ^
kernel/bpf/verifier.c:19002:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19002 | {
| ^
kernel/bpf/verifier.c:19015:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19015 | {
| ^
kernel/bpf/verifier.c:19139:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19139 | {
| ^
kernel/bpf/verifier.c:19212:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19212 | {
| ^
kernel/bpf/verifier.c:19232:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19232 | {
| ^
kernel/bpf/verifier.c:19241:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19241 | {
| ^
kernel/bpf/verifier.c:19282:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19282 | {
| ^
kernel/bpf/verifier.c:19343:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19343 | {
| ^
kernel/bpf/verifier.c:19373:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19373 | {
| ^
kernel/bpf/verifier.c:19390:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19390 | {
| ^
kernel/bpf/verifier.c:19453:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19453 | {
| ^
kernel/bpf/verifier.c:19477:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19477 | {
| ^
kernel/bpf/verifier.c:19833:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19833 | {
| ^
kernel/bpf/verifier.c:19861:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19861 | {
| ^
kernel/bpf/verifier.c:19867:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19867 | {
| ^
kernel/bpf/verifier.c:19878:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19878 | {
| ^
kernel/bpf/verifier.c:19884:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
19884 | {
| ^ kernel/bpf/verifier.c:19930:1: warning: empty declaration
vim +/asm +256 include/linux/init.h
a8cccdd954732a5 Sami Tolvanen 2020-12-11 252
1b1eeca7e4c19fa Ard Biesheuvel 2018-08-21 253 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
3578ad11f3fba07 Sami Tolvanen 2020-12-11 254 #define ____define_initcall(fn, __stub, __name, __sec) \
3578ad11f3fba07 Sami Tolvanen 2020-12-11 255 __define_initcall_stub(__stub, fn) \
a8cccdd954732a5 Sami Tolvanen 2020-12-11 @256 asm(".section \"" __sec "\", \"a\" \n" \
a8cccdd954732a5 Sami Tolvanen 2020-12-11 257 __stringify(__name) ": \n" \
3578ad11f3fba07 Sami Tolvanen 2020-12-11 258 ".long " __stringify(__stub) " - . \n" \
1cb61759d407166 Marco Elver 2021-05-21 259 ".previous \n"); \
1cb61759d407166 Marco Elver 2021-05-21 260 static_assert(__same_type(initcall_t, &fn));
1b1eeca7e4c19fa Ard Biesheuvel 2018-08-21 261 #else
3578ad11f3fba07 Sami Tolvanen 2020-12-11 262 #define ____define_initcall(fn, __unused, __name, __sec) \
a8cccdd954732a5 Sami Tolvanen 2020-12-11 263 static initcall_t __name __used \
a8cccdd954732a5 Sami Tolvanen 2020-12-11 264 __attribute__((__section__(__sec))) = fn;
1b1eeca7e4c19fa Ard Biesheuvel 2018-08-21 265 #endif1b1eeca7e4c19fa Ard Biesheuvel 2018-08-21 266
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Ihor Solodraikernel test robot noticed the following build warnings:
Hi Ihor,
kernel test robot noticed the following build warnings:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/bpf-Add-BTF_ID_LIST_END-and-BTF_ID_LIST_SIZE-macros/20251030-030608 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20251029190113.3323406-4-ihor.solodrai%40linux.dev patch subject: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS config: s390-randconfig-001-20251030 (https://download.01.org/0day-ci/archive/20251030/[email protected]/config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251030/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
kernel/bpf/verifier.c:3273:1: error: invalid storage class specifier in function declarator
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
kernel/bpf/verifier.c:3273:12: error: parameter named 'magic_kfunc_by_impl' is missing
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
kernel/bpf/verifier.c:3273:49: error: expected ';' at end of declaration
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
| ;
kernel/bpf/verifier.c:3271:17: error: parameter 'magic_kfuncs' was not declared, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
3272 |
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
3274 | {
kernel/bpf/verifier.c:3271:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
| int
kernel/bpf/verifier.c:3277:18: error: call to undeclared function 'BTF_ID_LIST_SIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3277 | for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ^
kernel/bpf/verifier.c:3277:18: note: did you mean 'BTF_ID_LIST_END'?
kernel/bpf/verifier.c:3271:1: note: 'BTF_ID_LIST_END' declared here
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
3272 |
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
3274 | {
3275 | int i;
3276 |
3277 | for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ~~~~~~~~~~~~~~~~
| BTF_ID_LIST_END
kernel/bpf/verifier.c:3278:19: error: subscripted value is not an array, pointer, or vector
3278 | if (magic_kfuncs[i] == impl_func_id)
| ~~~~~~~~~~~~^~
kernel/bpf/verifier.c:3278:26: error: use of undeclared identifier 'impl_func_id'
3278 | if (magic_kfuncs[i] == impl_func_id)
| ^
kernel/bpf/verifier.c:3279:23: error: subscripted value is not an array, pointer, or vector
3279 | return magic_kfuncs[i - 1];
| ~~~~~~~~~~~~^~~~~~
>> kernel/bpf/verifier.c:3271:1: warning: no previous prototype for function 'BTF_ID_LIST_END' [-Wmissing-prototypes]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
kernel/bpf/verifier.c:3271:16: note: declare 'static' if the function is not intended to be used outside of this translation unit
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
| static
kernel/bpf/verifier.c:3271:1: error: a function definition without a prototype is deprecated in all versions of C and is not supported in C2x [-Werror,-Wdeprecated-non-prototype]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
kernel/bpf/verifier.c:3288:18: error: call to undeclared function 'BTF_ID_LIST_SIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3288 | for (i = 0; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ^
kernel/bpf/verifier.c:3409:17: error: call to undeclared function 'magic_kfunc_by_impl'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3409 | tmp_func_id = magic_kfunc_by_impl(func_id);
| ^
kernel/bpf/verifier.c:13726:17: error: call to undeclared function 'magic_kfunc_by_impl'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
13726 | tmp_func_id = magic_kfunc_by_impl(func_id);
| ^1 warning and 13 errors generated.
vim +/BTF_ID_LIST_END +3271 kernel/bpf/verifier.c
3265
3266 /*
3267 * magic_kfuncs is used as a list of (foo, foo_impl) pairs
3268 */
3269 BTF_ID_LIST(magic_kfuncs)
3270 BTF_ID_UNUSED
> 3271 BTF_ID_LIST_END(magic_kfuncs)
3272
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS
In reply to Ihor Solodraikernel test robot noticed the following build errors:
Hi Ihor,
kernel test robot noticed the following build errors:
[auto build test ERROR on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/bpf-Add-BTF_ID_LIST_END-and-BTF_ID_LIST_SIZE-macros/20251030-030608 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20251029190113.3323406-4-ihor.solodrai%40linux.dev patch subject: [PATCH bpf-next v1 3/8] bpf: Support for kfuncs with KF_MAGIC_ARGS config: s390-randconfig-001-20251030 (https://download.01.org/0day-ci/archive/20251030/[email protected]/config) compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251030/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
kernel/bpf/verifier.c:3273:1: error: invalid storage class specifier in function declarator
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
kernel/bpf/verifier.c:3273:12: error: parameter named 'magic_kfunc_by_impl' is missing
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
kernel/bpf/verifier.c:3273:49: error: expected ';' at end of declaration
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
| ^
| ;
kernel/bpf/verifier.c:3271:17: error: parameter 'magic_kfuncs' was not declared, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
3272 |
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
3274 | {
kernel/bpf/verifier.c:3271:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
| int
kernel/bpf/verifier.c:3277:18: error: call to undeclared function 'BTF_ID_LIST_SIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3277 | for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ^
kernel/bpf/verifier.c:3277:18: note: did you mean 'BTF_ID_LIST_END'?
kernel/bpf/verifier.c:3271:1: note: 'BTF_ID_LIST_END' declared here
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
3272 |
3273 | static s32 magic_kfunc_by_impl(s32 impl_func_id)
3274 | {
3275 | int i;
3276 |
3277 | for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ~~~~~~~~~~~~~~~~
| BTF_ID_LIST_END
kernel/bpf/verifier.c:3278:19: error: subscripted value is not an array, pointer, or vector
3278 | if (magic_kfuncs[i] == impl_func_id)
| ~~~~~~~~~~~~^~
kernel/bpf/verifier.c:3278:26: error: use of undeclared identifier 'impl_func_id'
3278 | if (magic_kfuncs[i] == impl_func_id)
| ^
kernel/bpf/verifier.c:3279:23: error: subscripted value is not an array, pointer, or vector
3279 | return magic_kfuncs[i - 1];
| ~~~~~~~~~~~~^~~~~~
kernel/bpf/verifier.c:3271:1: warning: no previous prototype for function 'BTF_ID_LIST_END' [-Wmissing-prototypes]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
kernel/bpf/verifier.c:3271:16: note: declare 'static' if the function is not intended to be used outside of this translation unit
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
| static
>> kernel/bpf/verifier.c:3271:1: error: a function definition without a prototype is deprecated in all versions of C and is not supported in C2x [-Werror,-Wdeprecated-non-prototype]
3271 | BTF_ID_LIST_END(magic_kfuncs)
| ^
kernel/bpf/verifier.c:3288:18: error: call to undeclared function 'BTF_ID_LIST_SIZE'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3288 | for (i = 0; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
| ^
>> kernel/bpf/verifier.c:3409:17: error: call to undeclared function 'magic_kfunc_by_impl'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
3409 | tmp_func_id = magic_kfunc_by_impl(func_id);
| ^
kernel/bpf/verifier.c:13726:17: error: call to undeclared function 'magic_kfunc_by_impl'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
13726 | tmp_func_id = magic_kfunc_by_impl(func_id);
| ^1 warning and 13 errors generated.
vim +3271 kernel/bpf/verifier.c
3265
3266 /*
3267 * magic_kfuncs is used as a list of (foo, foo_impl) pairs
3268 */
3269 BTF_ID_LIST(magic_kfuncs)
3270 BTF_ID_UNUSED
> 3271 BTF_ID_LIST_END(magic_kfuncs)
3272
3273 static s32 magic_kfunc_by_impl(s32 impl_func_id)
3274 {
3275 int i;
3276
3277 for (i = 1; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
3278 if (magic_kfuncs[i] == impl_func_id)
3279 return magic_kfuncs[i - 1];
3280 }
3281 return -ENOENT;
3282 }
3283
3284 static s32 impl_by_magic_kfunc(s32 func_id)
3285 {
3286 int i;
3287
3288 for (i = 0; i < BTF_ID_LIST_SIZE(magic_kfuncs); i += 2) {
3289 if (magic_kfuncs[i] == func_id)
3290 return magic_kfuncs[i + 1];
3291 }
3292 return -ENOENT;
3293 }
3294
3295 static const struct btf_type *find_magic_kfunc_proto(struct btf *desc_btf, s32 func_id)
3296 {
3297 const struct btf_type *impl_func, *func_proto;
3298 u32 impl_func_id;
3299
3300 impl_func_id = impl_by_magic_kfunc(func_id);
3301 if (impl_func_id < 0)
3302 return NULL;
3303
3304 impl_func = btf_type_by_id(desc_btf, impl_func_id);
3305 if (!impl_func || !btf_type_is_func(impl_func))
3306 return NULL;
3307
3308 func_proto = btf_type_by_id(desc_btf, impl_func->type);
3309 if (!func_proto || !btf_type_is_func_proto(func_proto))
3310 return NULL;
3311
3312 return func_proto;
3313 }
3314
3315 static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset)
3316 {
3317 const struct btf_type *func, *func_proto, *tmp_func;
3318 struct bpf_kfunc_btf_tab *btf_tab;
3319 const char *func_name, *tmp_name;
3320 struct btf_func_model func_model;
3321 struct bpf_kfunc_desc_tab *tab;
3322 struct bpf_prog_aux *prog_aux;
3323 struct bpf_kfunc_desc *desc;
3324 struct btf *desc_btf;
3325 unsigned long addr;
3326 u32 *kfunc_flags;
3327 s32 tmp_func_id;
3328 int err;
3329
3330 prog_aux = env->prog->aux;
3331 tab = prog_aux->kfunc_tab;
3332 btf_tab = prog_aux->kfunc_btf_tab;
3333 if (!tab) {
3334 if (!btf_vmlinux) {
3335 verbose(env, "calling kernel function is not supported without CONFIG_DEBUG_INFO_BTF\n");
3336 return -ENOTSUPP;
3337 }
3338
3339 if (!env->prog->jit_requested) {
3340 verbose(env, "JIT is required for calling kernel function\n");
3341 return -ENOTSUPP;
3342 }
3343
3344 if (!bpf_jit_supports_kfunc_call()) {
3345 verbose(env, "JIT does not support calling kernel function\n");
3346 return -ENOTSUPP;
3347 }
3348
3349 if (!env->prog->gpl_compatible) {
3350 verbose(env, "cannot call kernel function from non-GPL compatible program\n");
3351 return -EINVAL;
3352 }
3353
3354 tab = kzalloc(sizeof(*tab), GFP_KERNEL_ACCOUNT);
3355 if (!tab)
3356 return -ENOMEM;
3357 prog_aux->kfunc_tab = tab;
3358 }
3359
3360 /* func_id == 0 is always invalid, but instead of returning an error, be
3361 * conservative and wait until the code elimination pass before returning
3362 * error, so that invalid calls that get pruned out can be in BPF programs
3363 * loaded from userspace. It is also required that offset be untouched
3364 * for such calls.
3365 */
3366 if (!func_id && !offset)
3367 return 0;
3368
3369 if (!btf_tab && offset) {
3370 btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL_ACCOUNT);
3371 if (!btf_tab)
3372 return -ENOMEM;
3373 prog_aux->kfunc_btf_tab = btf_tab;
3374 }
3375
3376 desc_btf = find_kfunc_desc_btf(env, offset);
3377 if (IS_ERR(desc_btf)) {
3378 verbose(env, "failed to find BTF for kernel function\n");
3379 return PTR_ERR(desc_btf);
3380 }
3381
3382 if (find_kfunc_desc(env->prog, func_id, offset))
3383 return 0;
3384
3385 if (tab->nr_descs == MAX_KFUNC_DESCS) {
3386 verbose(env, "too many different kernel function calls\n");
3387 return -E2BIG;
3388 }
3389
3390 func = btf_type_by_id(desc_btf, func_id);
3391 if (!func || !btf_type_is_func(func)) {
3392 verbose(env, "kernel btf_id %u is not a function\n",
3393 func_id);
3394 return -EINVAL;
3395 }
3396 func_proto = btf_type_by_id(desc_btf, func->type);
3397 if (!func_proto || !btf_type_is_func_proto(func_proto)) {
3398 verbose(env, "kernel function btf_id %u does not have a valid func_proto\n",
3399 func_id);
3400 return -EINVAL;
3401 }
3402
3403 kfunc_flags = btf_kfunc_flags(desc_btf, func_id, env->prog);
3404 func_name = btf_name_by_offset(desc_btf, func->name_off);
3405 addr = kallsyms_lookup_name(func_name);
3406
3407 /* This may be an _impl kfunc with KF_MAGIC_ARGS counterpart */
3408 if (unlikely(!addr && !kfunc_flags)) {
> 3409 tmp_func_id = magic_kfunc_by_impl(func_id); 3410 if (tmp_func_id < 0)
3411 return -EACCES;
3412 tmp_func = btf_type_by_id(desc_btf, tmp_func_id);
3413 if (!tmp_func || !btf_type_is_func(tmp_func))
3414 return -EACCES;
3415 tmp_name = btf_name_by_offset(desc_btf, tmp_func->name_off);
3416 addr = kallsyms_lookup_name(tmp_name);
3417 }
3418
3419 /*
3420 * Note that kfunc_flags may be NULL at this point, which means that we couldn't find
3421 * func_id in any relevant kfunc_id_set. This most likely indicates an invalid kfunc call.
3422 * However we don't want to fail the verification here, because invalid calls may be
3423 * eliminated as dead code later.
3424 */
3425 if (unlikely(kfunc_flags && KF_MAGIC_ARGS & *kfunc_flags)) {
3426 func_proto = find_magic_kfunc_proto(desc_btf, func_id);
3427 if (!func_proto) {
3428 verbose(env, "cannot find _impl proto for kernel function %s\n",
3429 func_name);
3430 return -EINVAL;
3431 }
3432 }
3433
3434 if (!addr) {
3435 verbose(env, "cannot find address for kernel function %s\n",
3436 func_name);
3437 return -EINVAL;
3438 }
3439
3440 if (bpf_dev_bound_kfunc_id(func_id)) {
3441 err = bpf_dev_bound_kfunc_check(&env->log, prog_aux);
3442 if (err)
3443 return err;
3444 }
3445
3446 err = btf_distill_func_proto(&env->log, desc_btf,
3447 func_proto, func_name,
3448 &func_model);
3449 if (err)
3450 return err;
3451
3452 desc = &tab->descs[tab->nr_descs++];
3453 desc->func_id = func_id;
3454 desc->offset = offset;
3455 desc->addr = addr;
3456 desc->func_model = func_model;
3457 sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]),
3458 kfunc_desc_cmp_by_id_off, NULL);
3459 return 0;
3460 }
3461
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Ihor Solodrai
reviewerPATCH 4/8 bpf: Support __magic prog_aux arguments for kfuncs
In reply to Ihor SolodraiTeach the verifier that the prog_aux argument of a kfunc can be specified with __magic suffix, in which case the type of the function parameter must be checked.
--- kernel/bpf/verifier.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +11 -1
@@ -12136,9 +12136,12 @@ static bool is_kfunc_arg_magic(const struct btf *btf, const struct btf_param *arreturn btf_param_match_suffix (btf, arg,"__magic" ); } +static bool is_kfunc_arg_prog_aux (const struct btf *btf,const struct btf_param *arg); +static bool is_kfunc_arg_prog (const struct btf *btf,const struct btf_param *arg) { -return btf_param_match_suffix (btf, arg,"__prog" ); +return btf_param_match_suffix (btf, arg,"__prog" ) || + (is_kfunc_arg_magic (btf, arg) &&is_kfunc_arg_prog_aux (btf, arg)); } static bool is_kfunc_arg_scalar_with_name(const struct btf *btf,
@@ -12169,6 +12172,7 @@ enum {KF_ARG_WORKQUEUE_ID ,KF_ARG_RES_SPIN_LOCK_ID ,KF_ARG_TASK_WORK_ID , + KF_ARG_PROG_AUX_ID };BTF_ID_LIST (kf_arg_btf_ids)
@@ -12180,6 +12184,7 @@ BTF_ID(struct, bpf_rb_node) BTF_ID(struct, bpf_wq) BTF_ID(struct, bpf_res_spin_lock) BTF_ID(struct, bpf_task_work) +BTF_ID(struct, bpf_prog_aux)static bool __is_kfunc_ptr_arg_type (const struct btf *btf,const struct btf_param *arg,int type)
@@ -12260,6 +12265,11 @@ static bool is_kfunc_arg_callback(struct bpf_verifier_env *env, const struct btfreturn true; } +static bool is_kfunc_arg_prog_aux (const struct btf *btf,const struct btf_param *arg) +{ +return __is_kfunc_ptr_arg_type (btf, arg,KF_ARG_PROG_AUX_ID ); +} +/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, const struct btf *btf,
Signature · 1 line
2.51.1
Ihor Solodrai
reviewerPATCH 5/8 bpf: Re-define bpf_wq_set_callback as magic kfunc
In reply to Ihor Solodrai* Rename bpf_wq_set_callback_impl to bpf_wq_set_callback
* void *aux__prog => struct bpf_prog_aux *aux__magic
* Set KF_MAGIC_ARGS kfunc flag
* Add bpf_wq_set_callback and _impl to magic_kfuncs BTF_ID_LIST
* Update special kfunc checks in the verifier to accept both _impl and non-_impl BTF ids
In the selftests, a bpf_wq_set_callback_impl() call is intentionally introduced to verify that both signatures are handled correctly.
--- kernel/bpf/helpers.c | 13 ++++++------- kernel/bpf/verifier.c | 18 +++++++++++------- tools/testing/selftests/bpf/bpf_experimental.h | 5 ----- tools/testing/selftests/bpf/progs/wq.c | 2 +- .../testing/selftests/bpf/progs/wq_failures.c | 4 ++-- 5 files changed, 20 insertions(+), 22 deletions(-)
kernel/bpf/helpers.c +6 -7
@@ -3119,18 +3119,17 @@ __bpf_kfunc int bpf_wq_start(struct bpf_wq *wq, unsigned int flags)return 0 ; } -__bpf_kfunc int bpf_wq_set_callback_impl(struct bpf_wq *wq, -int (callback_fn )(void *map,int *key,void *value), -unsigned int flags, -void *aux__prog) +__bpf_kfunc int bpf_wq_set_callback(struct bpf_wq *wq, +int (callback_fn )(void *map,int *key,void *value), +unsigned int flags, +struct bpf_prog_aux *aux__magic) { -struct bpf_prog_aux *aux = (struct bpf_prog_aux *)aux__prog;struct bpf_async_kern *async = (struct bpf_async_kern *)wq;if (flags)return -EINVAL ; -return __bpf_async_set_callback (async, callback_fn, aux, flags,BPF_ASYNC_TYPE_WQ ); +return __bpf_async_set_callback (async, callback_fn, aux__magic, flags,BPF_ASYNC_TYPE_WQ ); } __bpf_kfunc void bpf_preempt_disable(void)
@@ -4483,7 +4482,7 @@ BTF_ID_FLAGS(func, bpf_dynptr_memset) BTF_ID_FLAGS(func, bpf_modify_return_test_tp)#endif BTF_ID_FLAGS(func, bpf_wq_init) -BTF_ID_FLAGS(func, bpf_wq_set_callback_impl) +BTF_ID_FLAGS(func, bpf_wq_set_callback, KF_MAGIC_ARGS) BTF_ID_FLAGS(func, bpf_wq_start) BTF_ID_FLAGS(func, bpf_preempt_disable)BTF_ID_FLAGS (func, bpf_preempt_enable)
kernel/bpf/verifier.c +11 -7
@@ -512,7 +512,7 @@ static bool is_async_callback_calling_kfunc(u32 btf_id);static bool is_callback_calling_kfunc (u32 btf_id);static bool is_bpf_throw_kfunc (struct bpf_insn *insn); -static bool is_bpf_wq_set_callback_impl_kfunc (u32 btf_id); +static bool is_bpf_wq_set_callback_kfunc (u32 btf_id);static bool is_task_work_add_kfunc (u32 func_id);static bool is_sync_callback_calling_function (enum bpf_func_id func_id)
@@ -554,7 +554,7 @@ static bool is_async_cb_sleepable(struct bpf_verifier_env *env, struct bpf_insn/* bpf_wq and bpf_task_work callbacks are always sleepable. */ if (bpf_pseudo_kfunc_call (insn) && insn->off ==0 && - (is_bpf_wq_set_callback_impl_kfunc (insn->imm) ||is_task_work_add_kfunc (insn->imm))) + (is_bpf_wq_set_callback_kfunc (insn->imm) ||is_task_work_add_kfunc (insn->imm)))return true;verifier_bug (env,"unhandled async callback in is_async_cb_sleepable" );
@@ -3267,7 +3267,8 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset)* magic_kfuncs is used as a list of (foo, foo_impl) pairs */ BTF_ID_LIST(magic_kfuncs) -BTF_ID_UNUSED +BTF_ID(func, bpf_wq_set_callback) +BTF_ID(func, bpf_wq_set_callback_impl) BTF_ID_LIST_END(magic_kfuncs) static s32 magic_kfunc_by_impl(s32 impl_func_id)
@@ -12385,6 +12386,7 @@ enum special_kfunc_type { KF___bpf_trap, KF_bpf_task_work_schedule_signal, KF_bpf_task_work_schedule_resume, + KF_bpf_wq_set_callback, };BTF_ID_LIST (special_kfunc_list)
@@ -12459,6 +12461,7 @@ BTF_ID(func, bpf_dynptr_file_discard) BTF_ID(func, __bpf_trap) BTF_ID(func, bpf_task_work_schedule_signal) BTF_ID(func, bpf_task_work_schedule_resume) +BTF_ID(func, bpf_wq_set_callback)static bool is_task_work_add_kfunc (u32 func_id) {
@@ -12906,7 +12909,7 @@ static bool is_sync_callback_calling_kfunc(u32 btf_id)static bool is_async_callback_calling_kfunc (u32 btf_id) { -return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] || +return is_bpf_wq_set_callback_kfunc (btf_id) ||is_task_work_add_kfunc (btf_id); }
@@ -12916,9 +12919,10 @@ static bool is_bpf_throw_kfunc(struct bpf_insn *insn) insn->imm == special_kfunc_list[KF_bpf_throw]; } -static bool is_bpf_wq_set_callback_impl_kfunc (u32 btf_id) +static bool is_bpf_wq_set_callback_kfunc (u32 btf_id) { -return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl]; +return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] || + btf_id == special_kfunc_list[KF_bpf_wq_set_callback]; }static bool is_callback_calling_kfunc (u32 btf_id)
@@ -14035,7 +14039,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, meta.r0_rdonly = false; } -if (is_bpf_wq_set_callback_impl_kfunc (meta.func_id)) { +if (is_bpf_wq_set_callback_kfunc (meta.func_id)) { err =push_callback_call (env, insn, insn_idx, meta.subprogno, set_timer_callback_state);if (err) {
tools/testing/selftests/bpf/bpf_experimental.h +0 -5
@@ -580,11 +580,6 @@ extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; extern int bpf_wq_init(struct bpf_wq *wq, void *p__map, unsigned int flags) __weak __ksym; extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; -extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, - int (callback_fn)(void *map, int *key, void *value), - unsigned int flags__k, void *aux__ign) __ksym; -#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL)struct bpf_iter_kmem_cache ; extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym;
tools/testing/selftests/bpf/progs/wq.c +1 -1
@@ -107,7 +107,7 @@ static int test_hmap_elem_callback(void *map, int *key,if (bpf_wq_init (wq, map,0 ) !=0 )return -3 ; -if (bpf_wq_set_callback (wq, callback_fn,0 )) +if (bpf_wq_set_callback_impl (wq, callback_fn,0 ,NULL ))return -4 ;if (bpf_wq_start (wq,0 ))
tools/testing/selftests/bpf/progs/wq_failures.c +2 -2
@@ -97,7 +97,7 @@ __failure/* check that the first argument of bpf_wq_set_callback() * is a correct bpf_wq pointer. */ -__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */ +__msg(": (85) call bpf_wq_set_callback#") /* anchor message */ __msg("arg#0 doesn't point to a map value")long test_wrong_wq_pointer (void *ctx) {
@@ -123,7 +123,7 @@ __failure/* check that the first argument of bpf_wq_set_callback() * is a correct bpf_wq pointer. */ -__msg(": (85) call bpf_wq_set_callback_impl#") /* anchor message */ +__msg(": (85) call bpf_wq_set_callback#") /* anchor message */ __msg("off 1 doesn't point to 'struct bpf_wq' that is at 0")long test_wrong_wq_pointer_offset (void *ctx) {
Signature · 1 line
2.51.1
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 5/8] bpf: Re-define bpf_wq_set_callback as magic kfunc
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
[...]
Quoted 4 lines
kernel/bpf/verifier.c +0
-0
[...]
kernel/bpf/verifier.c static bool is_bpf_throw_kfunc(struct bpf_insn *insn) · line 12925
Quoted 10 lines
kernel/bpf/verifier.c +3
-2@@ -12916,9 +12919,10 @@ static bool is_bpf_throw_kfunc(struct bpf_insn *insn) insn->imm == special_kfunc_list[KF_bpf_throw]; } -static bool is_bpf_wq_set_callback_impl_kfunc (u32 btf_id) +static bool is_bpf_wq_set_callback_kfunc (u32 btf_id) { -return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl]; +return btf_id == special_kfunc_list[KF_bpf_wq_set_callback_impl] || + btf_id == special_kfunc_list[KF_bpf_wq_set_callback];
If insn->imm is patched in add_kfunc_call() there will be no need to handle two cases here.
tools/testing/selftests/bpf/bpf_experimental.h
Quoted 16 lines
kernel/bpf/verifier.c +1
-1}static bool is_callback_calling_kfunc (u32 btf_id)@@ -14035,7 +14039,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, meta.r0_rdonly = false; } -if (is_bpf_wq_set_callback_impl_kfunc (meta.func_id)) { +if (is_bpf_wq_set_callback_kfunc (meta.func_id)) { err =push_callback_call (env, insn, insn_idx, meta.subprogno, set_timer_callback_state);if (err) {tools/testing/selftests/bpf/bpf_experimental.h +0
-0
Nit: The tests won't fail with the above changes, right?
If so, we usually ship changes to selftests in separate patches.
Quoted 9 lines
tools/testing/selftests/bpf/bpf_experimental.h +0
-5@@ -580,11 +580,6 @@ extern void bpf_iter_css_destroy(struct bpf_iter_css *it) __weak __ksym; extern int bpf_wq_init(struct bpf_wq *wq, void *p__map, unsigned int flags) __weak __ksym; extern int bpf_wq_start(struct bpf_wq *wq, unsigned int flags) __weak __ksym; -extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, - int (callback_fn)(void *map, int *key, void *value), - unsigned int flags__k, void *aux__ign) __ksym; -#define bpf_wq_set_callback (timer, cb, flags) \ - bpf_wq_set_callback_impl(timer, cb, flags, NULL)
...
Ihor Solodrai
reviewerPATCH 6/8 bpf,docs: Document KF_MAGIC_ARGS flag and __magic annotation
In reply to Ihor SolodraiAdd sections explaining KF_MAGIC_ARGS kfunc flag and __magic argument annotation. Mark __prog annotation as deprecated.
--- Documentation/bpf/kfuncs.rst | 49 +++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-)
Documentation/bpf/kfuncs.rst +48 -1
@@ -160,7 +160,7 @@ Or:: ... } -2.2.6 __prog Annotation +2.2.6 __prog Annotation (deprecated, use __magic instead) --------------------------- This annotation is used to indicate that the argument needs to be fixed up to the bpf_prog_aux of the caller BPF program. Any value passed into this argument
@@ -177,6 +177,37 @@ An example is given below:: ... } +.. _magic_annotation: + +2.2.7 __magic Annotation +--------------------------- +This annotation is used in kfuncs with KF_MAGIC_ARGS flag to indicate the +arguments that are omitted in the BPF signature of the kfunc. The actual +values for __magic arguments are set by the verifier at load time, and +depend on the argument type. + +Currently only ``struct bpf_prog_aux *`` type is supported. + +Example declaration: + +.. code-block:: c + + __bpf_kfunc int bpf_wq_set_callback(struct bpf_wq *wq, + int (callback_fn)(void *map, int *key, void *value), + unsigned int flags, + struct bpf_prog_aux *aux__magic) + { + ... + } + +Example usage: + +.. code-block:: c + + /* note the last argument is omitted */ + if (bpf_wq_set_callback(wq, callback_fn, 0)) + return -1; + .. _BPF_kfunc_nodef: 2.3 Using an existing kernel function
@@ -374,6 +405,22 @@ encouraged to make their use-cases known as early as possible, and participate in upstream discussions regarding whether to keep, change, deprecate, or remove those kfuncs if and when such discussions occur. +2.4.10 KF_MAGIC_ARGS flag +------------------------------------ + +The KF_MAGIC_ARGS flag is used to indicate that the BPF signature of the kfunc +is different from it's kernel signature, and the values for arguments annotated +with __magic suffix are provided at load time by the verifier. + +A kfunc with KF_MAGIC_ARGS flag therefore has two types in BTF: one function +matching the kernel declaration (with _impl suffix by convention), and another +matching the intended BPF API. + +Verifier allows calls to both _impl and non-_impl versions of a magic kfunc. + +Note that only arguments of particular types can be __magic. +See :ref:`magic_annotation`. + 2.5 Registering the kfuncs --------------------------
Signature · 1 line
2.51.1
Eduard Zingerman
maintainerRe: [PATCH bpf-next v1 6/8] bpf,docs: Document KF_MAGIC_ARGS flag and __magic annotation
Quoted 13 lines · Ihor Solodrai
Add sections explaining
KF_MAGIC_ARGSkfunc flag and__magicargument annotation. Mark__progannotation as deprecated.Signed-off-by Ihor Solodrai <[email protected]>--- Documentation/bpf/kfuncs.rst | 49 +++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-)Documentation/bpf/kfuncs.rst +0
-0
Documentation/bpf/kfuncs.rst Or:: · line 164
Quoted 6 lines
Documentation/bpf/kfuncs.rst +1
-1@@ -160,7 +160,7 @@ Or:: } -2.2.6 __prog Annotation +2.2.6 __prog Annotation (deprecated, use __magic instead) ---------------------------
I don't see any kfuncs that use __prog parameter suffix after your changes, this section can be dropped altogether.
Quoted 2 lines
This annotation is used to indicate that the argument needs to be fixed up to the
bpf_prog_auxof the caller BPF program. Any value passed into this argument
[...]
Ihor Solodrai
reviewerPATCH 7/8 bpf: Re-define bpf_task_work_schedule_* kfuncs as magic
In reply to Ihor Solodrai* void *aux__prog => struct bpf_prog_aux *aux__magic
* Set KF_MAGIC_ARGS flag
* Add relevant symbols to magic_kfuncs list
* Update selftests to use the new signature
--- kernel/bpf/helpers.c | 16 ++++++++-------- kernel/bpf/verifier.c | 12 +++++++++++- tools/testing/selftests/bpf/progs/file_reader.c | 2 +- tools/testing/selftests/bpf/progs/task_work.c | 6 +++--- .../testing/selftests/bpf/progs/task_work_fail.c | 8 ++++---- .../selftests/bpf/progs/task_work_stress.c | 2 +- .../bpf/progs/verifier_async_cb_context.c | 4 ++-- 7 files changed, 30 insertions(+), 20 deletions(-)
kernel/bpf/helpers.c +8 -8
@@ -4278,15 +4278,15 @@ static int bpf_task_work_schedule(struct task_struct *task, struct bpf_task_work* @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping * @map__map: bpf_map that embeds struct bpf_task_work in the values * @callback: pointer to BPF subprogram to call -* @aux__prog: user should pass NULL +* @aux__magic: pointer to bpf_prog_aux of the caller BPF program, set by the verifier * * Return: 0 if task work has been scheduled successfully, negative error code otherwise */ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct bpf_task_work *tw,void *map__map,bpf_task_work_callback_t callback, -void *aux__prog) +struct bpf_prog_aux *aux__magic) { -return bpf_task_work_schedule (task, tw, map__map, callback, aux__prog,TWA_SIGNAL ); +return bpf_task_work_schedule (task, tw, map__map, callback, aux__magic,TWA_SIGNAL ); }/**
@@ -4295,15 +4295,15 @@ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct b* @tw: Pointer to struct bpf_task_work in BPF map value for internal bookkeeping * @map__map: bpf_map that embeds struct bpf_task_work in the values * @callback: pointer to BPF subprogram to call -* @aux__prog: user should pass NULL +* @aux__magic: pointer to bpf_prog_aux of the caller BPF program, set by the verifier * * Return: 0 if task work has been scheduled successfully, negative error code otherwise */ __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw,void *map__map,bpf_task_work_callback_t callback, -void *aux__prog) +struct bpf_prog_aux *aux__magic) { -return bpf_task_work_schedule (task, tw, map__map, callback, aux__prog,TWA_RESUME ); +return bpf_task_work_schedule (task, tw, map__map, callback, aux__magic,TWA_RESUME ); } static int make_file_dynptr(struct file *file, u32 flags, bool may_sleep,
@@ -4529,8 +4529,8 @@ BTF_ID_FLAGS(func, bpf_strncasestr); BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS) -BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_TRUSTED_ARGS) -BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_MAGIC_ARGS | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_MAGIC_ARGS | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_dynptr_from_file, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_dynptr_file_discard)BTF_KFUNCS_END (common_btf_ids)
kernel/bpf/verifier.c +11 -1
@@ -3269,6 +3269,10 @@ static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, s16 offset) BTF_ID_LIST(magic_kfuncs) BTF_ID(func, bpf_wq_set_callback) BTF_ID(func, bpf_wq_set_callback_impl) +BTF_ID(func, bpf_task_work_schedule_signal) +BTF_ID(func, bpf_task_work_schedule_signal_impl) +BTF_ID(func, bpf_task_work_schedule_resume) +BTF_ID(func, bpf_task_work_schedule_resume_impl) BTF_ID_LIST_END(magic_kfuncs) static s32 magic_kfunc_by_impl(s32 impl_func_id)
@@ -12387,6 +12391,8 @@ enum special_kfunc_type { KF_bpf_task_work_schedule_signal, KF_bpf_task_work_schedule_resume, KF_bpf_wq_set_callback, + KF_bpf_task_work_schedule_signal_impl, + KF_bpf_task_work_schedule_resume_impl, };BTF_ID_LIST (special_kfunc_list)
@@ -12462,11 +12468,15 @@ BTF_ID(func, __bpf_trap) BTF_ID(func, bpf_task_work_schedule_signal) BTF_ID(func, bpf_task_work_schedule_resume) BTF_ID(func, bpf_wq_set_callback) +BTF_ID(func, bpf_task_work_schedule_signal_impl) +BTF_ID(func, bpf_task_work_schedule_resume_impl)static bool is_task_work_add_kfunc (u32 func_id) {return func_id == special_kfunc_list[KF_bpf_task_work_schedule_signal] || - func_id == special_kfunc_list[KF_bpf_task_work_schedule_resume]; + func_id == special_kfunc_list[KF_bpf_task_work_schedule_signal_impl] || + func_id == special_kfunc_list[KF_bpf_task_work_schedule_resume] || + func_id == special_kfunc_list[KF_bpf_task_work_schedule_resume_impl]; }static bool is_kfunc_ret_null (struct bpf_kfunc_call_arg_meta *meta)
tools/testing/selftests/bpf/progs/file_reader.c +1 -1
@@ -77,7 +77,7 @@ int on_open_validate_file_read(void *c) err =1 ;return 0 ; } -bpf_task_work_schedule_signal (task, &work->tw, &arrmap, task_work_callback,NULL ); +bpf_task_work_schedule_signal (task, &work->tw, &arrmap, task_work_callback);return 0 ; }
tools/testing/selftests/bpf/progs/task_work.c +3 -3
@@ -66,7 +66,7 @@ int oncpu_hash_map(struct pt_regs *args)if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work);return 0 ; }
@@ -80,7 +80,7 @@ int oncpu_array_map(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_signal (task, &work->tw, &arrmap, process_work,NULL ); +bpf_task_work_schedule_signal (task, &work->tw, &arrmap, process_work);return 0 ; }
@@ -102,6 +102,6 @@ int oncpu_lru_map(struct pt_regs *args) work =bpf_map_lookup_elem (&lrumap, &key);if (!work || work->data[0 ])return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &lrumap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &lrumap, process_work);return 0 ; }
tools/testing/selftests/bpf/progs/task_work_fail.c +4 -4
@@ -53,7 +53,7 @@ int mismatch_map(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw, &hmap, process_work);return 0 ; }
@@ -65,7 +65,7 @@ int no_map_task_work(struct pt_regs *args)struct bpf_task_work tw; task =bpf_get_current_task_btf (); -bpf_task_work_schedule_resume (task, &tw, &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task, &tw, &hmap, process_work);return 0 ; }
@@ -76,7 +76,7 @@ int task_work_null(struct pt_regs *args)struct task_struct *task; task =bpf_get_current_task_btf (); -bpf_task_work_schedule_resume (task,NULL , &hmap, process_work,NULL ); +bpf_task_work_schedule_resume (task,NULL , &hmap, process_work);return 0 ; }
@@ -91,6 +91,6 @@ int map_null(struct pt_regs *args) work =bpf_map_lookup_elem (&arrmap, &key);if (!work)return 0 ; -bpf_task_work_schedule_resume (task, &work->tw,NULL , process_work,NULL ); +bpf_task_work_schedule_resume (task, &work->tw,NULL , process_work);return 0 ; }
tools/testing/selftests/bpf/progs/task_work_stress.c +1 -1
@@ -52,7 +52,7 @@ int schedule_task_work(void *ctx)return 0 ; } err =bpf_task_work_schedule_signal (bpf_get_current_task_btf (), &work->tw, &hmap, - process_work,NULL ); + process_work);if (err)__sync_fetch_and_add (&schedule_error,1 );else
tools/testing/selftests/bpf/progs/verifier_async_cb_context.c +2 -2
@@ -156,7 +156,7 @@ int task_work_non_sleepable_prog(void *ctx)if (!task)return 0 ; -bpf_task_work_schedule_resume (task, &val->tw, &task_work_map, task_work_cb,NULL ); +bpf_task_work_schedule_resume (task, &val->tw, &task_work_map, task_work_cb);return 0 ; }
@@ -176,6 +176,6 @@ int task_work_sleepable_prog(void *ctx)if (!task)return 0 ; -bpf_task_work_schedule_resume (task, &val->tw, &task_work_map, task_work_cb,NULL ); +bpf_task_work_schedule_resume (task, &val->tw, &task_work_map, task_work_cb);return 0 ; }
Signature · 1 line
2.51.1
Ihor Solodrai
reviewerPATCH 8/8 bpf: Re-define bpf_stream_vprintk as a magic kfunc
In reply to Ihor Solodrai* void *aux__prog => struct bpf_prog_aux *aux__magic
* Set KF_MAGIC_ARGS flag
* Add relevant symbols to magic_kfuncs list
* Update selftests to use the new signature
bpf_stream_vprintk macro is changed to use bpf_stream_vprintk_impl, and the extern definition of bpf_stream_vprintk is replaced with _impl version in bpf_helpers.h
This should help with backwards compatibility, as the API of bpf_stream_vprintk macro hasn't changed.
--- kernel/bpf/helpers.c | 2 +- kernel/bpf/stream.c | 9 ++++++--- kernel/bpf/verifier.c | 2 ++ tools/lib/bpf/bpf_helpers.h | 7 ++++--- tools/testing/selftests/bpf/progs/stream_fail.c | 6 +++--- 5 files changed, 16 insertions(+), 10 deletions(-)
kernel/bpf/helpers.c +1 -1
@@ -4528,7 +4528,7 @@ BTF_ID_FLAGS(func, bpf_strncasestr);#if defined(CONFIG_BPF_LSM ) && defined(CONFIG_CGROUPS ) BTF_ID_FLAGS(func, bpf_cgroup_read_xattr, KF_RCU)#endif -BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_stream_vprintk, KF_MAGIC_ARGS | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_work_schedule_signal, KF_MAGIC_ARGS | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_work_schedule_resume, KF_MAGIC_ARGS | KF_TRUSTED_ARGS)BTF_ID_FLAGS (func, bpf_dynptr_from_file,KF_TRUSTED_ARGS )
kernel/bpf/stream.c +6 -3
@@ -355,19 +355,22 @@ __bpf_kfunc_start_defs();* Avoid using enum bpf_stream_id so that kfunc users don't have to pull in the * enum in headers. */ -__bpf_kfunc int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, u32 len__sz, void *aux__prog) +__bpf_kfunc int bpf_stream_vprintk(int stream_id, +const char *fmt__str, +const void *args, +u32 len__sz, +struct bpf_prog_aux *aux__magic) {struct bpf_bprintf_data data = { .get_bin_args = true, .get_buf = true, }; -struct bpf_prog_aux *aux = aux__prog;u32 fmt_size =strlen (fmt__str) +1 ;struct bpf_stream *stream;u32 data_len = len__sz;int ret, num_args; - stream =bpf_stream_get (stream_id, aux); + stream =bpf_stream_get (stream_id, aux__magic);if (!stream)return -ENOENT ;
kernel/bpf/verifier.c +2 -0
@@ -3273,6 +3273,8 @@ BTF_ID(func, bpf_task_work_schedule_signal) BTF_ID(func, bpf_task_work_schedule_signal_impl) BTF_ID(func, bpf_task_work_schedule_resume) BTF_ID(func, bpf_task_work_schedule_resume_impl) +BTF_ID(func, bpf_stream_vprintk) +BTF_ID(func, bpf_stream_vprintk_impl)BTF_ID_LIST_END (magic_kfuncs) static s32 magic_kfunc_by_impl(s32 impl_func_id)
tools/lib/bpf/bpf_helpers.h +4 -3
@@ -315,8 +315,9 @@ enum libbpf_tristate { ___param, sizeof(___param)); \ }) -extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *args, - __u32 len__sz, void *aux__prog) __weak __ksym; +struct bpf_prog_aux ; +extern int bpf_stream_vprintk_impl(int stream_id, const char *fmt__str, const void *args, + __u32 len__sz, struct bpf_prog_aux *aux__magic) __weak __ksym; #define bpf_stream_printk(stream_id, fmt, args...) \ ({ \
@@ -328,7 +329,7 @@ extern int bpf_stream_vprintk(int stream_id, const char *fmt__str, const void *a___bpf_fill (___param, args); \ _Pragma("GCC diagnostic pop") \ \ -bpf_stream_vprintk (stream_id, ___fmt, ___param,sizeof (___param),NULL );\ +bpf_stream_vprintk_impl (stream_id, ___fmt, ___param,sizeof (___param),NULL );\ })/* Use __bpf_printk when bpf_printk call has 3 or fewer fmt args
tools/testing/selftests/bpf/progs/stream_fail.c +3 -3
@@ -10,7 +10,7 @@ SEC("syscall") __failure __msg("Possibly NULL pointer passed")int stream_vprintk_null_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT ,"" ,NULL ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT ,"" ,NULL ,0 );return 0 ; }
@@ -18,7 +18,7 @@ SEC("syscall") __failure __msg("R3 type=scalar expected=")int stream_vprintk_scalar_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT ,"" , (void *)46 ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT ,"" , (void *)46 ,0 );return 0 ; }
@@ -26,7 +26,7 @@ SEC("syscall") __failure __msg("arg#1 doesn't point to a const string")int stream_vprintk_string_arg (void *ctx) { -bpf_stream_vprintk (BPF_STDOUT , ctx,NULL ,0 ,NULL ); +bpf_stream_vprintk (BPF_STDOUT , ctx,NULL ,0 );return 0 ; }
Signature · 1 line
2.51.1
Eduard Zingerman
maintainerOn Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the _impl kfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no _impl kfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be unable to find non-_impl function names for existing kfuncs.
[...]
Eduard Zingerman
maintainerQuoted 11 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the
_implkfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no_implkfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be
unable to find non-_impl function names for existing kfuncs.[...]
Point being, if we are going to break backwards compatibility the following things need an update:
- Documentation/process/changes.rst
minimal pahole version has to be bumped
- scripts/Makefile.btf
All the different flags and options for different pahole versions can be dropped.
---
On the other hand, I'm not sure this useful but relatively obscure feature grants such a compatibility break. Some time ago Ihor advocated for just having two functions in the kernel, so that BTF will be generated for both. And I think that someone suggested putting the fake function to a discard-able section.
This way the whole thing can be done in kernel only.
E.g. it will look like so:
__bpf_kfunc void btf_foo_impl(struct bpf_prog_aux p__implicit) { /* real impl here */ }
__bpf_kfunc_proto void btf_foo(void) {}
Assuming that __bpf_kfunc_proto hides all the necessary attributes. Not much boilerplate, and a tad easier to understand where second prototype comes from, no need to read pahole.
Eduard Zingerman
maintainerQuoted 31 lines · Eduard Zingerman
Quoted 11 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the
_implkfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no_implkfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be
unable to find non-_impl function names for existing kfuncs.[...]
Point being, if we are going to break backwards compatibility the following things need an update:
-Documentation/process/changes.rst
minimal pahole version has to be bumped
-scripts/Makefile.btf
All the different flags and options for different pahole versions can be dropped.---
On the other hand, I'm not sure this useful but relatively obscure feature grants such a compatibility break. Some time ago Ihor advocated for just having two functions in the kernel, so that BTF will be generated for both. And I think that someone suggested putting the fake function to a discard-able section.
This way the whole thing can be done in kernel only.
E.g. it will look like so:
Quoted 8 lines
__bpf_kfuncvoidbtf_foo_impl(struct bpf_prog_aux p__implicit){ /* real impl here */ }
__bpf_kfunc_protovoidbtf_foo(void){}Assuming that
__bpf_kfunc_protohides all the necessary attributes. Not much boilerplate, and a tad easier to understand where second prototype comes from, no need to read pahole.
Scheme discussed off-list for new functions with __implicit args:
- kernel source code:
__bpf_kfunc void foo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)
- pahole:
- renames foo to foo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:
__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit);
void foo(void);- resolve_btfids puts the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags with KF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.
This will break the following scenario:
- new kfunc is added with __implicit arg
- kernel is built with old pahole
- vmlinux.h is generated for such kernel
- bpf program is compiled against such vmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.
Andrei and Alexei deemed this acceptable.
Ihor Solodrai
reviewerQuoted 47 lines · Eduard Zingerman
Quoted 31 lines · Eduard Zingerman
Quoted 11 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the
_implkfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no_implkfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be
unable to find non-_impl function names for existing kfuncs.[...]
Point being, if we are going to break backwards compatibility the following things need an update:
-Documentation/process/changes.rst
minimal pahole version has to be bumped
-scripts/Makefile.btf
All the different flags and options for different pahole versions can be dropped.---
On the other hand, I'm not sure this useful but relatively obscure feature grants such a compatibility break. Some time ago Ihor advocated for just having two functions in the kernel, so that BTF will be generated for both. And I think that someone suggested putting the fake function to a discard-able section.
This way the whole thing can be done in kernel only.
E.g. it will look like so:Quoted 8 lines
__bpf_kfuncvoidbtf_foo_impl(struct bpf_prog_aux p__implicit){ /* real impl here */ }
__bpf_kfunc_protovoidbtf_foo(void){}Assuming that
__bpf_kfunc_protohides all the necessary attributes. Not much boilerplate, and a tad easier to understand where second prototype comes from, no need to read pahole.Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)
Quoted 7 lines
- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);
I believe it's the other way around:
void foo_impl(struct bpf_prog_aux p__implicit);
__bpf_kfunc void foo(void);foo() is callable from BPF, but foo_impl() is not.
But we still want foo_impl() in BTF so that the verifier can find the correct prototype.
Andrii, please confirm.
Quoted 7 lines
-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.
Quoted 7 lines
This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.
I think you meant "new kernel compiled with old pahole".
Quoted 2 lines
Andrei and Alexei deemed this acceptable.
Eduard Zingerman
maintainerQuoted 57 lines · Ihor Solodrai
Quoted 47 lines · Eduard Zingerman
Quoted 31 lines · Eduard Zingerman
Quoted 11 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the
_implkfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no_implkfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be
unable to find non-_impl function names for existing kfuncs.[...]
Point being, if we are going to break backwards compatibility the following things need an update:
-Documentation/process/changes.rst
minimal pahole version has to be bumped
-scripts/Makefile.btf
All the different flags and options for different pahole versions can be dropped.---
On the other hand, I'm not sure this useful but relatively obscure feature grants such a compatibility break. Some time ago Ihor advocated for just having two functions in the kernel, so that BTF will be generated for both. And I think that someone suggested putting the fake function to a discard-able section.
This way the whole thing can be done in kernel only.
E.g. it will look like so:Quoted 8 lines
__bpf_kfuncvoidbtf_foo_impl(struct bpf_prog_aux p__implicit){ /* real impl here */ }
__bpf_kfunc_protovoidbtf_foo(void){}Assuming that
__bpf_kfunc_protohides all the necessary attributes. Not much boilerplate, and a tad easier to understand where second prototype comes from, no need to read pahole.Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)Quoted 7 lines
- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);I believe it's the other way around:
Quoted 8 lines
void foo_impl(struct bpf_prog_aux p__implicit); __bpf_kfunc void foo(void);
foo()is callable from BPF, butfoo_impl()is not.
But we still wantfoo_impl()in BTF so that the verifier can find the correct prototype.Andrii, please confirm.
Oops, yes, 'foo' gets __bpf_kfunc.
Quoted 14 lines
Quoted 6 lines
-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.Quoted 7 lines
This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.
Quoted 1 line
I think you meant "new kernel compiled with old pahole".
No, new kernel compiled with new pahole.
As it will have a different prototype for 'foo' compared to one encoded in the old program binary 'btf'.
Quoted 3 lines
Quoted 3 lines
Andrei and Alexei deemed this acceptable.
Quoted 69 lines · Eduard Zingerman
Quoted 31 lines · Eduard Zingerman
Quoted 11 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
Do we break compatibility with old pahole versions after this patch-set? Old paholes won't synthesize the
_implkfuncs, so:
- binary compatibility between new-kernel/old-pahole + old-bpf will be broken, as there would be no_implkfuncs;
- new-kernel/old-pahole + new-bpf won't work either, as kernel will be
unable to find non-_impl function names for existing kfuncs.[...]
Point being, if we are going to break backwards compatibility the following things need an update:
-Documentation/process/changes.rst
minimal pahole version has to be bumped
-scripts/Makefile.btf
All the different flags and options for different pahole versions can be dropped.---
On the other hand, I'm not sure this useful but relatively obscure feature grants such a compatibility break. Some time ago Ihor advocated for just having two functions in the kernel, so that BTF will be generated for both. And I think that someone suggested putting the fake function to a discard-able section.
This way the whole thing can be done in kernel only.
E.g. it will look like so:Quoted 8 lines
__bpf_kfuncvoidbtf_foo_impl(struct bpf_prog_aux p__implicit){ /* real impl here */ }
__bpf_kfunc_protovoidbtf_foo(void){}Assuming that
__bpf_kfunc_protohides all the necessary attributes. Not much boilerplate, and a tad easier to understand where second prototype comes from, no need to read pahole.Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.
Quoted 1 line
Andrei and Alexei deemed this acceptable.
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the _impl function and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the _impl variant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc and kfunc_impl. Couldn't we leverage the kernel build machinery around resolve_btf_ids to construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much because resolve_btfids travels with the kernel, no changes needed for pahole.
I'm guessing the above is missing something though?
Thanks!
Alan
Eduard Zingerman
maintainerOn Thu, 2025-10-30 at 18:26 +0000, Alan Maguire wrote:
[...]
Quoted 14 lines
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the
_implfunction and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the_implvariant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc andkfunc_impl. Couldn't we leverage the kernel build machinery aroundresolve_btf_idsto construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much becauseresolve_btfidstravels with the kernel, no changes needed for pahole.
Quoted 1 line
I'm guessing the above is missing something though?
Andrii does not like having to declare two functions manually in C code:
__bpf_kfunc foo_impl(p__implicit); __bpf_kfunc foo(void); <-------- don't want to declare this explicitly
But prototypes for both functions are needed in the BTF. And the only place that can invent new BTFs is pahole. Imo, that's a bit of an over-complication ¯\_(ツ)_/¯.
Ihor Solodrai
reviewerQuoted 37 lines · Alan Maguire
Quoted 34 lines · Eduard Zingerman
Quoted 4 lines · Eduard Zingerman
Quoted 3 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
[...]
Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.Quoted 1 line
Andrei and Alexei deemed this acceptable.
Quoted 13 lines
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the
_implfunction and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the_implvariant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc andkfunc_impl. Couldn't we leverage the kernel build machinery aroundresolve_btf_idsto construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much becauseresolve_btfidstravels with the kernel, no changes needed for pahole.
We've had a couple of rounds of back and forth on this.
The reasoning here is that going forward we want to make a kfunc with implicit arguments easy to define. That is:
__bpf_kfunc int bpf_kfunc(int arg, struct bpf_prog_aux *aux__impl) {} BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)
That's it.
In order to keep pahole out of the loop, it's necessary to have both interface and implementation declarations in the kernel. Something like this:
__bpf_kfunc_interface int bpf_kfunc(int arg) {}
__bpf_kfunc int bpf_kfunc_impl(int arg, struct bpf_prog_aux *aux__impl) {}
BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)Which shifts most of the burden of resolving KF_IMPLICIT_ARGS to the verifier. But then of course both variants will be callable from BPF, which is another thing we'd like to avoid.
pahole provides an ability to modify BTF only, and make that bpf_kfunc_impl (almost) invisible to the user, which is great.
The downside is that maintaining backwards compatibility between kernel, pahole and BPF binaries is difficult.
Quoted 6 lines
I'm guessing the above is missing something though?
Thanks!
Alan
Quoted 53 lines · Ihor Solodrai
Quoted 37 lines · Alan Maguire
Quoted 34 lines · Eduard Zingerman
Quoted 4 lines · Eduard Zingerman
Quoted 3 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
[...]
Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.Quoted 1 line
Andrei and Alexei deemed this acceptable.
Quoted 13 lines
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the
_implfunction and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the_implvariant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc andkfunc_impl. Couldn't we leverage the kernel build machinery aroundresolve_btf_idsto construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much becauseresolve_btfidstravels with the kernel, no changes needed for pahole.
Quoted 6 lines
We've had a couple of rounds of back and forth on this.
The reasoning here is that going forward we want to make a kfunc with implicit arguments easy to define. That is:
__bpf_kfunc int bpf_kfunc(int arg, struct bpf_prog_aux *aux__impl) {}
I don't think we even need __impl suffix for argument name with KF_IMPLICIT_ARGS, right?
Quoted 16 lines
BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)That's it.
In order to keep pahole out of the loop, it's necessary to have both interface and implementation declarations in the kernel. Something like this:
__bpf_kfunc_interface int bpf_kfunc(int arg) {} __bpf_kfunc int bpf_kfunc_impl(int arg, struct bpf_prog_aux *aux__impl) {} BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)Which shifts most of the burden of resolving
KF_IMPLICIT_ARGSto the verifier. But then of course both variants will be callable from BPF, which is another thing we'd like to avoid.
Quoted 6 lines
pahole provides an ability to modify BTF only, and make that
bpf_kfunc_impl(almost) invisible to the user, which is great.The downside is that maintaining backwards compatibility between kernel, pahole and BPF binaries is difficult.
I think we should differentiate backwards compat for all existing _impl kfuncs and BPF programs that used to work with them, and then, separately, what happens going forward with newly added non-_impl kfuncs with KF_IMPLICIT_ARGS and BPF programs that want to make use of these _impl-less kfuncs.
For existing _impl kfuncs ("legacy" case), backwards compat is 100% preserved, even if the kernel was built with an old pahole that doesn't yet know about KF_IMPLICTI_ARGS. There will be BTF for both _impl and _impl-less func_protos, all that will work. And BPF programs are explicitly calling _impl kfuncs. So even if pahole didn't do anything special for KF_IMPLICIT_ARGS, verifier should work fine, it will find _impl BTF, etc.
For all new _impl-less kfunc definitions and/or BPF programs that switch to _impl-less calls, yes, we will document and require that kernel BTF has to be generated with a newer pahole. We can enforce that in Kconfig, but it's a bit too strict/too soon as it's irrelevant and unnecessary for the majority of BPF users that don't care about _impl-less stuff.
Keep in mind, right now we have 4-5 such _impl special functions, but going forward we will probably have lots. sched-ext is poised to use that very extensively throughout a lot of its kfuncs, so requiring these explicit _impl wrappers just to support older pahole with newer kernels I think doesn't make sense in the grand scheme of things. Getting the latest pahole released/packaged/used for kernel build for distros shouldn't be a big deal at all. It's not really like upgrading the compiler toolchain at all.
Quoted 8 lines
Quoted 7 lines
I'm guessing the above is missing something though?
Thanks!
Alan
Ihor Solodrai
reviewerQuoted 56 lines · Andrii Nakryiko
Quoted 55 lines · Ihor Solodrai
Quoted 37 lines · Alan Maguire
Quoted 34 lines · Eduard Zingerman
Quoted 4 lines · Eduard Zingerman
Quoted 3 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
[...]
Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.Quoted 1 line
Andrei and Alexei deemed this acceptable.
Quoted 13 lines
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the
_implfunction and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the_implvariant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc andkfunc_impl. Couldn't we leverage the kernel build machinery aroundresolve_btf_idsto construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much becauseresolve_btfidstravels with the kernel, no changes needed for pahole.We've had a couple of rounds of back and forth on this.
Quoted 7 lines
Quoted 4 lines
The reasoning here is that going forward we want to make a kfunc with implicit arguments easy to define. That is:
__bpf_kfunc int bpf_kfunc(int arg, struct bpf_prog_aux *aux__impl) {}I don't think we even need
__implsuffix for argument name withKF_IMPLICIT_ARGS, right?
I mentioned options that we discussed before in the cover letter.
Basically, pahole needs to figure out how many arguments to omit *somehow*. Using a name suffix (aka annotation) seems to be the most flexible way, as it allows to avoid changes in pahole if/when we add new implicit arg types.
Quoted 38 lines
Quoted 17 lines
BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)That's it.
In order to keep pahole out of the loop, it's necessary to have both interface and implementation declarations in the kernel. Something like this:
__bpf_kfunc_interface int bpf_kfunc(int arg) {} __bpf_kfunc int bpf_kfunc_impl(int arg, struct bpf_prog_aux *aux__impl) {} BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)Which shifts most of the burden of resolving
KF_IMPLICIT_ARGSto the verifier. But then of course both variants will be callable from BPF, which is another thing we'd like to avoid.Quoted 6 lines
pahole provides an ability to modify BTF only, and make that
bpf_kfunc_impl(almost) invisible to the user, which is great.The downside is that maintaining backwards compatibility between kernel, pahole and BPF binaries is difficult.
I think we should differentiate backwards compat for all existing
_implkfuncs and BPF programs that used to work with them, and then, separately, what happens going forward with newly added non-_impl kfuncs withKF_IMPLICIT_ARGSand BPF programs that want to make use of these _impl-less kfuncs.For existing
_implkfuncs ("legacy" case), backwards compat is 100% preserved, even if the kernel was built with an old pahole that doesn't yet know aboutKF_IMPLICTI_ARGS. There will be BTF for both_impland _impl-lessfunc_protos, all that will work. And BPF programs are explicitly calling_implkfuncs. So even if pahole didn't do anything special forKF_IMPLICIT_ARGS, verifier should work fine, it will find_implBTF, etc.
Quoted 6 lines
For all new _impl-less kfunc definitions and/or BPF programs that switch to _impl-less calls, yes, we will document and require that kernel BTF has to be generated with a newer pahole. We can enforce that in
Kconfig, but it's a bit too strict/too soon as it's irrelevant and unnecessary for the majority of BPF users that don't care about _impl-less stuff.
I think some kind of build-time enforcement will be necessary. It's *very* easy to unintentionally use old(-er) pahole version for kernel build, especially when the new version is recent.
Quoted 18 lines
Keep in mind, right now we have 4-5 such
_implspecial functions, but going forward we will probably have lots. sched-ext is poised to use that very extensively throughout a lot of its kfuncs, so requiring these explicit_implwrappers just to support older pahole with newer kernels I think doesn't make sense in the grand scheme of things. Getting the latest pahole released/packaged/used for kernel build for distros shouldn't be a big deal at all. It's not really like upgrading the compiler toolchain at all.Quoted 8 lines
Quoted 7 lines
I'm guessing the above is missing something though?
Thanks!
Alan
Quoted 63 lines · Ihor Solodrai
Quoted 61 lines · Andrii Nakryiko
Quoted 59 lines · Ihor Solodrai
Quoted 37 lines · Alan Maguire
Quoted 34 lines · Eduard Zingerman
Quoted 4 lines · Eduard Zingerman
Quoted 3 lines · Eduard Zingerman
On Wed, 2025-10-29 at 12:01 -0700, Ihor Solodrai wrote:
[...]
Scheme discussed off-list for new functions with
__implicitargs:
- kernel source code:
__bpf_kfuncvoidfoo(struct bpf_prog_aux p__implicit)
BTF_ID_FLAGS(foo, KF_IMPLICIT_ARGS)- pahole:
- renames foo tofoo_impl
- adds bpf-side definition for 'foo' w/o implicit args vmlinux btf:__bpf_kfunc void foo_impl(struct bpf_prog_aux p__implicit); void foo(void);-
resolve_btfidsputs the 'foo' (the one w/o implicit args) id to all id lists (no changes needed for this, follows from pahole changes);
- verifier.c:add_kfunc_call()
- Sees the id of 'foo' and kfunc flags withKF_IMPLICIT_ARGS
- Replaces the id with id of 'foo_impl'.This will break the following scenario:
- new kfunc is added with__implicitarg
- kernel is built with old pahole
-vmlinux.his generated for such kernel
- bpf program is compiled against suchvmlinux.h
- attempt to run such program on a new kernel compiled with new pahole will fail.Quoted 1 line
Andrei and Alexei deemed this acceptable.
Quoted 13 lines
Okay so bear with me as this is probably a massive over-simplification. It seems like what we want here is a way to establish a relationship between the BTF associated with the
_implfunction and the kfunc-visible form (without the implicit arguments), right? Once we have that relationship, it's sort of implicit which are the implicit arguments; they're the ones the_implvariant has and the non-impl variant doesn't have. So to me - and again I'm probably missing a lot - the key thing is to establish that relationship between kfunc andkfunc_impl. Couldn't we leverage the kernel build machinery aroundresolve_btf_idsto construct these pairwise mappings of BTF ids? That way we keep pahole out of the loop (aside from generating BTF for both variants as usual) and compatibility issues aren't there as much becauseresolve_btfidstravels with the kernel, no changes needed for pahole.We've had a couple of rounds of back and forth on this.
The reasoning here is that going forward we want to make a kfunc with implicit arguments easy to define. That is:
__bpf_kfunc int bpf_kfunc(int arg, struct bpf_prog_aux *aux__impl) {}
Quoted 8 lines
Quoted 2 lines
I don't think we even need
__implsuffix for argument name withKF_IMPLICIT_ARGS, right?I mentioned options that we discussed before in the cover letter.
Basically, pahole needs to figure out how many arguments to omit *somehow*. Using a name suffix (aka annotation) seems to be the most flexible way, as it allows to avoid changes in pahole if/when we add new implicit arg types.
Ah, right, I forgot we discussed this. Ok, arg suffix isn't too bad.
Quoted 46 lines
Quoted 39 lines
Quoted 18 lines
BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)That's it.
In order to keep pahole out of the loop, it's necessary to have both interface and implementation declarations in the kernel. Something like this:
__bpf_kfunc_interface int bpf_kfunc(int arg) {} __bpf_kfunc int bpf_kfunc_impl(int arg, struct bpf_prog_aux *aux__impl) {} BTF_ID_FLAGS(func, bpf_kfunc, KF_IMPLICIT_ARGS)Which shifts most of the burden of resolving
KF_IMPLICIT_ARGSto the verifier. But then of course both variants will be callable from BPF, which is another thing we'd like to avoid.Quoted 6 lines
pahole provides an ability to modify BTF only, and make that
bpf_kfunc_impl(almost) invisible to the user, which is great.The downside is that maintaining backwards compatibility between kernel, pahole and BPF binaries is difficult.
I think we should differentiate backwards compat for all existing
_implkfuncs and BPF programs that used to work with them, and then, separately, what happens going forward with newly added non-_impl kfuncs withKF_IMPLICIT_ARGSand BPF programs that want to make use of these _impl-less kfuncs.For existing
_implkfuncs ("legacy" case), backwards compat is 100% preserved, even if the kernel was built with an old pahole that doesn't yet know aboutKF_IMPLICTI_ARGS. There will be BTF for both_impland _impl-lessfunc_protos, all that will work. And BPF programs are explicitly calling_implkfuncs. So even if pahole didn't do anything special forKF_IMPLICIT_ARGS, verifier should work fine, it will find_implBTF, etc.Quoted 6 lines
For all new _impl-less kfunc definitions and/or BPF programs that switch to _impl-less calls, yes, we will document and require that kernel BTF has to be generated with a newer pahole. We can enforce that in
Kconfig, but it's a bit too strict/too soon as it's irrelevant and unnecessary for the majority of BPF users that don't care about _impl-less stuff.
Quoted 4 lines
I think some kind of build-time enforcement will be necessary. It's *very* easy to unintentionally use old(-er) pahole version for kernel build, especially when the new version is recent.
I'd try to avoid failing kernel build if pahole is not recent enough for this feature.
How about we teach resolve_btfids() to check whether kernel BTF is "conformant" (i.e., it has xxx and xxx_impl func_protos, and xxx doesn't have implicit args, while _impl has implicit args), and if it's not conformant, we just effectively make such _impl-less kfunc not a kfunc. Would resolving such BTF_ID to zero do the trick?
Quoted 19 lines
Quoted 18 lines
Keep in mind, right now we have 4-5 such
_implspecial functions, but going forward we will probably have lots. sched-ext is poised to use that very extensively throughout a lot of its kfuncs, so requiring these explicit_implwrappers just to support older pahole with newer kernels I think doesn't make sense in the grand scheme of things. Getting the latest pahole released/packaged/used for kernel build for distros shouldn't be a big deal at all. It's not really like upgrading the compiler toolchain at all.Quoted 8 lines
Quoted 7 lines
I'm guessing the above is missing something though?
Thanks!
Alan