bpf: Support struct btf_struct_meta via KF_IMPLICIT_ARGS
Subsystems (4)
Ihor Solodrai
reviewerPATCH v4 1/2 bpf: Support struct btf_struct_meta via KF_IMPLICIT_ARGS
The following kfuncs currently accept void *meta__ign argument:
* bpf_obj_new_impl
* bpf_obj_drop_impl
* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl
The __ign suffix is an indicator for the verifier to skip the argument in check_kfunc_args(). Then, in fixup_kfunc_call() the verifier may set the value of this argument to struct btf_struct_meta * kptr_struct_meta from insn_aux_data.
BPF programs must pass a dummy NULL value when calling these kfuncs.
Additionally, the list and rbtree _impl kfuncs also accept an implicit u64 argument, which doesn't require __ign suffix because it's a scalar, and BPF programs explicitly pass 0.
Add new kfuncs with KF_IMPLICIT_ARGS [1], that correspond to each _impl kfunc accepting meta__ign. The existing _impl kfuncs remain unchanged for backwards compatibility.
To support this, add "btf_struct_meta" to the list of recognized implicit argument types in resolve_btfids.
Implement is_kfunc_arg_implicit() in the verifier, that determines implicit args by inspecting both (_impl and non-_impl) BTF prototypes of the kfunc.
Update the special_kfunc_list in the verifier and relevant checks to support both the old _impl and the new KF_IMPLICIT_ARGS variants of btf_struct_meta users.
[1] https://lore.kernel.org/bpf/[email protected]/
---
v3->v4:
* Move and reformat docs comments for relevant kfuncs from bpf_experimental.h to kernel/bpf/helpers.c (Jiri)
* Move BTF_ID() entries of affected kfuncs from the end of the special_kfunc_list to pair them with legacy variants (Alexei)
* Bump BTF_ID_LIST stub size to avoid compiler warnings for builds with CONFIG_DEBUG_INFO_BTF=n (kernel test robot)
* https://lore.kernel.org/bpf/[email protected]/
* Misc typos
v3: https://lore.kernel.org/bpf/[email protected]/
v1->v3: Nits suggested by AI
v1: https://lore.kernel.org/bpf/[email protected]/
--- include/linux/btf_ids.h | 2 +- kernel/bpf/helpers.c | 178 +++++++++++++++-- kernel/bpf/verifier.c | 184 +++++++++++++----- tools/bpf/resolve_btfids/main.c | 1 + .../selftests/bpf/progs/percpu_alloc_fail.c | 4 +- 5 files changed, 292 insertions(+), 77 deletions(-)
include/linux/btf_ids.h +1 -1
@@ -217,7 +217,7 @@ BTF_SET8_END(name)#else -#define BTF_ID_LIST (name) static u32 __maybe_unused name[64]; +#define BTF_ID_LIST (name) static u32 __maybe_unused name[128];#define BTF_ID (prefix, name)#define BTF_ID_FLAGS (prefix, name, ...)#define BTF_ID_UNUSED
kernel/bpf/helpers.c +158 -20
271 lines not on this page · raw message
kernel/bpf/verifier.c +130 -54
412 lines not on this page · raw message
tools/bpf/resolve_btfids/main.c +1 -0
@@ -1065,6 +1065,7 @@ static bool is_kf_implicit_arg(const struct btf *btf, const struct btf_param *p) {static const char *const kf_implicit_arg_types[] = {"bpf_prog_aux" , +"btf_struct_meta" , };const struct btf_type *t;const char *name;
tools/testing/selftests/bpf/progs/percpu_alloc_fail.c +2 -2
@@ -110,7 +110,7 @@ int BPF_PROG(test_array_map_3) } SEC("?fentry.s/bpf_fentry_test1") -__failure __msg("arg#0 expected for bpf_percpu_obj_drop_impl()") +__failure __msg("arg#0 expected for bpf_percpu_obj_drop()") int BPF_PROG(test_array_map_4) { struct val_t __percpu_kptr *p;
@@ -124,7 +124,7 @@ int BPF_PROG(test_array_map_4) } SEC("?fentry.s/bpf_fentry_test1") -__failure __msg("arg#0 expected for bpf_obj_drop_impl()") +__failure __msg("arg#0 expected for bpf_obj_drop()") int BPF_PROG(test_array_map_5) {struct val_t *p;
Signature · 1 line
2.53.0
Ihor Solodrai
reviewerPATCH v4 2/2 selftests/bpf: Update kfuncs using btf_struct_meta to new variants
Update selftests to use the new non-_impl kfuncs marked with KF_IMPLICIT_ARGS by removing redundant declarations and macros from bpf_experimental.h (the new kfuncs are present in the vmlinux.h) and updating relevant callsites.
Fix spin_lock verifier-log matching for lock_id_kptr_preserve by accepting variable instruction numbers. The calls to kfuncs with implicit arguments do not have register moves (e.g. r5 = 0) corresponding to dummy arguments anymore, so the order of instructions has shifted.
--- .../testing/selftests/bpf/bpf_experimental.h | 156 +----------------- .../selftests/bpf/prog_tests/spin_lock.c | 5 +- .../selftests/bpf/progs/kptr_xchg_inline.c | 4 +- 3 files changed, 9 insertions(+), 156 deletions(-)
tools/testing/selftests/bpf/bpf_experimental.h +4 -152
@@ -8,156 +8,11 @@#define __contains (name, node) __attribute__((btf_decl_tag("contains:" #name ":" #node))) -/* Description -* Allocates an object of the type represented by 'local_type_id' in -* program BTF. User may use the bpf_core_type_id_local macro to pass the -* type ID of a struct in program BTF. -* -* The 'local_type_id' parameter must be a known constant. -* The 'meta' parameter is rewritten by the verifier, no need for BPF -* program to set it. -* Returns -* A pointer to an object of the type corresponding to the passed in -* 'local_type_id', or NULL on failure. -*/ -extern void *bpf_obj_new_impl (__u64 local_type_id,void *meta) __ksym; - -/* Convenience macro to wrap over bpf_obj_new_impl */ -#define bpf_obj_new (type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) - -/* Description -* Free an allocated object. All fields of the object that require -* destruction will be destructed before the storage is freed. -* -* The 'meta' parameter is rewritten by the verifier, no need for BPF -* program to set it. -* Returns -* Void. -*/ -extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; - -/* Convenience macro to wrap over bpf_obj_drop_impl */ -#define bpf_obj_drop (kptr) bpf_obj_drop_impl(kptr, NULL) - -/* Description -* Increment the refcount on a refcounted local kptr, turning the -* non-owning reference input into an owning reference in the process. -* -* The 'meta' parameter is rewritten by the verifier, no need for BPF -* program to set it. -* Returns -* An owning reference to the object pointed to by 'kptr' -*/ -extern void *bpf_refcount_acquire_impl (void *kptr,void *meta) __ksym; - -/* Convenience macro to wrap over bpf_refcount_acquire_impl */ -#define bpf_refcount_acquire (kptr) bpf_refcount_acquire_impl(kptr, NULL) - -/* Description -* Add a new entry to the beginning of the BPF linked list. -* -* The 'meta' and 'off' parameters are rewritten by the verifier, no need -* for BPF programs to set them -* Returns -* 0 if the node was successfully added -* -EINVAL if the node wasn't added because it's already in a list -*/ -extern int bpf_list_push_front_impl(struct bpf_list_head *head, - struct bpf_list_node *node, - void *meta, __u64 off) __ksym; - -/* Convenience macro to wrap over bpf_list_push_front_impl */ -#define bpf_list_push_front (head, node) bpf_list_push_front_impl(head, node, NULL, 0) - -/* Description -* Add a new entry to the end of the BPF linked list. -* -* The 'meta' and 'off' parameters are rewritten by the verifier, no need -* for BPF programs to set them -* Returns -* 0 if the node was successfully added -* -EINVAL if the node wasn't added because it's already in a list -*/ -extern int bpf_list_push_back_impl(struct bpf_list_head *head, - struct bpf_list_node *node, - void *meta, __u64 off) __ksym; - -/* Convenience macro to wrap over bpf_list_push_back_impl */ -#define bpf_list_push_back (head, node) bpf_list_push_back_impl(head, node, NULL, 0) - -/* Description -* Remove the entry at the beginning of the BPF linked list. -* Returns -* Pointer to bpf_list_node of deleted entry, or NULL if list is empty. -*/ -extern struct bpf_list_node *bpf_list_pop_front (struct bpf_list_head *head) __ksym; - -/* Description -* Remove the entry at the end of the BPF linked list. -* Returns -* Pointer to bpf_list_node of deleted entry, or NULL if list is empty. -*/ -extern struct bpf_list_node *bpf_list_pop_back (struct bpf_list_head *head) __ksym; - -/* Description -* Remove 'node' from rbtree with root 'root' -* Returns -* Pointer to the removed node, or NULL if 'root' didn't contain 'node' -*/ -extern struct bpf_rb_node *bpf_rbtree_remove (struct bpf_rb_root *root, -struct bpf_rb_node *node) __ksym; - -/* Description -* Add 'node' to rbtree with root 'root' using comparator 'less' -* -* The 'meta' and 'off' parameters are rewritten by the verifier, no need -* for BPF programs to set them -* Returns -* 0 if the node was successfully added -* -EINVAL if the node wasn't added because it's already in a tree -*/ -extern int bpf_rbtree_add_impl(struct bpf_rb_root *root, struct bpf_rb_node *node, - bool (less)(struct bpf_rb_node *a, const struct bpf_rb_node *b), - void *meta, __u64 off) __ksym; - -/* Convenience macro to wrap over bpf_rbtree_add_impl */ -#define bpf_rbtree_add (head, node, less) bpf_rbtree_add_impl(head, node, less, NULL, 0) +/* Convenience macro to wrap over bpf_obj_new */ +#define bpf_obj_new (type) ((type *)bpf_obj_new(bpf_core_type_id_local(type))) -/* Description -* Return the first (leftmost) node in input tree -* Returns -* Pointer to the node, which is _not_ removed from the tree. If the tree -* contains no nodes, returns NULL. -*/ -extern struct bpf_rb_node *bpf_rbtree_first (struct bpf_rb_root *root) __ksym; - -/* Description -* Allocates a percpu object of the type represented by 'local_type_id' in -* program BTF. User may use the bpf_core_type_id_local macro to pass the -* type ID of a struct in program BTF. -* -* The 'local_type_id' parameter must be a known constant. -* The 'meta' parameter is rewritten by the verifier, no need for BPF -* program to set it. -* Returns -* A pointer to a percpu object of the type corresponding to the passed in -* 'local_type_id', or NULL on failure. -*/ -extern void *bpf_percpu_obj_new_impl (__u64 local_type_id,void *meta) __ksym; - -/* Convenience macro to wrap over bpf_percpu_obj_new_impl */ -#define bpf_percpu_obj_new (type) ((type __percpu_kptr *)bpf_percpu_obj_new_impl(bpf_core_type_id_local(type), NULL)) - -/* Description -* Free an allocated percpu object. All fields of the object that require -* destruction will be destructed before the storage is freed. -* -* The 'meta' parameter is rewritten by the verifier, no need for BPF -* program to set it. -* Returns -* Void. -*/ -extern void bpf_percpu_obj_drop_impl(void *kptr, void *meta) __ksym; +/* Convenience macro to wrap over bpf_percpu_obj_new */ +#define bpf_percpu_obj_new (type) ((type __percpu_kptr *)bpf_percpu_obj_new(bpf_core_type_id_local(type)))struct bpf_iter_task_vma ;
@@ -167,9 +22,6 @@ extern int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,extern struct vm_area_struct *bpf_iter_task_vma_next (struct bpf_iter_task_vma *it) __ksym; extern void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it) __ksym; -/* Convenience macro to wrap over bpf_obj_drop_impl */ -#define bpf_percpu_obj_drop (kptr) bpf_percpu_obj_drop_impl(kptr, NULL) -/* Description * Throw a BPF exception from the program, immediately terminating its * execution and unwinding the stack. The supplied 'cookie' parameter
tools/testing/selftests/bpf/prog_tests/spin_lock.c +3 -2
@@ -13,8 +13,9 @@ static struct {const char *err_msg; } spin_lock_fail_tests[] = { {"lock_id_kptr_preserve" , -"5: (bf) r1 = r0 ; R0=ptr_foo(id=2,ref_obj_id=2) " -"R1=ptr_foo(id=2,ref_obj_id=2) refs=2\n6: (85) call bpf_this_cpu_ptr#154\n" +"[0-9]\\+: (bf) r1 = r0 ; R0=ptr_foo(id=2,ref_obj_id=2)" +" R1=ptr_foo(id=2,ref_obj_id=2) refs=2\n" +"[0-9]\\+: (85) call bpf_this_cpu_ptr#154\n" "R1 type=ptr_ expected=percpu_ptr_" }, { "lock_id_global_zero","; R1=map_value(map=.data.A,ks=4,vs=4)\n2: (85) call bpf_this_cpu_ptr#154\n"
tools/testing/selftests/bpf/progs/kptr_xchg_inline.c +2 -2
@@ -25,14 +25,14 @@ __naked int kptr_xchg_inline(void) "if r0 == 0 goto 1f;" "r1 = r0;" "r2 = 0;" - "call %[bpf_obj_drop_impl];" + "call %[bpf_obj_drop];" "1:" "r0 = 0;" "exit;" : : __imm_addr(ptr), __imm(bpf_kptr_xchg), - __imm(bpf_obj_drop_impl) + __imm(bpf_obj_drop) : __clobber_all ); }
Signature · 1 line
2.53.0
[email protected]
AI reviewKF_ARG_BTF_STRUCT_META is added to the enum and btf_struct_meta is added to kf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID…
Quoted 15 lines
kernel/bpf/verifier.c +2
-1@@ -12369,7 +12369,8 @@ enum {KF_ARG_RES_SPIN_LOCK_ID ,KF_ARG_TASK_WORK_ID ,KF_ARG_PROG_AUX_ID , - KF_ARG_TIMER_ID +KF_ARG_TIMER_ID , +KF_ARG_BTF_STRUCT_META , };BTF_ID_LIST (kf_arg_btf_ids)
kernel/bpf/verifier.c BTF_ID(struct, bpf_res_spin_lock) · line 12387
Quoted 4 lines
kernel/bpf/verifier.c +1
-0@@ -12383,6 +12384,7 @@ BTF_ID(struct, bpf_res_spin_lock) BTF_ID(struct, bpf_task_work) BTF_ID(struct, bpf_prog_aux) BTF_ID(struct, bpf_timer) +BTF_ID (struct, btf_struct_meta)
KF_ARG_BTF_STRUCT_META is added to the enum and btf_struct_meta is added to kf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID, KF_ARG_PROG_AUX_ID, etc.) are each used by a corresponding is_kfunc_arg_*() function.
Was this intended as scaffolding for a follow-up patch, or is there a missing is_kfunc_arg_btf_struct_meta() helper?
[ ... ]
The commit message lists 7 kfuncs accepting void *meta__ign:
Quoted 8 lines
kernel/bpf/verifier.c +0
-0The following kfuncs currently accept void *meta__ign argument: * bpf_obj_new_impl * bpf_obj_drop_impl*
bpf_percpu_obj_new_impl
*bpf_percpu_obj_drop_impl
*bpf_refcount_acquire_impl
*bpf_list_push_front_impl
*bpf_rbtree_add_impl
bpf_list_push_back_impl also accepts void *meta__ign and is converted in this patch, but it is not listed here.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/23625407551
Quoted 42 lines · On Thu, Mar 26, 2026 at 6:15 PM <[email protected]> wrote:
Quoted 16 lines
kernel/bpf/verifier.c +2
-1@@ -12369,7 +12369,8 @@ enum {KF_ARG_RES_SPIN_LOCK_ID ,KF_ARG_TASK_WORK_ID ,KF_ARG_PROG_AUX_ID , - KF_ARG_TIMER_ID +KF_ARG_TIMER_ID , +KF_ARG_BTF_STRUCT_META , };BTF_ID_LIST (kf_arg_btf_ids)kernel/bpf/verifier.c BTF_ID(struct, bpf_res_spin_lock) · line 12387
Quoted 4 lines
kernel/bpf/verifier.c +1
-0@@ -12383,6 +12384,7 @@ BTF_ID(struct, bpf_res_spin_lock) BTF_ID(struct, bpf_task_work) BTF_ID(struct, bpf_prog_aux) BTF_ID(struct, bpf_timer) +BTF_ID (struct, btf_struct_meta)
KF_ARG_BTF_STRUCT_METAis added to the enum andbtf_struct_metais added tokf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID,KF_ARG_PROG_AUX_ID, etc.) are each used by a corresponding is_kfunc_arg_*() function.Was this intended as scaffolding for a follow-up patch, or is there a missing
is_kfunc_arg_btf_struct_meta()helper?[ ... ]
The commit message lists 7 kfuncs accepting void
*meta__ign:Quoted 8 lines
kernel/bpf/verifier.c +0
-0The following kfuncs currently accept void *meta__ign argument: * bpf_obj_new_impl * bpf_obj_drop_impl*
bpf_percpu_obj_new_impl
*bpf_percpu_obj_drop_impl
*bpf_refcount_acquire_impl
*bpf_list_push_front_impl
*bpf_rbtree_add_impl
Quoted 2 lines
bpf_list_push_back_implalso accepts void*meta__ignand is converted in this patch, but it is not listed here.
Looks like AI is correct on both counts.
pw-bot: cr
Kumar Kartikeya Dwivedi
maintainerQuoted 17 lines · Ihor Solodrai
The following kfuncs currently accept void
*meta__ignargument:
*bpf_obj_new_impl
*bpf_obj_drop_impl
*bpf_percpu_obj_new_impl
*bpf_percpu_obj_drop_impl
*bpf_refcount_acquire_impl
*bpf_list_push_front_impl
*bpf_rbtree_add_implThe
__ignsuffix is an indicator for the verifier to skip the argument incheck_kfunc_args(). Then, infixup_kfunc_call()the verifier may set the value of this argument to structbtf_struct_meta*kptr_struct_metafrominsn_aux_data.BPF programs must pass a dummy NULL value when calling these kfuncs.
Quoted 7 lines
Additionally, the list and rbtree
_implkfuncs also accept an implicit u64 argument, which doesn't require__ignsuffix because it's a scalar, and BPF programs explicitly pass 0.Add new kfuncs with
KF_IMPLICIT_ARGS[1], that correspond to each_implkfunc acceptingmeta__ign. The existing_implkfuncs remain unchanged for backwards compatibility.
Just a drive-by idle thought, but is there a way we could drop these _impl variants too eventually?
I think the existing macros in selftests can be updated to use either version by detecting the presence of the kfuncs, and maybe we can emit a warning to the stderr stream of the program during verification when we see these being used that they're going to go away in N releases, and then just drop them after that?
Quoted 2 lines
[...]
Ihor Solodrai
reviewerQuoted 22 lines · Kumar Kartikeya Dwivedi
Quoted 21 lines · Ihor Solodrai
The following kfuncs currently accept void
*meta__ignargument:
*bpf_obj_new_impl
*bpf_obj_drop_impl
*bpf_percpu_obj_new_impl
*bpf_percpu_obj_drop_impl
*bpf_refcount_acquire_impl
*bpf_list_push_front_impl
*bpf_rbtree_add_implThe
__ignsuffix is an indicator for the verifier to skip the argument incheck_kfunc_args(). Then, infixup_kfunc_call()the verifier may set the value of this argument to structbtf_struct_meta*kptr_struct_metafrominsn_aux_data.BPF programs must pass a dummy NULL value when calling these kfuncs.
Additionally, the list and rbtree
_implkfuncs also accept an implicit u64 argument, which doesn't require__ignsuffix because it's a scalar, and BPF programs explicitly pass 0.
Quoted 6 lines
Quoted 3 lines
Add new kfuncs with
KF_IMPLICIT_ARGS[1], that correspond to each_implkfunc acceptingmeta__ign. The existing_implkfuncs remain unchanged for backwards compatibility.Just a drive-by idle thought, but is there a way we could drop these
_implvariants too eventually?
I think we *could*, since we don't promise stable API for kfuncs.
But practically speaking, turning this off abruptly will make many people sad:
$ git log -1 --oneline 958cf2e273f0 958cf2e273f0 bpf: Introduce bpf_obj_new
$ git tag --contains 958cf2e273f0 --sort=creatordate | grep -v rc | head -n3 v6.2 v6.3 v6.4
Quoted 6 lines
I think the existing macros in selftests can be updated to use either version by detecting the presence of the kfuncs, and maybe we can emit a warning to the stderr stream of the program during verification when we see these being used that they're going to go away in N releases, and then just drop them after that?
Selftests can just be moved to new API exclusively, since they are in tree. Although we probably want to know that "legacy" is working for now.
More difficult problem is how to properly warn the users if we decide to drop old _impl kunfcs, and for how long do we want to support both. Do we already have some kind of "warning: deprecated" mechanism in the verifier? I'm not aware.
Quoted 3 lines
Quoted 3 lines
[...]