Skip to contentSkip to conversation

bpf: Support struct btf_struct_meta via KF_IMPLICIT_ARGS

PATCH Superseded Ihor Solodrai 4 participants 6 messages Mar 27

Subsystems (4)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

PATCH v4 1/2 bpf: Support struct btf_struct_meta via KF_IMPLICIT_ARGS

The following kfuncs currently accept void *meta__ign argument:
* bpf_obj_new_impl
* bpf_obj_drop_impl
* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl

The __ign suffix is an indicator for the verifier to skip the argument in check_kfunc_args(). Then, in fixup_kfunc_call() the verifier may set the value of this argument to struct btf_struct_meta * kptr_struct_meta from insn_aux_data.

BPF programs must pass a dummy NULL value when calling these kfuncs.

Additionally, the list and rbtree _impl kfuncs also accept an implicit u64 argument, which doesn't require __ign suffix because it's a scalar, and BPF programs explicitly pass 0.

Add new kfuncs with KF_IMPLICIT_ARGS [1], that correspond to each _impl kfunc accepting meta__ign. The existing _impl kfuncs remain unchanged for backwards compatibility.

To support this, add "btf_struct_meta" to the list of recognized implicit argument types in resolve_btfids.

Implement is_kfunc_arg_implicit() in the verifier, that determines implicit args by inspecting both (_impl and non-_impl) BTF prototypes of the kfunc.

Update the special_kfunc_list in the verifier and relevant checks to support both the old _impl and the new KF_IMPLICIT_ARGS variants of btf_struct_meta users.

[1] https://lore.kernel.org/bpf/[email protected]/

---

v3->v4:
* Move and reformat docs comments for relevant kfuncs from bpf_experimental.h to kernel/bpf/helpers.c (Jiri)
* Move BTF_ID() entries of affected kfuncs from the end of the special_kfunc_list to pair them with legacy variants (Alexei)
* Bump BTF_ID_LIST stub size to avoid compiler warnings for builds with CONFIG_DEBUG_INFO_BTF=n (kernel test robot)
* https://lore.kernel.org/bpf/[email protected]/
* Misc typos
v3: https://lore.kernel.org/bpf/[email protected]/

v1->v3: Nits suggested by AI
v1: https://lore.kernel.org/bpf/[email protected]/

---
 include/linux/btf_ids.h                       |   2 +-
 kernel/bpf/helpers.c                          | 178 +++++++++++++++--
 kernel/bpf/verifier.c                         | 184 +++++++++++++-----
 tools/bpf/resolve_btfids/main.c               |   1 +
 .../selftests/bpf/progs/percpu_alloc_fail.c   |   4 +-
 5 files changed, 292 insertions(+), 77 deletions(-)
include/linux/btf_ids.h +1 -1
index 139bdececdcf..af011db39ab3 100644
@@ -217,7 +217,7 @@ BTF_SET8_END(name)
 
 #else
 
-#define BTF_ID_LIST(name) static u32 __maybe_unused name[64];
+#define BTF_ID_LIST(name) static u32 __maybe_unused name[128];
 #define BTF_ID(prefix, name)
 #define BTF_ID_FLAGS(prefix, name, ...)
 #define BTF_ID_UNUSED
kernel/bpf/helpers.c +158 -20

271 lines not on this page · raw message

kernel/bpf/verifier.c +130 -54

412 lines not on this page · raw message

tools/bpf/resolve_btfids/main.c +1 -0
index 5208f650080f..f8a91fa7584f 100644
@@ -1065,6 +1065,7 @@ static bool is_kf_implicit_arg(const struct btf *btf, const struct btf_param *p)
 {
 	static const char *const kf_implicit_arg_types[] = {
 		"bpf_prog_aux",
+		"btf_struct_meta",
 	};
 	const struct btf_type *t;
 	const char *name;
tools/testing/selftests/bpf/progs/percpu_alloc_fail.c +2 -2
index f2b8eb2ff76f..81813c724fa9 100644
@@ -110,7 +110,7 @@ int BPF_PROG(test_array_map_3)
 }
 
 SEC("?fentry.s/bpf_fentry_test1")
-__failure __msg("arg#0 expected for bpf_percpu_obj_drop_impl()")
+__failure __msg("arg#0 expected for bpf_percpu_obj_drop()")
 int BPF_PROG(test_array_map_4)
 {
 	struct val_t __percpu_kptr *p;
@@ -124,7 +124,7 @@ int BPF_PROG(test_array_map_4)
 }
 
 SEC("?fentry.s/bpf_fentry_test1")
-__failure __msg("arg#0 expected for bpf_obj_drop_impl()")
+__failure __msg("arg#0 expected for bpf_obj_drop()")
 int BPF_PROG(test_array_map_5)
 {
 	struct val_t *p;
Signature · 1 line
2.53.0
author reviewer
#1 lore raw

PATCH v4 2/2 selftests/bpf: Update kfuncs using btf_struct_meta to new variants

Update selftests to use the new non-_impl kfuncs marked with KF_IMPLICIT_ARGS by removing redundant declarations and macros from bpf_experimental.h (the new kfuncs are present in the vmlinux.h) and updating relevant callsites.

Fix spin_lock verifier-log matching for lock_id_kptr_preserve by accepting variable instruction numbers. The calls to kfuncs with implicit arguments do not have register moves (e.g. r5 = 0) corresponding to dummy arguments anymore, so the order of instructions has shifted.

---
 .../testing/selftests/bpf/bpf_experimental.h  | 156 +-----------------
 .../selftests/bpf/prog_tests/spin_lock.c      |   5 +-
 .../selftests/bpf/progs/kptr_xchg_inline.c    |   4 +-
 3 files changed, 9 insertions(+), 156 deletions(-)
tools/testing/selftests/bpf/bpf_experimental.h +4 -152
index 44466acf8083..2234bd6bc9d3 100644
@@ -8,156 +8,11 @@
 
 #define __contains(name, node) __attribute__((btf_decl_tag("contains:" #name ":" #node)))
 
-/* Description
- *	Allocates an object of the type represented by 'local_type_id' in
- *	program BTF. User may use the bpf_core_type_id_local macro to pass the
- *	type ID of a struct in program BTF.
- *
- *	The 'local_type_id' parameter must be a known constant.
- *	The 'meta' parameter is rewritten by the verifier, no need for BPF
- *	program to set it.
- * Returns
- *	A pointer to an object of the type corresponding to the passed in
- *	'local_type_id', or NULL on failure.
- */
-extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym;
-
-/* Convenience macro to wrap over bpf_obj_new_impl */
-#define bpf_obj_new(type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL))
-
-/* Description
- *	Free an allocated object. All fields of the object that require
- *	destruction will be destructed before the storage is freed.
- *
- *	The 'meta' parameter is rewritten by the verifier, no need for BPF
- *	program to set it.
- * Returns
- *	Void.
- */
-extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym;
-
-/* Convenience macro to wrap over bpf_obj_drop_impl */
-#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL)
-
-/* Description
- *	Increment the refcount on a refcounted local kptr, turning the
- *	non-owning reference input into an owning reference in the process.
- *
- *	The 'meta' parameter is rewritten by the verifier, no need for BPF
- *	program to set it.
- * Returns
- *	An owning reference to the object pointed to by 'kptr'
- */
-extern void *bpf_refcount_acquire_impl(void *kptr, void *meta) __ksym;
-
-/* Convenience macro to wrap over bpf_refcount_acquire_impl */
-#define bpf_refcount_acquire(kptr) bpf_refcount_acquire_impl(kptr, NULL)
-
-/* Description
- *	Add a new entry to the beginning of the BPF linked list.
- *
- *	The 'meta' and 'off' parameters are rewritten by the verifier, no need
- *	for BPF programs to set them
- * Returns
- *	0 if the node was successfully added
- *	-EINVAL if the node wasn't added because it's already in a list
- */
-extern int bpf_list_push_front_impl(struct bpf_list_head *head,
-				    struct bpf_list_node *node,
-				    void *meta, __u64 off) __ksym;
-
-/* Convenience macro to wrap over bpf_list_push_front_impl */
-#define bpf_list_push_front(head, node) bpf_list_push_front_impl(head, node, NULL, 0)
-
-/* Description
- *	Add a new entry to the end of the BPF linked list.
- *
- *	The 'meta' and 'off' parameters are rewritten by the verifier, no need
- *	for BPF programs to set them
- * Returns
- *	0 if the node was successfully added
- *	-EINVAL if the node wasn't added because it's already in a list
- */
-extern int bpf_list_push_back_impl(struct bpf_list_head *head,
-				   struct bpf_list_node *node,
-				   void *meta, __u64 off) __ksym;
-
-/* Convenience macro to wrap over bpf_list_push_back_impl */
-#define bpf_list_push_back(head, node) bpf_list_push_back_impl(head, node, NULL, 0)
-
-/* Description
- *	Remove the entry at the beginning of the BPF linked list.
- * Returns
- *	Pointer to bpf_list_node of deleted entry, or NULL if list is empty.
- */
-extern struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head) __ksym;
-
-/* Description
- *	Remove the entry at the end of the BPF linked list.
- * Returns
- *	Pointer to bpf_list_node of deleted entry, or NULL if list is empty.
- */
-extern struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) __ksym;
-
-/* Description
- *	Remove 'node' from rbtree with root 'root'
- * Returns
- * 	Pointer to the removed node, or NULL if 'root' didn't contain 'node'
- */
-extern struct bpf_rb_node *bpf_rbtree_remove(struct bpf_rb_root *root,
-					     struct bpf_rb_node *node) __ksym;
-
-/* Description
- *	Add 'node' to rbtree with root 'root' using comparator 'less'
- *
- *	The 'meta' and 'off' parameters are rewritten by the verifier, no need
- *	for BPF programs to set them
- * Returns
- *	0 if the node was successfully added
- *	-EINVAL if the node wasn't added because it's already in a tree
- */
-extern int bpf_rbtree_add_impl(struct bpf_rb_root *root, struct bpf_rb_node *node,
-			       bool (less)(struct bpf_rb_node *a, const struct bpf_rb_node *b),
-			       void *meta, __u64 off) __ksym;
-
-/* Convenience macro to wrap over bpf_rbtree_add_impl */
-#define bpf_rbtree_add(head, node, less) bpf_rbtree_add_impl(head, node, less, NULL, 0)
+/* Convenience macro to wrap over bpf_obj_new */
+#define bpf_obj_new(type) ((type *)bpf_obj_new(bpf_core_type_id_local(type)))
 
-/* Description
- *	Return the first (leftmost) node in input tree
- * Returns
- *	Pointer to the node, which is _not_ removed from the tree. If the tree
- *	contains no nodes, returns NULL.
- */
-extern struct bpf_rb_node *bpf_rbtree_first(struct bpf_rb_root *root) __ksym;
-
-/* Description
- *	Allocates a percpu object of the type represented by 'local_type_id' in
- *	program BTF. User may use the bpf_core_type_id_local macro to pass the
- *	type ID of a struct in program BTF.
- *
- *	The 'local_type_id' parameter must be a known constant.
- *	The 'meta' parameter is rewritten by the verifier, no need for BPF
- *	program to set it.
- * Returns
- *	A pointer to a percpu object of the type corresponding to the passed in
- *	'local_type_id', or NULL on failure.
- */
-extern void *bpf_percpu_obj_new_impl(__u64 local_type_id, void *meta) __ksym;
-
-/* Convenience macro to wrap over bpf_percpu_obj_new_impl */
-#define bpf_percpu_obj_new(type) ((type __percpu_kptr *)bpf_percpu_obj_new_impl(bpf_core_type_id_local(type), NULL))
-
-/* Description
- *	Free an allocated percpu object. All fields of the object that require
- *	destruction will be destructed before the storage is freed.
- *
- *	The 'meta' parameter is rewritten by the verifier, no need for BPF
- *	program to set it.
- * Returns
- *	Void.
- */
-extern void bpf_percpu_obj_drop_impl(void *kptr, void *meta) __ksym;
+/* Convenience macro to wrap over bpf_percpu_obj_new */
+#define bpf_percpu_obj_new(type) ((type __percpu_kptr *)bpf_percpu_obj_new(bpf_core_type_id_local(type)))
 
 struct bpf_iter_task_vma;
 
@@ -167,9 +22,6 @@ extern int bpf_iter_task_vma_new(struct bpf_iter_task_vma *it,
 extern struct vm_area_struct *bpf_iter_task_vma_next(struct bpf_iter_task_vma *it) __ksym;
 extern void bpf_iter_task_vma_destroy(struct bpf_iter_task_vma *it) __ksym;
 
-/* Convenience macro to wrap over bpf_obj_drop_impl */
-#define bpf_percpu_obj_drop(kptr) bpf_percpu_obj_drop_impl(kptr, NULL)
-
 /* Description
  *	Throw a BPF exception from the program, immediately terminating its
  *	execution and unwinding the stack. The supplied 'cookie' parameter
tools/testing/selftests/bpf/prog_tests/spin_lock.c +3 -2
index 254fbfeab06a..bbe476f4c47d 100644
@@ -13,8 +13,9 @@ static struct {
 	const char *err_msg;
 } spin_lock_fail_tests[] = {
 	{ "lock_id_kptr_preserve",
-	  "5: (bf) r1 = r0                       ; R0=ptr_foo(id=2,ref_obj_id=2) "
-	  "R1=ptr_foo(id=2,ref_obj_id=2) refs=2\n6: (85) call bpf_this_cpu_ptr#154\n"
+	  "[0-9]\\+: (bf) r1 = r0                       ; R0=ptr_foo(id=2,ref_obj_id=2)"
+	  " R1=ptr_foo(id=2,ref_obj_id=2) refs=2\n"
+	  "[0-9]\\+: (85) call bpf_this_cpu_ptr#154\n"
 	  "R1 type=ptr_ expected=percpu_ptr_" },
 	{ "lock_id_global_zero",
 	  "; R1=map_value(map=.data.A,ks=4,vs=4)\n2: (85) call bpf_this_cpu_ptr#154\n"
tools/testing/selftests/bpf/progs/kptr_xchg_inline.c +2 -2
index 2414ac20b6d5..ca5943166057 100644
@@ -25,14 +25,14 @@ __naked int kptr_xchg_inline(void)
 		"if r0 == 0 goto 1f;"
 		"r1 = r0;"
 		"r2 = 0;"
-		"call %[bpf_obj_drop_impl];"
+		"call %[bpf_obj_drop];"
 	"1:"
 		"r0 = 0;"
 		"exit;"
 		:
 		: __imm_addr(ptr),
 		  __imm(bpf_kptr_xchg),
-		  __imm(bpf_obj_drop_impl)
+		  __imm(bpf_obj_drop)
 		: __clobber_all
 	);
 }
Signature · 1 line
2.53.0
#2 lore raw
In reply to Ihor Solodrai
KF_ARG_BTF_STRUCT_META is added to the enum and btf_struct_meta is added to kf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID…
Quoted 15 lines
kernel/bpf/verifier.c +2 -1
index 11207e63c94e..33f6c226d528 100644
@@ -12369,7 +12369,8 @@ enum {
 	KF_ARG_RES_SPIN_LOCK_ID,
 	KF_ARG_TASK_WORK_ID,
 	KF_ARG_PROG_AUX_ID,
-	KF_ARG_TIMER_ID
+	KF_ARG_TIMER_ID,
+	KF_ARG_BTF_STRUCT_META,
 };
 
 BTF_ID_LIST(kf_arg_btf_ids)

kernel/bpf/verifier.c BTF_ID(struct, bpf_res_spin_lock) · line 12387

Quoted 4 lines
kernel/bpf/verifier.c +1 -0
@@ -12383,6 +12384,7 @@ BTF_ID(struct, bpf_res_spin_lock)
 BTF_ID(struct, bpf_task_work)
 BTF_ID(struct, bpf_prog_aux)
 BTF_ID(struct, bpf_timer)
+BTF_ID(struct, btf_struct_meta)

KF_ARG_BTF_STRUCT_META is added to the enum and btf_struct_meta is added to kf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID, KF_ARG_PROG_AUX_ID, etc.) are each used by a corresponding is_kfunc_arg_*() function.

Was this intended as scaffolding for a follow-up patch, or is there a missing is_kfunc_arg_btf_struct_meta() helper?

[ ... ]

The commit message lists 7 kfuncs accepting void *meta__ign:

Quoted 8 lines
kernel/bpf/verifier.c +0 -0
 The following kfuncs currently accept void *meta__ign argument:
   * bpf_obj_new_impl
   * bpf_obj_drop_impl

* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl

bpf_list_push_back_impl also accepts void *meta__ign and is converted in this patch, but it is not listed here.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/23625407551

#3 lore raw
Quoted 42 lines · On Thu, Mar 26, 2026 at 6:15 PM <[email protected]> wrote:
Quoted 16 lines
kernel/bpf/verifier.c +2 -1
index 11207e63c94e..33f6c226d528 100644
@@ -12369,7 +12369,8 @@ enum {
      KF_ARG_RES_SPIN_LOCK_ID,
      KF_ARG_TASK_WORK_ID,
      KF_ARG_PROG_AUX_ID,
-     KF_ARG_TIMER_ID
+     KF_ARG_TIMER_ID,
+     KF_ARG_BTF_STRUCT_META,
 };
 
 BTF_ID_LIST(kf_arg_btf_ids)

kernel/bpf/verifier.c BTF_ID(struct, bpf_res_spin_lock) · line 12387

Quoted 4 lines
kernel/bpf/verifier.c +1 -0
@@ -12383,6 +12384,7 @@ BTF_ID(struct, bpf_res_spin_lock)
 BTF_ID(struct, bpf_task_work)
 BTF_ID(struct, bpf_prog_aux)
 BTF_ID(struct, bpf_timer)
+BTF_ID(struct, btf_struct_meta)

KF_ARG_BTF_STRUCT_META is added to the enum and btf_struct_meta is added to kf_arg_btf_ids, but neither appears to be referenced anywhere in the verifier. The other entries (KF_ARG_TIMER_ID, KF_ARG_PROG_AUX_ID, etc.) are each used by a corresponding is_kfunc_arg_*() function.

Was this intended as scaffolding for a follow-up patch, or is there a missing is_kfunc_arg_btf_struct_meta() helper?

[ ... ]

The commit message lists 7 kfuncs accepting void *meta__ign:

Quoted 8 lines
kernel/bpf/verifier.c +0 -0
 The following kfuncs currently accept void *meta__ign argument:
   * bpf_obj_new_impl
   * bpf_obj_drop_impl

* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl

Quoted 2 lines

bpf_list_push_back_impl also accepts void *meta__ign and is converted in this patch, but it is not listed here.

Looks like AI is correct on both counts.

pw-bot: cr

#4 lore raw
In reply to Ihor Solodrai
Quoted 17 lines · Ihor Solodrai

The following kfuncs currently accept void *meta__ign argument:
* bpf_obj_new_impl
* bpf_obj_drop_impl
* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl

The __ign suffix is an indicator for the verifier to skip the argument in check_kfunc_args(). Then, in fixup_kfunc_call() the verifier may set the value of this argument to struct btf_struct_meta * kptr_struct_meta from insn_aux_data.

BPF programs must pass a dummy NULL value when calling these kfuncs.

Quoted 7 lines

Additionally, the list and rbtree _impl kfuncs also accept an implicit u64 argument, which doesn't require __ign suffix because it's a scalar, and BPF programs explicitly pass 0.

Add new kfuncs with KF_IMPLICIT_ARGS [1], that correspond to each _impl kfunc accepting meta__ign. The existing _impl kfuncs remain unchanged for backwards compatibility.

Just a drive-by idle thought, but is there a way we could drop these _impl variants too eventually?

I think the existing macros in selftests can be updated to use either version by detecting the presence of the kfuncs, and maybe we can emit a warning to the stderr stream of the program during verification when we see these being used that they're going to go away in N releases, and then just drop them after that?

Quoted 2 lines

[...]

author reviewer
#5 lore raw
Quoted 22 lines · Kumar Kartikeya Dwivedi
Quoted 21 lines · Ihor Solodrai

The following kfuncs currently accept void *meta__ign argument:
* bpf_obj_new_impl
* bpf_obj_drop_impl
* bpf_percpu_obj_new_impl
* bpf_percpu_obj_drop_impl
* bpf_refcount_acquire_impl
* bpf_list_push_front_impl
* bpf_rbtree_add_impl

The __ign suffix is an indicator for the verifier to skip the argument in check_kfunc_args(). Then, in fixup_kfunc_call() the verifier may set the value of this argument to struct btf_struct_meta * kptr_struct_meta from insn_aux_data.

BPF programs must pass a dummy NULL value when calling these kfuncs.

Additionally, the list and rbtree _impl kfuncs also accept an implicit u64 argument, which doesn't require __ign suffix because it's a scalar, and BPF programs explicitly pass 0.

Quoted 6 lines
Quoted 3 lines

Add new kfuncs with KF_IMPLICIT_ARGS [1], that correspond to each _impl kfunc accepting meta__ign. The existing _impl kfuncs remain unchanged for backwards compatibility.

Just a drive-by idle thought, but is there a way we could drop these _impl variants too eventually?

I think we *could*, since we don't promise stable API for kfuncs.

But practically speaking, turning this off abruptly will make many people sad:

  $ git log -1 --oneline  958cf2e273f0 
  958cf2e273f0 bpf: Introduce bpf_obj_new
  $ git tag --contains 958cf2e273f0 --sort=creatordate | grep -v rc | head -n3
  v6.2
  v6.3
  v6.4
Quoted 6 lines

I think the existing macros in selftests can be updated to use either version by detecting the presence of the kfuncs, and maybe we can emit a warning to the stderr stream of the program during verification when we see these being used that they're going to go away in N releases, and then just drop them after that?

Selftests can just be moved to new API exclusively, since they are in tree. Although we probably want to know that "legacy" is working for now.

More difficult problem is how to properly warn the users if we decide to drop old _impl kunfcs, and for how long do we want to support both. Do we already have some kind of "warning: deprecated" mechanism in the verifier? I'm not aware.

Quoted 3 lines
Quoted 3 lines

[...]