Skip to contentSkip to conversation

Support passing BPF iterator to kfuncs

PATCH Applied Andrii Nakryiko 4 participants 10 messages Aug 8, 2024 – Aug 21, 2024

Subsystems (5)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

cover Support passing BPF iterator to kfuncs

Add support for passing BPF iterator state to any kfunc. Such kfunc has to declare such argument with valid struct bpf_iter_<type> * type and should use "__iter" suffix in argument name, following the established suffix-based convention. We add a simple test/demo iterator getter in bpf_testmod.

Andrii Nakryiko (3):
bpf: extract iterator argument type and name validation logic
bpf: allow passing struct bpf_iter_<type> as kfunc arguments selftests/bpf: test passing iterator to a kfunc

 include/linux/btf.h                           |  5 ++
 kernel/bpf/btf.c                              | 50 +++++++++++++------
 kernel/bpf/verifier.c                         | 35 +++++++++----
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   | 16 ++++--
 .../selftests/bpf/progs/iters_testmod_seq.c   | 50 +++++++++++++++++++
 5 files changed, 127 insertions(+), 29 deletions(-)
Signature · 1 line
2.43.5
author maintainer
#1 lore raw

PATCH 1/3 bpf: extract iterator argument type and name validation logic

Verifier enforces that all iterator structs are named bpf_iter_<name> and that whenever iterator is passed to a kfunc it's passed as a valid PTR -> STRUCT chain (with potentially const modifiers in between).

We'll need this check for upcoming changes, so instead of duplicating the logic, extract it into a helper function.

---
 include/linux/btf.h |  5 +++++
 kernel/bpf/btf.c    | 50 ++++++++++++++++++++++++++++++++-------------
 2 files changed, 41 insertions(+), 14 deletions(-)
include/linux/btf.h +5 -0
index cffb43133c68..b8a583194c4a 100644
@@ -580,6 +580,7 @@ bool btf_is_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf,
 int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_type);
 bool btf_types_are_same(const struct btf *btf1, u32 id1,
 			const struct btf *btf2, u32 id2);
+int btf_check_iter_arg(struct btf *btf, const struct btf_type *func, int arg_idx);
 #else
 static inline const struct btf_type *btf_type_by_id(const struct btf *btf,
 						    u32 type_id)
@@ -654,6 +655,10 @@ static inline bool btf_types_are_same(const struct btf *btf1, u32 id1,
 {
 	return false;
 }
+static inline int btf_check_iter_arg(struct btf *btf, const struct btf_type *func, int arg_idx)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 
 static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t)
kernel/bpf/btf.c +36 -14

79 lines not on this page · raw message

Signature · 1 line
2.43.5
maintainer Acked-by on v1 1/3
#2 lore raw

Re: [PATCH bpf-next 1/3] bpf: extract iterator argument type and name validation logic

Quoted 9 lines · Andrii Nakryiko

Verifier enforces that all iterator structs are named bpf_iter_<name> and that whenever iterator is passed to a kfunc it's passed as a valid PTR -> STRUCT chain (with potentially const modifiers in between).

We'll need this check for upcoming changes, so instead of duplicating the logic, extract it into a helper function.

---

author maintainer
#3 lore raw

PATCH 2/3 bpf: allow passing struct bpf_iter_<type> as kfunc arguments

In reply to Andrii Nakryiko

There are potentially useful cases where a specific iterator type might need to be passed into some kfunc. So, in addition to existing bpf_iter_<type>_{new,next,destroy}() kfuncs, allow to pass iterator pointer to any kfunc.

We employ "__iter" naming suffix for arguments that are meant to accept iterators. We also enforce that they accept PTR -> STRUCT btf_iter_<type> type chain and point to a valid initialized on-the-stack iterator state.

---
 kernel/bpf/verifier.c | 35 ++++++++++++++++++++++++-----------
 1 file changed, 24 insertions(+), 11 deletions(-)
kernel/bpf/verifier.c +24 -11
index df3be12096cf..920d7c5fe944 100644
@@ -7970,12 +7970,17 @@ static bool is_iter_destroy_kfunc(struct bpf_kfunc_call_arg_meta *meta)
 	return meta->kfunc_flags & KF_ITER_DESTROY;
 }
 
-static bool is_kfunc_arg_iter(struct bpf_kfunc_call_arg_meta *meta, int arg)
+static bool is_kfunc_arg_iter(struct bpf_kfunc_call_arg_meta *meta, int arg_idx,
+			      const struct btf_param *arg)
 {
 	/* btf_check_iter_kfuncs() guarantees that first argument of any iter
 	 * kfunc is iter state pointer
 	 */
-	return arg == 0 && is_iter_kfunc(meta);
+	if (is_iter_kfunc(meta))
+		return arg_idx == 0;
+
+	/* iter passed as an argument to a generic kfunc */
+	return btf_param_match_suffix(meta->btf, arg, "__iter");
 }
 
 static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_idx,
@@ -7983,14 +7988,20 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id
 {
 	struct bpf_reg_state *regs = cur_regs(env), *reg = &regs[regno];
 	const struct btf_type *t;
-	const struct btf_param *arg;
-	int spi, err, i, nr_slots;
-	u32 btf_id;
+	int spi, err, i, nr_slots, btf_id;
 
-	/* btf_check_iter_kfuncs() ensures we don't need to validate anything here */
-	arg = &btf_params(meta->func_proto)[0];
-	t = btf_type_skip_modifiers(meta->btf, arg->type, NULL);	/* PTR */
-	t = btf_type_skip_modifiers(meta->btf, t->type, &btf_id);	/* STRUCT */
+	/* For iter_{new,next,destroy} functions, btf_check_iter_kfuncs()
+	 * ensures struct convention, so we wouldn't need to do any BTF
+	 * validation here. But given iter state can be passed as a parameter
+	 * to any kfunc, if arg has "__iter" suffix, we need to be a bit more
+	 * conservative here.
+	 */
+	btf_id = btf_check_iter_arg(meta->btf, meta->func_proto, regno - 1);
+	if (btf_id < 0) {
+		verbose(env, "expected valid iter pointer as arg #%d\n", regno);
+		return -EINVAL;
+	}
+	t = btf_type_by_id(meta->btf, btf_id);
 	nr_slots = t->size / BPF_REG_SIZE;
 
 	if (is_iter_new_kfunc(meta)) {
@@ -8012,7 +8023,9 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id
 		if (err)
 			return err;
 	} else {
-		/* iter_next() or iter_destroy() expect initialized iter state*/
+		/* iter_next() or iter_destroy(), as well as any kfunc
+		 * accepting iter argument, expect initialized iter state
+		 */
 		err = is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots);
 		switch (err) {
 		case 0:
@@ -11382,7 +11395,7 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env,
 	if (is_kfunc_arg_dynptr(meta->btf, &args[argno]))
 		return KF_ARG_PTR_TO_DYNPTR;
 
-	if (is_kfunc_arg_iter(meta, argno))
+	if (is_kfunc_arg_iter(meta, argno, &args[argno]))
 		return KF_ARG_PTR_TO_ITER;
 
 	if (is_kfunc_arg_list_head(meta->btf, &args[argno]))
Signature · 1 line
2.43.5
maintainer
#4 lore raw

Re: [PATCH bpf-next 2/3] bpf: allow passing struct bpf_iter_<type> as kfunc arguments

Quoted 11 lines · Andrii Nakryiko

There are potentially useful cases where a specific iterator type might need to be passed into some kfunc. So, in addition to existing bpf_iter_<type>_{new,next,destroy}() kfuncs, allow to pass iterator pointer to any kfunc.

We employ "__iter" naming suffix for arguments that are meant to accept iterators. We also enforce that they accept PTR -> STRUCT btf_iter_<type> type chain and point to a valid initialized on-the-stack iterator state.

---

In current form this allows the following usage:

    SEC("?socket")
    __success
    int testmod_seq_getter_good(const void *ctx)
    {
    	struct bpf_iter_testmod_seq it;
    	s64 sum = 0;
    
    	bpf_iter_testmod_seq_new(&it, 100, 100);
    	sum *= bpf_iter_testmod_seq_value(0, &it);
    	bpf_iter_testmod_seq_destroy(&it);
    
    	return sum;
    }

Do we want to ensure that iterator is not drained before the call to bpf_iter_testmod_seq_value()?

Otherwise this patch lgtm.

[...]

#5 lore raw

Re: [PATCH bpf-next 2/3] bpf: allow passing struct bpf_iter_<type> as kfunc arguments

Quoted 30 lines · Eduard Zingerman
Quoted 11 lines · Andrii Nakryiko

There are potentially useful cases where a specific iterator type might need to be passed into some kfunc. So, in addition to existing bpf_iter_<type>_{new,next,destroy}() kfuncs, allow to pass iterator pointer to any kfunc.

We employ "__iter" naming suffix for arguments that are meant to accept iterators. We also enforce that they accept PTR -> STRUCT btf_iter_<type> type chain and point to a valid initialized on-the-stack iterator state.

---

In current form this allows the following usage:

    SEC("?socket")
    __success
    int testmod_seq_getter_good(const void *ctx)
    {
        struct bpf_iter_testmod_seq it;
        s64 sum = 0;

        bpf_iter_testmod_seq_new(&it, 100, 100);
        sum *= bpf_iter_testmod_seq_value(0, &it);
        bpf_iter_testmod_seq_destroy(&it);

        return sum;
    }
Quoted 3 lines

Do we want to ensure that iterator is not drained before the call to bpf_iter_testmod_seq_value()?

I'm not sure I follow your question. Drained or not it's still a valid iterator state. I don't want to put any restrictions, the user is free to pass it at any point between new and destroy.

Quoted 4 lines

Otherwise this patch lgtm.

[...]

maintainer
#6 lore raw

Re: [PATCH bpf-next 2/3] bpf: allow passing struct bpf_iter_<type> as kfunc arguments

On Fri, 2024-08-09 at 12:28 -0700, Andrii Nakryiko wrote:

[...]

Quoted 2 lines

I'm not sure I follow your question. Drained or not it's still a valid iterator state.

E.g. make sure that some such functions might be called only after a call to next() that returned a value.

Quoted 2 lines

I don't want to put any restrictions, the user is free to pass it at any point between new and destroy.

Ok, as you say.

[...]

#7 lore raw

Re: [PATCH bpf-next 2/3] bpf: allow passing struct bpf_iter_<type> as kfunc arguments

Quoted 10 lines · Eduard Zingerman

On Fri, 2024-08-09 at 12:28 -0700, Andrii Nakryiko wrote:

[...]

Quoted 2 lines

I'm not sure I follow your question. Drained or not it's still a valid iterator state.

E.g. make sure that some such functions might be called only after a call to next() that returned a value.

if that's important for some specific kfunc, it can handle that easily internally, I don't think we need to complicate verifier with something like that, even if that might have been useful for some niche use cases.

Quoted 8 lines
Quoted 3 lines

I don't want to put any restrictions, the user is free to pass it at any point between new and destroy.

Ok, as you say.

[...]

author maintainer
#8 lore raw

PATCH 3/3 selftests/bpf: test passing iterator to a kfunc

In reply to Andrii Nakryiko

Define BPF iterator "getter" kfunc, which accepts iterator pointer as one of the arguments. Make sure that argument passed doesn't have to be the very first argument (unlike new-next-destroy combo).

---
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   | 16 ++++--
 .../selftests/bpf/progs/iters_testmod_seq.c   | 50 +++++++++++++++++++
 2 files changed, 62 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +12 -4
index 3687a40b61c6..c04b7dec2ab9 100644
@@ -141,13 +141,12 @@ bpf_testmod_test_mod_kfunc(int i)
 
 __bpf_kfunc int bpf_iter_testmod_seq_new(struct bpf_iter_testmod_seq *it, s64 value, int cnt)
 {
-	if (cnt < 0) {
-		it->cnt = 0;
+	it->cnt = cnt;
+
+	if (cnt < 0)
 		return -EINVAL;
-	}
 
 	it->value = value;
-	it->cnt = cnt;
 
 	return 0;
 }
@@ -162,6 +161,14 @@ __bpf_kfunc s64 *bpf_iter_testmod_seq_next(struct bpf_iter_testmod_seq* it)
 	return &it->value;
 }
 
+__bpf_kfunc s64 bpf_iter_testmod_seq_value(int val, struct bpf_iter_testmod_seq* it__iter)
+{
+	if (it__iter->cnt < 0)
+		return 0;
+
+	return val + it__iter->value;
+}
+
 __bpf_kfunc void bpf_iter_testmod_seq_destroy(struct bpf_iter_testmod_seq *it)
 {
 	it->cnt = 0;
@@ -531,6 +538,7 @@ BTF_KFUNCS_START(bpf_testmod_common_kfunc_ids)
 BTF_ID_FLAGS(func, bpf_iter_testmod_seq_new, KF_ITER_NEW)
 BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY)
+BTF_ID_FLAGS(func, bpf_iter_testmod_seq_value)
 BTF_ID_FLAGS(func, bpf_kfunc_common_test)
 BTF_ID_FLAGS(func, bpf_kfunc_dynptr_test)
 BTF_ID_FLAGS(func, bpf_testmod_ctx_create, KF_ACQUIRE | KF_RET_NULL)
tools/testing/selftests/bpf/progs/iters_testmod_seq.c +50 -0

60 lines not on this page · raw message

Signature · 1 line
2.43.5
#9 lore raw
In reply to Andrii Nakryiko
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 11 lines · On Thu, 8 Aug 2024 16:22:27 -0700 you wrote:

Add support for passing BPF iterator state to any kfunc. Such kfunc has to declare such argument with valid struct bpf_iter_<type> * type and should use "__iter" suffix in argument name, following the established suffix-based convention. We add a simple test/demo iterator getter in bpf_testmod.

Andrii Nakryiko (3):
bpf: extract iterator argument type and name validation logic
bpf: allow passing struct bpf_iter_<type> as kfunc arguments selftests/bpf: test passing iterator to a kfunc

[...]

Here is the summary with links:
- [bpf-next,1/3] bpf: extract iterator argument type and name validation logic

    https://git.kernel.org/bpf/bpf-next/c/496ddd19a0fa
  - [bpf-next,2/3] bpf: allow passing struct bpf_iter_<type> as kfunc arguments
    https://git.kernel.org/bpf/bpf-next/c/baebe9aaba1e
  - [bpf-next,3/3] selftests/bpf: test passing iterator to a kfunc
    https://git.kernel.org/bpf/bpf-next/c/b0cd726f9a82

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html