Enable struct_ops programs to be sleepable
Subsystems (6)
v4 cover Enable struct_ops programs to be sleepable
This is part 4 of https://lore.kernel.org/bpf/[email protected]/
Part 3: https://lore.kernel.org/all/[email protected]/
Part 2: https://lore.kernel.org/all/[email protected]/
Changelog:
----------
v3 -> v4:
- Fix accidental typo in name of dummy_st_ops introduced in v2, moving it back to dummy_st_ops from dummy_st_ops_success. Should fix s390x testruns.
v2 -> v3:
- Don't call a KF_SLEEPABLE kfunc from the dummy_st_ops testsuite, and remove the newly added bpf_kfunc_call_test_sleepable() test kfunc (Martin).
- Include vmlinux.h from progs/dummy_st_ops_success.c (previously progs/dummy_st_ops.c) rather than manually defining struct bpf_dummy_ops_state and struct bpf_dummy_ops. (Martin).
- Fix a typo added to prog_tests/dummy_st_ops.c in a previous version:
s/trace_dummy_st_ops_success__open/trace_dummy_st_ops__open.
v1 -> v2:
- Add support for specifying sleepable struct_ops programs with struct_ops.s in libbpf (Alexei).
- Move failure test case into new dummy_st_ops_fail.c prog file.
- Update test_dummy_sleepable() to use struct_ops.s instead of manually setting prog flags. Also remove open_load_skel() helper which is no longer needed.
- Fix verifier tests to expect new sleepable prog failure message.
David Vernet (4):
bpf: Allow BPF_PROG_TYPE_STRUCT_OPS programs to be sleepable
libbpf: Support sleepable struct_ops.s section
bpf: Pass const struct bpf_prog * to .check_member bpf/selftests: Verify struct_ops prog sleepable behavior
include/linux/bpf.h | 4 +-
kernel/bpf/verifier.c | 7 +--
net/bpf/bpf_dummy_struct_ops.c | 18 +++++++
net/ipv4/bpf_tcp_ca.c | 3 +-
tools/lib/bpf/libbpf.c | 1 +
.../selftests/bpf/prog_tests/dummy_st_ops.c | 52 ++++++++++++++-----
.../selftests/bpf/progs/dummy_st_ops_fail.c | 27 ++++++++++
...{dummy_st_ops.c => dummy_st_ops_success.c} | 19 +++----
.../selftests/bpf/verifier/sleepable.c | 2 +-
9 files changed, 103 insertions(+), 30 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/dummy_st_ops_fail.c
rename tools/testing/selftests/bpf/progs/{dummy_st_ops.c => dummy_st_ops_success.c} (72%)Signature · 1 line
2.39.0
PATCH v4 1/4 bpf: Allow BPF_PROG_TYPE_STRUCT_OPS programs to be sleepable
BPF struct_ops programs currently cannot be marked as sleepable. This need not be the case -- struct_ops programs can be sleepable, and e.g. invoke kfuncs that export the KF_SLEEPABLE flag. So as to allow future struct_ops programs to invoke such kfuncs, this patch updates the verifier to allow struct_ops programs to be sleepable. A follow-on patch will add support to libbpf for specifying struct_ops.s as a sleepable struct_ops program, and then another patch will add testcases to the dummy_st_ops selftest suite which test sleepable struct_ops behavior.
--- kernel/bpf/verifier.c | 5 +++-- tools/testing/selftests/bpf/verifier/sleepable.c | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)
kernel/bpf/verifier.c +3 -2
@@ -17114,7 +17114,8 @@ static bool can_be_sleepable(struct bpf_prog *prog) } }return prog->type ==BPF_PROG_TYPE_LSM || - prog->type ==BPF_PROG_TYPE_KPROBE ;/* only for uprobes */ + prog->type ==BPF_PROG_TYPE_KPROBE /* only for uprobes */ || + prog->type ==BPF_PROG_TYPE_STRUCT_OPS ; }static int check_attach_btf_id (struct bpf_verifier_env *env)
@@ -17136,7 +17137,7 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) }if (prog->aux->sleepable && !can_be_sleepable (prog)) { -verbose (env,"Only fentry/fexit/fmod_ret, lsm, iter and uprobe programs can be sleepable\n" ); +verbose (env,"Only fentry/fexit/fmod_ret, lsm, iter, uprobe, and struct_ops programs can be sleepable\n" );return -EINVAL ; }
tools/testing/selftests/bpf/verifier/sleepable.c +1 -1
@@ -85,7 +85,7 @@ .expected_attach_type = BPF_TRACE_RAW_TP, .kfunc = "sched_switch", .result = REJECT, - .errstr = "Only fentry/fexit/fmod_ret, lsm, iter and uprobe programs can be sleepable", + .errstr = "Only fentry/fexit/fmod_ret, lsm, iter, uprobe, and struct_ops programs can be sleepable", .flags = BPF_F_SLEEPABLE, .runs = -1, },
Signature · 1 line
2.39.0
PATCH v4 2/4 libbpf: Support sleepable struct_ops.s section
In reply to David VernetIn a prior change, the verifier was updated to support sleepable BPF_PROG_TYPE_STRUCT_OPS programs. A caller could set the program as sleepable with bpf_program__set_flags(), but it would be more ergonomic and more in-line with other sleepable program types if we supported suffixing a struct_ops section name with .s to indicate that it's sleepable.
--- tools/lib/bpf/libbpf.c | 1 + 1 file changed, 1 insertion(+)
tools/lib/bpf/libbpf.c +1 -0
@@ -8605,6 +8605,7 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("cgroup/setsockopt" ,CGROUP_SOCKOPT ,BPF_CGROUP_SETSOCKOPT ,SEC_ATTACHABLE ),SEC_DEF ("cgroup/dev" ,CGROUP_DEVICE ,BPF_CGROUP_DEVICE ,SEC_ATTACHABLE_OPT ),SEC_DEF ("struct_ops+" ,STRUCT_OPS ,0 ,SEC_NONE ), +SEC_DEF ("struct_ops.s+" ,STRUCT_OPS ,0 ,SEC_SLEEPABLE ), SEC_DEF("sk_lookup", SK_LOOKUP, BPF_SK_LOOKUP, SEC_ATTACHABLE), };
Signature · 1 line
2.39.0
PATCH v4 3/4 bpf: Pass const struct bpf_prog * to .check_member
In reply to David VernetThe .check_member field of struct bpf_struct_ops is currently passed the member's btf_type via const struct btf_type *t, and a const struct btf_member *member. This allows the struct_ops implementation to check whether e.g. an ops is supported, but it would be useful to also enforce that the struct_ops prog being loaded for that member has other qualities, like being sleepable (or not). This patch therefore updates the .check_member() callback to also take a const struct bpf_prog *prog argument.
--- include/linux/bpf.h | 3 ++- kernel/bpf/verifier.c | 2 +- net/ipv4/bpf_tcp_ca.c | 3 ++- 3 files changed, 5 insertions(+), 3 deletions(-)
include/linux/bpf.h +2 -1
@@ -1422,7 +1422,8 @@ struct bpf_struct_ops {const struct bpf_verifier_ops *verifier_ops;int (*init)(struct btf *btf);int (*check_member)(const struct btf_type *t, -const struct btf_member *member); +const struct btf_member *member, +const struct bpf_prog *prog);int (*init_member)(const struct btf_type *t,const struct btf_member *member,void *kdata,const void *udata);
kernel/bpf/verifier.c +1 -1
@@ -16792,7 +16792,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) }if (st_ops->check_member) { -int err = st_ops->check_member (t, member); +int err = st_ops->check_member (t, member, prog);if (err) { verbose(env, "attach to unsupported member %s of struct %s\n",
net/ipv4/bpf_tcp_ca.c +2 -1
@@ -248,7 +248,8 @@ static int bpf_tcp_ca_init_member(const struct btf_type *t, }static int bpf_tcp_ca_check_member (const struct btf_type *t, -const struct btf_member *member) +const struct btf_member *member, +const struct bpf_prog *prog) {if (is_unsupported (__btf_member_bit_offset (t, member) /8 ))return -ENOTSUPP ;
Signature · 1 line
2.39.0
PATCH v4 4/4 bpf/selftests: Verify struct_ops prog sleepable behavior
In reply to David VernetIn a set of prior changes, we added the ability for struct_ops programs to be sleepable. This patch enhances the dummy_st_ops selftest suite to validate this behavior by adding a new sleepable struct_ops entry to dummy_st_ops.
---
include/linux/bpf.h | 1 +
net/bpf/bpf_dummy_struct_ops.c | 18 +++++++
.../selftests/bpf/prog_tests/dummy_st_ops.c | 52 ++++++++++++++-----
.../selftests/bpf/progs/dummy_st_ops_fail.c | 27 ++++++++++
...{dummy_st_ops.c => dummy_st_ops_success.c} | 19 +++----
5 files changed, 93 insertions(+), 24 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/dummy_st_ops_fail.c
rename tools/testing/selftests/bpf/progs/{dummy_st_ops.c => dummy_st_ops_success.c} (72%)include/linux/bpf.h +1 -0
@@ -1474,6 +1474,7 @@ struct bpf_dummy_ops {int (*test_1)(struct bpf_dummy_ops_state *cb);int (*test_2)(struct bpf_dummy_ops_state *cb,int a1,unsigned short a2,char a3,unsigned long a4); +int (*test_sleepable)(struct bpf_dummy_ops_state *cb); }; int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
net/bpf/bpf_dummy_struct_ops.c +18 -0
@@ -154,6 +154,23 @@ static bool bpf_dummy_ops_is_valid_access(int off, int size,return bpf_tracing_btf_ctx_access (off, size, type, prog, info); } +static int bpf_dummy_ops_check_member (const struct btf_type *t, +const struct btf_member *member, +const struct bpf_prog *prog) +{ +u32 moff =__btf_member_bit_offset (t, member) /8 ; + +switch (moff) { +case offsetof(struct bpf_dummy_ops , test_sleepable): +break ; +default : +if (prog->aux->sleepable) +return -EINVAL ; + } + +return 0 ; +} + static int bpf_dummy_ops_btf_struct_access(struct bpf_verifier_log *log, const struct bpf_reg_state *reg, int off, int size, enum bpf_access_type atype,
@@ -208,6 +225,7 @@ static void bpf_dummy_unreg(void *kdata)struct bpf_struct_ops bpf_bpf_dummy_ops = { .verifier_ops = &bpf_dummy_verifier_ops, .init = bpf_dummy_init, + .check_member = bpf_dummy_ops_check_member, .init_member = bpf_dummy_init_member, .reg = bpf_dummy_reg, .unreg = bpf_dummy_unreg,
tools/testing/selftests/bpf/prog_tests/dummy_st_ops.c +39 -13
120 lines not on this page · raw message
tools/testing/selftests/bpf/progs/dummy_st_ops_fail.c new file +27 -0
@@ -0,0 +1,27 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_tracing.h> + +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +SEC("struct_ops.s/test_2") +__failure __msg("attach to unsupported member test_2 of struct bpf_dummy_ops") +int BPF_PROG(test_unsupported_field_sleepable, + struct bpf_dummy_ops_state *state, int a1, unsigned short a2, + char a3, unsigned long a4) +{ +/* Tries to mark an unsleepable field in struct bpf_dummy_ops as sleepable. */ +return 0 ; +} + +SEC(".struct_ops") +struct bpf_dummy_ops dummy_1 = { + .test_1 =NULL , + .test_2 = (void *)test_unsupported_field_sleepable, + .test_sleepable = (void *)NULL , +};
tools/testing/selftests/bpf/progs/dummy_st_ops_success.c renamed from tools/testing/selftests/bpf/progs/dummy_st_ops.c +8 -11
@@ -1,19 +1,9 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright (C) 2021. Huawei Technologies Co., Ltd */ -#include <linux/bpf.h> +#include "vmlinux.h" #include <bpf/bpf_helpers.h> #include <bpf/bpf_tracing.h> -struct bpf_dummy_ops_state { -int val; -} __attribute__((preserve_access_index)); - -struct bpf_dummy_ops { -int (*test_1)(struct bpf_dummy_ops_state *state); -int (*test_2)(struct bpf_dummy_ops_state *state,int a1,unsigned short a2, -char a3,unsigned long a4); -}; - char _license[] SEC("license") = "GPL";SEC ("struct_ops/test_1" )
@@ -43,8 +33,15 @@ int BPF_PROG(test_2, struct bpf_dummy_ops_state *state, int a1, unsigned short areturn 0 ; } +SEC("struct_ops.s/test_sleepable") +int BPF_PROG (test_sleepable ,struct bpf_dummy_ops_state *state) +{ +return 0 ; +} + SEC(".struct_ops")struct bpf_dummy_ops dummy_1 = { .test_1 = (void *)test_1, .test_2 = (void *)test_2, + .test_sleepable = (void *)test_sleepable, };
Signature · 1 line
2.39.0
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Wed, 25 Jan 2023 10:47:31 -0600 you wrote:
This is part 4 of https://lore.kernel.org/bpf/[email protected]/
Part 3: https://lore.kernel.org/all/[email protected]/
Part 2: https://lore.kernel.org/all/[email protected]/Changelog:
[...]
Here is the summary with links:
- [bpf-next,v4,1/4] bpf: Allow BPF_PROG_TYPE_STRUCT_OPS programs to be sleepable
https://git.kernel.org/bpf/bpf-next/c/1e12d3ef47d2 - [bpf-next,v4,2/4] libbpf: Support sleepable struct_ops.s section https://git.kernel.org/bpf/bpf-next/c/913b2255c3d8 - [bpf-next,v4,3/4] bpf: Pass const struct bpf_prog * to .check_member https://git.kernel.org/bpf/bpf-next/c/51a52a29ebaa - [bpf-next,v4,4/4] bpf/selftests: Verify struct_ops prog sleepable behavior https://git.kernel.org/bpf/bpf-next/c/7dd880592a88
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html