Support storing struct cgroup * objects as kptrs
Subsystems (3)
cover Support storing struct cgroup * objects as kptrs
In [0], we added support for storing struct task_struct * objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.
As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.
[0]: https://lore.kernel.org/all/[email protected]/
David Vernet (4):
bpf: Enable cgroups to be used as kptrs selftests/bpf: Add cgroup kfunc / kptr selftests
bpf: Add bpf_cgroup_ancestor() kfunc selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc
kernel/bpf/helpers.c | 103 ++++++- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/cgrp_kfunc.c | 175 ++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_common.h | 72 +++++ .../selftests/bpf/progs/cgrp_kfunc_failure.c | 260 ++++++++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_success.c | 170 ++++++++++++ 6 files changed, 778 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c
Signature · 1 line
2.38.1
PATCH 1/4 bpf: Enable cgroups to be used as kptrs
Now that tasks can be used as kfuncs, and the PTR_TRUSTED flag is available for us to easily add basic acquire / get / release kfuncs, we can do the same for cgroups. This patch set adds the following kfuncs which enable using cgroups as kptrs:
struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp); struct cgroup *bpf_cgroup_kptr_get(struct cgroup **cgrpp); void bpf_cgroup_release(struct cgroup *cgrp);
A follow-on patch will add a selftest suite which validates these kfuncs.
--- kernel/bpf/helpers.c | 83 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 3 deletions(-)
kernel/bpf/helpers.c +80 -3
@@ -4,6 +4,7 @@#include <linux/bpf.h> #include <linux/btf.h> #include <linux/bpf-cgroup.h> +#include <linux/cgroup.h> #include <linux/rcupdate.h> #include <linux/random.h> #include <linux/smp.h>
@@ -1879,6 +1880,66 @@ void bpf_task_release(struct task_struct *p)put_task_struct_rcu_user (p); } +#ifdef CONFIG_CGROUPS +/** +* bpf_cgroup_acquire - Acquire a reference to a cgroup. A cgroup acquired by +* this kfunc which is not stored in a map as a kptr, must be released by +* calling bpf_cgroup_release(). +* @cgrp: The cgroup on which a reference is being acquired. +*/ +struct cgroup *bpf_cgroup_acquire (struct cgroup *cgrp) +{ +cgroup_get (cgrp); +return cgrp; +} + +/** +* bpf_cgroup_kptr_get - Acquire a reference on a struct cgroup kptr. A cgroup +* kptr acquired by this kfunc which is not subsequently stored in a map, must +* be released by calling bpf_cgroup_release(). +* @cgrpp: A pointer to a cgroup kptr on which a reference is being acquired. +*/ +struct cgroup *bpf_cgroup_kptr_get (struct cgroup **cgrpp) +{ +struct cgroup *cgrp; + +rcu_read_lock (); +/* Another context could remove the cgroup from the map and release it +* at any time, including after we've done the lookup above. This is +* safe because we're in an RCU read region, so the cgroup is +* guaranteed to remain valid until at least the rcu_read_unlock() +* below. +*/ + cgrp =READ_ONCE (*cgrpp); + +if (cgrp && !cgroup_tryget (cgrp)) +/* If the cgroup had been removed from the map and freed as +* described above, cgroup_tryget() will return false. The +* cgroup will be freed at some point after the current RCU gp +* has ended, so just return NULL to the user. +*/ + cgrp =NULL ; +rcu_read_unlock (); + +return cgrp; +} + +/** +* bpf_cgroup_release - Release the reference acquired on a struct cgroup *. +* If this kfunc is invoked in an RCU read region, the cgroup is guaranteed to +* not be freed until the current grace period has ended, even if its refcount +* drops to 0. +* @cgrp: The cgroup on which a reference is being released. +*/ +void bpf_cgroup_release (struct cgroup *cgrp) +{ +if (!cgrp) +return ; + +cgroup_put (cgrp); +} +#endif /* CONFIG_CGROUPS */ +void *bpf_cast_to_kern_ctx (void *obj) {return obj;
@@ -1904,6 +1965,12 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_task_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) + +#ifdef CONFIG_CGROUPS +BTF_ID_FLAGS(func, bpf_cgroup_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_cgroup_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_cgroup_release, KF_RELEASE) +#endif BTF_SET8_END (generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = {
@@ -1915,6 +1982,10 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { BTF_ID_LIST(generic_dtor_ids) BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release) +#ifdef CONFIG_CGROUPS +BTF_ID(struct, cgroup) +BTF_ID(func, bpf_cgroup_release) +#endif BTF_SET8_START(common_btf_ids) BTF_ID_FLAGS(func, bpf_cast_to_kern_ctx)
@@ -1928,12 +1999,18 @@ static const struct btf_kfunc_id_set common_kfunc_set = {static int __init kfunc_init (void ) { -int ret; +int ret, idx =0 ;const struct btf_id_dtor_kfunc generic_dtors[] = { { - .btf_id = generic_dtor_ids[0 ], - .kfunc_btf_id = generic_dtor_ids[1 ] + .btf_id = generic_dtor_ids[idx++], + .kfunc_btf_id = generic_dtor_ids[idx++] }, +#ifdef CONFIG_CGROUPS + { + .btf_id = generic_dtor_ids[idx++], + .kfunc_btf_id = generic_dtor_ids[idx++] + }, +#endif }; ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set);
Signature · 1 line
2.38.1
PATCH 3/4 bpf: Add bpf_cgroup_ancestor() kfunc
In reply to David Vernetstruct cgroup * objects have a variably sized struct cgroup *ancestors[] field which stores pointers to their ancestor cgroups. If using a cgroup as a kptr, it can be useful to access these ancestors, but doing so requires variable offset accesses for PTR_TO_BTF_ID, which is currently unsupported.
This is a very useful field to access for cgroup kptrs, as programs may wish to walk their ancestor cgroups when determining e.g. their proportional cpu.weight. So as to enable this functionality with cgroup kptrs before var_off is supported for PTR_TO_BTF_ID, this patch adds a bpf_cgroup_ancestor() kfunc which accesses the cgroup node on behalf of the caller, and acquires a reference on it. Once var_off is supported for PTR_TO_BTF_ID, and fields inside a struct can be marked as trusted so they retain the PTR_TRUSTED modifier when walked, this can be removed.
--- kernel/bpf/helpers.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
kernel/bpf/helpers.c +20 -0
@@ -1938,6 +1938,25 @@ void bpf_cgroup_release(struct cgroup *cgrp) cgroup_put(cgrp); } + +/** +* bpf_cgroup_ancestor - Perform a lookup on an entry in a cgroup's ancestor +* array. A cgroup returned by this kfunc which is not subsequently stored in a +* map, must be released by calling bpf_cgroup_release(). +* @cgrp: The cgroup for which we're performing a lookup. +* @level: The level of ancestor to look up. +*/ +struct cgroup *bpf_cgroup_ancestor (struct cgroup *cgrp,int level) +{ +struct cgroup *ancestor; + +if (level > cgrp->level || level <0 ) +return NULL ; + + ancestor = cgrp->ancestors[level]; +cgroup_get (ancestor); +return ancestor; +}#endif /* CONFIG_CGROUPS */ void *bpf_cast_to_kern_ctx (void *obj)
@@ -1970,6 +1989,7 @@ BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE) BTF_ID_FLAGS(func, bpf_cgroup_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_cgroup_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_cgroup_release, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_cgroup_ancestor, KF_ACQUIRE | KF_TRUSTED_ARGS | KF_RET_NULL)#endif BTF_SET8_END (generic_btf_ids)
Signature · 1 line
2.38.1
PATCH 2/4 selftests/bpf: Add cgroup kfunc / kptr selftests
In reply to David VernetThis patch adds a selftest suite to validate the cgroup kfuncs that were added in the prior patch.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/cgrp_kfunc.c | 174 ++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_common.h | 71 +++++ .../selftests/bpf/progs/cgrp_kfunc_failure.c | 260 ++++++++++++++++++ .../selftests/bpf/progs/cgrp_kfunc_success.c | 125 +++++++++ 5 files changed, 631 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -10,6 +10,7 @@ bpf_nf # JIT does not support calling kernel f bpf_tcp_ca # JIT does not support calling kernel function (kfunc) cb_refs # expected error message unexpected error: -524 (trampoline) cgroup_hierarchical_stats # JIT does not support calling kernel function (kfunc) +cgrp_kfunc # JIT does not support calling kernel function cgrp_local_storage # prog_attach unexpected error: -524 (trampoline) core_read_macros # unknown func bpf_probe_read#4 (overlapping) d_path # failed to auto-attach program 'prog_stat': -524 (trampoline)
tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c new file +174 -0
174 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h new file +71 -0
71 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c new file +260 -0
260 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c new file +125 -0
125 lines not on this page · raw message
Signature · 1 line
2.38.1
PATCH 4/4 selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc
In reply to David Vernetbpf_cgroup_ancestor() allows BPF programs to access the ancestor of a struct cgroup *. This patch adds selftests that validate its expected behavior.
--- .../selftests/bpf/prog_tests/cgrp_kfunc.c | 1 + .../selftests/bpf/progs/cgrp_kfunc_common.h | 1 + .../selftests/bpf/progs/cgrp_kfunc_success.c | 45 +++++++++++++++++++ 3 files changed, 47 insertions(+)
tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c +1 -0
@@ -86,6 +86,7 @@ static const char * const success_tests[] = {"test_cgrp_acquire_leave_in_map" ,"test_cgrp_xchg_release" ,"test_cgrp_get_release" , + "test_cgrp_get_ancestors", }; static struct {
tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h +1 -0
@@ -23,6 +23,7 @@ struct hash_map {struct cgroup *bpf_cgroup_acquire (struct cgroup *p) __ksym;struct cgroup *bpf_cgroup_kptr_get (struct cgroup **pp) __ksym; void bpf_cgroup_release(struct cgroup *p) __ksym; +struct cgroup *bpf_cgroup_ancestor (struct cgroup *cgrp,int level) __ksym;static inline struct __cgrps_kfunc_map_value *cgrps_kfunc_map_value_lookup (struct cgroup *cgrp) {
tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c +45 -0
48 lines not on this page · raw message
Signature · 1 line
2.38.1
Quoted 9 lines · David Vernet
In [0], we added support for storing struct
task_struct* objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.
Quoted 7 lines
[0]: https://lore.kernel.org/all/[email protected]/
David Vernet (4):
bpf: Enable cgroups to be used as kptrs selftests/bpf: Add cgroup kfunc / kptr selftests
bpf: Addbpf_cgroup_ancestor()kfunc selftests/bpf: Add selftests forbpf_cgroup_ancestor()kfunc
From cgroup POV:
Thanks.
Signature · 1 line
tejun
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 10 lines · On Mon, 21 Nov 2022 23:54:54 -0600 you wrote:
In [0], we added support for storing struct
task_struct* objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.
[...]
Here is the summary with links:
- [bpf-next,1/4] bpf: Enable cgroups to be used as kptrs
https://git.kernel.org/bpf/bpf-next/c/fda01efc6160 - [bpf-next,2/4] selftests/bpf: Add cgroup kfunc / kptr selftests https://git.kernel.org/bpf/bpf-next/c/f583ddf15e57 - [bpf-next,3/4] bpf: Add bpf_cgroup_ancestor() kfunc https://git.kernel.org/bpf/bpf-next/c/5ca786707829 - [bpf-next,4/4] selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc https://git.kernel.org/bpf/bpf-next/c/227a89cf5041
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html