Skip to contentSkip to conversation

Support storing struct cgroup * objects as kptrs

PATCH Applied David Vernet 3 participants 7 messages Nov 22, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Support storing struct cgroup * objects as kptrs

In [0], we added support for storing struct task_struct * objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.

As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.

[0]: https://lore.kernel.org/all/[email protected]/

David Vernet (4):
bpf: Enable cgroups to be used as kptrs selftests/bpf: Add cgroup kfunc / kptr selftests
bpf: Add bpf_cgroup_ancestor() kfunc selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc

 kernel/bpf/helpers.c                          | 103 ++++++-
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/cgrp_kfunc.c     | 175 ++++++++++++
 .../selftests/bpf/progs/cgrp_kfunc_common.h   |  72 +++++
 .../selftests/bpf/progs/cgrp_kfunc_failure.c  | 260 ++++++++++++++++++
 .../selftests/bpf/progs/cgrp_kfunc_success.c  | 170 ++++++++++++
 6 files changed, 778 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c
Signature · 1 line
2.38.1
#1 lore raw

PATCH 1/4 bpf: Enable cgroups to be used as kptrs

Now that tasks can be used as kfuncs, and the PTR_TRUSTED flag is available for us to easily add basic acquire / get / release kfuncs, we can do the same for cgroups. This patch set adds the following kfuncs which enable using cgroups as kptrs:

struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp); struct cgroup *bpf_cgroup_kptr_get(struct cgroup **cgrpp); void bpf_cgroup_release(struct cgroup *cgrp);

A follow-on patch will add a selftest suite which validates these kfuncs.

---
 kernel/bpf/helpers.c | 83 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 3 deletions(-)
kernel/bpf/helpers.c +80 -3
index 9e8a5557ea8d..420a4251cccc 100644
@@ -4,6 +4,7 @@
 #include <linux/bpf.h>
 #include <linux/btf.h>
 #include <linux/bpf-cgroup.h>
+#include <linux/cgroup.h>
 #include <linux/rcupdate.h>
 #include <linux/random.h>
 #include <linux/smp.h>
@@ -1879,6 +1880,66 @@ void bpf_task_release(struct task_struct *p)
 	put_task_struct_rcu_user(p);
 }
 
+#ifdef CONFIG_CGROUPS
+/**
+ * bpf_cgroup_acquire - Acquire a reference to a cgroup. A cgroup acquired by
+ * this kfunc which is not stored in a map as a kptr, must be released by
+ * calling bpf_cgroup_release().
+ * @cgrp: The cgroup on which a reference is being acquired.
+ */
+struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp)
+{
+	cgroup_get(cgrp);
+	return cgrp;
+}
+
+/**
+ * bpf_cgroup_kptr_get - Acquire a reference on a struct cgroup kptr. A cgroup
+ * kptr acquired by this kfunc which is not subsequently stored in a map, must
+ * be released by calling bpf_cgroup_release().
+ * @cgrpp: A pointer to a cgroup kptr on which a reference is being acquired.
+ */
+struct cgroup *bpf_cgroup_kptr_get(struct cgroup **cgrpp)
+{
+	struct cgroup *cgrp;
+
+	rcu_read_lock();
+	/* Another context could remove the cgroup from the map and release it
+	 * at any time, including after we've done the lookup above. This is
+	 * safe because we're in an RCU read region, so the cgroup is
+	 * guaranteed to remain valid until at least the rcu_read_unlock()
+	 * below.
+	 */
+	cgrp = READ_ONCE(*cgrpp);
+
+	if (cgrp && !cgroup_tryget(cgrp))
+		/* If the cgroup had been removed from the map and freed as
+		 * described above, cgroup_tryget() will return false. The
+		 * cgroup will be freed at some point after the current RCU gp
+		 * has ended, so just return NULL to the user.
+		 */
+		cgrp = NULL;
+	rcu_read_unlock();
+
+	return cgrp;
+}
+
+/**
+ * bpf_cgroup_release - Release the reference acquired on a struct cgroup *.
+ * If this kfunc is invoked in an RCU read region, the cgroup is guaranteed to
+ * not be freed until the current grace period has ended, even if its refcount
+ * drops to 0.
+ * @cgrp: The cgroup on which a reference is being released.
+ */
+void bpf_cgroup_release(struct cgroup *cgrp)
+{
+	if (!cgrp)
+		return;
+
+	cgroup_put(cgrp);
+}
+#endif /* CONFIG_CGROUPS */
+
 void *bpf_cast_to_kern_ctx(void *obj)
 {
 	return obj;
@@ -1904,6 +1965,12 @@ BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_task_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE)
+
+#ifdef CONFIG_CGROUPS
+BTF_ID_FLAGS(func, bpf_cgroup_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
+BTF_ID_FLAGS(func, bpf_cgroup_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_cgroup_release, KF_RELEASE)
+#endif
 BTF_SET8_END(generic_btf_ids)
 
 static const struct btf_kfunc_id_set generic_kfunc_set = {
@@ -1915,6 +1982,10 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {
 BTF_ID_LIST(generic_dtor_ids)
 BTF_ID(struct, task_struct)
 BTF_ID(func, bpf_task_release)
+#ifdef CONFIG_CGROUPS
+BTF_ID(struct, cgroup)
+BTF_ID(func, bpf_cgroup_release)
+#endif
 
 BTF_SET8_START(common_btf_ids)
 BTF_ID_FLAGS(func, bpf_cast_to_kern_ctx)
@@ -1928,12 +1999,18 @@ static const struct btf_kfunc_id_set common_kfunc_set = {
 
 static int __init kfunc_init(void)
 {
-	int ret;
+	int ret, idx = 0;
 	const struct btf_id_dtor_kfunc generic_dtors[] = {
 		{
-			.btf_id       = generic_dtor_ids[0],
-			.kfunc_btf_id = generic_dtor_ids[1]
+			.btf_id       = generic_dtor_ids[idx++],
+			.kfunc_btf_id = generic_dtor_ids[idx++]
 		},
+#ifdef CONFIG_CGROUPS
+		{
+			.btf_id       = generic_dtor_ids[idx++],
+			.kfunc_btf_id = generic_dtor_ids[idx++]
+		},
+#endif
 	};
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set);
Signature · 1 line
2.38.1
#2 lore raw

PATCH 2/4 selftests/bpf: Add cgroup kfunc / kptr selftests

In reply to David Vernet

This patch adds a selftest suite to validate the cgroup kfuncs that were added in the prior patch.

---
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/cgrp_kfunc.c     | 174 ++++++++++++
 .../selftests/bpf/progs/cgrp_kfunc_common.h   |  71 +++++
 .../selftests/bpf/progs/cgrp_kfunc_failure.c  | 260 ++++++++++++++++++
 .../selftests/bpf/progs/cgrp_kfunc_success.c  | 125 +++++++++
 5 files changed, 631 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c
 create mode 100644 tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
index 12cf2159975e..b9a3d80204c6 100644
@@ -10,6 +10,7 @@ bpf_nf                                   # JIT does not support calling kernel f
 bpf_tcp_ca                               # JIT does not support calling kernel function                                (kfunc)
 cb_refs                                  # expected error message unexpected error: -524                               (trampoline)
 cgroup_hierarchical_stats                # JIT does not support calling kernel function                                (kfunc)
+cgrp_kfunc                               # JIT does not support calling kernel function
 cgrp_local_storage                       # prog_attach unexpected error: -524                                          (trampoline)
 core_read_macros                         # unknown func bpf_probe_read#4                                               (overlapping)
 d_path                                   # failed to auto-attach program 'prog_stat': -524                             (trampoline)
tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c new file +174 -0

174 lines not on this page · raw message

tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h new file +71 -0

71 lines not on this page · raw message

tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c new file +260 -0

260 lines not on this page · raw message

tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c new file +125 -0

125 lines not on this page · raw message

Signature · 1 line
2.38.1
#3 lore raw

PATCH 3/4 bpf: Add bpf_cgroup_ancestor() kfunc

In reply to David Vernet

struct cgroup * objects have a variably sized struct cgroup *ancestors[] field which stores pointers to their ancestor cgroups. If using a cgroup as a kptr, it can be useful to access these ancestors, but doing so requires variable offset accesses for PTR_TO_BTF_ID, which is currently unsupported.

This is a very useful field to access for cgroup kptrs, as programs may wish to walk their ancestor cgroups when determining e.g. their proportional cpu.weight. So as to enable this functionality with cgroup kptrs before var_off is supported for PTR_TO_BTF_ID, this patch adds a bpf_cgroup_ancestor() kfunc which accesses the cgroup node on behalf of the caller, and acquires a reference on it. Once var_off is supported for PTR_TO_BTF_ID, and fields inside a struct can be marked as trusted so they retain the PTR_TRUSTED modifier when walked, this can be removed.

---
 kernel/bpf/helpers.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
kernel/bpf/helpers.c +20 -0
index 420a4251cccc..e4e9db301db5 100644
@@ -1938,6 +1938,25 @@ void bpf_cgroup_release(struct cgroup *cgrp)
 
 	cgroup_put(cgrp);
 }
+
+/**
+ * bpf_cgroup_ancestor - Perform a lookup on an entry in a cgroup's ancestor
+ * array. A cgroup returned by this kfunc which is not subsequently stored in a
+ * map, must be released by calling bpf_cgroup_release().
+ * @cgrp: The cgroup for which we're performing a lookup.
+ * @level: The level of ancestor to look up.
+ */
+struct cgroup *bpf_cgroup_ancestor(struct cgroup *cgrp, int level)
+{
+	struct cgroup *ancestor;
+
+	if (level > cgrp->level || level < 0)
+		return NULL;
+
+	ancestor = cgrp->ancestors[level];
+	cgroup_get(ancestor);
+	return ancestor;
+}
 #endif /* CONFIG_CGROUPS */
 
 void *bpf_cast_to_kern_ctx(void *obj)
@@ -1970,6 +1989,7 @@ BTF_ID_FLAGS(func, bpf_task_release, KF_RELEASE)
 BTF_ID_FLAGS(func, bpf_cgroup_acquire, KF_ACQUIRE | KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_cgroup_kptr_get, KF_ACQUIRE | KF_KPTR_GET | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_cgroup_release, KF_RELEASE)
+BTF_ID_FLAGS(func, bpf_cgroup_ancestor, KF_ACQUIRE | KF_TRUSTED_ARGS | KF_RET_NULL)
 #endif
 BTF_SET8_END(generic_btf_ids)
 
Signature · 1 line
2.38.1
#4 lore raw

PATCH 4/4 selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc

In reply to David Vernet

bpf_cgroup_ancestor() allows BPF programs to access the ancestor of a struct cgroup *. This patch adds selftests that validate its expected behavior.

---
 .../selftests/bpf/prog_tests/cgrp_kfunc.c     |  1 +
 .../selftests/bpf/progs/cgrp_kfunc_common.h   |  1 +
 .../selftests/bpf/progs/cgrp_kfunc_success.c  | 45 +++++++++++++++++++
 3 files changed, 47 insertions(+)
tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c +1 -0
index a59b166bbcc4..973f0c5af965 100644
@@ -86,6 +86,7 @@ static const char * const success_tests[] = {
 	"test_cgrp_acquire_leave_in_map",
 	"test_cgrp_xchg_release",
 	"test_cgrp_get_release",
+	"test_cgrp_get_ancestors",
 };
 
 static struct {
tools/testing/selftests/bpf/progs/cgrp_kfunc_common.h +1 -0
index 3f18def0e45c..7d30855bfe78 100644
@@ -23,6 +23,7 @@ struct hash_map {
 struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym;
 struct cgroup *bpf_cgroup_kptr_get(struct cgroup **pp) __ksym;
 void bpf_cgroup_release(struct cgroup *p) __ksym;
+struct cgroup *bpf_cgroup_ancestor(struct cgroup *cgrp, int level) __ksym;
 
 static inline struct __cgrps_kfunc_map_value *cgrps_kfunc_map_value_lookup(struct cgroup *cgrp)
 {
tools/testing/selftests/bpf/progs/cgrp_kfunc_success.c +45 -0

48 lines not on this page · raw message

Signature · 1 line
2.38.1
Acked-by on the v1 series
#5 lore raw
In reply to David Vernet
Quoted 9 lines · David Vernet

In [0], we added support for storing struct task_struct * objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.

As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.

Quoted 7 lines

[0]: https://lore.kernel.org/all/[email protected]/

David Vernet (4):
bpf: Enable cgroups to be used as kptrs selftests/bpf: Add cgroup kfunc / kptr selftests
bpf: Add bpf_cgroup_ancestor() kfunc selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc

From cgroup POV:

Thanks.

Signature · 1 line
tejun
#6 lore raw
In reply to David Vernet
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 10 lines · On Mon, 21 Nov 2022 23:54:54 -0600 you wrote:

In [0], we added support for storing struct task_struct * objects as kptrs. This patch set extends this effort to also include storing struct cgroup * object as kptrs.

As with tasks, there are many possible use cases for storing cgroups in maps. During tracing, for example, it could be useful to query cgroup statistics such as PSI averages, or tracking which tasks are migrating to and from the cgroup.

[...]

Here is the summary with links:
- [bpf-next,1/4] bpf: Enable cgroups to be used as kptrs

    https://git.kernel.org/bpf/bpf-next/c/fda01efc6160
  - [bpf-next,2/4] selftests/bpf: Add cgroup kfunc / kptr selftests
    https://git.kernel.org/bpf/bpf-next/c/f583ddf15e57
  - [bpf-next,3/4] bpf: Add bpf_cgroup_ancestor() kfunc
    https://git.kernel.org/bpf/bpf-next/c/5ca786707829
  - [bpf-next,4/4] selftests/bpf: Add selftests for bpf_cgroup_ancestor() kfunc
    https://git.kernel.org/bpf/bpf-next/c/227a89cf5041

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html