bpf: cgroup: support writing and freezing cgroups from BPF
Subsystems (4)
v2 cover bpf: cgroup: support writing and freezing cgroups from BPF
This patch series add support to write cgroup interfaces from BPF.
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
The writing happens via kernfs nodes and the cgroup must be on the default hierarchy. It implements the requests and feedback from v1 [1] where now we use a unified path for cgroup user space and BPF writing.
So I want to validate that this is the right approach first.
Todo:
* Limit size of data to be written.
* Further tests.
* Add cgroup kill support.
# RFC v1 -> v2
* Implemented Alexei and Tejun requests [1].
* Unified path where user space or BPF writing end up taking directly a kernfs_node with an example on the "cgroup.freeze" interface.
[1] https://lore.kernel.org/bpf/[email protected]/
Djalal Harouni (3):
kernfs: cgroup: support writing cgroup interfaces from a kernfs node
bpf: cgroup: Add BPF Kfunc to write cgroup interfaces selftests/bpf: add selftest for bpf_cgroup_write_interface
include/linux/cgroup.h | 3 ++ kernel/bpf/helpers.c | 45 +++++ kernel/cgroup/cgroup.c | 102 +++++++ tools/testing/selftests/bpf/prog_tests/task_freeze_cgroup.c | 172 ++++++++++++ tools/testing/selftests/bpf/progs/test_task_freeze_cgroup.c | 155 ++++++++++ 5 files changed, 471 insertions(+), 6 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_freeze_cgroup.c create mode 100644 tools/testing/selftests/bpf/progs/test_task_freeze_cgroup.c
Signature · 1 line
2.34.1
RFC PATCH v2 1/3 kernfs: cgroup: support writing cgroup interfaces from a kernfs node
Freezing a cgroup of a task from BPF is better than user space which could be too late and is subject to races. To achieve this allow writing to cgroup core interfaces from BPF by adding a new kfunc helper that take a kernfs node directly.
Currently only writing to "cgroup.freeze" on the default hierarchy is allowed. The writing goes directly via a kernfs_node which allows to share the same path as if a kernfs_node was opened from userspace.
--- include/linux/cgroup.h | 3 ++ kernel/cgroup/cgroup.c | 102 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 99 insertions(+), 6 deletions(-)
include/linux/cgroup.h +3 -0
@@ -125,6 +125,9 @@ int cgroup_rm_cftypes(struct cftype *cfts);void cgroup_file_notify (struct cgroup_file *cfile);void cgroup_file_show (struct cgroup_file *cfile,bool show); +ssize_t cgroup_kn_interface_write (struct kernfs_node *kn,const char *name__str, +const char *buf,size_t nbytes,loff_t off); +int cgroupstats_build (struct cgroupstats *stats,struct dentry *dentry);int proc_cgroup_show (struct seq_file *m,struct pid_namespace *ns,struct pid *pid,struct task_struct *tsk);
kernel/cgroup/cgroup.c +96 -6
137 lines not on this page · raw message
Signature · 1 line
2.43.0
RFC PATCH v2 2/3 bpf: cgroup: Add BPF Kfunc to write and freeze a cgroup
In reply to Djalal HarouniAdd bpf_cgroup_write_interface() kfunc that writes to a cgroup interface. Takes a cgroup on the default hierarchy as argument, and writes to the specified interface file of that cgroup.
Freezing a cgroup of a task from BPF is better than user space which could be too late and is subject to races. Hence, add support for writing to "cgroup.freeze" interface using the mentioned bpf kfunc.
Planned users of this feature are: systemd and BPF tools. Taking the freezing example, we could freeze a cgroup hierarchy on suspicious activity for a more thorough analysis. The cgroup hierarchies could be system services, user sessions, K8s pods or containers.
--- kernel/bpf/helpers.c | 45 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
kernel/bpf/helpers.c +45 -0
@@ -2605,6 +2605,50 @@ bpf_task_get_cgroup1(struct task_struct *task, int hierarchy_id)return NULL ;return cgrp; } + +#define BPF_CGROUP_MAX_WRITE ((1UL << 24) - 1) + +/** +* bpf_cgroup_write_interface - Writes to a cgroup interface file. +* @cgrp: The target cgroup +* @name__str: name of the cgroup core interface file +* @value_p: value to write +* @off: offset +* +* Return: number of bytes written on success, a negative value on error. +*/ +__bpf_kfunc int +bpf_cgroup_write_interface (struct cgroup *cgrp,const char *name__str, +const struct bpf_dynptr *value_p,loff_t off) +{ +struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p; +struct kernfs_node *kn; +const void *value; +u32 value_len; +int ret; + + value_len =__bpf_dynptr_size (value_ptr); +if (!value_len) +return 0 ; + +if (value_len >BPF_CGROUP_MAX_WRITE ) +return -E2BIG ; + + value =__bpf_dynptr_data (value_ptr, value_len); +if (!value) +return -EINVAL ; + +rcu_read_lock (); + kn = cgrp->kn; +rcu_read_unlock (); + +kernfs_get (kn); + ret =cgroup_kn_interface_write (kn, name__str, value, value_len, off); +kernfs_put (kn); + +return ret; +} +#endif /* CONFIG_CGROUPS */ /**
@@ -3736,6 +3780,7 @@ BTF_ID_FLAGS(func, bpf_cgroup_ancestor, KF_ACQUIRE | KF_RCU | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_cgroup_from_id, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_task_under_cgroup, KF_RCU) BTF_ID_FLAGS(func, bpf_task_get_cgroup1, KF_ACQUIRE | KF_RCU | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_cgroup_write_interface, KF_TRUSTED_ARGS | KF_SLEEPABLE)#endif BTF_ID_FLAGS(func, bpf_task_from_pid, KF_ACQUIRE | KF_RET_NULL)BTF_ID_FLAGS (func, bpf_task_from_vpid,KF_ACQUIRE |KF_RET_NULL )
Signature · 1 line
2.43.0
RFC PATCH v2 3/3 selftests/bpf: add selftest for bpf_cgroup_write_interface
In reply to Djalal HarouniThis adds a selftest for bpf_cgroup_write_interface kfunc. The test works by forking a child then:
1. Child:
- Migrate to a new cgroup
- Loads bpf programs
- Trigger the 'lsm_freeze_cgroup' bpf program so it freeze itself.
<- wait for parent to unthaw
- On unthaw it continues, forks another process and triggers the 'tp_newchild' bpf program to set some monitored pids of the new process, that assert that the user space resumed correctly.
2. Parent:
- Keeps reading the 'cgroup.freeze' file of the child cgroup until it prints 1 which means the child cgroup is frozen.
- Attaches the sample 'lsm_task_free' so it triggers the bpf program to unthaw the child task cgroup.
- Then waits for a clean exit of the child process.
The scenario allows to test multiple sides of: freeze and unthaw a cgroup.
--- .../bpf/prog_tests/task_freeze_cgroup.c | 172 ++++++++++++++++++ .../bpf/progs/test_task_freeze_cgroup.c | 155 ++++++++++++++++ 2 files changed, 327 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/task_freeze_cgroup.c create mode 100644 tools/testing/selftests/bpf/progs/test_task_freeze_cgroup.c
tools/testing/selftests/bpf/prog_tests/task_freeze_cgroup.c new file +172 -0
@@ -0,0 +1,172 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <sys/syscall.h> +#include <test_progs.h> +#include <cgroup_helpers.h> +#include <unistd.h> +#include "test_task_freeze_cgroup.skel.h" + +#define CGROUP_PATH "/test-task-freeze-cgroup" + +static int bpf_sleepable (struct test_task_freeze_cgroup *skel) +{ +int err, cgroup_fd; +pid_t new_pid2; + + cgroup_fd =cgroup_setup_and_join (CGROUP_PATH ); +if (!ASSERT_OK (cgroup_fd <0 ,"cgroup_setup_and_join" )) +return -errno; + + skel =test_task_freeze_cgroup__open (); +if (!ASSERT_OK_PTR (skel,"test_task_freeze_cgroup__open" )) { + err = -errno; + goto cleanup_cgroup; + } + + skel->rodata->parent_pid =getppid (); + skel->rodata->monitor_pid =getpid (); + skel->rodata->cgid =get_cgroup_id (CGROUP_PATH ); + skel->bss->new_pid =getpid (); + skel->bss->freeze =1 ; + + err =test_task_freeze_cgroup__load (skel); +if (!ASSERT_OK (err,"test_task_freeze_cgroup__load" )) { + err = -errno; + goto cleanup_skel; + } + +/* First attach the LSM Program that is triggered on bpf() calls +* especially on TP_BTF programs when attached. +*/ + skel->links.lsm_freeze_cgroup = +bpf_program__attach_lsm (skel->progs.lsm_freeze_cgroup); +if (!ASSERT_OK_PTR (skel->links.lsm_freeze_cgroup,"attach_lsm" )) { + err = -errno; + goto cleanup_detach; + } + +/* Attaching this must fail with -EPERM and freeze current task */ + skel->links.tp_newchild = +bpf_program__attach_trace (skel->progs.tp_newchild); +if (!ASSERT_EQ (errno,EPERM ,"attach_trace() must fail here" )) { + err = -EINVAL ; + goto cleanup_detach; + } + +/* Continue */ + +/* Attach again now with success */ + skel->links.tp_newchild = +bpf_program__attach_trace (skel->progs.tp_newchild); +if (!ASSERT_OK_PTR (skel->links.tp_newchild,"attach_trace" )) { + err = -EINVAL ; + goto cleanup_detach; + } + +/* Fork, update vars from BPF and assert the unfrozen state */ + new_pid2 =fork (); +if (new_pid2 ==0 ) +exit (0 ); + + err = (new_pid2 ==-1 ); +if (ASSERT_OK (err,"fork process" )) +wait (NULL ); + +/* Now assert that new_pid2 reflects this new child */ +ASSERT_NEQ (0 , skel->bss->new_pid, +"test task_freeze_cgroup failed at new_pid != 0" ); +ASSERT_NEQ (skel->rodata->monitor_pid, skel->bss->new_pid, +"test task_freeze_cgroup failed at old monitor_pid != new_pid" ); +/* Assert that bpf sets new_pid to new forked child new_pid2 */ +ASSERT_EQ (skel->bss->new_pid, new_pid2, +"test task_freeze_cgroup failed first child new_pid == new_pid2" ); + +cleanup_detach: +test_task_freeze_cgroup__detach (skel); +cleanup_skel: +test_task_freeze_cgroup__destroy (skel); +cleanup_cgroup: +close (cgroup_fd); +cleanup_cgroup_environment (); +return err; +} + +void test_task_freeze_cgroup (void ) +{ +pid_t pid, result; +char buf[512 ] = {0 }; +char path[PATH_MAX ] = {0 }; +int ret, status, attempts, frozen =0 , fd; +struct test_task_freeze_cgroup *skel =NULL ; + + pid =fork (); + ret = (pid ==-1 ); +if (!ASSERT_OK (ret,"fork process" )) +return ; + +if (pid ==0 ) { + ret =bpf_sleepable (skel); +ASSERT_EQ (0 , ret,"child bpf_sleepable failed" ); +exit (ret); + } + + skel =test_task_freeze_cgroup__open (); +if (!ASSERT_OK_PTR (skel,"test_task_freeze_cgroup__open" )) + goto out; + +snprintf (path,sizeof (path), +"/sys/fs/cgroup/cgroup-test-work-dir%d%s/cgroup.freeze" , + pid,CGROUP_PATH ); + +for (attempts =10 ; attempts >=0 ; attempts--) { + ret =0 ; + + fd =open (path,O_RDONLY ); +if (fd >0 ) + ret =read (fd, buf,sizeof (buf) -1 ); +if (ret >0 ) { + errno =0 ; + frozen =strtol (buf,NULL ,10 ); +if (errno) + frozen =0 ; + } + +close (fd); +if (frozen) +break ; +sleep (1 ); + } + +/* Assert that child cgroup is frozen */ +if (!ASSERT_EQ (1 , frozen,"child cgroup not frozen" )) + goto out; + + ret =test_task_freeze_cgroup__load (skel); +if (!ASSERT_OK (ret,"test_task_freeze_cgroup__load" )) + goto out; + +/* Trigger the unthaw child cgroup from parent */ + skel->links.lsm_task_free = +bpf_program__attach_lsm (skel->progs.lsm_task_free); +if (!ASSERT_OK_PTR (skel->links.lsm_task_free,"attach_lsm" )) + goto out; + + result =waitpid (pid, &status,WUNTRACED ); +if (!ASSERT_NEQ (result,-1 ,"waitpid" )) + goto detach; + + result =WIFEXITED (status); +if (!ASSERT_EQ (result,1 ,"forked process did not terminate normally" )) + goto detach; + + result =WEXITSTATUS (status); +if (!ASSERT_EQ (result,0 ,"forked process did not exit successfully" )) + goto detach; + +detach: +test_task_freeze_cgroup__detach (skel); + +out: +if (skel) +test_task_freeze_cgroup__destroy (skel); +}
tools/testing/selftests/bpf/progs/test_task_freeze_cgroup.c new file +155 -0
155 lines not on this page · raw message
Signature · 1 line
2.43.0
Tejun Heo
maintainerQuoted 7 lines · Djalal Harouni
This patch series add support to write cgroup interfaces from BPF.
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
Quoted 5 lines
The writing happens via kernfs nodes and the cgroup must be on the default hierarchy. It implements the requests and feedback from v1 [1] where now we use a unified path for cgroup user space and BPF writing.
So I want to validate that this is the right approach first.
I don't see any reason to object to the feature but the way it's constructed seems rather odd to me. If it's going to need per-feature code, might as well bypass the write part and implement a simpler interface - ie. bpf_cgroup_freeze(). Otherwise, can't it actually write to kernfs files so that we don't need to add code per enabled feature?
Thanks.
Signature · 1 line
tejun
Quoted 12 lines · Tejun Heo
Quoted 11 lines · Djalal Harouni
This patch series add support to write cgroup interfaces from BPF.
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
The writing happens via kernfs nodes and the cgroup must be on the default hierarchy. It implements the requests and feedback from v1 [1] where now we use a unified path for cgroup user space and BPF writing.
Quoted 6 lines
Quoted 1 line
So I want to validate that this is the right approach first.
I don't see any reason to object to the feature but the way it's constructed seems rather odd to me. If it's going to need per-feature code, might as well bypass the write part and implement a simpler interface - ie.
bpf_cgroup_freeze().
Approach 1:
First RFC months ago was something like that "bpf_task_freeze_cgroup" [1], can make it bpf_cgroup_freeze() as a proper kfunc, so resurrect approach 1?
Internally it used an ugly path to workaround kernfs active reference since we don't hold a kernfs_open_file coming from userspace kernfs->write path.
I can improve it, but let's discuss please approach (2) since you suggested it ;-)
Approach 2:
Per the old suggestions from you and Alexei [2] [3] you wanted something like:
s32 bpf_kernfs_knob_write(struct kernfs_node *dir,
const char *knob, char *buf);
I didn't make it generic for kernfs, since don't know yet about sysfs use cases and named it "bpf_cgroup_write_interface" to focus on cgroup base interfaces.
Doing something that generic now including sysfs without a proper valid use cases seems a bit too much. Also we have some cgroup kfunc to acquire and release that integrate well, so I kept it focused.
Alexei suggested to refactor the cgroup_base_file[] [4][5] to take "kernfs_node" as argument instead of "kernfs_open_file", which will open other possibilities for BPF.
However, instead of going full change on cgroup_base_files[], I added a minimalist: cgroup_kn_cftype kn_cfts[] that for now hold only "cgroup.freeze".
I see three possibilities here:
A. Minimal change with approach presented here:
add dedicated array cgroup_kn_cftype kn_cfts[] with only "cgroup.freeze" and later try to unify it inside cgroup_base_file[].
B. Add "->bpf_write()" handler to cgroup_base_file[] and start only with
"cgroup.freeze".
C. Refactor all cgroup_base_file[] to take a kernfs_node directly
instead of kernfs_open_file as suggested.
I took (C) the simple one since I wanted to do cgroup freeze first. You
also suggested maybe in future "cgroup.kill" well if we have it, we definitely will start using it. Not sure if we are allowed to BPF sleep that path, however we can also start doing "cgroup.freeze" from BPF and kill from user space as a first step. But we definitly want more BPF operations on cgroup interfaces, I can think of a companion cgroup where we migrate tasks there on specific signals...
So more or less current proposed approach (2) followed the suggestions, but focused only on writing cgroup kernfs knobs.
Thoughts, did I miss something?
Quoted 2 lines
Otherwise, can't it actually write to kernfs files so that we don't need to add code per enabled feature?
I'm not sure how we would write to kernfs files? As pointed by Alexei [6] it is more involved if we want to open files...
About "that we don't need to add code per enabled feature?" well if we
go the path of "bpf_cgroup_write_interface" or "bpf_cgroup_write_knob" adding a new write interface will involve theoretically:
1. Check if program can sleep or/and the calling context.
2. Add the new "cgroup.x" either in cgroup_base_file[] or in the new array.
3. New handlers or refactor old ones to take a "kernfs_node" instead of "kernfs_open_file".
Compared to having multiple bpf_cgroup_(freeze|kill|...) kfunc seems fair too, and not that much code from BPF part.
BTW current patches contain a bug, after testing. In normal writes from user context we break kernfs active protection to avoid nesting cgroup locking under.
Forget this part. In next series since we don't grab the kernfs_node active protection like userspace kernfs_write, then no need to break it... will add a parameter to check like the revalidate one that checks things is cgroup on dfl and not root, things that are automatically handled from normal userspace ->write.
Thank you!
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://lore.kernel.org/bpf/CAADnVQK970_Nx3918V41ue031RkGs+WsteOAm6EJOY7oSwzS1A@mail.gmail.com/
[5] https://lore.kernel.org/bpf/CAADnVQ+WmaPG1WOaSDbjxNPVzVape_JfG_CNSRy188ni076Mog@mail.gmail.com/
[6] https://lore.kernel.org/bpf/CAADnVQLhWDcX-7XCdo-W=jthU=9iPqODwrE6c9fvU8sfAJ5ARg@mail.gmail.com/
Quoted 2 lines
Thanks.
Quoted 53 lines · Djalal Harouni
Quoted 12 lines · Tejun Heo
Quoted 11 lines · Djalal Harouni
This patch series add support to write cgroup interfaces from BPF.
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
The writing happens via kernfs nodes and the cgroup must be on the default hierarchy. It implements the requests and feedback from v1 [1] where now we use a unified path for cgroup user space and BPF writing.
Quoted 6 lines
Quoted 1 line
So I want to validate that this is the right approach first.
I don't see any reason to object to the feature but the way it's constructed seems rather odd to me. If it's going to need per-feature code, might as well bypass the write part and implement a simpler interface - ie.
bpf_cgroup_freeze().Approach 1:
First RFC months ago was something like that "bpf_task_freeze_cgroup" [1], can make itbpf_cgroup_freeze()as a proper kfunc, so resurrect approach 1?Internally it used an ugly path to workaround kernfs active reference since we don't hold a
kernfs_open_filecoming from userspacekernfs->writepath.I can improve it, but let's discuss please approach (2) since you suggested it ;-)
Approach 2:
Per the old suggestions from you and Alexei [2] [3] you wanted something like:s32
bpf_kernfs_knob_write(struct kernfs_node *dir, const char *knob, char *buf);I didn't make it generic for kernfs, since don't know yet about sysfs use cases and named it "
bpf_cgroup_write_interface" to focus on cgroup base interfaces.
Doing something that generic now including sysfs without a proper valid use cases seems a bit too much. Also we have some cgroup kfunc to acquire and release that integrate well, so I kept it focused.Alexei suggested to refactor the cgroup_base_file[] [4][5] to take "
kernfs_node" as argument instead of "kernfs_open_file", which will open other possibilities for BPF.However, instead of going full change on cgroup_base_files[], I added a minimalist:
cgroup_kn_cftypekn_cfts[] that for now hold only "cgroup.freeze".I see three possibilities here:
A. Minimal change with approach presented here:
add dedicated arraycgroup_kn_cftypekn_cfts[] with only "cgroup.freeze" and later try to unify it inside cgroup_base_file[].
B. Add "->bpf_write()" handler to cgroup_base_file[] and start only with
"cgroup.freeze".
C. Refactor all cgroup_base_file[] to take akernfs_nodedirectly
instead ofkernfs_open_fileas suggested.
Quoted 1 line
I took (C) the simple one since I wanted to do cgroup freeze first. You
Sorry here I meant (A).
Thanks.
Quoted 42 lines
also suggested maybe in future "cgroup.kill" well if we have it, we definitely will start using it. Not sure if we are allowed to BPF sleep that path, however we can also start doing "cgroup.freeze" from BPF and kill from user space as a first step. But we definitly want more BPF operations on cgroup interfaces, I can think of a companion cgroup where we migrate tasks there on specific signals...
So more or less current proposed approach (2) followed the suggestions, but focused only on writing cgroup kernfs knobs.
Thoughts, did I miss something?
Quoted 2 lines
Otherwise, can't it actually write to kernfs files so that we don't need to add code per enabled feature?
I'm not sure how we would write to kernfs files? As pointed by Alexei [6] it is more involved if we want to open files...
About "that we don't need to add code per enabled feature?" well if we
go the path of "bpf_cgroup_write_interface" or "bpf_cgroup_write_knob" adding a new write interface will involve theoretically:1. Check if program can sleep or/and the calling context.
2. Add the new "cgroup.x" either in cgroup_base_file[] or in the new array.
3. New handlers or refactor old ones to take a "kernfs_node" instead of "kernfs_open_file".Compared to having multiple
bpf_cgroup_(freeze|kill|...)kfunc seems fair too, and not that much code from BPF part.BTW current patches contain a bug, after testing. In normal writes from user context we break kernfs active protection to avoid nesting cgroup locking under.
Forget this part. In next series since we don't grab thekernfs_nodeactive protection like userspacekernfs_write, then no need to break it... will add a parameter to check like the revalidate one that checks things is cgroup on dfl and not root, things that are automatically handled from normal userspace ->write.Thank you!
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://lore.kernel.org/bpf/ CAADnVQK970_Nx3918V41ue031RkGs+WsteOAm6EJOY7oSwzS1A@mail.gmail.com/
[5] https://lore.kernel.org/bpf/ CAADnVQ+WmaPG1WOaSDbjxNPVzVape_JfG_CNSRy188ni076Mog@mail.gmail.com/
[6] https://lore.kernel.org/bpf/CAADnVQLhWDcX-7XCdo- [email protected]/Quoted 2 lines
Thanks.
Tejun Heo
maintainerHello,
On Wed, Aug 20, 2025 at 12:31:01AM +0100, Djalal Harouni wrote:
...
Quoted 3 lines
Approach 1:
First RFC months ago was something like that "bpf_task_freeze_cgroup" [1], can make itbpf_cgroup_freeze()as a proper kfunc, so resurrect approach 1?
Thanks for reminding me. I often feel like my memory is a ring buffer which lasts a few weeks at most.
Quoted 21 lines
Internally it used an ugly path to workaround kernfs active reference since we don't hold a
kernfs_open_filecoming from userspacekernfs->writepath.I can improve it, but let's discuss please approach (2) since you suggested it ;-)
Approach 2:
Per the old suggestions from you and Alexei [2] [3] you wanted something like:s32 bpf_kernfs_knob_write(struct
kernfs_node*dir,const char *knob, char *buf);I didn't make it generic for kernfs, since don't know yet about sysfs use cases and named it "
bpf_cgroup_write_interface" to focus on cgroup base interfaces.
Doing something that generic now including sysfs without a proper valid use cases seems a bit too much. Also we have some cgroup kfunc to acquire and release that integrate well, so I kept it focused.
Quoted 7 lines
Alexei suggested to refactor the cgroup_base_file[] [4][5] to take "
kernfs_node" as argument instead of "kernfs_open_file", which will open other possibilities for BPF.However, instead of going full change on cgroup_base_files[], I added a
minimalist:cgroup_kn_cftypekn_cfts[] that for now hold only "cgroup.freeze".
I think there's some misunderstanding here. IIUC, Alexei didn't want to expose direct file interface because none of the necessary abstractions are available and the effort becomes too big and wide. However, I don't think the suggestion to use @kn as the path designator (or @cgroup for that matter) means that we want to pipe that all the way down to each op that's to be written to. That'd be rather pointless - why add generic interface if each needs a dedicated backend anyway? Can't you make kernfs to create open_file from kn and follow the usual write path?
Thanks.
Signature · 1 line
tejun
Quoted 10 lines · Tejun Heo
Hello,
On Wed, Aug 20, 2025 at 12:31:01AM +0100, Djalal Harouni wrote:
...Quoted 3 lines
Approach 1:
First RFC months ago was something like that "bpf_task_freeze_cgroup" [1], can make itbpf_cgroup_freeze()as a proper kfunc, so resurrect approach 1?Thanks for reminding me. I often feel like my memory is a ring buffer which lasts a few weeks at most.
yeh been a while...
Quoted 29 lines
Quoted 21 lines
Internally it used an ugly path to workaround kernfs active reference since we don't hold a
kernfs_open_filecoming from userspacekernfs->writepath.I can improve it, but let's discuss please approach (2) since you suggested it ;-)
Approach 2:
Per the old suggestions from you and Alexei [2] [3] you wanted something like:s32 bpf_kernfs_knob_write(struct
kernfs_node*dir,const char *knob, char *buf);I didn't make it generic for kernfs, since don't know yet about sysfs use cases and named it "
bpf_cgroup_write_interface" to focus on cgroup base interfaces.
Doing something that generic now including sysfs without a proper valid use cases seems a bit too much. Also we have some cgroup kfunc to acquire and release that integrate well, so I kept it focused.Quoted 7 lines
Alexei suggested to refactor the cgroup_base_file[] [4][5] to take "
kernfs_node" as argument instead of "kernfs_open_file", which will open other possibilities for BPF.However, instead of going full change on cgroup_base_files[], I added a
minimalist:cgroup_kn_cftypekn_cfts[] that for now hold only "cgroup.freeze".
Quoted 7 lines
I think there's some misunderstanding here. IIUC, Alexei didn't want to expose direct file interface because none of the necessary abstractions are available and the effort becomes too big and wide. However, I don't think the suggestion to use @kn as the path designator (or @cgroup for that matter) means that we want to pipe that all the way down to each op that's to be written to. That'd be rather pointless - why add generic interface if each needs a dedicated backend anyway?
I took time to check...
I thought to avoid precisely creating a kernfs_open_file, since we don't have the corresponding context. Beside the user information who opened the file, kernfs_open_file assumes a corresponding opened file and a corresponding kernfs_open_node for each kernfs_node with one or more open files.
Also it seems each kernfs_open_file is added back into kernfs_open_node->files...
Quoted 2 lines
Can't you make kernfs to create
open_filefrom kn and follow the usual write path?
Seems tricky, needs to handle also kernfs_global_locks->open_file_mutex compared with writing directly to the dedicated cgroup backend. The only protection there beside cgroup_lock() is break kernfs active protection so kernfs nodes can be drained later...
The cgroup_kn_lock_live() grabs a ref on the cgroup, breaks the active protection, takes the cgroup_lock() and ensures the cgroup is not dead, from there we don't dereference kn-> until we restore the protection. Plus we have the kernfs_get(cgrp->kn) on cgroup_mkdir which ensures it is always accessible.
I do realize taking the same usual path with write is the obvious thing, but we don't have the corresponding open context, and faking it seems more trouble than calling directly cgroup backends...
Allow me please to do it again directly on cgroup_base_file[] assuming it was Alexei suggestion and see how it looks.
Also Tejun, could you please point me to extra cgroup or kernfs tests you run? much appreciated!
Thanks!
Quoted 2 lines
Thanks.
Tejun Heo
maintainerHello,
On Fri, Aug 22, 2025 at 07:16:15PM +0100, Djalal Harouni wrote:
...
Quoted 6 lines
I do realize taking the same usual path with write is the obvious thing, but we don't have the corresponding open context, and faking it seems more trouble than calling directly cgroup backends...
Allow me please to do it again directly on cgroup_base_file[] assuming it was Alexei suggestion and see how it looks.
I'm probably missing something but what prevents you from getting a dentry from kernfs_node and then calling vfs_open() on it and then do vfs_write() on the returned file?
If there are some fundamental reasons that we can't do something like that, let's go back to the simple approach where we just have bpf helpers for freezing and unfreezing cgroups outside of fs interface.
Quoted 2 lines
Also Tejun, could you please point me to extra cgroup or kernfs tests you run? much appreciated!
I'm afraid there isn't much outside what's in the selftest directory.
Thanks.
Signature · 1 line
tejun
Quoted 11 lines · Tejun Heo
Hello,
On Fri, Aug 22, 2025 at 07:16:15PM +0100, Djalal Harouni wrote:
...Quoted 6 lines
I do realize taking the same usual path with write is the obvious thing, but we don't have the corresponding open context, and faking it seems more trouble than calling directly cgroup backends...
Allow me please to do it again directly on cgroup_base_file[] assuming it was Alexei suggestion and see how it looks.
It's been 1.5 year since v1. It's safe to assume that all opinions have changed, including mine.
Quoted 3 lines
I'm probably missing something but what prevents you from getting a dentry from
kernfs_nodeand then callingvfs_open()on it and then dovfs_write()on the returned file?
Generic vfs ops from kfunc feels like a can of worms.
It will require a lot more thinking and coordination with vfs folks. I'd rather keep things simple especially, since this thread might continue in 2026.
Quoted 3 lines
If there are some fundamental reasons that we can't do something like that, let's go back to the simple approach where we just have bpf helpers for freezing and unfreezing cgroups outside of fs interface.
I'd just do whatever version of cgroup_lock*() is necessary from kfunc followed by cgroup_freeze(),
and limit kfunc to sleepable progs, of course.
Hello,
Quoted 8 lines · Tejun Heo
Hello,
On Fri, Aug 22, 2025 at 07:16:15PM +0100, Djalal Harouni wrote:
...Quoted 4 lines
I do realize taking the same usual path with write is the obvious thing, but we don't have the corresponding open context, and faking it seems more trouble than calling directly cgroup backends...
Quoted 6 lines
Quoted 2 lines
Allow me please to do it again directly on cgroup_base_file[] assuming it was Alexei suggestion and see how it looks.
I'm probably missing something but what prevents you from getting a dentry from
kernfs_nodeand then callingvfs_open()on it and then dovfs_write()on the returned file?
If we include the open path then don't have the right context, first example in vfs_open() will use the wrong current cred context to perform permission checks, current could have dropped privileges while the cgroup hierarchy is still root owned...
The thing here is that the bpf program will be called from arbitrary paths, not a single pre-defined path/function were we could control the context...
Quoted 3 lines
If there are some fundamental reasons that we can't do something like that, let's go back to the simple approach where we just have bpf helpers for freezing and unfreezing cgroups outside of fs interface.
Alright, seems Alexei also agree on this. Thanks will prepare another version.
Quoted 4 lines
Quoted 2 lines
Also Tejun, could you please point me to extra cgroup or kernfs tests you run? much appreciated!
I'm afraid there isn't much outside what's in the selftest directory.
Ok, thank you!
Quoted 2 lines
Thanks.
Michal Koutný
maintainerHi Djalal.
Quoted 6 lines · Djalal Harouni
This patch series add support to write cgroup interfaces from BPF.
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
Could you please give more specific example of the "suspicious activity"? The last time (v1) it was referring to LSM hooks where such asynchronous approach wasn't ideal.
Also why couldn't all these tools execute the cgroup actions themselves through traditional userspace API?
One more point (for possible interference with lifecycles) -- what is the relation between cgroup in which the BPF code "runs" and cgroup that's target of the operation? (I hope this isn't supposed to run from BPF without process context.)
Quoted 4 lines
Todo:
* Limit size of data to be written.
* Further tests.
* Add cgroup kill support.
I'm missing the retrieval of freeze result in this plan :) cgroup kill would be simpler for PoC (and maybe even sufficient for your use case?).
Regards,
Michal
Hi Michal,
Quoted 5 lines · Michal Koutný
Hi Djalal.
Quoted 2 lines · Djalal Harouni
This patch series add support to write cgroup interfaces from BPF.
Quoted 8 lines
Quoted 4 lines
It is useful to freeze a cgroup hierarchy on suspicious activity for a more thorough analysis before killing it. Planned users of this feature are: systemd and BPF tools where the cgroup hierarchy could be a system service, user session, k8s pod or a container.
Could you please give more specific example of the "suspicious activity"? The last time (v1) it was referring to LSM hooks where such asynchronous approach wasn't ideal.
It solves the case perfectly, you detect something you fail the security hook return -EPERM and optionally freeze the cgroup, snapshot the runtime state.
Oh I thought the attached example is an obvious one, customers want to restrict bpf() usage per cgroup specific container/pod, so when we detect bpf() that's not per allowed cgroup we fail it and freeze it.
Take this and build on top, detect bash/shell exec or any other new dropped binaries, fail and freeze the exec early at linux_bprm object checks.
Quoted 2 lines
Also why couldn't all these tools execute the cgroup actions themselves through traditional userspace API?
- Freezing at BPF is obviously better, less race since you don't need access to the corresponding cgroup fs and namespace. Not all tools run as supervisor/container manager.
- The bpf_send_signal in some cases is not enough, what if you race with a task clone as an example? however freezing the cgroup hierarchy or the one above is a catch all...
Quoted 4 lines
One more point (for possible interference with lifecycles) -- what is the relation between cgroup in which the BPF code "runs" and cgroup that's target of the operation? (I hope this isn't supposed to run from BPF without process context.)
The feature is supposed to be used by sleepable BPF programs, I don't think we need extra checks here?
It could be that this BPF code runs in a process that is under pod-x/container-y/cgroup-z/ and maybe you want to freeze "cgroup-z" or "container-y" and so on... or in case of delegated hierarchies, freezing the parent is a catch all.
Quoted 7 lines
Quoted 5 lines
Todo:
* Limit size of data to be written.
* Further tests.
* Add cgroup kill support.I'm missing the retrieval of freeze result in this plan :) cgroup kill
Indeed you are right a small kfunc to read back, yes ;) !
Quoted 2 lines
would be simpler for PoC (and maybe even sufficient for your use case?).
I think both are useful cases.
Thank you!
Quoted 2 lines
Regards,
Michal
Michal Koutný
maintainerQuoted 3 lines · Djalal Harouni
It solves the case perfectly, you detect something you fail the security hook return
-EPERMand optionally freeze the cgroup, snapshot the runtime state.
So -EPERM is the right way to cut off such tasks.
Quoted 8 lines
Oh I thought the attached example is an obvious one, customers want to restrict
bpf()usage per cgroup specific container/pod, so when we detectbpf()that's not per allowed cgroup we fail it and freeze it.Take this and build on top, detect bash/shell exec or any other new dropped binaries, fail and freeze the exec early at
linux_bprmobject checks.
Or if you want to do some followup analysis, the process can be killed and coredump'd (at least seccomp allows this, it'd be good to have such a possibility with LSMs if there isn't (I'm not that familiar)). Freezing the groups sounds like a way to DoS the system (not only because of hanging the faulty process itself but possibly spreading via IPC dependencies to unrelated processes).
Quoted 6 lines
Quoted 2 lines
Also why couldn't all these tools execute the cgroup actions themselves through traditional userspace API?
- Freezing at BPF is obviously better, less race since you don't need access to the corresponding cgroup fs and namespace. Not all tools run as supervisor/container manager.
Less race or more race -- I know the race window size may vary but strictly speaking , there is a race or isn't (depends on having proper synchronization or not). (And when intentionally misbehaving processes are considered even tiny window is potential risk.)
Quoted 3 lines
- The
bpf_send_signalin some cases is not enough, what if you race with a task clone as an example? however freezing the cgroup hierarchy or the one above is a catch all...
Yeah, this might be part that I don't internalize well. If you're running the hook in particular task's process context, it cannot do clone at the same time. If they are independent tasks, there's no ordering, so there's always possibility of the race (so why not embrace it and do whatever is possible with userspace monitoring audit log or similar and respond based on that).
Quoted 2 lines
The feature is supposed to be used by sleepable BPF programs, I don't think we need extra checks here?
Good.
Quoted 4 lines
It could be that this BPF code runs in a process that is under pod-x/container-y/cgroup-z/ and maybe you want to freeze "cgroup-z" or "container-y" and so on... or in case of delegated hierarchies, freezing the parent is a catch all.
OK, this would be good. Could it also be pod-x/container-y/cgroup-z2?
---
I acknowledge that sooner or later some kind of access to cgroup through BPF will be added, I'd prefer if it was done in a generic way (so that it doesn't become cgroup's problem but someone else's e.g. VFS's or kernfs's ;-)).
I can even imagine some usefulness of helpers for selected specific cgroup (core) operations (which is the direction brought up in the other discussion), I just don't think it solves the problem as you present it.
HTH,
Michal
Hi Michal,
Quoted 7 lines · Michal Koutný
Quoted 3 lines · Djalal Harouni
It solves the case perfectly, you detect something you fail the security hook return
-EPERMand optionally freeze the cgroup, snapshot the runtime state.So
-EPERMis the right way to cut off such tasks.
Indeed. A process can retry x y z paths, at some point we just want to stop the process or container before snapshot.
Quoted 9 lines
Quoted 8 lines
Oh I thought the attached example is an obvious one, customers want to restrict
bpf()usage per cgroup specific container/pod, so when we detectbpf()that's not per allowed cgroup we fail it and freeze it.Take this and build on top, detect bash/shell exec or any other new dropped binaries, fail and freeze the exec early at
linux_bprmobject checks.
Quoted 6 lines
Or if you want to do some followup analysis, the process can be killed and coredump'd (at least seccomp allows this, it'd be good to have such a possibility with LSMs if there isn't (I'm not that familiar)). Freezing the groups sounds like a way to DoS the system (not only because of hanging the faulty process itself but possibly spreading via IPC dependencies to unrelated processes).
Well mis-using things is possible, but here nothing is new.
Pausing a container, freezing the cgroup, or criu have been there for years, no new features here ;-)
Quoted 17 lines
Quoted 7 lines
Quoted 3 lines
Also why couldn't all these tools execute the cgroup actions themselves through traditional userspace API?
- Freezing at BPF is obviously better, less race since you don't need access to the corresponding cgroup fs and namespace. Not all tools run as supervisor/container manager.
Less race or more race -- I know the race window size may vary but strictly speaking , there is a race or isn't (depends on having proper synchronization or not). (And when intentionally misbehaving processes are considered even tiny window is potential risk.)
Quoted 3 lines
- The
bpf_send_signalin some cases is not enough, what if you race with a task clone as an example? however freezing the cgroup hierarchy or the one above is a catch all...
Quoted 6 lines
Yeah, this might be part that I don't internalize well. If you're running the hook in particular task's process context, it cannot do clone at the same time. If they are independent tasks, there's no ordering, so there's always possibility of the race (so why not embrace it and do whatever is possible with userspace monitoring audit log or similar and respond based on that).
The complexity to do that from userspace for an eBPF security tool
that is not running as a supervisor in other namespaces is high.
Basically the race window is reduced... we trigger in the task context that we want to freeze and set the freeze jobctl bit of the other tasks.
The extra offending syscalls by that cgroup tasks are reduced.
Quoted 4 lines
Quoted 3 lines
The feature is supposed to be used by sleepable BPF programs, I don't think we need extra checks here?
Quoted 9 lines
Good.
Quoted 4 lines
It could be that this BPF code runs in a process that is under pod-x/container-y/cgroup-z/ and maybe you want to freeze "cgroup-z" or "container-y" and so on... or in case of delegated hierarchies, freezing the parent is a catch all.
OK, this would be good. Could it also be pod-x/container-y/cgroup-z2?
Yes, basically the cgroup of the task, or any parent by fetching its cgroup.
BPF is already the core interface of cgroup device controller.
Thanks!
Quoted 12 lines
---
I acknowledge that sooner or later some kind of access to cgroup through BPF will be added, I'd prefer if it was done in a generic way (so that it doesn't become cgroup's problem but someone else's e.g. VFS's or kernfs's ;-)).
I can even imagine some usefulness of helpers for selected specific cgroup (core) operations (which is the direction brought up in the other discussion), I just don't think it solves the problem as you present it.HTH,
Michal