mm: BPF OOM
Subsystems (11)
Roman Gushchin
maintainercover mm: BPF OOM
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.
The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html
----
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for the bpf_handle_out_of_memory() hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to pass oom_control down to bpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for the oom_control pointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepable bpf_handle_out_of_memory() hook calls might_fault() under oom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introduce bpf_get_root_mem_cgroup() bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?
4) Documentation is lacking and will be added in the next version.
Roman Gushchin (12):
mm: introduce a bpf hook for OOM handling
bpf: mark struct oom_control's memcg field as TRUSTED_OR_NULL
bpf: treat fmodret tracing program's arguments as trusted
mm: introduce bpf_oom_kill_process() bpf kfunc
mm: introduce bpf kfuncs to deal with memcg pointers
mm: introduce bpf_get_root_mem_cgroup() bpf kfunc
bpf: selftests: introduce read_cgroup_file() helper
bpf: selftests: bpf OOM handler test
sched: psi: bpf hook to handle psi events
mm: introduce bpf_out_of_memory() bpf kfunc
bpf: selftests: introduce open_cgroup_file() helper
bpf: selftests: psi handler test
include/linux/memcontrol.h | 2 + include/linux/oom.h | 5 + kernel/bpf/btf.c | 9 +- kernel/bpf/verifier.c | 5 + kernel/sched/psi.c | 36 ++- mm/Makefile | 3 + mm/bpf_memcontrol.c | 108 +++++++++ mm/oom_kill.c | 140 +++++++++++ tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 3 + tools/testing/selftests/bpf/prog_tests/oom.c | 227 ++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/psi.c | 234 +++++++++++++++++++ tools/testing/selftests/bpf/progs/test_oom.c | 103 ++++++++ tools/testing/selftests/bpf/progs/test_psi.c | 43 ++++ 14 files changed, 983 insertions(+), 2 deletions(-) create mode 100644 mm/bpf_memcontrol.c create mode 100644 tools/testing/selftests/bpf/prog_tests/oom.c create mode 100644 tools/testing/selftests/bpf/prog_tests/psi.c create mode 100644 tools/testing/selftests/bpf/progs/test_oom.c create mode 100644 tools/testing/selftests/bpf/progs/test_psi.c
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 1/12 mm: introduce a bpf hook for OOM handling
Introduce a bpf hook for implementing custom OOM handling policies.
The hook is int bpf_handle_out_of_memory(struct oom_control *oc) function, which expected to return 1 if it was able to free some memory and 0 otherwise. In the latter case it's guaranteed that the in-kernel OOM killer will be invoked. Otherwise the kernel also checks the bpf_memory_freed field of the oom_control structure, which is expected to be set by kfuncs suitable for releasing memory. It's a safety mechanism which prevents a bpf program to claim forward progress without actually releasing memory.
The hook program is sleepable to enable using iterators, e.g. cgroup iterators.
The hook is executed just before the kernel victim task selection algorithm, so all heuristics and sysctls like panic on oom, sysctl_oom_kill_allocating_task and sysctl_oom_kill_allocating_task are respected.
--- include/linux/oom.h | 5 ++++ mm/oom_kill.c | 68 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+)
include/linux/oom.h +5 -0
@@ -51,6 +51,11 @@ struct oom_control {/* Used to print the constraint info. */ enum oom_constraint constraint; + +#ifdef CONFIG_BPF_SYSCALL +/* Used by the bpf oom implementation to mark the forward progress */ +bool bpf_memory_freed; +#endif };extern struct mutex oom_lock;
mm/oom_kill.c +68 -0
89 lines not on this page · raw message
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 2/12 bpf: mark struct oom_control's memcg field as TRUSTED_OR_NULL
In reply to Roman GushchinStruct oom_control is used to describe the OOM context. It's memcg field defines the scope of OOM: it's NULL for global OOMs and a valid memcg pointer for memcg-scoped OOMs. Teach bpf verifier to recognize it as trusted or NULL pointer. It will provide the bpf OOM handler a trusted memcg pointer, which for example is required for iterating the memcg's subtree.
--- kernel/bpf/verifier.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/bpf/verifier.c +5 -0
@@ -7047,6 +7047,10 @@ BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct socket) {struct sock *sk; }; +BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct oom_control) { +struct mem_cgroup *memcg; +}; +static bool type_is_rcu (struct bpf_verifier_env *env,struct bpf_reg_state *reg,const char *field_name,u32 btf_id)
@@ -7087,6 +7091,7 @@ static bool type_is_trusted_or_null(struct bpf_verifier_env *env, const char *field_name, u32 btf_id) {BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED_OR_NULL (struct socket )); +BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct oom_control ));return btf_nested_type_is_trusted (&env->log, reg, field_name, btf_id,"__safe_trusted_or_null" );
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 3/12 bpf: treat fmodret tracing program's arguments as trusted
In reply to Roman Gushchin*** DO NOT MERGE! ***
This is a temporarily workaround, which will be fixed/replaced in the next version.
--
Bpf oom handler hook has to:
1) have a trusted pointer to the oom_control structure,
2) return a value,
3) be sleepable to use cgroup iterator functions.
fmodret tracing programs fulfill 2) and 3).
This patch enables 1), however this change contradicts the commit c6b0337f0120 ("bpf: Don't mark arguments to fentry/fexit programs as trusted.").
--- kernel/bpf/btf.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +8 -1
@@ -6424,7 +6424,14 @@ static bool prog_args_trusted(const struct bpf_prog *prog)switch (prog->type) {case BPF_PROG_TYPE_TRACING : -return atype ==BPF_TRACE_RAW_TP || atype ==BPF_TRACE_ITER ; +switch (atype) { +case BPF_TRACE_RAW_TP : +case BPF_TRACE_ITER : +case BPF_MODIFY_RETURN : +return true; +default : +return false; + }case BPF_PROG_TYPE_LSM :return bpf_lsm_is_trusted (prog);case BPF_PROG_TYPE_STRUCT_OPS :
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 4/12 mm: introduce bpf_oom_kill_process() bpf kfunc
In reply to Roman GushchinIntroduce bpf_oom_kill_process() bpf kfunc, which is supposed to be used by bpf OOM programs. It allows to kill a process in exactly the same way the OOM killer does: using the OOM reaper, bumping corresponding memcg and global statistics, respecting memory.oom.group etc.
On success, it sets om_control's bpf_memory_freed field to true, enabling the bpf program to bypass the kernel OOM killer.
--- mm/oom_kill.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+)
mm/oom_kill.c +50 -0
@@ -1299,6 +1299,42 @@ SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags)#ifdef CONFIG_BPF_SYSCALL +__bpf_kfunc_start_defs (); +/* +* Kill a process in a way similar to the kernel OOM killer. +* This means dump the necessary information to dmesg, adjust memcg +* statistics, leverage the oom reaper, respect memory.oom.group etc. +* +* bpf_oom_kill_process() marks the forward progress by setting +* oc->bpf_memory_freed. If the progress was made, the bpf program +* is free to decide if the kernel oom killer should be invoked. +* Otherwise it's enforced, so that a bad bpf program can't +* deadlock the machine on memory. +*/ +__bpf_kfunc int bpf_oom_kill_process(struct oom_control *oc, +struct task_struct *task, +const char *message__str) +{ +if (oom_unkillable_task (task)) +return -EPERM ; + +/* paired with put_task_struct() in oom_kill_process() */ + task =tryget_task_struct (task); +if (!task) +return -EINVAL ; + + oc->chosen = task; + +oom_kill_process (oc, message__str); + + oc->chosen =NULL ; + oc->bpf_memory_freed = true; + +return 0 ; +} + +__bpf_kfunc_end_defs (); +__bpf_hook_start ();/*
@@ -1319,6 +1355,16 @@ static const struct btf_kfunc_id_set bpf_oom_hook_set = { .owner = THIS_MODULE, .set = &bpf_oom_hooks, }; + +BTF_KFUNCS_START (bpf_oom_kfuncs) +BTF_ID_FLAGS(func, bpf_oom_kill_process, KF_SLEEPABLE | KF_TRUSTED_ARGS) +BTF_KFUNCS_END (bpf_oom_kfuncs) + +static const struct btf_kfunc_id_set bpf_oom_kfunc_set = { + .owner =THIS_MODULE , + .set = &bpf_oom_kfuncs, +}; + static int __init bpf_oom_init(void) {int err;
@@ -1326,6 +1372,10 @@ static int __init bpf_oom_init(void) err =register_btf_fmodret_id_set (&bpf_oom_hook_set);if (err)pr_warn ("error while registering bpf oom hooks: %d" , err); + err =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , + &bpf_oom_kfunc_set); +if (err) +pr_warn ("error while registering bpf oom kfuncs: %d" , err);return err; }
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 5/12 mm: introduce bpf kfuncs to deal with memcg pointers
In reply to Roman GushchinTo effectively operate with memory cgroups in bpf there is a need to convert css pointers to memcg pointers. A simple container_of cast which is used in the kernel code can't be used in bpf because from the verifier's point of view that's a out-of-bounds memory access.
Introduce helper get/put kfuncs which can be used to get a refcounted memcg pointer from the css pointer:
- bpf_get_mem_cgroup,
- bpf_put_mem_cgroup.
bpf_get_mem_cgroup() can take both memcg's css and the corresponding cgroup's "self" css. It allows it to be used with the existing cgroup iterator which iterates over cgroup tree, not memcg tree.
--- include/linux/memcontrol.h | 2 + mm/Makefile | 3 ++ mm/bpf_memcontrol.c | 101 +++++++++++++++++++++++++++++++++++++ 3 files changed, 106 insertions(+) create mode 100644 mm/bpf_memcontrol.c
include/linux/memcontrol.h +2 -0
@@ -932,6 +932,8 @@ static inline void mod_memcg_page_state(struct page *page,rcu_read_unlock (); } +unsigned long memcg_events (struct mem_cgroup *memcg,int event); +unsigned long mem_cgroup_usage (struct mem_cgroup *memcg,bool swap);unsigned long memcg_page_state (struct mem_cgroup *memcg,int idx);unsigned long lruvec_page_state (struct lruvec *lruvec,enum node_stat_item idx); unsigned long lruvec_page_state_local(struct lruvec *lruvec,
mm/Makefile +3 -0
@@ -105,6 +105,9 @@ obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o ifdef CONFIG_SWAP obj-$(CONFIG_MEMCG) += swap_cgroup.o endif +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_MEMCG) += bpf_memcontrol.o +endif obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o obj-$(CONFIG_GUP_TEST) += gup_test.o obj-$(CONFIG_DMAPOOL_TEST) += dmapool_test.o
mm/bpf_memcontrol.c new file +101 -0
101 lines not on this page · raw message
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 6/12 mm: introduce bpf_get_root_mem_cgroup() bpf kfunc
In reply to Roman GushchinIntroduce a bpf kfunc to get a trusted pointer to the root memory cgroup. It's very handy to traverse the full memcg tree, e.g. for handling a system-wide OOM.
It's possible to obtain this pointer by traversing the memcg tree up from any known memcg, but it's sub-optimal and makes bpf programs more complex and less efficient.
bpf_get_root_mem_cgroup() has a KF_ACQUIRE | KF_RET_NULL semantics, however in reality it's not necessarily to bump the corresponding reference counter - root memory cgroup is immortal, reference counting is skipped, see css_get(). Once set, root_mem_cgroup is always a valid memcg pointer. It's safe to call bpf_put_mem_cgroup() for the pointer obtained with bpf_get_root_mem_cgroup(), it's effectively a no-op.
--- mm/bpf_memcontrol.c | 7 +++++++ 1 file changed, 7 insertions(+)
mm/bpf_memcontrol.c +7 -0
@@ -10,6 +10,12 @@__bpf_kfunc_start_defs (); +__bpf_kfunc struct mem_cgroup *bpf_get_root_mem_cgroup(void) +{ +/* css_get() is not needed */ +return root_mem_cgroup; +} + __bpf_kfunc struct mem_cgroup *bpf_get_mem_cgroup (struct cgroup_subsys_state *css) {
@@ -72,6 +78,7 @@ __bpf_kfunc void bpf_mem_cgroup_flush_stats(struct mem_cgroup *memcg)__bpf_kfunc_end_defs (); BTF_KFUNCS_START(bpf_memcontrol_kfuncs) +BTF_ID_FLAGS(func, bpf_get_root_mem_cgroup, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_get_mem_cgroup, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_put_mem_cgroup, KF_RELEASE)
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 7/12 bpf: selftests: introduce read_cgroup_file() helper
In reply to Roman GushchinImplement read_cgroup_file() helper to read from cgroup control files, e.g. statistics.
--- tools/testing/selftests/bpf/cgroup_helpers.c | 39 ++++++++++++++++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 2 + 2 files changed, 41 insertions(+)
tools/testing/selftests/bpf/cgroup_helpers.c +39 -0
@@ -125,6 +125,45 @@ int enable_controllers(const char *relative_path, const char *controllers)return __enable_controllers (cgroup_path, controllers); } +static size_t __read_cgroup_file (const char *cgroup_path,const char *file, +char *buf,size_t size) +{ +char file_path[PATH_MAX +1 ]; +size_t ret; +int fd; + +snprintf (file_path,sizeof (file_path),"%s/%s" , cgroup_path, file); + fd =open (file_path,O_RDONLY ); +if (fd <0 ) { +log_err ("Opening %s" , file_path); +return -1 ; + } + + ret =read (fd, buf, size); +close (fd); +return ret; +} + +/** +* read_cgroup_file() - Read to a cgroup file +* @relative_path: The cgroup path, relative to the workdir +* @file: The name of the file in cgroupfs to read to +* @buf: Buffer to read from the file +* @size: Size of the buffer +* +* Read to a file in the given cgroup's directory. +* +* If successful, the number of read bytes is returned. +*/ +size_t read_cgroup_file (const char *relative_path,const char *file, +char *buf,size_t size) +{ +char cgroup_path[PATH_MAX -24 ]; + +format_cgroup_path (cgroup_path, relative_path); +return __read_cgroup_file (cgroup_path, file, buf, size); +} +static int __write_cgroup_file (const char *cgroup_path,const char *file,const char *buf) {
tools/testing/selftests/bpf/cgroup_helpers.h +2 -0
@@ -11,6 +11,8 @@/* cgroupv2 related */ int enable_controllers (const char *relative_path,const char *controllers); +size_t read_cgroup_file (const char *relative_path,const char *file, +char *buf,size_t size);int write_cgroup_file (const char *relative_path,const char *file,const char *buf); int write_cgroup_file_parent(const char *relative_path, const char *file,
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 8/12 bpf: selftests: bpf OOM handler test
In reply to Roman GushchinImplement a pseudo-realistic test for the OOM handling functionality.
The OOM handling policy which is implemented in bpf is to kill all tasks belonging to the biggest leaf cgroup, which doesn't contain unkillable tasks (tasks with oom_score_adj set to -1000). Pagecache size is excluded from the accounting.
The test creates a hierarchy of memory cgroups, causes an OOM at the top level, checks that the expected process will be killed and checks memcg's oom statistics.
--- tools/testing/selftests/bpf/prog_tests/oom.c | 227 +++++++++++++++++++ tools/testing/selftests/bpf/progs/test_oom.c | 103 +++++++++ 2 files changed, 330 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/oom.c create mode 100644 tools/testing/selftests/bpf/progs/test_oom.c
tools/testing/selftests/bpf/prog_tests/oom.c new file +227 -0
@@ -0,0 +1,227 @@ +// SPDX-License-Identifier: GPL-2.0-only +#define _GNU_SOURCE + +#include <stdio.h> +#include <fcntl.h> +#include <unistd.h> +#include <stdlib.h> +#include <signal.h> +#include <sys/stat.h> +#include <test_progs.h> +#include <bpf/btf.h> +#include <bpf/bpf.h> + +#include "cgroup_helpers.h" +#include "test_oom.skel.h" + +struct cgroup_desc { +const char *path; +int fd; +unsigned long long id; +int pid; +size_t target; +size_t max; +int oom_score_adj; +bool victim; +}; + +#define MB (1024 * 1024) +#define OOM_SCORE_ADJ_MIN (-1000) +#define OOM_SCORE_ADJ_MAX 1000 + +static struct cgroup_desc cgroups[] = { + { .path ="/oom_test" , .max =80 *MB }, + { .path ="/oom_test/cg1" , .target =10 *MB , + .oom_score_adj =OOM_SCORE_ADJ_MAX }, + { .path ="/oom_test/cg2" , .target =40 *MB , + .oom_score_adj =OOM_SCORE_ADJ_MIN }, + { .path ="/oom_test/cg3" }, + { .path ="/oom_test/cg3/cg4" , .target =30 *MB , + .victim = true }, + { .path ="/oom_test/cg3/cg5" , .target =20 *MB }, +}; + +static int spawn_task (struct cgroup_desc *desc) +{ +char *ptr; +int pid; + + pid =fork (); +if (pid <0 ) +return pid; + +if (pid >0 ) { +/* parent */ + desc->pid = pid; +return 0 ; + } + +/* child */ +if (desc->oom_score_adj) { +char buf[64 ]; +int fd =open ("/proc/self/oom_score_adj" ,O_WRONLY ); + +if (fd <0 ) +return -1 ; + +snprintf (buf,sizeof (buf),"%d" , desc->oom_score_adj); +write (fd, buf,sizeof (buf)); +close (fd); + } + + ptr = (char *)malloc (desc->target); +if (!ptr) +return -ENOMEM ; + +memset (ptr,'a' , desc->target); + +while (1 ) +sleep (1000 ); + +return 0 ; +} + +static void setup_environment (void ) +{ +int i, err; + + err =setup_cgroup_environment (); +if (!ASSERT_OK (err,"setup_cgroup_environment" )) + goto cleanup; + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) { + cgroups[i].fd =create_and_get_cgroup (cgroups[i].path); +if (!ASSERT_GE (cgroups[i].fd,0 ,"create_and_get_cgroup" )) + goto cleanup; + + cgroups[i].id =get_cgroup_id (cgroups[i].path); +if (!ASSERT_GT (cgroups[i].id,0 ,"get_cgroup_id" )) + goto cleanup; + +if (i ==0 ) { +/* Freeze the top-level cgroup */ + err =write_cgroup_file (cgroups[i].path,"cgroup.freeze" ,"1" ); +if (!ASSERT_OK (err,"freeze cgroup" )) + goto cleanup; + } + +if (!cgroups[i].target) { +/* Recursively enable the memory controller */ + err =write_cgroup_file (cgroups[i].path,"cgroup.subtree_control" , +"+memory" ); +if (!ASSERT_OK (err,"enable memory controller" )) + goto cleanup; + } + +if (cgroups[i].max) { +char buf[256 ]; + +snprintf (buf,sizeof (buf),"%lu" , cgroups[i].max); + err =write_cgroup_file (cgroups[i].path,"memory.max" , buf); +if (!ASSERT_OK (err,"set memory.max" )) + goto cleanup; + +snprintf (buf,sizeof (buf),"0" ); +write_cgroup_file (cgroups[i].path,"memory.swap.max" , buf); + + } + +if (cgroups[i].target) { +char buf[256 ]; + + err =spawn_task (&cgroups[i]); +if (!ASSERT_OK (err,"spawn task" )) + goto cleanup; + +snprintf (buf,sizeof (buf),"%d" , cgroups[i].pid); + err =write_cgroup_file (cgroups[i].path,"cgroup.procs" , buf); +if (!ASSERT_OK (err,"put child into a cgroup" )) + goto cleanup; + } + } + +return ; + +cleanup: +cleanup_cgroup_environment (); +} + +static int run_and_wait_for_oom (void ) +{ +int ret =-1 ; +bool first = true; +char buf[4096 ] = {}; +size_t size; + + ret =write_cgroup_file (cgroups[0 ].path,"cgroup.freeze" ,"0" ); +if (!ASSERT_OK (ret,"freeze cgroup" )) +return -1 ; + +for (;;) { +int i, status; +pid_t pid =wait (&status); + +if (pid ==-1 ) { +if (errno ==EINTR ) +continue ; +/* ECHILD */ +break ; + } + +if (!first) +continue ; + + first = false; + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) { +if (!ASSERT_OK (cgroups[i].victim != + (pid == cgroups[i].pid), +"correct process was killed" )) { + ret =-1 ; +break ; + } + +if (!cgroups[i].victim) +continue ; + + size =read_cgroup_file (cgroups[i].path, +"memory.events" , + buf,sizeof (buf)); +if (!ASSERT_OK (size <=0 ,"read memory.events" )) { + ret =-1 ; +break ; + } + +if (!ASSERT_OK (strstr (buf,"oom_kill 1" ) ==NULL , +"oom_kill count check" )) { + ret =-1 ; +break ; + } + } + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) +if (cgroups[i].pid && cgroups[i].pid != pid) +kill (cgroups[i].pid,SIGKILL ); + } + +return ret; +} + +void test_oom (void ) +{ +struct test_oom *skel; +int err; + + skel =test_oom__open_and_load (); + err =test_oom__attach (skel); +if (!ASSERT_OK (err,"test_oom__attach" )) + goto cleanup; + +setup_environment (); + +run_and_wait_for_oom (); + +cleanup_cgroup_environment (); +cleanup: +test_oom__destroy (skel); +}
tools/testing/selftests/bpf/progs/test_oom.c new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 9/12 sched: psi: bpf hook to handle psi events
In reply to Roman GushchinIntroduce a bpf hook to handle psi events. The primary intended purpose of this hook is to declare OOM events based on the reaching a certain memory pressure level, similar to what systemd-oomd and oomd are doing in userspace.
--- kernel/sched/psi.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)
kernel/sched/psi.c +35 -1
@@ -176,6 +176,32 @@ static void psi_avgs_work(struct work_struct *work);static void poll_timer_fn (struct timer_list *t); +#ifdef CONFIG_BPF_SYSCALL +__bpf_hook_start (); + +__weak noinline int bpf_handle_psi_event(struct psi_trigger *t) +{ +return 0 ; +} + +__bpf_hook_end (); + +BTF_KFUNCS_START (bpf_psi_hooks) +BTF_ID_FLAGS (func ,bpf_handle_psi_event ,KF_SLEEPABLE ) +BTF_KFUNCS_END (bpf_psi_hooks) + +static const struct btf_kfunc_id_set bpf_psi_hook_set = { + .owner =THIS_MODULE , + .set = &bpf_psi_hooks, +}; + +#else +static inline int bpf_handle_psi_event (struct psi_trigger *t) +{ +return 0 ; +} +#endif +static void group_init (struct psi_group *group) {int cpu;
@@ -489,6 +515,7 @@ static void update_triggers(struct psi_group *group, u64 now,/* Generate an event */ if (cmpxchg (&t->event,0 ,1 ) ==0 ) { +bpf_handle_psi_event (t);if (t->of)kernfs_notify (t->of->kn);else
@@ -1655,6 +1682,8 @@ static const struct proc_ops psi_irq_proc_ops = {static int __init psi_proc_init (void ) { +int err =0 ; +if (psi_enable) {proc_mkdir ("pressure" ,NULL );proc_create ("pressure/io" ,0666 ,NULL , &psi_io_proc_ops);
@@ -1662,9 +1691,14 @@ static int __init psi_proc_init(void)proc_create ("pressure/cpu" ,0666 ,NULL , &psi_cpu_proc_ops);#ifdef CONFIG_IRQ_TIME_ACCOUNTING proc_create ("pressure/irq" ,0666 ,NULL , &psi_irq_proc_ops); +#endif +#ifdef CONFIG_BPF_SYSCALL + err =register_btf_fmodret_id_set (&bpf_psi_hook_set); +if (err) +pr_err ("error while registering bpf psi hooks: %d" , err);#endif } -return 0 ; +return err; }module_init (psi_proc_init);
Signature · 1 line
2.49.0.901.g37484f566f-goog
kernel test robot
build botRe: [PATCH rfc 09/12] sched: psi: bpf hook to handle psi events
kernel test robot noticed the following build warnings:
Hi Roman,
kernel test robot noticed the following build warnings:
[auto build test WARNING on akpm-mm/mm-everything]
url: https://github.com/intel-lab-lkp/linux/commits/Roman-Gushchin/mm-introduce-a-bpf-hook-for-OOM-handling/20250428-113742 base: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-everything patch link: https://lore.kernel.org/r/20250428033617.3797686-10-roman.gushchin%40linux.dev patch subject: [PATCH rfc 09/12] sched: psi: bpf hook to handle psi events config: sh-randconfig-001-20250428 (https://download.01.org/0day-ci/archive/20250428/[email protected]/config) compiler: sh4-linux-gcc (GCC) 10.5.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250428/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from kernel/sched/build_utility.c:94:
>> kernel/sched/psi.c:193:38: warning: 'bpf_psi_hook_set' defined but not used [-Wunused-const-variable=]
193 | static const struct btf_kfunc_id_set bpf_psi_hook_set = {
| ^~~~~~~~~~~~~~~~vim +/bpf_psi_hook_set +193 kernel/sched/psi.c
192
> 193 static const struct btf_kfunc_id_set bpf_psi_hook_set = { 194 .owner = THIS_MODULE,
195 .set = &bpf_psi_hooks,
196 };
197
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Suren Baghdasaryan
maintainerRe: [PATCH rfc 09/12] sched: psi: bpf hook to handle psi events
In reply to Roman GushchinQuoted 5 lines · Roman Gushchin
Introduce a bpf hook to handle psi events. The primary intended purpose of this hook is to declare OOM events based on the reaching a certain memory pressure level, similar to what systemd-oomd and oomd are doing in userspace.
It's a bit awkward that this requires additional userspace action to create PSI triggers. I have almost no experience with BPF, so this might be a stupid question, but maybe we could provide a bpf kfunc for the BPF handler to register its PSI trigger(s) upon handler registration?
Quoted 79 lines
Signed-off-by Roman Gushchin <[email protected]>--- kernel/sched/psi.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-)kernel/sched/psi.c +35
-2@@ -176,6 +176,32 @@ static void psi_avgs_work(struct work_struct *work);static void poll_timer_fn (struct timer_list *t); +#ifdef CONFIG_BPF_SYSCALL +__bpf_hook_start (); + +__weak noinline int bpf_handle_psi_event(struct psi_trigger *t) +{ +return 0 ; +} + +__bpf_hook_end (); + +BTF_KFUNCS_START (bpf_psi_hooks) +BTF_ID_FLAGS (func ,bpf_handle_psi_event ,KF_SLEEPABLE ) +BTF_KFUNCS_END (bpf_psi_hooks) + +static const struct btf_kfunc_id_set bpf_psi_hook_set = { + .owner =THIS_MODULE , + .set = &bpf_psi_hooks, +}; + +#else +static inline int bpf_handle_psi_event (struct psi_trigger *t) +{ +return 0 ; +} +#endif +static void group_init (struct psi_group *group) {int cpu;@@ -489,6 +515,7 @@ static void update_triggers(struct psi_group *group, u64 now,/* Generate an event */ if (cmpxchg (&t->event,0 ,1 ) ==0 ) { +bpf_handle_psi_event (t);if (t->of)kernfs_notify (t->of->kn);else @@ -1655,6 +1682,8 @@ static const struct proc_ops psi_irq_proc_ops = {static int __init psi_proc_init (void ) { +int err =0 ; +if (psi_enable) {proc_mkdir ("pressure" ,NULL );proc_create ("pressure/io" ,0666 ,NULL , &psi_io_proc_ops);@@ -1662,9 +1691,14 @@ static int __init psi_proc_init(void)proc_create ("pressure/cpu" ,0666 ,NULL , &psi_cpu_proc_ops);#ifdef CONFIG_IRQ_TIME_ACCOUNTING proc_create ("pressure/irq" ,0666 ,NULL , &psi_irq_proc_ops); +#endif +#ifdef CONFIG_BPF_SYSCALL + err =register_btf_fmodret_id_set (&bpf_psi_hook_set); +if (err) +pr_err ("error while registering bpf psi hooks: %d" , err);#endif } -return 0 ; +return err; }module_init (psi_proc_init); --2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRe: [PATCH rfc 09/12] sched: psi: bpf hook to handle psi events
Quoted 7 lines · Suren Baghdasaryan
Quoted 5 lines · Roman Gushchin
Introduce a bpf hook to handle psi events. The primary intended purpose of this hook is to declare OOM events based on the reaching a certain memory pressure level, similar to what systemd-oomd and oomd are doing in userspace.
Quoted 5 lines
It's a bit awkward that this requires additional userspace action to create PSI triggers. I have almost no experience with BPF, so this might be a stupid question, but maybe we could provide a bpf kfunc for the BPF handler to register its PSI trigger(s) upon handler registration?
It looks like it's doable using struct_ops path: the .init callback can create psi triggers and "attach" them to the loaded bpf program. But I need to figure out the details.
Good point, thank you!
Roman Gushchin
maintainerRFC PATCH 10/12 mm: introduce bpf_out_of_memory() bpf kfunc
In reply to Roman GushchinIntroduce bpf_out_of_memory() bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.
It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with -EBUSY.
The function is declared as sleepable. It guarantees that it won't be called from an atomic context. It's required by the OOM handling code, which is not guaranteed to work in a non-blocking context.
Handling of a memcg OOM almost always requires taking of the css_set_lock spinlock. The fact that bpf_out_of_memory() is sleepable also guarantees that it can't be called with acquired css_set_lock, so the kernel can't deadlock on it.
--- mm/oom_kill.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
mm/oom_kill.c +22 -0
@@ -1333,6 +1333,27 @@ __bpf_kfunc int bpf_oom_kill_process(struct oom_control *oc,return 0 ; } +__bpf_kfunc int bpf_out_of_memory(struct mem_cgroup *memcg__nullable, int order) +{ +struct oom_control oc = { + .memcg = memcg__nullable, + .order = order, + }; +int ret = -EINVAL ; + +if (oc.order <0 || oc.order >MAX_PAGE_ORDER ) + goto out; + + ret = -EBUSY ; +if (mutex_trylock (&oom_lock)) { + ret =out_of_memory (&oc); +mutex_unlock (&oom_lock); + } + +out: +return ret; +} +__bpf_kfunc_end_defs ();__bpf_hook_start ();
@@ -1358,6 +1379,7 @@ static const struct btf_kfunc_id_set bpf_oom_hook_set = { BTF_KFUNCS_START(bpf_oom_kfuncs) BTF_ID_FLAGS(func, bpf_oom_kill_process, KF_SLEEPABLE | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_out_of_memory, KF_SLEEPABLE | KF_TRUSTED_ARGS) BTF_KFUNCS_END(bpf_oom_kfuncs) static const struct btf_kfunc_id_set bpf_oom_kfunc_set = {
Signature · 1 line
2.49.0.901.g37484f566f-goog
Michal Hocko
maintainerRe: [PATCH rfc 10/12] mm: introduce bpf_out_of_memory() bpf kfunc
Quoted 10 lines · Roman Gushchin
Introduce
bpf_out_of_memory()bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with
-EBUSY.
This makes sense for the global OOM handler because concurrent handlers are cooperative. But is this really correct for memcg ooms which could happen for different hierarchies? Currently we do block on oom_lock in that case to make sure one oom doesn't starve others. Do we want the same behavior for custom OOM handlers?
Signature · 2 lines
Michal Hocko SUSE Labs
Roman Gushchin
maintainerRe: [PATCH rfc 10/12] mm: introduce bpf_out_of_memory() bpf kfunc
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Roman Gushchin
Introduce
bpf_out_of_memory()bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with
-EBUSY.
Quoted 5 lines
This makes sense for the global OOM handler because concurrent handlers are cooperative. But is this really correct for memcg ooms which could happen for different hierarchies? Currently we do block on
oom_lockin that case to make sure one oom doesn't starve others. Do we want the same behavior for custom OOM handlers?
It's a good point and I had similar thoughts when I was working on it. But I think it's orthogonal to the customization of the oom handling. Even for the existing oom killer it makes no sense to serialize memcg ooms in independent memcg subtrees. But I'm worried about the dmesg reporting, it can become really messy for 2+ concurrent OOMs.
Also, some memory can be shared, so one OOM can eliminate a need for another OOM, even if they look independent.
So my conclusion here is to leave things as they are until we'll get signs of real world problems with the (lack of) concurrency between ooms.
Thanks
Michal Hocko
maintainerRe: [PATCH rfc 10/12] mm: introduce bpf_out_of_memory() bpf kfunc
Quoted 25 lines · Roman Gushchin
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Roman Gushchin
Introduce
bpf_out_of_memory()bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with
-EBUSY.Quoted 5 lines
This makes sense for the global OOM handler because concurrent handlers are cooperative. But is this really correct for memcg ooms which could happen for different hierarchies? Currently we do block on
oom_lockin that case to make sure one oom doesn't starve others. Do we want the same behavior for custom OOM handlers?It's a good point and I had similar thoughts when I was working on it. But I think it's orthogonal to the customization of the oom handling. Even for the existing oom killer it makes no sense to serialize memcg ooms in independent memcg subtrees. But I'm worried about the dmesg reporting, it can become really messy for 2+ concurrent OOMs.
Quoted 5 lines
Also, some memory can be shared, so one OOM can eliminate a need for another OOM, even if they look independent.
So my conclusion here is to leave things as they are until we'll get signs of real world problems with the (lack of) concurrency between ooms.
How do we learn about that happening though? I do not think we have any counters to watch to suspect that some oom handlers cannot run.
Signature · 2 lines
Michal Hocko SUSE Labs
Roman Gushchin
maintainerRe: [PATCH rfc 10/12] mm: introduce bpf_out_of_memory() bpf kfunc
Quoted 26 lines · Michal Hocko
Quoted 25 lines · Roman Gushchin
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Roman Gushchin
Introduce
bpf_out_of_memory()bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with
-EBUSY.Quoted 5 lines
This makes sense for the global OOM handler because concurrent handlers are cooperative. But is this really correct for memcg ooms which could happen for different hierarchies? Currently we do block on
oom_lockin that case to make sure one oom doesn't starve others. Do we want the same behavior for custom OOM handlers?It's a good point and I had similar thoughts when I was working on it. But I think it's orthogonal to the customization of the oom handling. Even for the existing oom killer it makes no sense to serialize memcg ooms in independent memcg subtrees. But I'm worried about the dmesg reporting, it can become really messy for 2+ concurrent OOMs.
Quoted 8 lines
Quoted 5 lines
Also, some memory can be shared, so one OOM can eliminate a need for another OOM, even if they look independent.
So my conclusion here is to leave things as they are until we'll get signs of real world problems with the (lack of) concurrency between ooms.
How do we learn about that happening though? I do not think we have any counters to watch to suspect that some oom handlers cannot run.
The bpf program which declares an OOM can handle this: e.g. retry, wait and retry, etc. We can also try to mimick the existing behavior and wait on oom_lock (potentially splitting it into multiple locks to support concurrent ooms in various memcgs). Do you think it's preferable?
Michal Hocko
maintainerRe: [PATCH rfc 10/12] mm: introduce bpf_out_of_memory() bpf kfunc
Quoted 33 lines · Roman Gushchin
Quoted 32 lines · Michal Hocko
Quoted 30 lines · Roman Gushchin
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Roman Gushchin
Introduce
bpf_out_of_memory()bpf kfunc, which allows to declare an out of memory events and trigger the corresponding kernel OOM handling mechanism.It takes a trusted memcg pointer (or NULL for system-wide OOMs) as an argument, as well as the page order.
Only one OOM can be declared and handled in the system at once, so if the function is called in parallel to another OOM handling, it bails out with
-EBUSY.Quoted 5 lines
This makes sense for the global OOM handler because concurrent handlers are cooperative. But is this really correct for memcg ooms which could happen for different hierarchies? Currently we do block on
oom_lockin that case to make sure one oom doesn't starve others. Do we want the same behavior for custom OOM handlers?It's a good point and I had similar thoughts when I was working on it. But I think it's orthogonal to the customization of the oom handling. Even for the existing oom killer it makes no sense to serialize memcg ooms in independent memcg subtrees. But I'm worried about the dmesg reporting, it can become really messy for 2+ concurrent OOMs.
Also, some memory can be shared, so one OOM can eliminate a need for another OOM, even if they look independent.
So my conclusion here is to leave things as they are until we'll get signs of real world problems with the (lack of) concurrency between ooms.
Quoted 7 lines
Quoted 2 lines
How do we learn about that happening though? I do not think we have any counters to watch to suspect that some oom handlers cannot run.
The bpf program which declares an OOM can handle this: e.g. retry, wait and retry, etc. We can also try to mimick the existing behavior and wait on
oom_lock(potentially splitting it into multiple locks to support concurrent ooms in various memcgs). Do you think it's preferable?
Yes, I would just provide different callbacks for global and memcg ooms and do the blockin for the latter. It will be consistent with the in kernel implementation (therefore less surprising behavior).
Signature · 2 lines
Michal Hocko SUSE Labs
Roman Gushchin
maintainerRFC PATCH 11/12 bpf: selftests: introduce open_cgroup_file() helper
In reply to Roman GushchinImplement the open_cgroup_file() helper which opens a cgroup control file with the given flags and returns a file descriptor.
It's useful when a test needs to do something more sophisticated than read/write, e.g. listen for poll events or keep the file descriptor open.
--- tools/testing/selftests/bpf/cgroup_helpers.c | 28 ++++++++++++++++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 1 + 2 files changed, 29 insertions(+)
tools/testing/selftests/bpf/cgroup_helpers.c +28 -0
@@ -125,6 +125,34 @@ int enable_controllers(const char *relative_path, const char *controllers)return __enable_controllers (cgroup_path, controllers); } +static int __open_cgroup_file (const char *cgroup_path,const char *file, +int flags) +{ +char file_path[PATH_MAX +1 ]; + +snprintf (file_path,sizeof (file_path),"%s/%s" , cgroup_path, file); +return open (file_path, flags); +} + +/** +* open_cgroup_file() - Open a cgroup file +* @relative_path: The cgroup path, relative to the workdir +* @file: The name of the file in cgroupfs to open to +* @flags: Flags +* +* Open a file in the given cgroup's directory. +* +* If successful, fd is returned. +*/ +int open_cgroup_file (const char *relative_path,const char *file, +int flags) +{ +char cgroup_path[PATH_MAX -24 ]; + +format_cgroup_path (cgroup_path, relative_path); +return __open_cgroup_file (cgroup_path, file, flags); +} +static size_t __read_cgroup_file (const char *cgroup_path,const char *file,char *buf,size_t size) {
tools/testing/selftests/bpf/cgroup_helpers.h +1 -0
@@ -11,6 +11,7 @@/* cgroupv2 related */ int enable_controllers (const char *relative_path,const char *controllers); +int open_cgroup_file (const char *relative_path,const char *file,int flags);size_t read_cgroup_file (const char *relative_path,const char *file,char *buf,size_t size); int write_cgroup_file(const char *relative_path, const char *file,
Signature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerRFC PATCH 12/12 bpf: selftests: psi handler test
In reply to Roman GushchinAdd a psi handler test. The test creates a cgroup with two child sub-cgroups, sets up memory.high for one of those and puts memory hungry processes in each of them.
Then it sets up a psi trigger for one of cgroups and waits till the process in this cgroup will be killed by the OOM killer. To make sure there was indeed an OOM event, it checks the corresponding memcg statistics.
--- tools/testing/selftests/bpf/prog_tests/psi.c | 234 +++++++++++++++++++ tools/testing/selftests/bpf/progs/test_psi.c | 43 ++++ 2 files changed, 277 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/psi.c create mode 100644 tools/testing/selftests/bpf/progs/test_psi.c
tools/testing/selftests/bpf/prog_tests/psi.c new file +234 -0
@@ -0,0 +1,234 @@ +// SPDX-License-Identifier: GPL-2.0-only +#define _GNU_SOURCE + +#include <stdio.h> +#include <fcntl.h> +#include <unistd.h> +#include <stdlib.h> +#include <signal.h> +#include <sys/stat.h> +#include <test_progs.h> +#include <bpf/btf.h> +#include <bpf/bpf.h> +#include <errno.h> +#include <string.h> + +#include "cgroup_helpers.h" +#include "test_psi.skel.h" + +struct cgroup_desc { +const char *path; +int fd; +unsigned long long id; +int pid; +size_t target; +size_t high; +bool victim; +bool psi; +}; + +#define MB (1024 * 1024) + +static struct cgroup_desc cgroups[] = { + { .path ="/oom_test" }, + { .path ="/oom_test/cg1" , .target =100 *MB }, + { .path ="/oom_test/cg2" , .target =500 *MB , + .high =40 *MB , .psi = true, .victim = true }, +}; + +static int spawn_task (struct cgroup_desc *desc) +{ +char *ptr; +int pid; + + pid =fork (); +if (pid <0 ) +return pid; + +if (pid >0 ) { +/* parent */ + desc->pid = pid; +return 0 ; + } + +/* child */ + ptr = (char *)malloc (desc->target); +if (!ptr) +return -ENOMEM ; + +memset (ptr,'a' , desc->target); + +while (1 ) +sleep (1000 ); + +return 0 ; +} + +static int setup_psi_alert (struct cgroup_desc *desc) +{ +const char *trig ="some 100000 1000000" ; +int fd; + + fd =open_cgroup_file (desc->path,"memory.pressure" ,O_RDWR ); +if (fd <0 ) { +printf ("memory.pressure open error: %s\n" ,strerror (errno)); +return 1 ; + } + +if (write (fd, trig,strlen (trig) +1 ) <0 ) { +printf ("memory.pressure write error: %s\n" ,strerror (errno)); +return 1 ; + } + +/* keep fd open, otherwise the psi trigger will be deleted */ +return 0 ; +} + +static void setup_environment (void ) +{ +int i, err; + + err =setup_cgroup_environment (); +if (!ASSERT_OK (err,"setup_cgroup_environment" )) + goto cleanup; + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) { + cgroups[i].fd =create_and_get_cgroup (cgroups[i].path); +if (!ASSERT_GE (cgroups[i].fd,0 ,"create_and_get_cgroup" )) + goto cleanup; + + cgroups[i].id =get_cgroup_id (cgroups[i].path); +if (!ASSERT_GT (cgroups[i].id,0 ,"get_cgroup_id" )) + goto cleanup; + +if (i ==0 ) { +/* Freeze the top-level cgroup */ + err =write_cgroup_file (cgroups[i].path,"cgroup.freeze" ,"1" ); +if (!ASSERT_OK (err,"freeze cgroup" )) + goto cleanup; + } + +if (!cgroups[i].target) { +/* Recursively enable the memory controller */ + err =write_cgroup_file (cgroups[i].path,"cgroup.subtree_control" , +"+memory" ); +if (!ASSERT_OK (err,"enable memory controller" )) + goto cleanup; + } + +if (cgroups[i].high) { +char buf[256 ]; + +snprintf (buf,sizeof (buf),"%lu" , cgroups[i].high); + err =write_cgroup_file (cgroups[i].path,"memory.high" , buf); +if (!ASSERT_OK (err,"set memory.high" )) + goto cleanup; + +snprintf (buf,sizeof (buf),"0" ); +write_cgroup_file (cgroups[i].path,"memory.swap.max" , buf); + } + +if (cgroups[i].target) { +char buf[256 ]; + + err =spawn_task (&cgroups[i]); +if (!ASSERT_OK (err,"spawn task" )) + goto cleanup; + +snprintf (buf,sizeof (buf),"%d" , cgroups[i].pid); + err =write_cgroup_file (cgroups[i].path,"cgroup.procs" , buf); +if (!ASSERT_OK (err,"put child into a cgroup" )) + goto cleanup; + } + +if (cgroups[i].psi) { + err =setup_psi_alert (&cgroups[i]); +if (!ASSERT_OK (err,"create psi trigger" )) + goto cleanup; + } + } + +return ; + +cleanup: +cleanup_cgroup_environment (); +} + +static int run_and_wait_for_oom (void ) +{ +int ret =-1 ; +bool first = true; +char buf[4096 ] = {}; +size_t size; + + ret =write_cgroup_file (cgroups[0 ].path,"cgroup.freeze" ,"0" ); +if (!ASSERT_OK (ret,"freeze cgroup" )) +return -1 ; + +for (;;) { +int i, status; +pid_t pid =wait (&status); + +if (pid ==-1 ) { +if (errno ==EINTR ) +continue ; +/* ECHILD */ +break ; + } + +if (!first) +continue ; + + first = false; + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) { +if (!ASSERT_OK (cgroups[i].victim != + (pid == cgroups[i].pid), +"correct process was killed" )) { + ret =-1 ; +break ; + } + +if (!cgroups[i].victim) +continue ; + + size =read_cgroup_file (cgroups[i].path,"memory.events" , + buf,sizeof (buf)); +if (!ASSERT_OK (size <=0 ,"read memory.events" )) { + ret =-1 ; +break ; + } + +if (!ASSERT_OK (strstr (buf,"oom_kill 1" ) ==NULL , +"oom_kill count check" )) { + ret =-1 ; +break ; + } + } + +for (i =0 ; i <ARRAY_SIZE (cgroups); i++) +if (cgroups[i].pid && cgroups[i].pid != pid) +kill (cgroups[i].pid,SIGKILL ); + } + +return ret; +} + +void test_psi (void ) +{ +struct test_psi *skel; +int err; + + skel =test_psi__open_and_load (); + err =test_psi__attach (skel); +if (!ASSERT_OK (err,"test_psi__attach" )) + goto cleanup; + +setup_environment (); + +run_and_wait_for_oom (); + +cleanup_cgroup_environment (); +cleanup: +test_psi__destroy (skel); +}
tools/testing/selftests/bpf/progs/test_psi.c new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.49.0.901.g37484f566f-goog
Quoted 50 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 9 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
Thinking out loud now, but rather than introducing and having a single BPF-specific function/interface, and BPF program for that matter, which can effectively be used to short-circuit steps from within out_of_memory(), why not introduce a tcp_congestion_ops/sched_ext_ops-like interface which essentially provides a multifaceted interface for controlling OOM killing (->select_bad_process, ->oom_kill_process, etc), optionally also from the context of a BPF program (BPF_PROG_TYPE_STRUCT_OPS)?
I don't know whether that's what you meant by introducing a new attachment, or program type, but an approach like this is what immediately comes to mind when wanting to provide more than a single implementation for a set of operations within the Linux kernel, particularly also from the context of a BPF program.
Quoted 51 lines
2) Currently lockdep complaints about a potential circular dependency because sleepable
bpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introducebpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?
4) Documentation is lacking and will be added in the next version.Roman Gushchin (12):
mm: introduce a bpf hook for OOM handling
bpf: mark struct oom_control's memcg field asTRUSTED_OR_NULL
bpf: treat fmodret tracing program's arguments as trusted
mm: introducebpf_oom_kill_process()bpf kfunc
mm: introduce bpf kfuncs to deal with memcg pointers
mm: introducebpf_get_root_mem_cgroup()bpf kfunc
bpf: selftests: introduceread_cgroup_file()helper
bpf: selftests: bpf OOM handler test
sched: psi: bpf hook to handle psi events
mm: introducebpf_out_of_memory()bpf kfunc
bpf: selftests: introduceopen_cgroup_file()helper
bpf: selftests: psi handler testinclude/linux/memcontrol.h | 2 + include/linux/oom.h | 5 + kernel/bpf/btf.c | 9 +- kernel/bpf/verifier.c | 5 + kernel/sched/psi.c | 36 ++- mm/Makefile | 3 + mm/bpf_memcontrol.c | 108 +++++++++ mm/oom_kill.c | 140 +++++++++++ tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 3 + tools/testing/selftests/bpf/prog_tests/oom.c | 227 ++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/psi.c | 234 +++++++++++++++++++ tools/testing/selftests/bpf/progs/test_oom.c | 103 ++++++++ tools/testing/selftests/bpf/progs/test_psi.c | 43 ++++ 14 files changed, 983 insertions(+), 2 deletions(-) create mode 100644 mm/bpf_memcontrol.c create mode 100644 tools/testing/selftests/bpf/prog_tests/oom.c create mode 100644 tools/testing/selftests/bpf/prog_tests/psi.c create mode 100644 tools/testing/selftests/bpf/progs/test_oom.c create mode 100644 tools/testing/selftests/bpf/progs/test_psi.cSignature · 1 line
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerQuoted 61 lines · Matt Bobrowski
Quoted 50 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 9 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
Quoted 8 lines
Thinking out loud now, but rather than introducing and having a single BPF-specific function/interface, and BPF program for that matter, which can effectively be used to short-circuit steps from within
out_of_memory(), why not introduce a tcp_congestion_ops/sched_ext_ops-like interface which essentially provides a multifaceted interface for controlling OOM killing (->select_bad_process, ->oom_kill_process, etc), optionally also from the context of a BPF program (BPF_PROG_TYPE_STRUCT_OPS)?
It's certainly an option and I thought about it. I don't think we need a bunch of hooks though. This patchset adds 2 and they belong to completely different subsystems (mm and sched/psi), so Idk how well they can be gathered into a single struct ops. But maybe it's fine.
The only potentially new hook I can envision now is one to customize the oom reporting.
Thanks for the suggestion!
Kumar Kartikeya Dwivedi
maintainerQuoted 72 lines · Roman Gushchin
Quoted 61 lines · Matt Bobrowski
Quoted 50 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 9 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.Quoted 8 lines
Thinking out loud now, but rather than introducing and having a single BPF-specific function/interface, and BPF program for that matter, which can effectively be used to short-circuit steps from within
out_of_memory(), why not introduce a tcp_congestion_ops/sched_ext_ops-like interface which essentially provides a multifaceted interface for controlling OOM killing (->select_bad_process, ->oom_kill_process, etc), optionally also from the context of a BPF program (BPF_PROG_TYPE_STRUCT_OPS)?
Quoted 8 lines
It's certainly an option and I thought about it. I don't think we need a bunch of hooks though. This patchset adds 2 and they belong to completely different subsystems (mm and sched/psi), so Idk how well they can be gathered into a single struct ops. But maybe it's fine.
The only potentially new hook I can envision now is one to customize the oom reporting.
If you're considering scoping it down to a particular cgroup (as you allude to in the TODO), or building a hierarchical interface, using struct_ops will be much better than fmod_ret etc., which is global in nature. Even if you don't support it now. I don't think a struct_ops is warranted only when you have more than a few callbacks. As an illustration, sched_ext started out without supporting hierarchical attachment, but will piggy-back on the struct_ops interface to do so in the near future.
Quoted 3 lines
Thanks for the suggestion!
Roman Gushchin
maintainerQuoted 82 lines · Kumar Kartikeya Dwivedi
Quoted 72 lines · Roman Gushchin
Quoted 61 lines · Matt Bobrowski
Quoted 50 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 9 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.Quoted 8 lines
Thinking out loud now, but rather than introducing and having a single BPF-specific function/interface, and BPF program for that matter, which can effectively be used to short-circuit steps from within
out_of_memory(), why not introduce a tcp_congestion_ops/sched_ext_ops-like interface which essentially provides a multifaceted interface for controlling OOM killing (->select_bad_process, ->oom_kill_process, etc), optionally also from the context of a BPF program (BPF_PROG_TYPE_STRUCT_OPS)?Quoted 8 lines
It's certainly an option and I thought about it. I don't think we need a bunch of hooks though. This patchset adds 2 and they belong to completely different subsystems (mm and sched/psi), so Idk how well they can be gathered into a single struct ops. But maybe it's fine.
The only potentially new hook I can envision now is one to customize the oom reporting.
Quoted 8 lines
If you're considering scoping it down to a particular cgroup (as you allude to in the TODO), or building a hierarchical interface, using
struct_opswill be much better thanfmod_retetc., which is global in nature. Even if you don't support it now. I don't think astruct_opsis warranted only when you have more than a few callbacks. As an illustration,sched_extstarted out without supporting hierarchical attachment, but will piggy-back on thestruct_opsinterface to do so in the near future.
Good point! I agree.
Thanks
Song Liu
reviewerOn Mon, Apr 28, 2025 at 6:57 PM Kumar Kartikeya Dwivedi <[email protected]> wrote:
[...]
Quoted 10 lines
Quoted 9 lines
It's certainly an option and I thought about it. I don't think we need a bunch of hooks though. This patchset adds 2 and they belong to completely different subsystems (mm and sched/psi), so Idk how well they can be gathered into a single struct ops. But maybe it's fine.
The only potentially new hook I can envision now is one to customize the oom reporting.
Quoted 8 lines
If you're considering scoping it down to a particular cgroup (as you allude to in the TODO), or building a hierarchical interface, using
struct_opswill be much better thanfmod_retetc., which is global in nature. Even if you don't support it now. I don't think astruct_opsis warranted only when you have more than a few callbacks. As an illustration,sched_extstarted out without supporting hierarchical attachment, but will piggy-back on thestruct_opsinterface to do so in the near future.
+1 for using struct_ops, which is the best way to enable BPF in existing use cases.
Song
Michal Hocko
maintainerQuoted 12 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
Quoted 5 lines
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
Makes sense to me. I still have a slight concern though. We have 3 different oom handlers smashed into a single one with special casing involved. This is manageable (although not great) for the in kernel code but I am wondering whether we should do better for BPF based OOM implementations. Would it make sense to have different callbacks for cpuset, memcg and global oom killer handlers?
I can see you have already added some helper functions to deal with memcgs but I do not see anything to iterate processes or find a process to kill etc. Is that functionality generally available (sorry I am not really familiar with BPF all that much so please bear with me)?
I like the way how you naturalely hooked into existing OOM primitives like oom_kill_process but I do not see tsk_is_oom_victim exposed. Are you waiting for a first user that needs to implement oom victim synchronization or do you plan to integrate that into tasks iterators? I am mostly asking because it is exactly these kind of details that make the current in kernel oom handler quite complex and it would be great if custom ones do not have to reproduce that complexity and only focus on the high level policy.
Quoted 8 lines
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls.
This makes sense to me as well. I have to admit I am not fully familiar with PSI integration into sched code but from what I can see the evaluation is done on regular bases from the worker context kicked off from the scheduler code. There shouldn't be any locking constrains which is good. Is there any risk if the oom handler took too long though?
Also an important question. I can see selftests which are using the infrastructure. But have you tried to implement a real OOM handler with this proposed infrastructure?
Quoted 7 lines
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 13 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepablebpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).
I cannot see this in the code. Could you be more specific please? Where is this might_fault coming from? Is this BPF constrain?
Quoted 6 lines
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introduce
bpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?
Yes, let's start simple and see where we get from there.
Quoted 1 line
4) Documentation is lacking and will be added in the next version.
+1
Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Roman Gushchin
maintainerQuoted 19 lines · Michal Hocko
Quoted 12 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
Quoted 5 lines
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
Quoted 6 lines
Makes sense to me. I still have a slight concern though. We have 3 different oom handlers smashed into a single one with special casing involved. This is manageable (although not great) for the in kernel code but I am wondering whether we should do better for BPF based OOM implementations. Would it make sense to have different callbacks for cpuset, memcg and global oom killer handlers?
Yes, it's certainly possible. If we go struct_ops path, we can even have both the common hook which handles all types of OOM's and separate hooks for each type. The user then can choose what's more convenient. Good point.
Quoted 5 lines
I can see you have already added some helper functions to deal with memcgs but I do not see anything to iterate processes or find a process to kill etc. Is that functionality generally available (sorry I am not really familiar with BPF all that much so please bear with me)?
Yes, task iterator is available since v6.7:
https://docs.ebpf.io/linux/kfuncs/bpf_iter_task_new/
Quoted 5 lines
I like the way how you naturalely hooked into existing OOM primitives like
oom_kill_processbut I do not seetsk_is_oom_victimexposed. Are you waiting for a first user that needs to implement oom victim synchronization or do you plan to integrate that into tasks iterators?
It can be implemented in bpf directly, but I agree that it probably deserves at least an example in the test or a separate in-kernel helper. In-kernel helper is probably a better idea.
Quoted 4 lines
I am mostly asking because it is exactly these kind of details that make the current in kernel oom handler quite complex and it would be great if custom ones do not have to reproduce that complexity and only focus on the high level policy.
Totally agree.
Quoted 10 lines
Quoted 9 lines
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls.
Quoted 5 lines
This makes sense to me as well. I have to admit I am not fully familiar with PSI integration into sched code but from what I can see the evaluation is done on regular bases from the worker context kicked off from the scheduler code. There shouldn't be any locking constrains which is good. Is there any risk if the oom handler took too long though?
It's a good question. In theory yes, it can affect the timing of other PSI events. An option here is to move it into a separate work, however I'm not sure if it worth the added complexity. I actually tried this approach in an earlier version of this patchset, but the problem was that the code for scheduling this work should be dynamically turned on/off when a bpf program is attached/detached, otherwise it's an obvious cpu overhead.
It's doable, but Idk if it's justified.
Quoted 4 lines
Also an important question. I can see selftests which are using the infrastructure. But have you tried to implement a real OOM handler with this proposed infrastructure?
Not yet. Given the size and complexity of the infrastructure of my current employer, it's not a short process. But we're working on it.
Quoted 22 lines
Quoted 8 lines
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 13 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepablebpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).
Quoted 2 lines
I cannot see this in the code. Could you be more specific please? Where is this
might_faultcoming from? Is this BPF constrain?
It's in __bpf_prog_enter_sleepable(). But I hope I can make this hook non-sleepable (by going struct_ops path) and the problem will go away.
Quoted 9 lines
Quoted 7 lines
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introduce
bpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?Yes, let's start simple and see where we get from there.
Agree.
Thank you for taking a look and your comments/ideas!
Suren Baghdasaryan
maintainerQuoted 86 lines · Roman Gushchin
Quoted 19 lines · Michal Hocko
Quoted 12 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
Quoted 5 lines
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
Quoted 6 lines
Makes sense to me. I still have a slight concern though. We have 3 different oom handlers smashed into a single one with special casing involved. This is manageable (although not great) for the in kernel code but I am wondering whether we should do better for BPF based OOM implementations. Would it make sense to have different callbacks for cpuset, memcg and global oom killer handlers?
Yes, it's certainly possible. If we go
struct_opspath, we can even have both the common hook which handles all types of OOM's and separate hooks for each type. The user then can choose what's more convenient. Good point.Quoted 5 lines
I can see you have already added some helper functions to deal with memcgs but I do not see anything to iterate processes or find a process to kill etc. Is that functionality generally available (sorry I am not really familiar with BPF all that much so please bear with me)?
Yes, task iterator is available since v6.7:
https://docs.ebpf.io/linux/kfuncs/bpf_iter_task_new/Quoted 5 lines
I like the way how you naturalely hooked into existing OOM primitives like
oom_kill_processbut I do not seetsk_is_oom_victimexposed. Are you waiting for a first user that needs to implement oom victim synchronization or do you plan to integrate that into tasks iterators?It can be implemented in bpf directly, but I agree that it probably deserves at least an example in the test or a separate in-kernel helper. In-kernel helper is probably a better idea.
Quoted 4 lines
I am mostly asking because it is exactly these kind of details that make the current in kernel oom handler quite complex and it would be great if custom ones do not have to reproduce that complexity and only focus on the high level policy.
Totally agree.
Quoted 10 lines
Quoted 9 lines
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls.
Quoted 5 lines
This makes sense to me as well. I have to admit I am not fully familiar with PSI integration into sched code but from what I can see the evaluation is done on regular bases from the worker context kicked off from the scheduler code. There shouldn't be any locking constrains which is good. Is there any risk if the oom handler took too long though?
It's a good question. In theory yes, it can affect the timing of other PSI events. An option here is to move it into a separate work, however I'm not sure if it worth the added complexity. I actually tried this approach in an earlier version of this patchset, but the problem was that the code for scheduling this work should be dynamically turned on/off when a bpf program is attached/detached, otherwise it's an obvious cpu overhead.
It's doable, but Idk if it's justified.Quoted 1 line
Quoted 6 lines
Quoted 3 lines
Also an important question. I can see selftests which are using the infrastructure. But have you tried to implement a real OOM handler with this proposed infrastructure?
Not yet. Given the size and complexity of the infrastructure of my current employer, it's not a short process. But we're working on it.
Hi Roman,
This might end up being very useful for Android. Since we have a shared current employer, we might be able to provide an earlier test environment for this concept on Android and speed up development of a real OOM handler. I'll be following the development of this patchset and will see if we can come up with an early prototype for testing.
Quoted 43 lines
Quoted 23 lines
Quoted 9 lines
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 13 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepablebpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).Quoted 2 lines
I cannot see this in the code. Could you be more specific please? Where is this
might_faultcoming from? Is this BPF constrain?It's in
__bpf_prog_enter_sleepable(). But I hope I can make this hook non-sleepable (by goingstruct_opspath) and the problem will go away.Quoted 9 lines
Quoted 7 lines
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introduce
bpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?Yes, let's start simple and see where we get from there.
Agree.
Thank you for taking a look and your comments/ideas!
Roman Gushchin
maintainerQuoted 94 lines · Suren Baghdasaryan
Quoted 86 lines · Roman Gushchin
Quoted 19 lines · Michal Hocko
Quoted 12 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
Quoted 5 lines
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
Quoted 6 lines
Makes sense to me. I still have a slight concern though. We have 3 different oom handlers smashed into a single one with special casing involved. This is manageable (although not great) for the in kernel code but I am wondering whether we should do better for BPF based OOM implementations. Would it make sense to have different callbacks for cpuset, memcg and global oom killer handlers?
Yes, it's certainly possible. If we go
struct_opspath, we can even have both the common hook which handles all types of OOM's and separate hooks for each type. The user then can choose what's more convenient. Good point.Quoted 5 lines
I can see you have already added some helper functions to deal with memcgs but I do not see anything to iterate processes or find a process to kill etc. Is that functionality generally available (sorry I am not really familiar with BPF all that much so please bear with me)?
Yes, task iterator is available since v6.7:
https://docs.ebpf.io/linux/kfuncs/bpf_iter_task_new/Quoted 5 lines
I like the way how you naturalely hooked into existing OOM primitives like
oom_kill_processbut I do not seetsk_is_oom_victimexposed. Are you waiting for a first user that needs to implement oom victim synchronization or do you plan to integrate that into tasks iterators?It can be implemented in bpf directly, but I agree that it probably deserves at least an example in the test or a separate in-kernel helper. In-kernel helper is probably a better idea.
Quoted 4 lines
I am mostly asking because it is exactly these kind of details that make the current in kernel oom handler quite complex and it would be great if custom ones do not have to reproduce that complexity and only focus on the high level policy.
Totally agree.
Quoted 10 lines
Quoted 9 lines
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls.
Quoted 5 lines
This makes sense to me as well. I have to admit I am not fully familiar with PSI integration into sched code but from what I can see the evaluation is done on regular bases from the worker context kicked off from the scheduler code. There shouldn't be any locking constrains which is good. Is there any risk if the oom handler took too long though?
It's a good question. In theory yes, it can affect the timing of other PSI events. An option here is to move it into a separate work, however I'm not sure if it worth the added complexity. I actually tried this approach in an earlier version of this patchset, but the problem was that the code for scheduling this work should be dynamically turned on/off when a bpf program is attached/detached, otherwise it's an obvious cpu overhead.
It's doable, but Idk if it's justified.Quoted 1 line
Quoted 6 lines
Quoted 3 lines
Also an important question. I can see selftests which are using the infrastructure. But have you tried to implement a real OOM handler with this proposed infrastructure?
Not yet. Given the size and complexity of the infrastructure of my current employer, it's not a short process. But we're working on it.
Quoted 6 lines
Hi Roman,
This might end up being very useful for Android. Since we have a shared current employer, we might be able to provide an earlier test environment for this concept on Android and speed up development of a real OOM handler. I'll be following the development of this patchset and will see if we can come up with an early prototype for testing.
Hi Suren,
Sounds great, thank you!
Suren Baghdasaryan
maintainerQuoted 76 lines · Roman Gushchin
Quoted 19 lines · Michal Hocko
Quoted 12 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
Quoted 5 lines
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
Quoted 6 lines
Makes sense to me. I still have a slight concern though. We have 3 different oom handlers smashed into a single one with special casing involved. This is manageable (although not great) for the in kernel code but I am wondering whether we should do better for BPF based OOM implementations. Would it make sense to have different callbacks for cpuset, memcg and global oom killer handlers?
Yes, it's certainly possible. If we go
struct_opspath, we can even have both the common hook which handles all types of OOM's and separate hooks for each type. The user then can choose what's more convenient. Good point.Quoted 5 lines
I can see you have already added some helper functions to deal with memcgs but I do not see anything to iterate processes or find a process to kill etc. Is that functionality generally available (sorry I am not really familiar with BPF all that much so please bear with me)?
Yes, task iterator is available since v6.7:
https://docs.ebpf.io/linux/kfuncs/bpf_iter_task_new/Quoted 5 lines
I like the way how you naturalely hooked into existing OOM primitives like
oom_kill_processbut I do not seetsk_is_oom_victimexposed. Are you waiting for a first user that needs to implement oom victim synchronization or do you plan to integrate that into tasks iterators?It can be implemented in bpf directly, but I agree that it probably deserves at least an example in the test or a separate in-kernel helper. In-kernel helper is probably a better idea.
Quoted 4 lines
I am mostly asking because it is exactly these kind of details that make the current in kernel oom handler quite complex and it would be great if custom ones do not have to reproduce that complexity and only focus on the high level policy.
Totally agree.
Quoted 10 lines
Quoted 9 lines
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls.
Quoted 5 lines
This makes sense to me as well. I have to admit I am not fully familiar with PSI integration into sched code but from what I can see the evaluation is done on regular bases from the worker context kicked off from the scheduler code. There shouldn't be any locking constrains which is good. Is there any risk if the oom handler took too long though?
Quoted 8 lines
It's a good question. In theory yes, it can affect the timing of other PSI events. An option here is to move it into a separate work, however I'm not sure if it worth the added complexity. I actually tried this approach in an earlier version of this patchset, but the problem was that the code for scheduling this work should be dynamically turned on/off when a bpf program is attached/detached, otherwise it's an obvious cpu overhead.
It's doable, but Idk if it's justified.
I think this is a legitimate concern. bpf_handle_psi_event() can block update_triggers() and delay other PSI triggers.
Quoted 51 lines
Quoted 5 lines
Also an important question. I can see selftests which are using the infrastructure. But have you tried to implement a real OOM handler with this proposed infrastructure?
Not yet. Given the size and complexity of the infrastructure of my current employer, it's not a short process. But we're working on it.
Quoted 22 lines
Quoted 8 lines
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
Quoted 13 lines
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepablebpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).Quoted 2 lines
I cannot see this in the code. Could you be more specific please? Where is this
might_faultcoming from? Is this BPF constrain?It's in
__bpf_prog_enter_sleepable(). But I hope I can make this hook non-sleepable (by goingstruct_opspath) and the problem will go away.Quoted 9 lines
Quoted 7 lines
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introduce
bpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?Yes, let's start simple and see where we get from there.
Agree.
Thank you for taking a look and your comments/ideas!
Suren Baghdasaryan
maintainerQuoted 35 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the
Quoted 8 lines
last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.
I didn't read the whole patchset yet but want to mention couple features that we should not forget:
- memory reaping. Maybe you already call oom_reap_task_mm() after BPF oom-handler kills a process or maybe BPF handler is expected to implement it?
- kill reporting to userspace. I think BPF handler would be expected to implement it?
Quoted 68 lines
[1]: https://lwn.net/ml/linux-kernel/[email protected]/
[2]: https://lore.kernel.org/lkml/[email protected]/
[3]: https://github.com/facebookincubator/oomd
[4]: https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.service.html----
This is an RFC version, which is not intended to be merged in the current form. Open questions/TODOs:
1) Program type/attachment type for thebpf_handle_out_of_memory()hook. It has to be able to return a value, to be sleepable (to use cgroup iterators) and to have trusted arguments to passoom_controldown tobpf_oom_kill_process(). Current patchset has a workaround (patch "bpf: treat fmodret tracing program's arguments as trusted"), which is not safe. One option is to fake acquire/release semantics for theoom_controlpointer. Other option is to introduce a completely new attachment or program type, similar to lsm hooks.
2) Currently lockdep complaints about a potential circular dependency because sleepablebpf_handle_out_of_memory()hook callsmight_fault()underoom_lock. One way to fix it is to make it non-sleepable, but then it will require some additional work to allow it using cgroup iterators. It's intervened with 1).
3) What kind of hierarchical features are required? Do we want to nest oom policies? Do we want to attach oom policies to cgroups? I think it's too complicated, but if we want a full hierarchical support, it might be required. Patch "mm: introducebpf_get_root_mem_cgroup()bpf kfunc" exposes the true root memcg, which is potentially outside of the ns of the loading process. Does it require some additional capabilities checks? Should it be removed?
4) Documentation is lacking and will be added in the next version.Roman Gushchin (12):
mm: introduce a bpf hook for OOM handling
bpf: mark struct oom_control's memcg field asTRUSTED_OR_NULL
bpf: treat fmodret tracing program's arguments as trusted
mm: introducebpf_oom_kill_process()bpf kfunc
mm: introduce bpf kfuncs to deal with memcg pointers
mm: introducebpf_get_root_mem_cgroup()bpf kfunc
bpf: selftests: introduceread_cgroup_file()helper
bpf: selftests: bpf OOM handler test
sched: psi: bpf hook to handle psi events
mm: introducebpf_out_of_memory()bpf kfunc
bpf: selftests: introduceopen_cgroup_file()helper
bpf: selftests: psi handler testinclude/linux/memcontrol.h | 2 + include/linux/oom.h | 5 + kernel/bpf/btf.c | 9 +- kernel/bpf/verifier.c | 5 + kernel/sched/psi.c | 36 ++- mm/Makefile | 3 + mm/bpf_memcontrol.c | 108 +++++++++ mm/oom_kill.c | 140 +++++++++++ tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 3 + tools/testing/selftests/bpf/prog_tests/oom.c | 227 ++++++++++++++++++ tools/testing/selftests/bpf/prog_tests/psi.c | 234 +++++++++++++++++++ tools/testing/selftests/bpf/progs/test_oom.c | 103 ++++++++ tools/testing/selftests/bpf/progs/test_psi.c | 43 ++++ 14 files changed, 983 insertions(+), 2 deletions(-) create mode 100644 mm/bpf_memcontrol.c create mode 100644 tools/testing/selftests/bpf/prog_tests/oom.c create mode 100644 tools/testing/selftests/bpf/prog_tests/psi.c create mode 100644 tools/testing/selftests/bpf/progs/test_oom.c create mode 100644 tools/testing/selftests/bpf/progs/test_psi.c--
2.49.0.901.g37484f566f-goog
Roman Gushchin
maintainerQuoted 45 lines · Suren Baghdasaryan
Quoted 35 lines · Roman Gushchin
This patchset adds an ability to customize the out of memory handling using bpf.
It focuses on two parts:
1) OOM handling policy,
2) PSI-based OOM invocation.The idea to use bpf for customizing the OOM handling is not new, but unlike the previous proposal [1], which augmented the existing task ranking-based policy, this one tries to be as generic as possible and leverage the full power of the modern bpf.
It provides a generic hook which is called before the existing OOM killer code and allows implementing any policy, e.g. picking a victim task or memory cgroup or potentially even releasing memory in other ways, e.g. deleting tmpfs files (the last one might require some additional but relatively simple changes).
The past attempt to implement memory-cgroup aware policy [2] showed that there are multiple opinions on what the best policy is. As it's highly workload-dependent and specific to a concrete way of organizing workloads, the structure of the cgroup tree etc, a customizable bpf-based implementation is preferable over a in-kernel implementation with a dozen on sysctls.
The second part is related to the fundamental question on when to declare the OOM event. It's a trade-off between the risk of unnecessary OOM kills and associated work losses and the risk of infinite trashing and effective soft lockups. In the last few years several PSI-based userspace solutions were developed (e.g. OOMd [3] or systemd-OOMd [4]). The common idea was to use userspace daemons to implement custom OOM logic as well as rely on PSI monitoring to avoid stalls. In this scenario the userspace daemon was supposed to handle the majority of OOMs, while the in-kernel OOM killer worked as the
Quoted 8 lines
last resort measure to guarantee that the system would never deadlock on the memory. But this approach creates additional infrastructure
churn: userspace OOM daemon is a separate entity which needs to be deployed, updated, monitored. A completely different pipeline needs to be built to monitor both types of OOM events and collect associated logs. A userspace daemon is more restricted in terms on what data is available to it. Implementing a daemon which can work reliably under a heavy memory pressure in the system is also tricky.
Quoted 7 lines
I didn't read the whole patchset yet but want to mention couple features that we should not forget:
- memory reaping. Maybe you already calloom_reap_task_mm()after BPF oom-handler kills a process or maybe BPF handler is expected to implement it?
- kill reporting to userspace. I think BPF handler would be expected to implement it?
The patchset implements the bpf_oom_kill_process() helper, which kills the desired process the same way as the kernel oom killer: with the help of the oom reaper, dumping corresponding stats into dmesg and bumping corresponding memcg- and system-level stats.
For additional reporting generic bpf<->userspace interaction mechanims can be used, e.g. ringbuffer.
Thanks!