mm/oom_kill: add sysctl_oom_dump_stack to control kernel stack dumping on OOM
Subsystems (2)
PATCH 1/1 mm/oom_kill: add sysctl_oom_dump_stack to control kernel stack dumping on OOM
Most OOM kills triggered by user-space processes produce kernel stack traces that are not helpful for diagnosing the root cause. These traces usually just show the page fault handler or system call entry.
Furthermore, dump_stack() can be expensive. It often runs with interrupts disabled or holds the console lock for a long time, potentially causing system latencies and preventing the system from responding to other events.
This patch adds a new sysctl vm.oom_dump_stack to control this behavior. Writing '0' to /proc/sys/vm/oom_dump_stack suppresses the kernel stack dump during OOM kills, while '1' (the default) preserves the existing behavior.
--- mm/oom_kill.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
mm/oom_kill.c +11 -1
@@ -56,6 +56,7 @@static int sysctl_panic_on_oom;static int sysctl_oom_kill_allocating_task;static int sysctl_oom_dump_tasks =1 ; +static int sysctl_oom_dump_stack =1 ;/* * Serializes oom killer invocations (out_of_memory()) from all contexts to
@@ -464,7 +465,9 @@ static void dump_header(struct oom_control *oc)if (!IS_ENABLED (CONFIG_COMPACTION ) && oc->order)pr_warn ("COMPACTION is disabled!!!\n" ); -dump_stack (); +if (sysctl_oom_dump_stack) +dump_stack (); +if (is_memcg_oom (oc))mem_cgroup_print_oom_meminfo (oc->memcg);else {
@@ -736,6 +739,13 @@ static const struct ctl_table vm_oom_kill_table[] = { .mode = 0644, .proc_handler = proc_dointvec, }, + { + .procname = "oom_dump_stack", + .data = &sysctl_oom_dump_stack, + .maxlen = sizeof(sysctl_oom_dump_stack), + .mode = 0644, + .proc_handler = proc_dointvec, + }, };#endif
Signature · 1 line
2.30.2
Michal Hocko
maintainerQuoted 9 lines · Yang Xin
Most OOM kills triggered by user-space processes produce kernel stack traces that are not helpful for diagnosing the root cause. These traces usually just show the page fault handler or system call entry.
Furthermore,
dump_stack()can be expensive. It often runs with interrupts disabled or holds the console lock for a long time, potentially causing system latencies and preventing the system from responding to other events.
Quoted 4 lines
This patch adds a new sysctl vm.oom_dump_stack to control this behavior. Writing '0' to
/proc/sys/vm/oom_dump_stacksuppresses the kernel stack dump during OOM kills, while '1' (the default) preserves the existing behavior.
While I fundamentally do not object to ways to suppress stacks traces for OOM I would really like to hear more what kind of overhead we are talking about here (stack traces are reported for tracing and other low latency situations) and why does this matter for as cold of a path as OOM is.
Also we are getting way too many of these sysctls. Maybe it is time to look for a more customizable way to configure oom output that doesn't require sysctl per output feature.
Quoted 45 lines
Signed-off-by Yang Xin <[email protected]>--- mm/oom_kill.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)mm/oom_kill.c +11
-1@@ -56,6 +56,7 @@static int sysctl_panic_on_oom;static int sysctl_oom_kill_allocating_task;static int sysctl_oom_dump_tasks =1 ; +static int sysctl_oom_dump_stack =1 ;/* * Serializes oom killer invocations (out_of_memory()) from all contexts to @@ -464,7 +465,9 @@ static void dump_header(struct oom_control *oc)if (!IS_ENABLED (CONFIG_COMPACTION ) && oc->order)pr_warn ("COMPACTION is disabled!!!\n" ); -dump_stack (); +if (sysctl_oom_dump_stack) +dump_stack (); +if (is_memcg_oom (oc))mem_cgroup_print_oom_meminfo (oc->memcg);else {@@ -736,6 +739,13 @@ static const struct ctl_table vm_oom_kill_table[] = { .mode = 0644, .proc_handler = proc_dointvec, }, + { + .procname = "oom_dump_stack", + .data = &sysctl_oom_dump_stack, + .maxlen = sizeof(sysctl_oom_dump_stack), + .mode = 0644, + .proc_handler = proc_dointvec, + }, };#endif Signature · 1 line
2.30.2
Signature · 2 lines
Michal Hocko SUSE Labs
David Rientjes
reviewerQuoted 21 lines · Michal Hocko
Quoted 9 lines · Yang Xin
Most OOM kills triggered by user-space processes produce kernel stack traces that are not helpful for diagnosing the root cause. These traces usually just show the page fault handler or system call entry.
Furthermore,
dump_stack()can be expensive. It often runs with interrupts disabled or holds the console lock for a long time, potentially causing system latencies and preventing the system from responding to other events.Quoted 4 lines
This patch adds a new sysctl vm.oom_dump_stack to control this behavior. Writing '0' to
/proc/sys/vm/oom_dump_stacksuppresses the kernel stack dump during OOM kills, while '1' (the default) preserves the existing behavior.While I fundamentally do not object to ways to suppress stacks traces for OOM I would really like to hear more what kind of overhead we are talking about here (stack traces are reported for tracing and other low latency situations) and why does this matter for as cold of a path as OOM is.
Quoted 4 lines
Also we are getting way too many of these sysctls. Maybe it is time to look for a more customizable way to configure oom output that doesn't require sysctl per output feature.
Strongly agree, I don't think this requires yet another sysctl. It's also global, so it will affect all oom kills including the ones where you might find a stack to actually be really helpful to understand the issue.