Skip to contentSkip to conversation

oom_kill: oom_score_adj broken for processes with small memory usage

PATCH In review [email protected] 4 participants 6 messages Jul 1, 2021 – Sep 3, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 oom_kill: oom_score_adj broken for processes with small memory usage

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Fix by adding 1000 to totalpages before dividing.

---
I ran across this trying to diagnose another problem where I set up a cgroup with a small amount of memory and couldn't get a test program to work right.

I'm not sure this is quite right, to keep closer to the current behavior you could do:

	if (totalpages >= 1000)
		adj *= totalpages / 1000;

but that would map 0-1999 to the same value. But this at least shows the issue. I can provide a test program the shows the issue, but I think it's pretty obvious from the code.

 mm/oom_kill.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
mm/oom_kill.c +5 -2
index eefd3f5fde46..4ae0b6193bcd 100644
@@ -233,8 +233,11 @@ long oom_badness(struct task_struct *p, unsigned long totalpages)
 		mm_pgtables_bytes(p->mm) / PAGE_SIZE;
 	task_unlock(p);
 
-	/* Normalize to oom_score_adj units */
-	adj *= totalpages / 1000;
+	/*
+	 * Normalize to oom_score_adj units.  You should never
+	 * multiply by zero here, or oom_score_adj will not work.
+	 */
+	adj *= (totalpages + 1000) / 1000;
 	points += adj;
 
 	return points;
Signature · 1 line
2.25.1
maintainer
#1 lore raw
Quoted 4 lines · On Thu 01-07-21 07:54:30, [email protected] wrote:

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

Quoted 8 lines
  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Fix by adding 1000 to totalpages before dividing.

Yes, this is a known limitation of the oom_score_adj and its scale. Is this a practical problem to be solved though? I mean 0-1000 pages is not really that much different from imprecision at a larger scale where tasks are effectively considered equal.

I have to say I do not really like the proposed workaround. It doesn't really solve the problem yet it adds another special case.

Signature · 2 lines
Michal Hocko
SUSE Labs
#2 lore raw
Quoted 12 lines · Michal Hocko
Quoted 11 lines · On Thu 01-07-21 07:54:30, [email protected] wrote:

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Quoted 6 lines
Quoted 1 line

Fix by adding 1000 to totalpages before dividing.

Yes, this is a known limitation of the oom_score_adj and its scale. Is this a practical problem to be solved though? I mean 0-1000 pages is not really that much different from imprecision at a larger scale where tasks are effectively considered equal.

Known limitation? Is this documented? I couldn't find anything that said "oom_score_adj doesn't work at all with programs with <1000 pages besides setting the value to -1000".

Quoted 3 lines

I have to say I do not really like the proposed workaround. It doesn't really solve the problem yet it adds another special case.

The problem is that if you have a small program, there is no way to set it's priority besides completely disablling the OOM killer for it.

I don't understand the special case comment. How is this adding a special case? This patch removes a special case. Small programs working different than big programs is a special case. Making them all work the same is removing an element of surprise from someone expecting things to work as documented.

-corey

maintainer
#3 lore raw
Quoted 32 lines · On Fri, 16 Jul 2021 07:25:47 -0500 Corey Minyard <[email protected]> wrote:
Quoted 12 lines · Michal Hocko
Quoted 11 lines · On Thu 01-07-21 07:54:30, [email protected] wrote:

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Quoted 6 lines
Quoted 1 line

Fix by adding 1000 to totalpages before dividing.

Yes, this is a known limitation of the oom_score_adj and its scale. Is this a practical problem to be solved though? I mean 0-1000 pages is not really that much different from imprecision at a larger scale where tasks are effectively considered equal.

Known limitation? Is this documented? I couldn't find anything that said "oom_score_adj doesn't work at all with programs with <1000 pages besides setting the value to -1000".

Quoted 3 lines

I have to say I do not really like the proposed workaround. It doesn't really solve the problem yet it adds another special case.

The problem is that if you have a small program, there is no way to set it's priority besides completely disablling the OOM killer for it.

Quoted 6 lines

I don't understand the special case comment. How is this adding a special case? This patch removes a special case. Small programs working different than big programs is a special case. Making them all work the same is removing an element of surprise from someone expecting things to work as documented.

Can we please get this resolved one way or the other?

#4 lore raw
Quoted 34 lines · Andrew Morton
Quoted 32 lines · On Fri, 16 Jul 2021 07:25:47 -0500 Corey Minyard <[email protected]> wrote:
Quoted 12 lines · Michal Hocko
Quoted 11 lines · On Thu 01-07-21 07:54:30, [email protected] wrote:

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Quoted 6 lines
Quoted 1 line

Fix by adding 1000 to totalpages before dividing.

Yes, this is a known limitation of the oom_score_adj and its scale. Is this a practical problem to be solved though? I mean 0-1000 pages is not really that much different from imprecision at a larger scale where tasks are effectively considered equal.

Known limitation? Is this documented? I couldn't find anything that said "oom_score_adj doesn't work at all with programs with <1000 pages besides setting the value to -1000".

Quoted 3 lines

I have to say I do not really like the proposed workaround. It doesn't really solve the problem yet it adds another special case.

The problem is that if you have a small program, there is no way to set it's priority besides completely disablling the OOM killer for it.

Quoted 8 lines
Quoted 6 lines

I don't understand the special case comment. How is this adding a special case? This patch removes a special case. Small programs working different than big programs is a special case. Making them all work the same is removing an element of surprise from someone expecting things to work as documented.

Can we please get this resolved one way or the other?

My goal in submitting this is to avoid someone having to go through what I went through. I know it now, so it's not going to affect me again.

We could document this, but to me it seems silly when something can just be made consistent to avoid having to document it. I got no response to my questions above, so I don't know what to make of it.

Thanks Andrew,

-corey

maintainer
#5 lore raw
In reply to Andrew Morton
Quoted 34 lines · Andrew Morton
Quoted 32 lines · On Fri, 16 Jul 2021 07:25:47 -0500 Corey Minyard <[email protected]> wrote:
Quoted 12 lines · Michal Hocko
Quoted 11 lines · On Thu 01-07-21 07:54:30, [email protected] wrote:

From: Corey Minyard <[email protected]>

If you have a process with less than 1000 totalpages, the calculation:

  adj = (long)p->signal->oom_score_adj;
  ...
  adj *= totalpages / 1000;

will always result in adj being zero no matter what oom_score_adj is, which could result in the wrong process being picked for killing.

Quoted 6 lines
Quoted 1 line

Fix by adding 1000 to totalpages before dividing.

Yes, this is a known limitation of the oom_score_adj and its scale. Is this a practical problem to be solved though? I mean 0-1000 pages is not really that much different from imprecision at a larger scale where tasks are effectively considered equal.

Known limitation? Is this documented? I couldn't find anything that said "oom_score_adj doesn't work at all with programs with <1000 pages besides setting the value to -1000".

Quoted 3 lines

I have to say I do not really like the proposed workaround. It doesn't really solve the problem yet it adds another special case.

The problem is that if you have a small program, there is no way to set it's priority besides completely disablling the OOM killer for it.

Quoted 8 lines
Quoted 6 lines

I don't understand the special case comment. How is this adding a special case? This patch removes a special case. Small programs working different than big programs is a special case. Making them all work the same is removing an element of surprise from someone expecting things to work as documented.

Can we please get this resolved one way or the other?

As I've already said, I do not see this practical enough problem to warrant special treatment. Do we really care about controlling the oom behavior for tasks with <4MB of memory?

I fully agree that the current situation is not ideal. The whole oom_score* API sucks but here we are with an user API that is effectivelly impossible to fix properly.

Signature · 2 lines
Michal Hocko
SUSE Labs