Skip to contentSkip to conversation

User space out of memory approach

DISCUSSION Bodo Eggert 2 participants 5 messages Jan 16, 2005 – Jan 20, 2005 2 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 2 lines · Edjard Souza Mota

What do you think about the point we are trying to make, i.e., moving the ranking of PIDs to be killed to user space?

If my system needs the OOM killer, it's usurally unresponsive to most userspace applications. A normal daemon would be swapped out before the runaway dhcpd grows larger than the web cache. It would have to be a mlocked RT task started from early userspace. It would be difficult to set up (unless you upgrade your distro), and almost nobody will feel like tweaking it to take the benefit (OOM == -ECANNOTHAPPEN).

What about creating a linked list of (stackable) algorhithms which can be extended by loading modules and resorted using {proc,sys}fs? It will avoid the extra process, the extra CPU time (and task switches) to frequently update the list and I think it will decrease the typical amount of used memory, too.

#3 lore raw

Hi,

Quoted 6 lines

If my system needs the OOM killer, it's usurally unresponsive to most userspace applications. A normal daemon would be swapped out before the runaway dhcpd grows larger than the web cache. It would have to be a mlocked RT task started from early userspace. It would be difficult to set up (unless you upgrade your distro), and almost nobody will feel like tweaking it to take the benefit (OOM == -ECANNOTHAPPEN).

Please correct me if I got it wrong: as deamon in this case is not a normal one, since it never gets rate for its own safety, then it needs an RT lock whenever system boots.

Quoted 5 lines

What about creating a linked list of (stackable) algorhithms which can be extended by loading modules and resorted using {proc,sys}fs? It will avoid the extra process, the extra CPU time (and task switches) to frequently update the list and I think it will decrease the typical amount of used memory, too.

Wouldn't this bring the (set of ) ranking algorithm(s) back to the kernel? This is exactly what we're trying to avoid. The way we see the potential for doing this is that kernel shouldn't worry about users decision on which process to kill but rather take her/his option into account. The computation of such a decision could be at user space (protected as you suggested above).

We'll think about it, although I'm not sure if there would be such a decrease in memory concumption.

br

Edjard

Signature · 1 line
"In a world without fences ... who needs Gates?"
#4 lore raw
Quoted 9 lines · Edjard Souza Mota
Quoted 6 lines

If my system needs the OOM killer, it's usurally unresponsive to most userspace applications. A normal daemon would be swapped out before the runaway dhcpd grows larger than the web cache. It would have to be a mlocked RT task started from early userspace. It would be difficult to set up (unless you upgrade your distro), and almost nobody will feel like tweaking it to take the benefit (OOM == -ECANNOTHAPPEN).

Please correct me if I got it wrong: as deamon in this case is not a normal one, since it never gets rate for its own safety,

That's it's own task, it must make sure not to commit suicide. I forgot about that.

Quoted 2 lines

then it needs an RT lock whenever
system boots.

It may not be blocked by a random RT task iff the RT task is supposed to be OOM-killed. Therefore it *MUST* run at the highest priority and be locked into the RAM.

It *SHOULD* be run at boot time, too, just in case it's needed early.

Quoted 8 lines
Quoted 5 lines

What about creating a linked list of (stackable) algorhithms which can be extended by loading modules and resorted using {proc,sys}fs? It will avoid the extra process, the extra CPU time (and task switches) to frequently update the list and I think it will decrease the typical amount of used memory, too.

Wouldn't this bring the (set of ) ranking algorithm(s) back to the kernel? This is exactly what we're trying to avoid.

You're trying to avoid it in order to let admins try other ranking algorhithms (at least that's what I read). The module approach seems to be flexible enough to do that, and it avoids the mentioned issues. If you really want a userspace daemon, it can be controled by a module.-)

I 'm thinking of something like that:

[X] support stacking of OOM killer ranking algorhythms
[X] Task blessing OOM filter
[X] Userspace OOM ranking daemon
[X] Default OOM killer ranking

-vs-

[ ] support stacking of OOM killer ranking algorhythms ( ) Userspace OOM ranking daemon
(o) Default OOM killer ranking

Signature · 1 line
Exceptions prove the rule, and destroy the battle plan. 
#5 lore raw
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

If my system needs the OOM killer, it's usurally unresponsive to most userspace applications. A normal daemon would be swapped out before the runaway dhcpd grows larger than the web cache. It would have to be a mlocked RT task started from early userspace. It would be difficult to set up (unless you upgrade your distro), and almost nobody will feel like tweaking it to take the benefit (OOM == -ECANNOTHAPPEN).

Quoted 5 lines
Quoted 2 lines

Please correct me if I got it wrong: as deamon in this case is not a normal one, since it never gets rate for its own safety,

That's it's own task, it must make sure not to commit suicide. I forgot about that.

Ok.

Quoted 8 lines
Quoted 2 lines

then it needs an RT lock whenever
system boots.

It may not be blocked by a random RT task iff the RT task is supposed to be OOM-killed. Therefore it *MUST* run at the highest priority and be locked into the RAM.

It *SHOULD* be run at boot time, too, just in case it's needed early.

Yes. That's the idea of the application we posted to test the oom killer ranking at
user space. At least, we are working to put it at boot time and these suggestions are very helpful.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

What about creating a linked list of (stackable) algorhithms which can be extended by loading modules and resorted using {proc,sys}fs? It will avoid the extra process, the extra CPU time (and task switches) to frequently update the list and I think it will decrease the typical amount of used memory, too.

Quoted 7 lines
Quoted 2 lines

Wouldn't this bring the (set of ) ranking algorithm(s) back to the kernel? This is exactly what we're trying to avoid.

You're trying to avoid it in order to let admins try other ranking algorhithms (at least that's what I read). The module approach seems to be flexible enough to do that, and it avoids the mentioned issues. If you really want a userspace daemon, it can be controled by a module.-)

Yes, your reading is correct, but this choice should take into account the "patterns"
of how memory is allocated for user's mostly used applications. Why? The closer the
ranking gets to "The Best choice" the longer it will take to invoke oom killer again.

I am wondering how could a module control a user space deamon if it hasn't started
yet? I mean, processes at user space are supposed to start only after all modules
are loaded (those loadable at boot time). So, this user space deamon would break
this standard. But if we manage to have a special module that takes care of loading
this stack of OOM Killer ranking algorithms, then the deamon would not need to break
the default order of loading modules. The init could be changed to start the deamon,
and then the module would start controlling it. Am I right?

So that's why people is complaining every distro would have to update the init and load this new module. Correct?

Quoted 8 lines

I 'm thinking of something like that:

[X] support stacking of OOM killer ranking algorhythms
[X] Task blessing OOM filter
[X] Userspace OOM ranking daemon
[X] Default OOM killer ranking

Quoted 6 lines

-vs-

[ ] support stacking of OOM killer ranking algorhythms ( ) Userspace OOM ranking daemon
(o) Default OOM killer ranking

Very interesting idea. Will take that into account. Thanks a lot.

Signature · 1 line
"In a world without fences ... who needs Gates?"
#6 lore raw
Quoted 14 lines · Edjard Souza Mota
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

What about creating a linked list of (stackable) algorhithms which can be extended by loading modules and resorted using {proc,sys}fs? It will avoid the extra process, the extra CPU time (and task switches) to frequently update the list and I think it will decrease the typical amount of used memory, too.

Quoted 7 lines
Quoted 2 lines

Wouldn't this bring the (set of ) ranking algorithm(s) back to the kernel? This is exactly what we're trying to avoid.

You're trying to avoid it in order to let admins try other ranking algorhithms (at least that's what I read). The module approach seems to be flexible enough to do that, and it avoids the mentioned issues. If you really want a userspace daemon, it can be controled by a module.-)

Quoted 6 lines

Yes, your reading is correct, but this choice should take into account the "patterns"
of how memory is allocated for user's mostly used applications. Why? The closer the
ranking gets to "The Best choice" the longer it will take to invoke oom killer again.

ACK.

Quoted 11 lines

I am wondering how could a module control a user space deamon if it hasn't started
yet? I mean, processes at user space are supposed to start only after all modules
are loaded (those loadable at boot time). So, this user space deamon would break
this standard. But if we manage to have a special module that takes care of loading
this stack of OOM Killer ranking algorithms, then the deamon would not need to break
the default order of loading modules.

I don't think there neeeds to be a special order while loading the modules, since each module will provide a defined interface which can be registered in a linked list and sorted on demand. Just init all compiled-in modules and sort using a kernel-parameter (remembering modprobe might be fubar), then modprobe (if compiled-in) all missing decision modules from the list (appending them) and resort again.

If the admin wants to add a module later, he can also change the order again, possibly after configuring the module. Disabeling may be either done by moving a decision past one without fall-through or by using a seperate list.

There will be a need for a controling instance which will build a list of candidates and pass it to each decision module in turn untill the victim is found. Maybe the list will need a field for a ranking offset and a scaling factor if a module is not supposed to do the final decision but to modify the ranking for some blessed processes.

Quoted 3 lines

The init could be changed to
start the deamon,
and then the module would start controlling it. Am I right?

It can, but it should be run from the (possibly autogenerated) initr{d,amfs} if it's used.

Quoted 2 lines

So that's why people is complaining every distro would have to update the init and load this new module. Correct?

ACK. (It's just me - for now)

Upgrading kernels used to be a drop-in replacement, except for ISDN and (for 2.4 -> 2.6) v4l. I like it that way.

Signature · 4 lines
Top 100 things you don't want the sysadmin to say:
66. What do you mean you needed that directory?

Friß, Spammer: [email protected] [email protected] [email protected]