Skip to contentSkip to conversation

user- vs kernel-level resource sandbox for Linux?

DISCUSSION Marek Habersack 5 participants 8 messages Nov 29, 2004 – Dec 2, 2004

lore
Share & tools
lore
#0 lore raw

Hello,

I am looking for advice on how to limit resource (memory in particular)
usage on a linux machine (running either kernel v2.4 or2.6) on the per-user (vs per-process) basis. I am aware that I could use Xen or UML for that purpose, but I am wondering whether anybody knows any solution that can implement that entirely in the userland (e.g. a monitor application that intercepts system calls responsible for resource allocation and controls the memory usage that way). My problem is apache which spawns a certain process on which sometimes runs away and causes the kernel to kill apache, the offending process and cause all fork(2) attempts to fail (which effectively disables ssh). I've tried limiting resources on the apache startup, but that isn't of much help since each apache process will get the same resources and it's enough that several of them allocate too much memory at the same time and the effect is as described above. I've also played with overcommit on the 2.6 kernel in hope that it will stop the process from allocating excessive amounts of memory, but it wasn't of much help either, alas...
I would appreciate any pointers to the userland solutions for that problem
(if any exist) before I resort to Xen/UML.

thanks in advance,

marek

#1 lore raw
Quoted 2 lines · [email protected] said:

I would appreciate any pointers to the userland solutions for that
problem (if any exist) before I resort to Xen/UML.

UML would be exactly what you're looking for.

Jeff

#2 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jeff" == Jeff Dike <[email protected]> writes:

Quoted 2 lines · Jeff> [email protected] said:
Quoted 2 lines

I would appreciate any pointers to the userland solutions for that problem (if any exist) before I resort to Xen/UML.

Jeff> UML would be exactly what you're looking for.

Jeff> Jeff

apart from the performance hit :-(

There have been a number of different approaches proposed in the past to limit real memory usage per-process; search for RSS limit in the archives.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au The technical we do immediately, the political takes *forever*

#3 lore raw

On Tue, Nov 30, 2004 at 09:13:03AM +1100, Peter Chubb scribbled:

Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Jeff" == Jeff Dike <[email protected]> writes:

Quoted 2 lines · Jeff> [email protected] said:
Quoted 2 lines

I would appreciate any pointers to the userland solutions for that problem (if any exist) before I resort to Xen/UML.

Jeff> UML would be exactly what you're looking for.

Jeff> Jeff

apart from the performance hit :-(

that's the problem...

Quoted 4 lines

There have been a number of different approaches proposed in the past to limit real memory usage per-process; search for RSS limit in the archives.

per-process isn't enough. I specifically need something to limit the memory usage on a more global scale - per user ID or per process group or a similar way of grouping related processes. That's the only way to tame processes like apache. At this point the option I'm considering is Xen, unless I can find a userland solution to the problem...

regards,

marek

#4 lore raw
Quoted 5 lines · Marek Habersack

per-process isn't enough. I specifically need something to limit the memory usage on a more global scale - per user ID or per process group or a similar way of grouping related processes. That's the only way to tame processes like apache. At this point the option I'm considering is Xen, unless I can find a userland solution to the problem...

I'd suggest playing with Xen - its very efficient and it really does come close to perfect constraint for resources.

#5 lore raw

On Tue, Nov 30, 2004 at 06:48:27PM +0000, Alan Cox scribbled:

Quoted 9 lines · Marek Habersack
Quoted 5 lines

per-process isn't enough. I specifically need something to limit the memory usage on a more global scale - per user ID or per process group or a similar way of grouping related processes. That's the only way to tame processes like apache. At this point the option I'm considering is Xen, unless I can find a userland solution to the problem...

I'd suggest playing with Xen - its very efficient and it really does come close to perfect constraint for resources.

That's my current impression. I also considered writing a simple kernel module to intercept sys_brk, but that seemed to be a bit clumsy. We have been running a test installation of Xen with 2 VMs under quite high load and it performs outstandingly well in "laboratory environment".
Also, I seem to recall there used to be a patch for the linux kernel to implement BSD-like jail environment, which would suit my purpose too, do you know what happened to the project/where it can be found? It would be a great addition to the kernel, just like the Zones in Solaris 10 are (which are based on the BSD jail concept as well).

regards,

marek

#6 lore raw
Quoted 2 lines · Marek Habersack

That's my current impression. I also considered writing a simple kernel module to intercept sys_brk, but that seemed to be a bit clumsy. We have

You have to consider kernel side resources too - page tables, memory maps
and the like which jails don't really fix.

#7 lore raw
In reply to Marek Habersack
Quoted 10 lines · Marek Habersack

On Tue, Nov 30, 2004 at 06:48:27PM +0000, Alan Cox scribbled:

Quoted 9 lines · Marek Habersack
Quoted 5 lines

per-process isn't enough. I specifically need something to limit the memory usage on a more global scale - per user ID or per process group or a similar way of grouping related processes. That's the only way to tame processes like apache. At this point the option I'm considering is Xen, unless I can find a userland solution to the problem...

I'd suggest playing with Xen - its very efficient and it really does come close to perfect constraint for resources.

Quoted 9 lines

That's my current impression. I also considered writing a simple kernel module to intercept sys_brk, but that seemed to be a bit clumsy. We have been running a test installation of Xen with 2 VMs under quite high load and it performs outstandingly well in "laboratory environment".
Also, I seem to recall there used to be a patch for the linux kernel to implement BSD-like jail environment, which would suit my purpose too, do you know what happened to the project/where it can be found? It would be a great addition to the kernel, just like the Zones in Solaris 10 are (which are based on the BSD jail concept as well).

maybe this might be of interest for you:

http://linux-vserver.org/
http://linux-vserver.org/Linux-VServer-Paper

best,
Herbert

Quoted 3 lines

regards,

marek