Skip to contentSkip to conversation

strict VM overcommit accounting for 2.4.32/2.4.33-pre1

PATCH Al Boldi 5 participants 14 messages Dec 30, 2005 – Jan 1, 2006

lore
Share & tools
lore

Al Boldi

author
#0 lore raw
Quoted 4 lines · Barry K. Nathan

This patch adds strict VM overcommit accounting to the mainline 2.4 kernel, thus allowing overcommit to be truly disabled. This feature has been in 2.4-ac, Red Hat Enterprise Linux 3 (RHEL 3) vendor kernels, and 2.6 for a long while.

Thanks a lot!

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Thanks!

--
Al

#1 lore raw
Quoted 8 lines · Al Boldi

Thanks a lot!

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

I mentioned in my original post (maybe I wasn't clear enough) that this is only in the documentation and not in the actual code (i.e. the code has no advantage over 2.6 in this regard). This error comes from the 2.4-ac/pac kernels from which this documentation originally comes. Yes, I need to fix the documentation, I didn't get to do that yet.

I think you can get paranoid overcommit with either my patch or 2.6 by setting /proc/sys/vm/overcommit_mode to 2 *and* /proc/sys/vm/overcommit_ratio to 0, however.

--
-Barry K. Nathan <[email protected]>

author
#2 lore raw
Quoted 8 lines · Barry K. Nathan
Quoted 7 lines · Al Boldi

Thanks a lot!

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)
Quoted 5 lines
Quoted 1 line

This one isn't in 2.6, which is critical for a stable system.

I think you can get paranoid overcommit with either my patch or 2.6 by setting /proc/sys/vm/overcommit_mode to 2 *and* /proc/sys/vm/overcommit_ratio to 0, however.

Not really in 2.6.
And even if this were made to work, what would it imply to a system running w/o swap?

Thanks!

--
Al

#3 lore raw
Quoted 11 lines · Al Boldi
Quoted 10 lines · Barry K. Nathan
Quoted 8 lines · Al Boldi

Thanks a lot!

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Quoted 7 lines
Quoted 3 lines

I think you can get paranoid overcommit with either my patch or 2.6 by setting /proc/sys/vm/overcommit_mode to 2 *and* /proc/sys/vm/overcommit_ratio to 0, however.

Not really in 2.6.
And even if this were made to work, what would it imply to a system running w/o swap?

I can clearly reply on this one :

root@pcw:vm# swapoff -a
root@pcw:vm# free
             total       used       free     shared    buffers     cached
Mem:       1031752      84992     946760          0       3144      38564
-/+ buffers/cache:      43284     988468
Swap:            0          0          0
root@pcw:vm# echo 2 > overcommit_memory
root@pcw:vm# echo 0 > overcommit_ratio
root@pcw:vm# uname -a
bash: fork: Cannot allocate memory
root@pcw:vm# echo 10 > overcommit_ratio
root@pcw:vm# uname -a
Linux pcw 2.4.33-pre1 #1 SMP Fri Dec 30 21:52:06 CET 2005 i686 unknown
root@pcw:vm#

So as one could expect, if you're limited to allocation of (swap_size)+0% of RAM, then you have a limit of 0 kB, so that malloc() always fails. I'll do some crash tests with mmap, shm and 100 ratio to see how it behaves, but at the moment it's clearly good. Multi-process malloc cannot allocate more than specified and that's good.

Quoted 4 lines

Thanks!

--
Al

Regards,
Willy

#4 lore raw
In reply to Al Boldi
Quoted 6 lines · Al Boldi

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Actually it is

In the 2.4 case we took "50% RAM + swap" as the safe sane world 'never OOM kill' and to all intents and purposes it works. We also had a 100% paranoia mode.

When it was ported to 2.6 (not by me) whoever did it very sensibly made the percentage tunable and removed "mode 3" since its mode 2 0% ram and can be set that way.

Alan

author
#5 lore raw
Quoted 10 lines · Alan Cox
Quoted 6 lines · Al Boldi

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Actually it is

Quoted 7 lines

In the 2.4 case we took "50% RAM + swap" as the safe sane world 'never OOM kill' and to all intents and purposes it works. We also had a 100% paranoia mode.

When it was ported to 2.6 (not by me) whoever did it very sensibly made the percentage tunable and removed "mode 3" since its mode 2 0% ram and can be set that way.

Only, doesn't this imply that you cannot control overcommit unless backed by swap? i.e Without swap the kernel cannot use all of ram, because it would overcommit no-matter what, thus invoking OOM-killer.

Which raises an important question: What's overcommit to do with limiting access to physical RAM?

Thanks!

--
Al

#6 lore raw
Quoted 19 lines · Al Boldi
Quoted 10 lines · Alan Cox
Quoted 6 lines · Al Boldi

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Actually it is

Quoted 7 lines

In the 2.4 case we took "50% RAM + swap" as the safe sane world 'never OOM kill' and to all intents and purposes it works. We also had a 100% paranoia mode.

When it was ported to 2.6 (not by me) whoever did it very sensibly made the percentage tunable and removed "mode 3" since its mode 2 0% ram and can be set that way.

Quoted 6 lines

Only, doesn't this imply that you cannot control overcommit unless backed by swap? i.e Without swap the kernel cannot use all of ram, because it would overcommit no-matter what, thus invoking OOM-killer.

Which raises an important question: What's overcommit to do with limiting access to physical RAM?

As shown in my previous mail, it allows malloc() to return NULL. I've also successfully verified that it allows mmap() to fail if there is not enough memory. I disabled swap, and set the overcommit_ratio to 95 and could not kill the system. Above this, it becomes tricky. At 97, I see the last malloc() calls take a very long time, and at 98, the system still hangs. But 95% without swap seems stable here.

Quoted 4 lines

Thanks!

--
Al

Cheers,
Willy

author
#7 lore raw
Quoted 27 lines · Willy Tarreau
Quoted 19 lines · Al Boldi
Quoted 10 lines · Alan Cox
Quoted 6 lines · Al Boldi

Review

Quoted 4 lines

+3 - (NEW) paranoid overcommit The total address space commit

(quoted diff) +3 -0
+      for the system is not permitted to exceed swap. The machine
+      will never kill a process accessing pages it has mapped
+      except due to a bug (ie report it!)

This one isn't in 2.6, which is critical for a stable system.

Actually it is

Quoted 7 lines

In the 2.4 case we took "50% RAM + swap" as the safe sane world 'never OOM kill' and to all intents and purposes it works. We also had a 100% paranoia mode.

When it was ported to 2.6 (not by me) whoever did it very sensibly made the percentage tunable and removed "mode 3" since its mode 2 0% ram and can be set that way.

Quoted 6 lines

Only, doesn't this imply that you cannot control overcommit unless backed by swap? i.e Without swap the kernel cannot use all of ram, because it would overcommit no-matter what, thus invoking OOM-killer.

Which raises an important question: What's overcommit to do with limiting access to physical RAM?

Quoted 6 lines

As shown in my previous mail, it allows malloc() to return NULL. I've also successfully verified that it allows mmap() to fail if there is not enough memory. I disabled swap, and set the overcommit_ratio to 95 and could not kill the system. Above this, it becomes tricky. At 97, I see the last malloc() calls take a very long time, and at 98, the system still hangs. But 95% without swap seems stable here.

Thanks, for confirming this! And I agree that this patch and 2.6 offer an important and necessary workaround to inhibit OOM-killer, but it's no more than a workaround.

And so the question remains: Why should overcommit come into play at all when dealing with physical RAM only?

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

Thanks!

--
Al

#8 lore raw
Quoted 1 line · Al Boldi

Willy Tarreau wrote:

(...)

Quoted 7 lines
Quoted 6 lines

As shown in my previous mail, it allows malloc() to return NULL. I've also successfully verified that it allows mmap() to fail if there is not enough memory. I disabled swap, and set the overcommit_ratio to 95 and could not kill the system. Above this, it becomes tricky. At 97, I see the last malloc() calls take a very long time, and at 98, the system still hangs. But 95% without swap seems stable here.

Quoted 6 lines

Thanks, for confirming this! And I agree that this patch and 2.6 offer an important and necessary workaround to inhibit OOM-killer, but it's no more than a workaround.

And so the question remains: Why should overcommit come into play at all when dealing with physical RAM only?

It is very important when you have many processes wasting lots of unused memory. For instance, when firefox allocates 10 MB and uses only 3, then the remaining 7 MB can be used by another process. But if finally firefox tries to use them, and there is no memory left, then chances are that some processes will be killed. I believe the same problem happens after a fork() because data gets COWed but I'm not certain about this.

I'd bet that using a heavy GUI under X with no swap an overcommit_ratio set around 95%, you could get occasionnal malloc() failures. But once again, I may be wrong.

Quoted 3 lines

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

I think it's going unstable above 95% because there are lots of other areas consuming memory. I don't know for example if dentry caches, network buffers, various hash tables, etc... are accounted for.

Quoted 4 lines

Thanks!

--
Al

Willy

#9 lore raw
In reply to Al Boldi
Quoted 3 lines · Al Boldi

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

shared mappings make this impractical. To disable overcommit completely, each process would need to account for all its own shared libraries, eg each process gets glibc added etc. You'll find that on any non-extremely-stripped system you then end up with much more memory needed than you have ram.

#10 lore raw
Quoted 11 lines · Arjan van de Ven
Quoted 3 lines · Al Boldi

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

shared mappings make this impractical. To disable overcommit completely, each process would need to account for all its own shared libraries, eg each process gets glibc added etc. You'll find that on any non-extremely-stripped system you then end up with much more memory needed than you have ram.

Arjan, is this true even for read-only mappings such as shared libs ? It seems to me that those ones can precisely be mapped once because they are areas the process cannot extend. Am I wrong ?

Willy

#11 lore raw
Quoted 7 lines · Willy Tarreau
Quoted 6 lines · Arjan van de Ven
Quoted 3 lines · Al Boldi

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

Quoted 7 lines
Quoted 5 lines

shared mappings make this impractical. To disable overcommit completely, each process would need to account for all its own shared libraries, eg each process gets glibc added etc. You'll find that on any non-extremely-stripped system you then end up with much more memory needed than you have ram.

Arjan, is this true even for read-only mappings such as shared libs ?

shared libs aren't read only! they have all the relocations applied for example. (the fact that glibc mprotects it read only again afterwards doesn't change things.. part of the memory is already written to). and mprotect can be called again.... what would mprotect need to do if it's asked to make memory writable and the overcommit says "no space" ....

author
#12 lore raw
In reply to Arjan van de Ven
Quoted 11 lines · Arjan van de Ven
Quoted 4 lines · Al Boldi

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

shared mappings make this impractical. To disable overcommit completely, each process would need to account for all its own shared libraries, eg each process gets glibc added etc. You'll find that on any non-extremely-stripped system you then end up with much more memory needed than you have ram.

Are you implying shared maps are implemented by way of overcommitting?

Really, overcommit is an add-on feature like swapping, only overcommit is free because it's a lier. So removing an add-on feature should not affect the underlying system in any way, such as shared mappings or swapping.

It should be possible to allow swapping to handle all memory requests exceeding physical RAM. OverCommit should be a tuning option for those who like to live on the edge, because it really is a gamble.

In the case where swap = physical RAM and overcommit_ratio = 0, the kernel is in effect hiding the fact that it is overcommitting.

Can you see the overhead involved here?

Thanks for your input!

--
Al

#13 lore raw
Quoted 7 lines · Al Boldi
Quoted 6 lines · Arjan van de Ven
Quoted 4 lines · Al Boldi

Shouldn't it be possible to disable overcommit completely, thus giving kswapd a break from running wild trying to find something to swap/page, which is the reason why the system gets unstable going over 95% in your example.

Quoted 7 lines
Quoted 5 lines

shared mappings make this impractical. To disable overcommit completely, each process would need to account for all its own shared libraries, eg each process gets glibc added etc. You'll find that on any non-extremely-stripped system you then end up with much more memory needed than you have ram.

Are you implying shared maps are implemented by way of overcommitting?

yes. Using only 1 page shared is already overcommitting because in principle each user can cause a COW on that page and cause a memory allocation for it in the future. (just like "traditional" overcommit can cause a pagefault with an allocation)

Quoted 3 lines

Really, overcommit is an add-on feature like swapping, only overcommit is free because it's a lier. So removing an add-on feature should not affect the underlying system in any way, such as shared mappings or swapping.

then I think you're misunderstanding how the linux VM works

Quoted 3 lines

It should be possible to allow swapping to handle all memory requests exceeding physical RAM. OverCommit should be a tuning option for those who like to live on the edge, because it really is a gamble.

but it's a worthwhile gamble to not need GOBBLES of memory you're not using 99.9999999999999% of the time

Quoted 3 lines

In the case where swap = physical RAM and overcommit_ratio = 0, the kernel is in effect hiding the fact that it is overcommitting.

swap==ram is not really relevant; you'll need a LOT more to cover the shared maps...