Kswapd flaw
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Thanks!
Quoted 4 lines · Al Boldi
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Al,
What do you mean by paging? Can you be more detailed on the problem description please?
Quoted 4 lines · Al Boldi
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Marcelo,
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
Quoted 2 lines · Al Boldi
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
The kernel can always discard pages of executables and read-only mapped files, because it can load them back if necessary. If you don't like this, call mlockall() in your program.
Michal
Quoted 1 line · Michal Schmidt
The kernel can always discard pages of executables and read-only mapped
files,
Quoted 2 lines
because it can load them back if necessary.
If you don't like this, callmlockall()in your program.
Thanks for the pointer, but what if I don't own the source. Is there a way to tell kswapd to disable this feature?
Quoted 5 lines · Al Boldi
Quoted 1 line · Michal Schmidt
If you don't like this, call
mlockall()in your program.Thanks for the pointer, but what if I don't own the source.
An LD_PRELOAD trick might work.
Michal
On 28 Jun 2005, Al Boldi murmured woefully:
Quoted 2 lines
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
This is how the kernel has always worked. If you have no swap, memory pressure is higher than it otherwise might be, because dirty pages must be kept in physical RAM even if they are rarely used. So when memory gets low the kernel *has* to evict pages which aren't dirty, and write out and flush dirty pages corresponding to files on disk.
What else could it possibly do? Stop evicting everything?
I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no time.
Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
at people who weren't supposed to be in your machine room.'
--- Anthony de BoerOn 28 Jun 2005, Al Boldi murmured woefully:
Quoted 2 lines
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
Quoted 1 line · Nix
I can't even tell what you're expecting. Surely not that no pages are ever
evicted or flushed;
Quoted 1 line
your memory would fill up with page cache in no time.
Nix,
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
On 28 Jun 2005, Al Boldi yowled:
Quoted 9 lines
Quoted 5 lines · Nix
On 28 Jun 2005, Al Boldi murmured woefully:
Quoted 2 lines
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no time.
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
But processes (really, mapped text pages; really, read-only mapped pages of all kinds) are loaded piecemeal in any case. Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.
A system in which pages can be faulted in *and* out is consistent: one in which they can only be faulted in is both inconsistent and very deadlock-prone.
Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
at people who weren't supposed to be in your machine room.'
--- Anthony de BoerHi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:
Quoted 6 lines
Quoted 5 lines · Nix
On 28 Jun 2005, Al Boldi murmured woefully:
Quoted 2 lines
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no
time.
Quoted 3 lines
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.
}
Nix,
You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.
Is it possible to change kswapd's default behaviour to not fault if there is no swap?
Thanks!
Quoted 6 lines · Al Boldi
Hi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:Quoted 2 lines
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
Quoted 6 lines
Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.
}You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.
That's `swapping', i.e. writing out dirty data pages (and text pages dirtied by e.g. relocation) which otherwise have nowhere to go.
Pages mapped from files with read-only mappings and non-dirty pages of read-write mappings get discarded when memory is tight; pages mapped with read-write non-private maps and which have been modified get written back to their file and discarded in the same situation.
(Private modified read-write mappings of data files have to stay in
memory: the only place they could go is swap, and there is none.)
Quoted 2 lines
Is it possible to change kswapd's default behaviour to not fault if there is no swap?
I don't think so, except on a process-by-process basis via mlockall(). (/proc/sys/vm/swappiness lets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)
You can kludge it to some extent by putting your binaries in a tmpfs and running from there: then you'll be paging from memory to memory ;)
But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things. Surely slow execution is generally preferable to *no* execution?
(Or is it that you're trying to suppress *disk activity*, perhaps to keep power consumption down? If so, running from tmpfs seems the best option.)
Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
at people who weren't supposed to be in your machine room.'
--- Anthony de BoerNix wrote:{
Quoted 10 lines · Al Boldi
Hi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:Quoted 2 lines
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast. }
Quoted 5 lines
You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.
Is it possible to change kswapd's default behaviour to not fault if there is no swap?
I don't think so, except on a process-by-process basis via mlockall(). (/proc/sys/vm/swappiness lets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)
But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things.
}
Nix,
You start killing things because you overcommitted, when you were not supposed to fault in the first place because you have no swap.
( I couldn't find /proc/sys/vm/swappiness in 2.4, although I heard about it in 2.6.)
On Tue, 28 Jun 2005, Al Boldi murmured woefully:
Quoted 6 lines
Nix wrote:{
I don't think so, except on a process-by-process basis viamlockall(). (/proc/sys/vm/swappinesslets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)
Quoted 8 lines
But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things.
}Nix,
You start killing things because you overcommitted, when you were not supposed to fault in the first place because you have no swap.
The only thing that is blocked by having no swap is swapping. The only things that go to swap are things that can't go anywhere else (because swap is comparatievly slow). As far as I know, the only things that land in swap are modified private writable file-backed mappings (e.g. relocations in text sections of programs), and allocated anonymous mappings (what most people think of as `data'). (Data allocated via brk() is of the latter type.)
So if swap is turned off, the kernel can still acquire more memory when short by discarding file-backed pages from the page cache --- this includes non-modified text pages of programs and modified non-private file mappings, as well as data read/written via read() and write().
That is, with no swapfile, the kernel can't swap. It can still *page*, pushing stuff out to files if modified, throwing them out of memory if not, and reloading them from their original file when needed.
Quoted 2 lines
( I couldn't find
/proc/sys/vm/swappinessin 2.4, although I heard about it in 2.6.)
Yeah, it's a 2.6 thing.
Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
at people who weren't supposed to be in your machine room.'
--- Anthony de BoerQuoted 11 lines · Al Boldi
On 28 Jun 2005, Al Boldi murmured woefully:
Quoted 2 lines
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
Quoted 1 line · Nix
I can't even tell what you're expecting. Surely not that no pages are ever
evicted or flushed;
Quoted 1 line
your memory would fill up with page cache in no time.
Nix,
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.
Something running rarely gets thrown out - It is usually only pages that see little use that gets evicted. Such as startup code.
Of course you can improve the situation a lot by adding swap - the kernel will then be free to swap out dirty but little-used pages instead of not-dirty but more used executable mappings. That will improve your performance.
Getting more memory or running fewer programs also helps, obviously.
Helge Hafting
Quoted 11 lines · Al Boldi
Quoted 3 lines · Al Boldi
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Marcelo,
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
I think what you really want is to adjust your "overcommit" settings.
See: "Documentation/vm/overcommit-accounting"
Signature · 5 lines
Paulo Marques - www.grupopie.com It is a mistake to think you can solve any major problems just with potatoes. Douglas Adams
Quoted 8 lines · Paulo Marques
Quoted 7 lines · Al Boldi
Quoted 3 lines · Al Boldi
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Quoted 8 lines
Quoted 4 lines
Marcelo,
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
I think what you really want is to adjust your "overcommit" settings.
See: "
Documentation/vm/overcommit-accounting"
Al,
You should set overcommit accordingly as Paulo mentions.
You might also want to disable the OOM killer (CONFIG_OOM_KILLER).
Marcelo wrote: {
Quoted 8 lines · Paulo Marques
Quoted 7 lines · Al Boldi
Quoted 3 lines · Al Boldi
In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.
Is there a way to fix/disable this behaviour?
Quoted 8 lines
Quoted 4 lines
Marcelo,
Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.
I think what you really want is to adjust your "overcommit" settings.
See: "
Documentation/vm/overcommit-accounting"
You might also want to disable the OOM killer (CONFIG_OOM_KILLER). }
Paulo,
Thanks for the pointer! Overcommit is the problem, and kswapd is not honoring it.
Can this be fixed/adjusted?
Marcelo,
Disable OOM killer? Is that an option in 2.4 or 2.6?
Thanks!