Skip to contentSkip to conversation

Kswapd flaw

DISCUSSION Al Boldi 6 participants 17 messages Jun 27, 2005 – Jun 28, 2005

lore
Share & tools
lore

Al Boldi

author
#0 lore raw

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Thanks!

#1 lore raw
Quoted 4 lines · Al Boldi

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Al,

What do you mean by paging? Can you be more detailed on the problem description please?

author
#2 lore raw
Quoted 4 lines · Al Boldi

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Marcelo,

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

#3 lore raw
Quoted 2 lines · Al Boldi

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

The kernel can always discard pages of executables and read-only mapped files, because it can load them back if necessary. If you don't like this, call mlockall() in your program.

Michal

author
#4 lore raw
Quoted 1 line · Michal Schmidt

The kernel can always discard pages of executables and read-only mapped

files,

Quoted 2 lines

because it can load them back if necessary.
If you don't like this, call mlockall() in your program.

Thanks for the pointer, but what if I don't own the source. Is there a way to tell kswapd to disable this feature?

#5 lore raw
Quoted 5 lines · Al Boldi
Quoted 1 line · Michal Schmidt

If you don't like this, call mlockall() in your program.

Thanks for the pointer, but what if I don't own the source.

An LD_PRELOAD trick might work.

Michal

#6 lore raw
In reply to Al Boldi

On 28 Jun 2005, Al Boldi murmured woefully:

Quoted 2 lines

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

This is how the kernel has always worked. If you have no swap, memory pressure is higher than it otherwise might be, because dirty pages must be kept in physical RAM even if they are rarely used. So when memory gets low the kernel *has* to evict pages which aren't dirty, and write out and flush dirty pages corresponding to files on disk.

What else could it possibly do? Stop evicting everything?

I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no time.

Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
 at people who weren't supposed to be in your machine room.'
    --- Anthony de Boer
author
#7 lore raw

On 28 Jun 2005, Al Boldi murmured woefully:

Quoted 2 lines

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

Quoted 1 line · Nix

I can't even tell what you're expecting. Surely not that no pages are ever

evicted or flushed;

Quoted 1 line

your memory would fill up with page cache in no time.

Nix,
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

#8 lore raw

On 28 Jun 2005, Al Boldi yowled:

Quoted 9 lines
Quoted 5 lines · Nix

On 28 Jun 2005, Al Boldi murmured woefully:

Quoted 2 lines

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no time.

Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

But processes (really, mapped text pages; really, read-only mapped pages of all kinds) are loaded piecemeal in any case. Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.

A system in which pages can be faulted in *and* out is consistent: one in which they can only be faulted in is both inconsistent and very deadlock-prone.

Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
 at people who weren't supposed to be in your machine room.'
    --- Anthony de Boer
author
#9 lore raw

Hi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:

Quoted 6 lines
Quoted 5 lines · Nix

On 28 Jun 2005, Al Boldi murmured woefully:

Quoted 2 lines

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

I can't even tell what you're expecting. Surely not that no pages are ever evicted or flushed; your memory would fill up with page cache in no

time.

Quoted 3 lines

Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.
}

Nix,
You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.

Is it possible to change kswapd's default behaviour to not fault if there is no swap?

Thanks!

#10 lore raw
Quoted 6 lines · Al Boldi

Hi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:

Quoted 2 lines

Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

Quoted 6 lines

Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast.
}

You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.

That's `swapping', i.e. writing out dirty data pages (and text pages dirtied by e.g. relocation) which otherwise have nowhere to go.

Pages mapped from files with read-only mappings and non-dirty pages of read-write mappings get discarded when memory is tight; pages mapped with read-write non-private maps and which have been modified get written back to their file and discarded in the same situation.

(Private modified read-write mappings of data files have to stay in
memory: the only place they could go is swap, and there is none.)

Quoted 2 lines

Is it possible to change kswapd's default behaviour to not fault if there is no swap?

I don't think so, except on a process-by-process basis via mlockall(). (/proc/sys/vm/swappiness lets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)

You can kludge it to some extent by putting your binaries in a tmpfs and running from there: then you'll be paging from memory to memory ;)

But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things. Surely slow execution is generally preferable to *no* execution?

(Or is it that you're trying to suppress *disk activity*, perhaps to keep power consumption down? If so, running from tmpfs seems the best option.)

Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
 at people who weren't supposed to be in your machine room.'
    --- Anthony de Boer
author
#11 lore raw

Nix wrote:{

Quoted 10 lines · Al Boldi

Hi Nix, how are you?
You wrote: {
On 28 Jun 2005, Al Boldi yowled:

Quoted 2 lines

Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

Would you really like a system where once something was faulted in, it could never leave? You'd run out of memory *awfully* fast. }

Quoted 5 lines

You should only fault if you have a place to fault to, as into a swap. Without swap faulting is overkill.

Is it possible to change kswapd's default behaviour to not fault if there is no swap?

I don't think so, except on a process-by-process basis via mlockall(). (/proc/sys/vm/swappiness lets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)

But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things.
}

Nix,
You start killing things because you overcommitted, when you were not supposed to fault in the first place because you have no swap.

( I couldn't find /proc/sys/vm/swappiness in 2.4, although I heard about it in 2.6.)

#12 lore raw

On Tue, 28 Jun 2005, Al Boldi murmured woefully:

Quoted 6 lines

Nix wrote:{
I don't think so, except on a process-by-process basis via mlockall(). (/proc/sys/vm/swappiness lets you say that swapping is more or less desirable, but under enough memory pressure paging *will* happen regardless of the value of that variable.)

Quoted 8 lines

But I'm mystified as to why you might want to suppress paging. The only effect of suppressing it is to reduce the amount of memory you can allocate before you run completely out and start killing things.
}

Nix,
You start killing things because you overcommitted, when you were not supposed to fault in the first place because you have no swap.

The only thing that is blocked by having no swap is swapping. The only things that go to swap are things that can't go anywhere else (because swap is comparatievly slow). As far as I know, the only things that land in swap are modified private writable file-backed mappings (e.g. relocations in text sections of programs), and allocated anonymous mappings (what most people think of as `data'). (Data allocated via brk() is of the latter type.)

So if swap is turned off, the kernel can still acquire more memory when short by discarding file-backed pages from the page cache --- this includes non-modified text pages of programs and modified non-private file mappings, as well as data read/written via read() and write().

That is, with no swapfile, the kernel can't swap. It can still *page*, pushing stuff out to files if modified, throwing them out of memory if not, and reloading them from their original file when needed.

Quoted 2 lines

( I couldn't find /proc/sys/vm/swappiness in 2.4, although I heard about it in 2.6.)

Yeah, it's a 2.6 thing.

Signature · 3 lines
`I lost interest in "blade servers" when I found they didn't throw knives
 at people who weren't supposed to be in your machine room.'
    --- Anthony de Boer
#13 lore raw
In reply to Al Boldi
Quoted 11 lines · Al Boldi

On 28 Jun 2005, Al Boldi murmured woefully:

Quoted 2 lines

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

Quoted 1 line · Nix

I can't even tell what you're expecting. Surely not that no pages are ever

evicted or flushed;

Quoted 1 line

your memory would fill up with page cache in no time.

Nix,
Please do flush anytime, and do it in sync during OOMs; but don't evict procs especially not RUNNING procs, that is overkill.

Something running rarely gets thrown out - It is usually only pages that see little use that gets evicted. Such as startup code.

Of course you can improve the situation a lot by adding swap - the kernel will then be free to swap out dirty but little-used pages instead of not-dirty but more used executable mappings. That will improve your performance.

Getting more memory or running fewer programs also helps, obviously.

Helge Hafting

#14 lore raw
In reply to Al Boldi
Quoted 11 lines · Al Boldi
Quoted 3 lines · Al Boldi

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Marcelo,

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

I think what you really want is to adjust your "overcommit" settings.

See: "Documentation/vm/overcommit-accounting"

Signature · 5 lines
Paulo Marques - www.grupopie.com

It is a mistake to think you can solve any major problems
just with potatoes.
Douglas Adams
#15 lore raw
Quoted 8 lines · Paulo Marques
Quoted 7 lines · Al Boldi
Quoted 3 lines · Al Boldi

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Quoted 8 lines
Quoted 4 lines

Marcelo,

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

I think what you really want is to adjust your "overcommit" settings.

See: "Documentation/vm/overcommit-accounting"

Al,

You should set overcommit accordingly as Paulo mentions.

You might also want to disable the OOM killer (CONFIG_OOM_KILLER).

author
#16 lore raw

Marcelo wrote: {

Quoted 8 lines · Paulo Marques
Quoted 7 lines · Al Boldi
Quoted 3 lines · Al Boldi

In 2.4.31 kswapd starts paging during OOMs even w/o swap enabled.

Is there a way to fix/disable this behaviour?

Quoted 8 lines
Quoted 4 lines

Marcelo,

Kswapd starts evicting processes to fullfil a malloc, when it should just deny it because there is no swap.

I think what you really want is to adjust your "overcommit" settings.

See: "Documentation/vm/overcommit-accounting"

You might also want to disable the OOM killer (CONFIG_OOM_KILLER). }

Paulo,
Thanks for the pointer! Overcommit is the problem, and kswapd is not honoring it.
Can this be fixed/adjusted?

Marcelo,
Disable OOM killer? Is that an option in 2.4 or 2.6?

Thanks!