why swap at all?
Subsystems (5)
Hi!
I've recently started to follow this list.
I read the swap discussion here, and I was wondering about what Nick Pigging said about grepping the kernel tree.
Quoted 3 lines · Nick Piggin
For example, I have 57MB swapped right now. It allows me to instantly grep the kernel tree. If I turned swap off, each grep would probably take 30 seconds.
Are the pages swapped to disk as a result of the grep run?
Im still running 2.4.25. And when I do a grep on the linux kernel tree, it always takes at least 2 minutes at every run. Almost all physical ram, and 21MB of swap is used. Should the files read by grep be cached in memory/swap?
Michael Brennan
Hi,
Quote from Michael Brennan <[email protected]>:
Quoted 11 lines
Hi!
I've recently started to follow this list.
I read the swap discussion here, and I was wondering about what Nick Pigging said about grepping the kernel tree.Quoted 3 lines · Nick Piggin
For example, I have 57MB swapped right now. It allows me to instantly grep the kernel tree. If I turned swap off, each grep would probably take 30 seconds.
Are the pages swapped to disk as a result of the grep run?
I'm not really sure what the above was intended to demonstrate, but I assume that it was that having swap allowed the first grep to fill physical RAM with cache at the expense of swapping other processes, which were using physical RAM to disk.
However, if 57 Mb of swap allows this, 57 Mb of extra physical RAM should also also allow the grep to be cached, without having to swap out anything.
Hence my comment about it not being a magical property of swap space.
John.
Quoted 15 lines · John Bradford
Hi,
Quote from Michael Brennan <[email protected]>:
Quoted 10 lines
Hi!
I've recently started to follow this list.
I read the swap discussion here, and I was wondering about what Nick Pigging said about grepping the kernel tree.Quoted 3 lines · Nick Piggin
For example, I have 57MB swapped right now. It allows me to instantly grep the kernel tree. If I turned swap off, each grep would probably take 30 seconds.
Are the pages swapped to disk as a result of the grep run?
The pages are gradually swapped to disk as I use the system.
Quoted 6 lines
I'm not really sure what the above was intended to demonstrate, but I assume that it was that having swap allowed the first grep to fill physical RAM with cache at the expense of swapping other processes, which were using physical RAM to disk.
Well, at the "expense" of paging out unused memory. I don't see any swapin.
Quoted 3 lines
However, if 57 Mb of swap allows this, 57 Mb of extra physical RAM should also also allow the grep to be cached, without having to swap out anything.
Well yes, but if I had another 57MB of physical memory then I would still turn on swap so that other 57MB of unused memory isn't wasted.
Quoted 2 lines · In article <[email protected]> you wrote:
Well, at the "expense" of paging out unused memory. I don't see any swapin.
On a slow system with small memory you quite often see swapped out applications like for example a kopete messenger windows. Once you click on it, it takes 10sec or more to get responsive again. Of course its a slow system, but gradually paging out and forgetting image pages has that effecct on faster systems too, makes the desktop sluggish.
Quoted 2 lines
Well yes, but if I had another 57MB of physical memory then I would still turn on swap so that other 57MB of unused memory isn't wasted.
Actually the number of totally unused memory is quite small. Therefore the pages get swapped in sooner or later anyway. And even if you turn of fswap completely, the image pages backed up by binaries on disk get still freeded, if the code is unused. So on my multimedia system I prefer to have no swap (1GB ram) and make sure the pages are not freeded so aggressivley to keep the system smooth and responsive (and allow spin down of the disk).
Bernd
Signature · 2 lines
eckes privat - http://www.eckes.org/ Project Freefire - http://www.freefire.org/
Quoted 9 lines · Bernd Eckenfels
Quoted 2 lines · In article <[email protected]> you wrote:
Well, at the "expense" of paging out unused memory. I don't see any swapin.
On a slow system with small memory you quite often see swapped out applications like for example a kopete messenger windows. Once you click on it, it takes 10sec or more to get responsive again. Of course its a slow system, but gradually paging out and forgetting image pages has that effecct on faster systems too, makes the desktop sluggish.
I guess we have a design problem right here.We could add per-process swappiness attribute.That swap thread becomes boring coz we're looking globally what's going wrong locally.
FabF
Quoted 3 lines · FabF
I guess we have a design problem right here.We could add per-process swappiness attribute.That swap thread becomes boring coz we're looking globally what's going wrong locally.
Hmm.. do we need to worry about the same DoS issues we need to worry about with mlock and friends? I know I can trust myself to not do stupid things to said flags on my laptop (well... not twice anyhow ;). On the other hand, I have systems with clueless users, and the even more dangerous half-clued users. And then I have a bunch of machines in our security lab, where Bad Things happen all the time...
Quoted 6 lines · On Tue, 2004-06-01 at 21:02, [email protected] wrote:
Quoted 3 lines · FabF
I guess we have a design problem right here.We could add per-process swappiness attribute.That swap thread becomes boring coz we're looking globally what's going wrong locally.
Quoted 6 lines
Hmm.. do we need to worry about the same DoS issues we need to worry about with mlock and friends? I know I can trust myself to not do stupid things to said flags on my laptop (well... not twice anyhow ;). On the other hand, I have systems with clueless users, and the even more dangerous half-clued users. And then I have a bunch of machines in our security lab, where Bad Things happen all the time...
I was thinking about some rule e.g. any process using libX* isn't swapped to disk until OOM ...
Quoted 2 lines · FabF
I was thinking about some rule e.g. any process using libX* isn't swapped to disk until OOM ...
Odd.. some of the processes that I'd want kept in memory use libX*, but others that also use it are at the top of my list of things to migrate out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
And that rule doesn't even help much - as it will cause at least some X servers themselves to get swapped out. Here's the list for my X server at the moment, as reported by lsof:
X 13886 root txt REG 254,1 1960870 1966 /usr/X11R6/bin/Xorg X 13886 root mem REG 254,5 105700 12388 /lib/ld-2.3.3.so X 13886 root mem REG 254,5 50944 12530 /lib/libnss_files-2.3.3.so X 13886 root mem REG 254,1 64040 1347 /usr/lib/libz.so.1.2.1.1 X 13886 root mem REG 254,5 212972 53335 /lib/tls/libm-2.3.3.so X 13886 root mem REG 254,5 28008 12513 /lib/libpam.so.0.77 X 13886 root mem REG 254,5 15008 12471 /lib/libdl-2.3.3.so X 13886 root mem REG 254,5 8332 12515 /lib/libpam_misc.so.0.77 X 13886 root mem REG 254,5 29660 12511 /lib/libgcc_s-3.3.3-20040413.so.1 X 13886 root mem REG 254,5 1451868 53258 /lib/tls/libc-2.3.3.so X 13886 root mem REG 254,1 647652 32015 /usr/X11R6/lib/modules/extensions/libglx.so.1.0.5341 X 13886 root mem REG 254,1 4954876 8362 /usr/lib/tls/libGLcore.so.1.0.5341
Nope, no libX* here... ;)
It's a lot harder than it looks, which explains why we haven't gotten it right yet...
Quoted 29 lines · On Tue, 2004-06-01 at 22:00, [email protected] wrote:
Quoted 2 lines · FabF
I was thinking about some rule e.g. any process using libX* isn't swapped to disk until OOM ...
Odd.. some of the processes that I'd want kept in memory use libX*, but others that also use it are at the top of my list of things to migrate out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
And that rule doesn't even help much - as it will cause at least some X servers themselves to get swapped out. Here's the list for my X server at the moment, as reported by lsof:
X 13886 root txt REG 254,1 1960870 1966 /usr/X11R6/bin/Xorg X 13886 root mem REG 254,5 105700 12388 /lib/ld-2.3.3.so X 13886 root mem REG 254,5 50944 12530 /lib/libnss_files-2.3.3.so X 13886 root mem REG 254,1 64040 1347 /usr/lib/libz.so.1.2.1.1 X 13886 root mem REG 254,5 212972 53335 /lib/tls/libm-2.3.3.so X 13886 root mem REG 254,5 28008 12513 /lib/libpam.so.0.77 X 13886 root mem REG 254,5 15008 12471 /lib/libdl-2.3.3.so X 13886 root mem REG 254,5 8332 12515 /lib/libpam_misc.so.0.77 X 13886 root mem REG 254,5 29660 12511 /lib/libgcc_s-3.3.3-20040413.so.1 X 13886 root mem REG 254,5 1451868 53258 /lib/tls/libc-2.3.3.so X 13886 root mem REG 254,1 647652 32015 /usr/X11R6/lib/modules/extensions/libglx.so.1.0.5341 X 13886 root mem REG 254,1 4954876 8362 /usr/lib/tls/libGLcore.so.1.0.5341
Quoted 5 lines
Nope, no libX* here... ;)
It's a lot harder than it looks, which explains why we haven't gotten it right yet...
Boring....You can't have X root layer swapped to disk as it's often used ! Some quick lsof | grep "libX" gives all frontal applications 'swapping sensible' .fuser can do 'user resource reverse'.Kernel _can_ 'appl. resource reverse' as well.
PS: I'm not talking about inactive desktop box.Such box has to be rl 3 and is not meant to be user (geek) relevant :)
FabF
Quoted 4 lines · FabF
Boring....You can't have X root layer swapped to disk as it's often used ! Some quick lsof | grep "libX" gives all frontal applications 'swapping sensible' .fuser can do 'user resource reverse'.Kernel _can_ 'appl. resource reverse' as well.
The point you're missing is that if you use a rule such as "everything using libX* isn't swappable", then the X *server* is suddenly the prime candidate for swapping out (as it's quite likely the biggest user of memory not using libX*). (Anybody who ever had the OOM killer whomp their X server to free up space fast when the *real* problem was a cluster of 6 or 8 "large but still smaller than the X server" processes knows exactly what I mean... ;)
Quoted 2 lines
PS: I'm not talking about inactive desktop box.Such box has to be rl 3 and is not meant to be user (geek) relevant :)
So you're saying that I should have kicked my laptop down to runlevel 3 just because I went across the hall to the machine room to help get a few servers into racks? Or every time I go into a meeting, or get stuck on a longish phone call?
Also, be *very* careful equating "user" with "geek" - at least some of us are trying to produce systems that suit the needs of non-geek users....
Quoted 17 lines · On Tue, 2004-06-01 at 22:22, [email protected] wrote:
Quoted 4 lines · FabF
Boring....You can't have X root layer swapped to disk as it's often used ! Some quick lsof | grep "libX" gives all frontal applications 'swapping sensible' .fuser can do 'user resource reverse'.Kernel _can_ 'appl. resource reverse' as well.
The point you're missing is that if you use a rule such as "everything using libX* isn't swappable", then the X *server* is suddenly the prime candidate for swapping out (as it's quite likely the biggest user of memory not using libX*). (Anybody who ever had the OOM killer whomp their X server to free up space fast when the *real* problem was a cluster of 6 or 8 "large but still smaller than the X server" processes knows exactly what I mean... ;)
Quoted 2 lines
PS: I'm not talking about inactive desktop box.Such box has to be rl 3 and is not meant to be user (geek) relevant :)
Quoted 8 lines
So you're saying that I should have kicked my laptop down to runlevel 3 just because I went across the hall to the machine room to help get a few servers into racks? Or every time I go into a meeting, or get stuck on a longish phone call?
Also, be *very* careful equating "user" with "geek" - at least some of us are trying to produce systems that suit the needs of non-geek users....
As I said, I think this thread is "becoming offtopic" but what can be
interesting is the swapping problem fragmentation :
1.Global inactivity (what you're talking about) 2.Application isolation (what we're talking about).
Geek or not, someone backgrounding an application doesn't want it to down the box for X seconds some minutes later when it comes back and such things arrive many times a day.Maybe you've got an idea about a better rule(s) then ? (I mean for the 2 cases)
Quoted 2 lines · FabF
1.Global inactivity (what you're talking about) 2.Application isolation (what we're talking about).
Again, be careful there - I wasn't the one who said inactive boxes should be in RL3. ;)
And just because I may not be typing on the keyboard doesn't mean that things are in fact globally inactive - gkrellm is still running, and it has a plugin monitoring the CPU temperature and adjusting the fan speed as needed, and new mail is arriving in the background and causing status changes in my MUA.
And yes, said activity tends to keep the gkrellm and MUA pages "hot" and prevent their swapping out. The problem is that other processes are also doing stuff in the background for me - but there's no really good way for the system to know that I consider the gkrellm lages to be "more important" than those pages taken up by xclock....
Quoted 3 lines
Geek or not, someone backgrounding an application doesn't want it to down the box for X seconds some minutes later when it comes back and such things arrive many times a day.
Yes, but a solution to that really *should* take into account that some things will only down the *app* (if OpenOffice is paging in, I can still interact with the system if X and my window manager and an xterm aren't paged out), whereas other things will effectively down the *system* as far as the user is concerned (if X and/or my window manager are paged out, I'm *stuck* till they come back in).
Quoted 2 lines
Maybe you've got an idea about a
better rule(s) then ? (I mean for the 2 cases)
I admit I have slacked and haven't tried Nick Piggin's MM patches - others have commented that those work well. I am however quite sure that the Really Right Answer will require much greater subtlety than a rule like "if it uses libX it shouldn't be swapped out"....
Quoted 10 lines · FabF
As I said, I think this thread is "becoming offtopic" but what can be
interesting is the swapping problem fragmentation :1.Global inactivity (what you're talking about) 2.Application isolation (what we're talking about).
Geek or not, someone backgrounding an application doesn't want it to down the box for X seconds some minutes later when it comes back and such things arrive many times a day.Maybe you've got an idea about a better rule(s) then ? (I mean for the 2 cases)
Maybe what we need is a per-process tuner like nice, for swap candidacy. Unfortunately doing it right is probably 2.7 material, you want users to be able to set it DOWN for seldom used things, but not UP where they could hog the system. And I think 'right' also means having a capability for setting it UP again, etc.
Note that there are some hooks which *might* be useful for quick user, there is a sticky bit which seems pretty unused in practice, and which might cause pages to be marked less likely to swap. You could implement in exec() to do the setting, with whatever access control seems useful.
Just a thought, I'm pretty well convinced that Nick's latest patches have reduced the problem, at least for me. I'll try to get some metrics on the measured effect, but the "feel" is better by far.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 2 lines · Bill Davidsen
Just a thought, I'm pretty well convinced that Nick's latest patches have reduced the problem, at least for me. I'll try to get some metrics on the measured effect, but the "feel" is better by far.
Well thanks for testing them. I have another version with minor bug fixes and a sync up to recent -mm changes.
Quoted 4 lines · In article <[email protected]> you wrote:
out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Greetings
Bernd
Signature · 2 lines
eckes privat - http://www.eckes.org/ Project Freefire - http://www.freefire.org/
Quoted 10 lines · Bernd Eckenfels
Quoted 4 lines · In article <[email protected]> you wrote:
out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?
FabF
Quoted 2 lines
Greetings
Bernd
Quoted 2 lines · FabF
Quoted 1 line · Bernd Eckenfels
In article <[email protected]> you
Quoted 11 lines · wrote:
Quoted 8 lines
Quoted 4 lines
out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?
Unless you are pushing the limits of your available ram by your usage pattern then yes the autoregulated swappiness patch should help.
available here:
http://ck.kolivas.org/patches/2.6/2.6.7-rc2/patch-2.6.7-rc2-am11
Just a brief word that might clarify things for people. It seems this huge swap discussion centres around 2 different arguments. Akpm has said that the correct way for the vm to behave is that of swappiness=100. Desktop users note they have less swap out of the programs they use with swappiness 0 or their swap turned off. When your swappiness is set high, the current vm decisions are the fastest they can be, but when you go back to your applications they will take longer to restart. When your swappiness is set low your applications will restart rapidly, but the current vm will be doing more work and be slower. Most benchmarks will show the latter, but most desktop users will feel the former and not really notice the latter.
Try the little experiment to see: Boot with mem=128M and try to compile a 2.6 kernel with all the debugging symbols option enabled - do this with swappiness set to 0 and then at 100. You'll see it compile much faster at
100. Yet you know that if you set your swappiness to 0 mozilla will load faster next time you use it on your desktop during your normal usage pattern (of course you'd probably be using mozilla on a system with a bit more than 128M ram but this helps demonstrate the point).
Does this explain in coarse examples to the desktop users why ideal systems shouldn't be swap disabled or swappiness=0 ?
The autoregulated swappiness patch tries to get some sort of common ground, where it sacrifices performance slightly currently to improve what happens the next time you use your machine substantially. Because it changes with the amount of application pages in ram, it will not increasingly sacrifice performance when your memory is full with application pages. What it will not do is improve the swap thrash situation when you have grossly overloaded your ram.
Con
Quote from Con Kolivas <[email protected]>:
Quoted 2 lines
Does this explain in coarse examples to the desktop users why ideal systems shouldn't be swap disabled or swappiness=0 ?
Yes, except in the case where you are processing a small, (relative to physical RAM), dataset, and not even touching all physical RAM.
(I admit, this isn't really typical desktop usage, though).
John.
Quoted 8 lines · John Bradford
Quote from Con Kolivas <[email protected]>:
Quoted 2 lines
Does this explain in coarse examples to the desktop users why ideal systems shouldn't be swap disabled or swappiness=0 ?
Yes, except in the case where you are processing a small, (relative to physical RAM), dataset, and not even touching all physical RAM.
(I admit, this isn't really typical desktop usage, though).
Well there is no doubt that there are some unique scenarios where an algorithmic setting will not be as good as a single static setting; and that's why I put in the option of disabling the auto swappiness. I believe our proc settings in the kernel should not need to be adjusted for the majority of cases, though.
Con
Quoted 52 lines · Con Kolivas
Quoted 2 lines · FabF
Quoted 1 line · Bernd Eckenfels
In article <[email protected]> you
Quoted 11 lines · wrote:
Quoted 8 lines
Quoted 4 lines
out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?Unless you are pushing the limits of your available ram by your usage pattern then yes the autoregulated swappiness patch should help.
available here:
http://ck.kolivas.org/patches/2.6/2.6.7-rc2/patch-2.6.7-rc2-am11Just a brief word that might clarify things for people. It seems this huge swap discussion centres around 2 different arguments. Akpm has said that the correct way for the vm to behave is that of swappiness=100. Desktop users note they have less swap out of the programs they use with swappiness 0 or their swap turned off. When your swappiness is set high, the current vm decisions are the fastest they can be, but when you go back to your applications they will take longer to restart. When your swappiness is set low your applications will restart rapidly, but the current vm will be doing more work and be slower. Most benchmarks will show the latter, but most desktop users will feel the former and not really notice the latter.
Try the little experiment to see: Boot with mem=128M and try to compile a 2.6 kernel with all the debugging symbols option enabled - do this with swappiness set to 0 and then at 100. You'll see it compile much faster at
100. Yet you know that if you set your swappiness to 0 mozilla will load faster next time you use it on your desktop during your normal usage pattern (of course you'd probably be using mozilla on a system with a bit more than 128M ram but this helps demonstrate the point).Does this explain in coarse examples to the desktop users why ideal systems shouldn't be swap disabled or swappiness=0 ?
The autoregulated swappiness patch tries to get some sort of common ground, where it sacrifices performance slightly currently to improve what happens the next time you use your machine substantially. Because it changes with the amount of application pages in ram, it will not increasingly sacrifice performance when your memory is full with application pages. What it will not do is improve the swap thrash situation when you have grossly overloaded your ram.
Quoted 1 line
Con
My box rocks with you patch Con ! Swappiness is floating between 50->65. I never saw a 2.6 box so quick in rl5.
Thanks !
FabF
Quoted 26 lines · Con Kolivas
Quoted 3 lines · FabF
Quoted 1 line · Bernd Eckenfels
In article <[email protected]> you
Quoted 11 lines · wrote:
Quoted 8 lines
Quoted 4 lines
out (unlike some, I don't mind if Mozilla or OpenOffice end up out on disk after extended inactivity - but if my window manager gets swapped out, I get peeved when focus-follows-mouse doesn't and my typing goes into the wrong window or some such... ;)
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?Unless you are pushing the limits of your available ram by your usage pattern then yes the autoregulated swappiness patch should help.
available here:
http://ck.kolivas.org/patches/2.6/2.6.7-rc2/patch-2.6.7-rc2-am11
Quoted 7 lines
Just a brief word that might clarify things for people. It seems this huge swap discussion centres around 2 different arguments. Akpm has said that the correct way for the vm to behave is that of swappiness=100. Desktop users note they have less swap out of the programs they use with swappiness 0 or their swap turned off. When your swappiness is set high, the current vm decisions are the fastest they can be, but when you go back to your applications they will take longer to restart. When your swappiness is set low your applications will restart rapidly, but the current vm will be doing more work and be slower. Most benchmarks will show the latter, but most desktop users will feel the former and not really notice the latter.
Try the little experiment to see: Boot with mem=128M and try to compile a 2.6 kernel with all the debugging symbols option enabled - do this with swappiness set to 0 and then at 100. You'll see it compile much faster at 100. Yet you know that if you set your swappiness to 0 mozilla will load faster next time you use it on your desktop during your normal usage pattern (of course you'd probably be using mozilla on a system with a bit more than 128M ram but this helps demonstrate the point).
Does this explain in coarse examples to the desktop users why ideal systems shouldn't be swap disabled or swappiness=0 ?
The autoregulated swappiness patch tries to get some sort of common ground, where it sacrifices performance slightly currently to improve what happens the next time you use your machine substantially. Because it changes with the amount of application pages in ram, it will not increasingly sacrifice performance when your memory is full with application pages. What it will not do is improve the swap thrash situation when you have grossly overloaded your ram.
But swap behaviour kills performance even when memory is more than adequate. Consider building a DVD image in a 4GB system. The i/o forces all of the unused programs out, in spite of the fact that an extra 100MB doesn't make a measurable difference in performance. But when I click Mozilla paging most of it in from disk make a big difference in performance to the user.
My perception is that the system is really bad at recognizing diminishing returns to be had by paging programs for the benefit of i/o. Not to mention what happens if you get 2-3GB of dirty buffers and then do a sync(). In practice my little RAID array will take at most 40MB/s, so the creation of a DVD runs fast, but the system bogs right after.
The problems with small memory are different in kind, when not even the programs will fit in memory at the same time, or will leave next to nothing for i/o, swap is required for performance. But on a large memory system I believe the gain to pain ratio is way too low with the current VM. The solution at the moment is to turn off swap, which as you note has other problems (can't move between zones without swap?) which in theory could really hang a system.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 6 lines
But swap behaviour kills performance even when memory is more than adequate. Consider building a DVD image in a 4GB system. The i/o forces all of the unused programs out, in spite of the fact that an extra 100MB doesn't make a measurable difference in performance. But when I click Mozilla paging most of it in from disk make a big difference in performance to the user.
We really need a server option. Something that ages out file backed pages naturally with less overhead than kswapd. One method would be to keep the pagecache on it's own list, and move pages to the head of the list any time they are modified or referenced, and reclaim from the tail.
All pages on this list can be considered as "free memory", because any new memory requests would just cause pages to be evicted from the tail of the list.
Anonymous memory would *not* be on this list. This way any time anonymous memory is allocated, the pages can be readily stolen from the pagecache list.
Lastly one nifty configuration parameter that could exist as a knob for sys-admins is the ability to tell the VM not to add file backed pages with the execute bit set to the page cache list but rather, leave them to be reclaimed if kswapd wakes up in a true low memory situation (pagecache is exhausted and memory is still low). This would require a sys-admin to make sure only executables have the execute bit set and "data files", etc... do not have the execute bit set.
A system that works like this is nice for the following reasons:
1) The system administrator can size a system so that all programs Safely run within physical RAM. Extra RAM
Could be added and sized based on the need
for caching files.
2) Anonymous pages (and possibly executable if you read the last paragraph above) will only be evicted if kswapd is awaken due to a true memory shortage (1/128th pagable memory?).
I like to view the VM system as always being full, because if enough unique file system IO takes place, that is exactly what eventually happens. A system that counts page cache as free memory and uses a gentler mechanism to evict pages from the page cache would benefit IO bound servers significantly IMHO.
--Buddy
Quoted 17 lines · Buddy Lumpkin
Quoted 1 line
But swap behaviour kills performance even when memory is more than adequate. Consider building a DVD image in a 4GB system. The i/o forces all of the unused programs out, in spite of the fact that an extra 100MB doesn't make a measurable difference in performance. But when I click Mozilla paging most of it in from disk make a big difference in performance to the user.
We really need a server option. Something that ages out file backed pages naturally with less overhead than kswapd. One method would be to keep the pagecache on it's own list, and move pages to the head of the list any time they are modified or referenced, and reclaim from the tail.
All pages on this list can be considered as "free memory", because any new memory requests would just cause pages to be evicted from the tail of the list.
Anonymous memory would *not* be on this list. This way any time anonymous memory is allocated, the pages can be readily stolen from the pagecache list.
Quoted 7 lines
Lastly one nifty configuration parameter that could exist as a knob for sys-admins is the ability to tell the VM not to add file backed pages with the execute bit set to the page cache list but rather, leave them to be reclaimed if kswapd wakes up in a true low memory situation (pagecache is exhausted and memory is still low). This would require a sys-admin to make sure only executables have the execute bit set and "data files", etc... do not have the execute bit set.
Or have the exec() call set a "part of a process" flag. That means that if I read an executable in as data it doesn't get locked, other than what part might be in my i/o buffers. And mmap can produce different effects than read/write which may be good, if they are GOOD different effects ;-) Before you ask, thing 'strings' as why avg user does this.
But I fail to make my point... I want to limit how much memory is used for i/o buffers, cache, or anything else which will produce memory pressure of my programs. The quick solution might be just a number from the admin, like the 2.2 patch, but some kernel logic to understand that while 20MB is much better than 10MB in a tiny system, 2GB is not a lot better than 1GB in a large memory system, and having a sync() bog the system for tens of seconds is undesirable. Well, maybe some folks don't agree, it could be that the admin set limit is really the way to go.
I regard this as a desktop issue, trading some i/o performance to keep window changes fast.
Quoted 14 lines
A system that works like this is nice for the following reasons:
1) The system administrator can size a system so that all programs Safely run within physical RAM. Extra RAM
Could be added and sized based on the need
for caching files.2) Anonymous pages (and possibly executable if you read the last paragraph above) will only be evicted if kswapd is awaken due to a true memory shortage (1/128th pagable memory?).
Quoted 5 lines
I like to view the VM system as always being full, because if enough unique file system IO takes place, that is exactly what eventually happens. A system that counts page cache as free memory and uses a gentler mechanism to evict pages from the page cache would benefit IO bound servers significantly IMHO.
That's what would be nice with tuning, the admin can optimize what is important on that system. I am usually happy with what the system does on i/o, but I want my 500MB or so of programs to stay resident in a 2GB machine, and if that adds a ms or two to i/o I can live with it, so that when I change windows it happens now, not eventually. And I bet there are a lot of others who would like better response to focus changes aswell.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 3 lines · Bill Davidsen
But I fail to make my point... I want to limit how much memory is used for i/o buffers, cache, or anything else which will produce memory pressure of my programs.
I would love to be able to limit this kind of memory use.
I've always liked how BSD works in this area, never using over a certain amount.
I find the Linux behavior of using all memory for things like buffercache is less than optimal. While there are situations where it helps, there are a great many where it hurts.
I frequently do work which fills memory with data I'll never use again, and it makes things slow.
Desktop work tends to do this kind of thing as well.
Quoted 6 lines
That's what would be nice with tuning, the admin can optimize what is important on that system. I am usually happy with what the system does on i/o, but I want my 500MB or so of programs to stay resident in a 2GB machine, and if that adds a ms or two to i/o I can live with it, so that when I change windows it happens now, not eventually. And I bet there are a lot of others who would like better response to focus changes aswell.
Not only that, but I wish certain bits of code could be locked into memory. Generally any code and data associated with the user interface should always be there.
It's annoying when a menu in X takes ten seconds of swapping to appear.
Signature · 2 lines
shannon "AT" widomaker.com -- [javalin: an unwieldy programming weapon used to stab a software project through the heart until dead]
Hello!
Quoted 1 line
But swap behaviour kills performance even when memory is more than adequate. Consider building a DVD image in a 4GB system. The i/o forces all of the unused programs out, in spite of the fact that an extra 100MB doesn't make a measurable difference in performance. But when I click Mozilla paging most of it in from disk make a big difference in performance to the user.
I think that kernel cannot know that you need some data once or more. This is fadvise for.
With my wrapper (http://kernel.umbrella.ro) for fadvise you can do this:
NOCA_SIZE=128 NOCA_READ=1 NOCA_WRITE=1 NOCA_RA=1 \
noca mkisofs -R -o /tmp/1.iso /tmp/data
This means:NOCA_SIZE: Call fadvise only after 128KiB was read/wrote.NOCA_RA: call fadvise with POSIX_FADV_SEQUENTIALNOCA_READ: use fadvise(POSIX_FADV_DONTNEED) for reads (because you don't need anymore the source files)NOCA_WRITE: use fadvise(POSIX_FADV_DONTNEED) for writes (because it's useless to cache the end of the ISO)
Do this program resolve your problem?
Quoted 12 lines
The problems with small memory are different in kind, when not even the programs will fit in memory at the same time, or will leave next to nothing for i/o, swap is required for performance. But on a large memory system I believe the gain to pain ratio is way too low with the current VM. The solution at the moment is to turn off swap, which as you note has other problems (can't move between zones without swap?) which in theory could really hang a system.
Signature · 8 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
---
Catalin(ux aka Dino) BOIE
catab at deuroconsult.ro
http://kernel.umbrella.ro/
Quoted 11 lines · Catalin BOIE
Hello!
Quoted 1 line
But swap behaviour kills performance even when memory is more than adequate. Consider building a DVD image in a 4GB system. The i/o forces all of the unused programs out, in spite of the fact that an extra 100MB doesn't make a measurable difference in performance. But when I click Mozilla paging most of it in from disk make a big difference in performance to the user.
I think that kernel cannot know that you need some data once or more. This is fadvise for.
With my wrapper (http://kernel.umbrella.ro) for fadvise you can do this:
NOCA_SIZE=128 NOCA_READ=1 NOCA_WRITE=1 NOCA_RA=1 \
noca mkisofs -R -o/tmp/1.iso/tmp/data
Quoted 7 lines
This means:
NOCA_SIZE: Call fadvise only after 128KiB was read/wrote.NOCA_RA: call fadvise withPOSIX_FADV_SEQUENTIALNOCA_READ: usefadvise(POSIX_FADV_DONTNEED)for reads (because you don't need anymore the source files)NOCA_WRITE: usefadvise(POSIX_FADV_DONTNEED)for writes (because it's useless to cache the end of the ISO)Do this program resolve your problem?
It addresses one of the cases which trigger problems, certainly. Thank you.
Quoted 2 lines
Quoted 2 lines
The problems with small memory are different in kind, when not even the programs will fit in memory at the same time, or will leave next to nothing for i/o, swap is required for performance. But on a large memory system I believe the gain to pain ratio is way too low with the current VM. The solution at the moment is to turn off swap, which as you note has other problems (can't move between zones without swap?) which in theory could really hang a system.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 2 lines · Bill Davidsen
My perception is that the system is really bad at recognizing diminishing returns to be had by paging programs for the benefit of i/o.
Currently the kernel has no mechanisms at all to do any kind of detection of bad pageout decisions it made in the past, and consequently no way to learn for the future.
Checks and balances ... those are what's missing ;)
Signature · 3 lines
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan
Quoted 6 lines · FabF
Quoted 4 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?
Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
Quoted 6 lines · On Wed, 2004-06-02 at 19:59, [email protected] wrote:
Quoted 4 lines · FabF
Quoted 4 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Quoted 6 lines
Quoted 2 lines
Exactly !
Does autoregulated VM swap. patch could help here ?Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
It helps inactive applications to re-ermerge smoothly, heavy I/O and global tuning.I've got 20 swapping delta from start to high usage. That patch rock'n'roll my box until updatedb makes sw climbs up to 80 and freezes my box :(
FabF
Quoted 9 lines · FabF
Quoted 8 lines · On Wed, 2004-06-02 at 19:59, [email protected] wrote:
Quoted 6 lines · FabF
Quoted 3 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?
Quoted 8 lines
Quoted 3 lines
Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
It helps inactive applications to re-ermerge smoothly, heavy I/O and global tuning.I've got 20 swapping delta from start to high usage. That patch rock'n'roll my box until updatedb makes sw climbs up to 80 and freezes my box :(
Try this version instead which biases it downwards.
Con
include/linux/swap.h +1 -0
@@ -175,6 +175,7 @@ extern void swap_setup(void);extern int try_to_free_pages (struct zone **,unsigned int ,unsigned int );extern int shrink_all_memory (int );extern int vm_swappiness; +extern int auto_swappiness;#ifdef CONFIG_MMU /* linux/mm/shmem.c */
include/linux/sysctl.h +1 -0
@@ -164,6 +164,7 @@ enumVM_LAPTOP_MODE =23 ,/* vm laptop mode */ VM_BLOCK_DUMP =24 ,/* block dump mode */ VM_HUGETLB_GROUP =25 ,/* permitted hugetlb group */ + VM_AUTO_SWAPPINESS=26, /* Make vm_swappiness autoregulated */ };
kernel/sysctl.c +8 -0
@@ -727,6 +727,14 @@ static ctl_table vm_table[] = { .extra1 = &zero, .extra2 = &one_hundred, }, + { + .ctl_name = VM_AUTO_SWAPPINESS, + .procname = "autoswappiness", + .data = &auto_swappiness, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec, + }, #ifdef CONFIG_HUGETLB_PAGE { .ctl_name = VM_HUGETLB_PAGES,
mm/vmscan.c +36 -0
@@ -43,6 +43,7 @@* From 0 .. 100. Higher means more swappy. */ int vm_swappiness =60 ; +int auto_swappiness =1 ;static long total_memory;#define lru_to_page (_head) (list_entry((_head)->prev, struct page, lru))
@@ -634,6 +635,41 @@ refill_inactive_zone(struct zone *zone,*/ mapped_ratio = (ps->nr_mapped *100 ) / total_memory; +if (auto_swappiness) { +#ifdef CONFIG_SWAP +int app_percent; +struct sysinfo i; + +si_swapinfo (&i); + +if (likely (i.totalswap >=100 )) { +int swap_centile; + +/* +* app_percent is the percentage of physical ram used +* by application pages. +*/ +si_meminfo (&i); + app_percent =100 - ((i.freeram +get_page_cache_size () - + swapper_space.nrpages) / (i.totalram /100 )); + +/* +* swap_centile is the percentage of the last (sizeof physical +* ram) of swap free. +*/ + swap_centile = i.freeswap / + (min (i.totalswap, i.totalram) /100 ); +/* +* Autoregulate vm_swappiness to be equal to the lowest of +* app_percent and swap_centile. Bias it downwards -ck +*/ + vm_swappiness =min (app_percent, swap_centile); + vm_swappiness = vm_swappiness * vm_swappiness /100 ; + }else + vm_swappiness =0 ; +#endif + } +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory
Quoted 107 lines · Con Kolivas
Quoted 9 lines · FabF
Quoted 8 lines · On Wed, 2004-06-02 at 19:59, [email protected] wrote:
Quoted 6 lines · FabF
Quoted 3 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?Quoted 8 lines
Quoted 3 lines
Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
It helps inactive applications to re-ermerge smoothly, heavy I/O and global tuning.I've got 20 swapping delta from start to high usage. That patch rock'n'roll my box until updatedb makes sw climbs up to 80 and freezes my box :(
Try this version instead which biases it downwards.
Con
______________________________________________________________________
diff -Naurp --exclude-from=dontdifflinux-2.6.7-rc2-base/include/linux/swap.hlinux-2.6.7-rc2-am11/include/linux/swap.hinclude/linux/swap.h +1
-0@@ -175,6 +175,7 @@ extern void swap_setup(void);extern int try_to_free_pages (struct zone **,unsigned int ,unsigned int );extern int shrink_all_memory (int );extern int vm_swappiness; +extern int auto_swappiness;#ifdef CONFIG_MMU /* linux/mm/shmem.c */ diff -Naurp --exclude-from=dontdiff
linux-2.6.7-rc2-base/include/linux/sysctl.hlinux-2.6.7-rc2-am11/include/linux/sysctl.hinclude/linux/sysctl.h +1
-0@@ -164,6 +164,7 @@ enumVM_LAPTOP_MODE =23 ,/* vm laptop mode */ VM_BLOCK_DUMP =24 ,/* block dump mode */ VM_HUGETLB_GROUP =25 ,/* permitted hugetlb group */ + VM_AUTO_SWAPPINESS=26, /* Make vm_swappiness autoregulated */ };diff -Naurp --exclude-from=dontdiff
linux-2.6.7-rc2-base/kernel/sysctl.clinux-2.6.7-rc2-am11/kernel/sysctl.ckernel/sysctl.c +8
-0@@ -727,6 +727,14 @@ static ctl_table vm_table[] = { .extra1 = &zero, .extra2 = &one_hundred, }, + { + .ctl_name = VM_AUTO_SWAPPINESS, + .procname = "autoswappiness", + .data = &auto_swappiness, + .maxlen = sizeof(int), + .mode = 0644, + .proc_handler = &proc_dointvec, + }, #ifdef CONFIG_HUGETLB_PAGE { .ctl_name = VM_HUGETLB_PAGES,diff -Naurp --exclude-from=dontdiff
linux-2.6.7-rc2-base/mm/vmscan.clinux-2.6.7-rc2-am11/mm/vmscan.cmm/vmscan.c +29
-0@@ -43,6 +43,7 @@* From 0 .. 100. Higher means more swappy. */ int vm_swappiness =60 ; +int auto_swappiness =1 ;static long total_memory;#define lru_to_page (_head) (list_entry((_head)->prev, struct page, lru))@@ -634,6 +635,41 @@ refill_inactive_zone(struct zone *zone,*/ mapped_ratio = (ps->nr_mapped *100 ) / total_memory; +if (auto_swappiness) { +#ifdef CONFIG_SWAP +int app_percent; +struct sysinfo i; + +si_swapinfo (&i); + +if (likely (i.totalswap >=100 )) { +int swap_centile; + +/* +* app_percent is the percentage of physical ram used +* by application pages. +*/ +si_meminfo (&i); + app_percent =100 - ((i.freeram +get_page_cache_size () - + swapper_space.nrpages) / (i.totalram /100 )); + +/* +* swap_centile is the percentage of the last (sizeof physical +* ram) of swap free. +*/ + swap_centile = i.freeswap / + (min (i.totalswap, i.totalram) /100 ); +/* +* Autoregulate vm_swappiness to be equal to the lowest of +* app_percent and swap_centile. Bias it downwards -ck +*/
mm/vmscan.c refill_inactive_zone(struct zone *zone, · line 675
Quoted 10 lines
mm/vmscan.c +7
-0+ vm_swappiness =min (app_percent, swap_centile); + vm_swappiness = vm_swappiness * vm_swappiness / 100; + }else + vm_swappiness =0 ; +#endif + } +/* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory
I've been unhappy with this one.sw range : 19->60.
So I've been playing slightly with sw curve replacing nerve centre with :vm_swappiness = vm_swappiness * vm_swappiness / 50;
if (vm_swappiness > 100)
vm_swappiness = 100;
Results :
Warmup :Smooth, 30
Under pressure, swap grows roughly 60*60/50->72, 70*70/50->98 .... and ... rock'n'roll .... Box seems to keep the road this time. updatedb gives side effects after 30 sec. though....sw drops to 70 ... but I've got good global response.I guess we could expose curve parameter.That one seems determinant :
vm_swappiness = vm_swappiness * vm_swappiness / swapcurve; if (vm_swappiness > 100)
vm_swappiness = 100;
and sysctl stuff as well :
{
.ctl_name = VM_AUTO_SWAPPINESS,
.procname = "swapcurvature",
.data = &swapcurve,
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_curvature,
},Regards,
FabF
Quoted 4 lines · FabF
Quoted 1 line · Con Kolivas
Try this version instead which biases it downwards.
I've been unhappy with this one.sw range : 19->60.
So I've been playing slightly with sw curve replacing nerve centre with
Are you unhappy with the numbers for swappiness it gives or the feel of it? It gives a range of 0-100 in meaningful ways. Your version gives swappiness > 100 at times (oops). If this version does not feel good, the last linear one is better and you simply dont have enough ram for it to feel good after updatedb.
Con
Quoted 4 lines · Con Kolivas
Quoted 3 lines · FabF
Quoted 1 line · Con Kolivas
Try this version instead which biases it downwards.
Quoted 8 lines
Quoted 2 lines
I've been unhappy with this one.sw range : 19->60.
So I've been playing slightly with sw curve replacing nerve centre withAre you unhappy with the numbers for swappiness it gives or the feel of it? It gives a range of 0-100 in meaningful ways. Your version gives swappiness
Quoted 1 line
100 at times (oops). If this version does not feel good, the last linear
one is better and you simply dont have enough ram for it to feel good after updatedb.
Oh and I forgot to say, if that's the case then you should try Nick's patches which are far more sophisticated than this.
Con
Quoted 7 lines · [email protected] wrote:
Quoted 4 lines · FabF
Quoted 3 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Quoted 7 lines
Quoted 2 lines
Exactly !
Does autoregulated VM swap. patch could help here ?Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
I haven't had a chance to try Con's stuff, the Nick patch is working VERY well for me, small memory and slow system, lots of memory pressure. Hopefully you can report a comparison.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 9 lines · Bill Davidsen
Quoted 8 lines · [email protected] wrote:
Quoted 6 lines · FabF
Quoted 3 lines
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Exactly !
Does autoregulated VM swap. patch could help here ?
Quoted 7 lines
Quoted 3 lines
Con's auto-adjusting swappiness patch did in fact help that quite a bit, especially for the case of heavy file I/O causing process images to be swapped out. I need to do some comparisons of that to Nick's MM work...
I haven't had a chance to try Con's stuff, the Nick patch is working VERY well for me, small memory and slow system, lots of memory pressure. Hopefully you can report a comparison.
Well note there are two revisions available now. The original linear design is here:
http://ck.kolivas.org/patches/2.6/2.6.7-rc2/patch-2.6.7-rc2-am11
and there is an exponential curve bias in this one which will probably deprecate the last one:
http://ck.kolivas.org/patches/2.6/2.6.7-rc2/patch-2.6.7-rc2-as
I am keen to get more feedback; apart from what I get off list there has been very little in the way of reports.
Con
Quoted 3 lines · Bernd Eckenfels
Yes but: your wm is so often used/activated it will not get swaped out. But if your mouse passes over mozilla and tries to focus it, then you will feel the pain of a swapped-out x program.
Yes, I'm quite familiar with what a swapped-out mozilla does to my laptop ;)
The point I was making (apparently poorly) was that if mozilla is swapping in, *that window* is hosed, but if the WM or the X server is swapping in, *everything* is hosed.
And I *have* had times when I've left for an extended period while Mozilla is downloading a Knoppix .iso or similar beastly large thing, and it managed to keep Mozilla pages hot because it was busy doing a download, and the WM pages got swapped out because the WM wasn't actually doing anything...
Yes, it's a rare state of affairs, but it *has* happened...
[email protected] said on Tue, 01 Jun 2004 15:02:48 -0400:
Quoted 9 lines
--==_Exmh_482188856P
Content-Type: text/plain; charset=us-asciiQuoted 3 lines · FabF
I guess we have a design problem right here.We could add per-process swappiness attribute.That swap thread becomes boring coz we're looking globally what's going wrong locally.
Quoted 6 lines
Hmm.. do we need to worry about the same DoS issues we need to worry about with mlock and friends? I know I can trust myself to not do stupid things to said flags on my laptop (well... not twice anyhow ;). On the other hand, I have systems with clueless users, and the even more dangerous half-clued users. And then I have a bunch of machines in our security lab, where Bad Things happen all the time...
I do often get frustrated that the DoS card is brought up to kill a potentially useful solution. I think there should be a flag in KConfig saying "This machine will be a server"/"This machine will be mostly a single user desktop machine". In the latter, you can enable all these vm/etc heuristics that will help out mozilla/X/your favourite bloat-ware, but potentially enable a DoS attack, and in the former, you stay conservative.
I can't rememeber the situation that I was last annoyed by someone saying "but what about a DoS?"...
Signature · 3 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/
Entropy requires no maintenance.
-- Markoff ChaneyQuoted 7 lines · Tim Connors
I do often get frustrated that the DoS card is brought up to kill a potentially useful solution. I think there should be a flag in KConfig saying "This machine will be a server"/"This machine will be mostly a single user desktop machine". In the latter, you can enable all these vm/etc heuristics that will help out mozilla/X/your favourite bloat-ware, but potentially enable a DoS attack, and in the former, you stay conservative.
And with that, you've worried about whether it's a potential DoS or not. I didn't bring it up to "kill" it - I brought it up to start a discussion, because I felt that including that sort of feature without at least thinking about the DoS issues was a bad idea. Shipping it with a Kconfig or sysctl flag, or using the capabilities framework, or any other similar "allow the sysadmin to control it" feature is a different matter entirely...
Quoted 6 lines
Quoted 3 lines
However, if 57 Mb of swap allows this, 57 Mb of extra physical RAM should also also allow the grep to be cached, without having to swap out anything.
Well yes, but if I had another 57MB of physical memory then I would still turn on swap so that other 57MB of unused memory isn't wasted.
Sure, but tell me, for example, what is the point of having swap on a system like this:
$ free
total used free shared buffers cached
Mem: 516688 19900 496788 0 628 11276
-/+ buffers/cache: 7996 508692
Swap: 0 0 0John.
Quoted 7 lines · John Bradford
Sure, but tell me, for example, what is the point of having swap on a system like this:
$ free total used free shared buffers cached Mem: 516688 19900 496788 0 628 11276 -/+ buffers/cache: 7996 508692 Swap: 0 0 0
So you can move userspace pages out of ZONE_DMA as-needed.
-- wli
Quote from William Lee Irwin III <[email protected]>:
Quoted 10 lines · John Bradford
Quoted 7 lines
Sure, but tell me, for example, what is the point of having swap on a system like this:
$ free total used free shared buffers cached Mem: 516688 19900 496788 0 628 11276 -/+ buffers/cache: 7996 508692 Swap: 0 0 0So you can move userspace pages out of
ZONE_DMAas-needed.
But how does that improve performance before untouched RAM, (496788 in this example), is exhausted?
In normal use, (almost always CPU bound), I've honestly never noticed any performance gain from having swap configured. I must admit I haven't put a lot of effort recently in to looking at this, but I have never been able to reproduce these 'swap increases performance even with untouched RAM' claims.
John.
Quote from William Lee Irwin III <[email protected]>:
Quoted 1 line
Quoted 1 line
So you can move userspace pages out of
ZONE_DMAas-needed.
Quoted 7 lines · John Bradford
But how does that improve performance before untouched RAM, (496788 in this example), is exhausted?
In normal use, (almost always CPU bound), I've honestly never noticed any performance gain from having swap configured. I must admit I haven't put a lot of effort recently in to looking at this, but I have never been able to reproduce these 'swap increases performance even with untouched RAM' claims.
Because ZONE_DMA, the lower 16MB is not all of RAM.
-- wli
Quote from William Lee Irwin III <[email protected]>:
Quoted 12 lines
Quote from William Lee Irwin III <[email protected]>:
Quoted 1 line
Quoted 1 line
So you can move userspace pages out of
ZONE_DMAas-needed.Quoted 7 lines · John Bradford
But how does that improve performance before untouched RAM, (496788 in this example), is exhausted?
In normal use, (almost always CPU bound), I've honestly never noticed any performance gain from having swap configured. I must admit I haven't put a lot of effort recently in to looking at this, but I have never been able to reproduce these 'swap increases performance even with untouched RAM' claims.
Quoted 1 line
Because
ZONE_DMA, the lower 16MB is not all of RAM.
Ah, OK, this isn't really my area of expertise so maybe this is a stupid, (for LKML), question, but can we only migrate data from low RAM via swap!?
Also, surely this is only relevant to X86 architectures?
John.
John Bradford <[email protected]> said on Tue, 1 Jun 2004 10:10:32 +0100:
Quoted 11 lines
Quote from William Lee Irwin III <[email protected]>:
Quoted 5 lines
Quote from William Lee Irwin III <[email protected]>:
Quoted 2 lines
to reproduce these 'swap increases performance even with untouched RAM' claims.
Because
ZONE_DMA, the lower 16MB is not all of RAM.Ah, OK, this isn't really my area of expertise so maybe this is a stupid, (for LKML), question, but can we only migrate data from low RAM via swap!?
Also, surely this is only relevant to X86 architectures?
I just got an interesting problem - possibly (or not?) related to this:
I have my laptop with 256MB of RAM, running 2.4.25-pre7, uptime 48 days. Every morning, I come in and have to wait 2 minutes while everything comes back into RAM, after the daily slocate. So I did a swapoff -a, and it failed despite all the applications and cache and tmpfs adding up to far less than 256MB (more like 128).
I closed mozilla, which let me do a swapoff -a.
All was well for a few days, but then thismorning, my partitions were mounted ro, and an oops was in syslog at the same time as all the slocate work:
Jun 8 07:55:24 scuzzie kernel: VM: killing process xemacs Jun 8 07:56:17 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:56:17 scuzzie kernel: VM: killing process mailoops Jun 8 07:57:30 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d0/0) Jun 8 07:57:34 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:35 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:35 scuzzie kernel: VM: killing process sh Jun 8 07:57:35 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:35 scuzzie kernel: VM: killing process sh Jun 8 07:57:38 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:38 scuzzie kernel: VM: killing process ssh-agent Jun 8 07:57:38 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:38 scuzzie kernel: VM: killing process gunzip Jun 8 07:57:40 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:40 scuzzie kernel: VM: killing process gunzip Jun 8 07:57:40 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:40 scuzzie kernel: VM: killing process gunzip Jun 8 07:57:41 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d0/0) Jun 8 07:57:41 scuzzie kernel: ERROR: (device ide0(3,5)): txAbortCommit Jun 8 07:57:41 scuzzie kernel: BUG at jfs_txnmgr.c:509 assert(tblk->next == 0) Jun 8 07:57:41 scuzzie kernel: kernel BUG at jfs_txnmgr.c:509! Jun 8 07:57:41 scuzzie kernel: invalid operand: 0000 Jun 8 07:57:41 scuzzie kernel: CPU: 0 Jun 8 07:57:41 scuzzie kernel: EIP: 0010:[txEnd+235/256] Not tainted Jun 8 07:57:41 scuzzie kernel: EFLAGS: 00010282 Jun 8 07:57:41 scuzzie kernel: eax: 00000033 ebx: d0811c98 ecx: c12dc000 edx: cd9f5f7c Jun 8 07:57:41 scuzzie kernel: esi: 00000102 edi: cfec1d60 ebp: cf4eb000 esp: c12ddf18 Jun 8 07:57:41 scuzzie kernel: ds: 0018 es: 0018 ss: 0018 Jun 8 07:57:41 scuzzie kernel: Process keventd (pid: 2, stackpage=c12dd000) Jun 8 07:57:41 scuzzie kernel: Stack: c028ed43 c028f190 000001fd c028f180 fffffffb 00000102 cf4eb05c c017db9e Jun 8 07:57:41 scuzzie kernel: 00000102 00000001 c12ddf54 00000000 00000000 c35c5220 c017dc0e c35c5220 Jun 8 07:57:41 scuzzie kernel: 00000000 00000001 c014d3f2 c35c5220 00000000 00000003 c12ddf88 c12ddf88 Jun 8 07:57:41 scuzzie kernel: Call Trace: [jfs_commit_inode+222/256] [jfs_write_inode+78/96] [try_to_sync_unused_inodes+466/480] [__run_task_queue+90/112] [cont ext_thread+442/448] Jun 8 07:57:41 scuzzie kernel: [context_thread+0/448] [rest_init+0/64] [arch_kernel_thread+46/64] [context_thread+0/448] Jun 8 07:57:41 scuzzie kernel: Jun 8 07:57:41 scuzzie kernel: Code: 0f 0b fd 01 90 f1 28 c0 e9 71 ff ff ff 90 8d b4 26 00 00 00 Jun 8 07:57:41 scuzzie kernel: <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:41 scuzzie kernel: VM: killing process gunzip Jun 8 07:57:49 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) Jun 8 07:57:49 scuzzie kernel: VM: killing process mozilla-bin Jun 8 07:57:49 scuzzie kernel: __alloc_pages: 0-order allocation failed (gfp=0x1f0/0) Jun 8 07:58:09 scuzzie wwwoffles[25878]: Cannot make a lock file for 'http/www.bom.gov.au/LkdPT+cpeS73Lf-picjpvUw' [Read-only file system]. Jun 8 07:58:58 scuzzie anacron[16529]: Job `cron.weekly' terminated (exit status: 1) (mailing output) Jun 8 07:58:59 scuzzie modprobe: modprobe: cannot create /var/log/ksymoops/20040608.log Read-only file system Jun 8 07:58:59 scuzzie modprobe: modprobe: cannot create /var/log/ksymoops/20040608.log Read-only file system Jun 8 07:58:59 scuzzie anacron[16529]: Tried to mail output of job `cron.weekly', but mailer process (/usr/sbin/sendmail) exited with ststus 1 Jun 8 07:58:59 scuzzie anacron[16529]: Normal exit (2 jobs run)
So OOM - but why? The cache was registering 65MB used. 24353,23> cat /proc/meminfo
total: used: free: shared: buffers: cached:
Mem: 262647808 256618496 6029312 0 4820992 67239936 Swap: 0 0 0 MemTotal: 256492 kB MemFree: 5888 kB MemShared: 0 kB Buffers: 4708 kB Cached: 65664 kB SwapCached: 0 kB Active: 77944 kB Inactive: 142308 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 256492 kB LowFree: 5888 kB SwapTotal: 0 kB SwapFree: 0 kB
Why was it so eager to kill applications, and not reclaim some of that swap space? Is this a problem that is known on 2.4, and can't be fixed (I can't use 2.6 on my laptop yet, far too many problems to even start - eg the suspend to ram on APM thread).
Is there another output of a /proc file you want? I'll try not to get the urge to use/reboot the box in the meantime.
Signature · 3 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Anyone seeking the "Relativistic Quantum Mechanics" soft option course, may wish to leave now. -- Intro lecture to RQM
Quoted 9 lines · Tim Connors
I just got an interesting problem - possibly (or not?) related to this:
I have my laptop with 256MB of RAM, running 2.4.25-pre7, uptime 48 days. Every morning, I come in and have to wait 2 minutes while everything comes back into RAM, after the daily slocate. So I did a swapoff -a, and it failed despite all the applications and cache and tmpfs adding up to far less than 256MB (more like 128).
Quoted 5 lines
I closed mozilla, which let me do a swapoff -a.
All was well for a few days, but then thismorning, my partitions were mounted ro, and an oops was in syslog at the same time as all the slocate work:
[alloc failures + oops snipped]
prolonged oom condition triggers lots of rarely user error paths in kernel (and applications). Most probably slocate hit one of bugs still living in one of them.
Quoted 19 lines
So OOM - but why? The cache was registering 65MB used. 24353,23> cat
/proc/meminfo
total: used: free: shared: buffers: cached:Mem: 262647808 256618496 6029312 0 4820992 67239936 Swap: 0 0 0 MemTotal: 256492 kB MemFree: 5888 kB MemShared: 0 kB Buffers: 4708 kB Cached: 65664 kB SwapCached: 0 kB Active: 77944 kB Inactive: 142308 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 256492 kB LowFree: 5888 kB SwapTotal: 0 kB SwapFree: 0 kB
Maybe this is not the state of the meminfo at the time of oom condition. oops killed the task and had freed its memory, which is now used by cache.
Quoted 7 lines
Why was it so eager to kill applications, and not reclaim some of that swap space? Is this a problem that is known on 2.4, and can't be fixed (I can't use 2.6 on my laptop yet, far too many problems to even start - eg the suspend to ram on APM thread).
Is there another output of a /proc file you want? I'll try not to get the urge to use/reboot the box in the meantime.
vmstat log of this event may be useful.
Signature · 1 line
vda
Quoted 7 lines
I'm not really sure what the above was intended to demonstrate, but I assume that it was that having swap allowed the first grep to fill physical RAM with
cache at the expense of swapping other processes, which were using physical RAM to disk.
Quoted 6 lines
However, if 57 Mb of swap allows this, 57 Mb of extra physical RAM should also allow the grep to be cached, without having to swap out anything.
Hence my comment about it not being a magical property of swap space.
John.
Exactly! Swap will allow for room to evict anonymous pages (heap, stack, shared memory, etc.) from memory to make room for other pages in memory. Those pages could be file backed pages and therefore qualify as "pagecache".
When you read or write to/from a file at a given offset, the file operations actually occur against memory.
Accessing an offset into the mapping of a file triggers a pagefault if a page for that offset doesn't currently exist in memory. The point is, file system I/O is actually achieved by accessing memory which triggers a major pagefault (disk access) if the page doesn't already exist in memory. If the page does already exist (minor fault) because the lookup of that device,inode,offset succeeded, the already memory resident page is used rather than incurring an I/O to disk.
This means if you grep thru the kernel tree, grep reads every line of every file in the kernel tree trying to do pattern matching, every file in the kernel tree is sitting in memory in it's entirety. Any subsequent reads of those files are quite snappy since they are already memory resident (only minor faults are incurred).
The above scenario where 57mb of swap allows for the entire kernel src tree to be memory resident may provide tremendous value on a workstation or a system with a small amount of disk I/O takes place, but it assumes that the anonymous pages aren't going to be faulted right back in as soon as the program that uses these pages becomes runnable again. It's not uncommon to have a system where certain pages continually are paged in and out to/from the swap device, simply because the system is very low on RAM, and filesystem I/O is filling up physical memory at a rate that exceeds the frequency that a process that is allocating anonymous memory becomes runnable.
The problem is that any system doing enough constant filesystem I/O, with enough data is eventually going to fill physical memory. This is not true for anonymous memory. When I choose the amount of memory for a system, it is first and foremost based on the resident set size of the application processes. If I care about caching filesystem I/O, then all I have to do is populate the system with enough "extra" physical RAM that files can be cached in physical memory.
If I know in advance that filesystem I/O will eventually fill physical memory with filesystem pages (pagecache), then why would I allow file system I/O to force out anonymous pages on the system? Also, why wake up an expensive algorithm (kswapd) that walks all pages in physical memory in order to determine which pages are "Least Recently Used" on a system where all resident set sizes of all processes add up to 100MB, and another 900MB of physical RAM is full of filesystem backed pages?
On a server, where lots of I/O is taking place and you are willing to size applications to fit completely within physical memory and add a little extra for pagecache, I very much prefer the way that Solaris was modified to work in Solaris 8 using a cyclical page cache.
The idea is "something" like this:
Filesystem backed pages are considered free memory. If you need to allocate more anonymous memory, you just grab from (evict) the tail of a linked list (called the cachelist) that represents the pagecache. If you update a page or create a new page in the cachelist, then you simply move that page to the head of the cachelist. This means there is always a small overhead in maintaining the list, but nothing compared to the two-handed clock algorithm that scans for pages to evict. The two-handed clock algorthim (the scanner) is kept, but only when freemem falls to lotsfree. In Solaris 8, freemem is the size of the cachelist + the size of free memory (pages that are not pagecache and are free).
This way, filesystem I/O CANNOT cause the scanner to wake up and start traversing main memory, eating up valuable CPU time. Also, anonymous pages will not be evicted on systems due to lots of filesystem I/O.
I won't try to imply that the Solaris 8 or later VM system outperforms the Linux VM because I haven't compared the two, but I can attest that the Solaris 8 VM beats the pants off the Solaris 7 VM system on systems where large amounts of filesystem I/O take place.
You don't get the supposed "benefit" of evicting anonymous memory to swap in order to cache filesystem pages, but quite frankly on a server, I would not want this bug ... err, ... I mean ... feature :) I would much rather size my system such that applications fit in physical memory and if I so desire, add a little extra for pagecache.
--Buddy
"Buddy Lumpkin" <[email protected]> said on Tue, 1 Jun 2004 02:38:42 -0700:
Quoted 5 lines
If I know in advance that filesystem I/O will eventually fill physical memory with filesystem pages (pagecache), then why would I allow file system I/O to force out anonymous pages on the system? Also, why wake up an expensive algorithm (kswapd) that walks all pages in physical memory in order to determine which pages are "Least Recently Used" on a system where
...
Incidentally, what happens when kswapd becomes a zombie? I've seen this a few times, and I am currently posting on a machine that has been up for 15 days, and which oopsed 10 or so days ago (something to do with nfs, but don't worry about that - the machine is running 2.4.20, and is not exactly up-to-date), killing kswapd.
But I don't notice anything at all different about how the system is behaving. However, I haven't been doing much more than running emacs and mozilla recently - I haven't been running my visualisation software that typically stresses the VM beyond usefullness.
Signature · 2 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Whip me. Beat me. Make me maintain AIX.
Quoted 9 lines · Tim Connors
Incidentally, what happens when kswapd becomes a zombie? I've seen this a few times, and I am currently posting on a machine that has been up for 15 days, and which oopsed 10 or so days ago (something to do with nfs, but don't worry about that - the machine is running 2.4.20, and is not exactly up-to-date), killing kswapd. But I don't notice anything at all different about how the system is behaving. However, I haven't been doing much more than running emacs and mozilla recently - I haven't been running my visualisation software that typically stresses the VM beyond usefullness.
Check your syslog for oopsen. That's the only known reason for kswapd to become a zombie.
-- wli
Quoted 4 lines · William Lee Irwin III
Quoted 3 lines · Tim Connors
Incidentally, what happens when kswapd becomes a zombie? I've seen this a few times, and I am currently posting on a machine that has been up for 15 days, and which oopsed 10 or so days ago (something to
^^^^^^^^^^^^^^^^^^^^^^^^
Quoted 9 lines
Quoted 6 lines
do with nfs, but don't worry about that - the machine is running 2.4.20, and is not exactly up-to-date), killing kswapd. But I don't notice anything at all different about how the system is behaving. However, I haven't been doing much more than running emacs and mozilla recently - I haven't been running my visualisation software that typically stresses the VM beyond usefullness.
Check your syslog for oopsen. That's the only known reason for kswapd to become a zombie.
What ill effects are meant to happen? I haven't noticed anything ill about the machine at all. No OOMs, no 'failed zero order allocation', etc. Swap is currently 508388k used, 543860k free, 175060k cached, mem is 498908k used, 15540k free.
Signature · 2 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/ Error: Fuzzy Pointer Exception