VM and swap - it's really unusable
Quoted 8 lines
Re: [2.4.17/18pre] VM and swap - it's really unusable
My observation:
Why does the kernel swap to get free memory for caching / buffering? I can't see any sense in this action. Wouldn't it be better to shrink the cashing / buffering-RAM to the amount of memory, which is obviously free?
Quoted 5 lines
Swapping should be principally used, if the RAM ends for real memory (memory, which is used for running applications). First of all, the memory-usage of cache and buffers should be reduced before starting to swap IMHO.
My main "problem" with 2.4 as well. Using free memory for Cache/Buffers
is great, but only as long as the memory is not needed for running tasks. As soon as a task is requesting more memory, it should be first taken from Cache/Buffer (they are caches, aren't they :-). Only if this has been used up (to a tunable minimum, see below), swapping and finally OOM killing should happen.
To prevent completely trashing IO performance, there should be tunable
parameters for minimum and maximum Cache/Buffer usage (lets say in percent of total memory). Maybe those tunables are even there today and I am just to stupid to find them :-))
In any case, 2.4 Caches/Buffers show to much persistance. This is
basically true for both branches of VM. I was using the -ac kernels because, being far from perfect, the VM gave considreabele better interactive behaviour.
Quoted 4 lines
Or would it be possible, to implement more than one swapping strategy, which could be configured during make menuconfig? This would give the user the chance to find the best swapping strategy for his purpose.
That is another option. But I would prefer something that could be
selected dynamically or at boot time.
Martin
Signature · 7 lines
+-----------------------------------------------------+ |Martin Knoblauch | |-----------------------------------------------------| |http://www.knobisoft.de/cats | |-----------------------------------------------------| |e-mail: [email protected] | +-----------------------------------------------------+
Quoted 12 lines · Martin Knoblauch
Quoted 12 lines
Re: [2.4.17/18pre] VM and swap - it's really unusable
My observation:
Why does the kernel swap to get free memory for caching / buffering? I can't see any sense in this action. Wouldn't it be better to shrink the cashing / buffering-RAM to the amount of memory, which is obviously free?Swapping should be principally used, if the RAM ends for real memory (memory, which is used for running applications). First of all, the memory-usage of cache and buffers should be reduced before starting to swap IMHO.
[...]
Quoted 4 lines
In any case, 2.4 Caches/Buffers show to much persistance. This is
basically true for both branches of VM. I was using the -ac kernels because, being far from perfect, the VM gave considreabele better interactive behaviour.
I did some tests with different VM-patches. I tested one ac-patch, too. I detected the same as you described - but the memory-consumption and the behaviour at all isn't better. If you want to, you can test another patch, which worked best in my test. It's nearly as good as kernel 2.2.x. Ask M.H.vanLeeuwen ([email protected]) for his oom-patch to kernel 2.4.17. But beware: maybe this strategy doesn't fit to your applications. And it's not for productive use.
I and some others surely too, would be interested in your experience with this patch.
Quoted 8 lines
Quoted 5 lines
Or would it be possible, to implement more than one swapping strategy, which could be configured during make menuconfig? This would give the user the chance to find the best swapping strategy for his purpose.
That is another option. But I would prefer something that could be
selected dynamically or at boot time.
Dynamic selection - this would be great. For all situations the best swapping strategy. On servers switched with a cronjob or a intelligent daemon.
Regards,
Andreas Hartmann
Quoted 11 lines
My main "problem" with 2.4 as well. Using free memory for Cache/Buffers is great, but only as long as the memory is not needed for running tasks. As soon as a task is requesting more memory, it should be first taken from Cache/Buffer (they are caches, aren't they :-). Only if this has been used up (to a tunable minimum, see below), swapping and finally OOM killing should happen.
To prevent completely trashing IO performance, there should be tunable parameters for minimum and maximum Cache/Buffer usage (lets say in percent of total memory). Maybe those tunables are even there today and I am just to stupid to find them :-))
Yes!! I second that motion!! On top of that, we need buffer/page cache hit rate statistics!! Once your read hit rate gets up into the high 90 percentages, more buffer/page cache memory is wasted.
If Linux is to succeed in enterprise-level usage, we *must* have tools to measure, manage and tune performance -- in short, to do capacity planning like we do on any other system. And the kernel variables that affect performance *must* be under control of the system administrator and ultimately the machine's *customers*, *not* a bunch of kernel geeks! That means keeping them in variables accessible by a system administrator, *not* #defines in code that must be entirely recompiled when you want to tweak a parameter.
If you build it, they will come :). If you *refuse* to build it, they will use something else -- it's as simple as that.
--
M. Edward Borasky
Quoted 6 lines · M. Edward Borasky
Yes!! I second that motion!! On top of that, we need buffer/page cache hit rate statistics!!
If Linux is to succeed in enterprise-level usage, we *must* have tools to measure, manage and tune performance -- in short, to do capacity planning like we do on any other system.
Indeed, VM statistics added back to the TODO list for my VM ;)
Rik
Signature · 3 lines
Shortwave goes a long way: irc.starchat.net #swl http://www.surriel.com/ http://distro.conectiva.com/
Quoted 6 lines · Rik van Riel
Quoted 2 lines · M. Edward Borasky
Yes!! I second that motion!! On top of that, we need buffer/page cache hit rate statistics!!
Yes, I forgot to mention them. And we need them not only to do the
production planning, but also for developing the VM[-strategies]. Most time these days we just say things like "it sucks", or "no, it does not" :-) Even worse if it comes to interactive "feeling".
Quoted 7 lines
Quoted 3 lines
If Linux is to succeed in enterprise-level usage, we *must* have tools to measure, manage and tune performance -- in short, to do capacity planning like we do on any other system.
Indeed, VM statistics added back to the TODO list for my VM ;)
Very good to hear.
Happy new year
Martin
Signature · 7 lines
+-----------------------------------------------------+ |Martin Knoblauch | |-----------------------------------------------------| |http://www.knobisoft.de/cats | |-----------------------------------------------------| |e-mail: [email protected] | +-----------------------------------------------------+