VM
Linus, this question is really to you...
Why is the simple vm system still in place on the linus tree? I would think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything. (The potential for less swapping is _always better_)
Signature · 1 line
Patrick "Diablo-D3" McFarland || [email protected]
In article <20011015211216.A1314@localhost>,
Quoted 2 lines · Patrick McFarland
Why is the simple vm system still in place on the linus tree? I would
think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything.
"complex" != "smart".
The benchmarks I've seen says that the simple VM performs better - both in terms of repeatability and in terms of absolute performance. Search this list yourself if you don't believe me.
Linus
Quoted 4 lines · In article <[email protected]> [email protected] wrote:
In article <20011015211216.A1314@localhost>,
Quoted 2 lines · Patrick McFarland
Why is the simple vm system still in place on the linus tree? I would
Quoted 8 lines
think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything.
"complex" != "smart".
The benchmarks I've seen says that the simple VM performs better - both in terms of repeatability and in terms of absolute performance. Search this list yourself if you don't believe me.
The problem may be one of perception. There has been a lot of effort
to fix large memory cases, and that's good. I have a load of 2-4GB machines in remote locations with heavy load. I would like dead stable and good performance, and that may be coming in either case.
But for busy little machines, uni, small memory, which represent the
majority of desktops, I find the -ac VM seems to be significantly faster in starting X, or netscape, or changing desktops. I haven't tried 2.4.12, because I want to be sure to avoid another 2.4.11oh-shit debacle. But at 2.4.10 the difference was significant, particularly on small machines with slow disk. I'll try 2.4.12 Thursday unless major bugs pop up and see how that does, 2.4.10-ac12 has been stable and responsive, and I'm out of the office tomorrow and can't reboot if I get an oops.
Signature · 4 lines
bill davidsen <[email protected]> "If I were a diplomat, in the best case I'd go hungry. In the worst case, people would die." -- Robert Lipe
from an interested bystander here's how I see it
much of the time the -ac VM is better, but sometimes it is MUCH worse. It's performance can vary substantually even while running the same test.
the linux VM may not be as fast in some cases, but it is far more predictable and repeatable, and doesn't have the same horrible 'worst case' performance that has been the bane of the 2.4 kernels.
say for the sake of argument that the linus VM is only 80% as good as the -ac VM. but the -ac VM has pathalogical conditions that hit is 5% of the time (both numbers out of thin air, I'm sure that aa would argue that it's better then the 80% and Rik would argue that it's less then 5% :-)
we're late in the 2.4 series, stability and predictability is better then raw performance. the 5% pathalogical problem is bad enough to make that VM unsuatable for many machines (and worse the conditions that trigger it aren't well understood making it untrustworthy for a much larger group of machines) while the slight performance hit on the 80% as good is much easier to deal with (buy a faster disk or more ram)
I have been impressed by the repeatability shown by the linus VM system and have just been waiting for the last of the gotchas to be hammered out before switching my production machines from 2.4.5 to a newer kernel.
David Lang
Quoted 17 lines · Patrick McFarland
Date: Mon, 15 Oct 2001 21:12:18 -0400
From: Patrick McFarland <[email protected]>
To: [email protected]
Subject: VMLinus, this question is really to you...
Why is the simple vm system still in place on the linus tree? I would think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything. (The potential for less swapping is
_alwaysbetter_)--
Patrick "Diablo-D3" McFarland || [email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
reading what lang wrote, ive been thinking
Im on the type of machine that swapping the least is most favorable. rik's vm seems that it would be able to swap less, and not swap the wrong things enough of the time. andrea's, if i try to do something major, it swaps like crazy, but I havent tested rik's because I dont trust the rest of the ac tree to mess around with it. Is there any chance of rik's vm being atleast an option to choose, and possibly see what the community wants? Maybe if rik's vm is cleaned up, that 5% of stupidity would go down to the less than 1% we all hope for.
Quoted 20 lines · Linus Torvalds
In article <20011015211216.A1314@localhost>,
Quoted 2 lines · Patrick McFarland
Why is the simple vm system still in place on the linus tree? I would
think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything.
"complex" != "smart".
The benchmarks I've seen says that the simple VM performs better - both in terms of repeatability and in terms of absolute performance. Search this list yourself if you don't believe me.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Patrick "Diablo-D3" McFarland || [email protected]
Quoted 8 lines · Patrick McFarland
Im on the type of machine that swapping the least is most favorable. rik's vm seems that it would be able to swap less, and not swap the wrong things enough of the time. andrea's, if i try to do something major, it swaps like crazy, but I havent tested rik's because I dont trust the rest of the ac tree to mess around with it. Is there any chance of rik's vm being atleast an option to choose, and possibly see what the community wants? Maybe if rik's vm is cleaned up, that 5% of stupidity would go down to the less than 1% we all hope for.
There really is nothing to fear in Alan's tree. Give it a tree. In fact, I view Alan as more stable (or at least, less experimental) at this stage in the game.
Robert Love
Quoted 3 lines · On Mon, 15 Oct 2001 23:08:38 -0400 Patrick McFarland <[email protected]> wrote:
reading what lang wrote, ive been thinking
Im on the type of machine that swapping the least is most favorable. rik's vm
seems that it would be able to swap less, and not swap the wrong things enough of the time.
Well, I cannot really comment on who does what based on the code. But I can see the results on my machine(s). And what I see is that the current linus-tree does not swap at all in my environment. Maybe one could say that 1 GB of RAM is a bit oversized for most of my business, but anyway the point I started complaining real loud about the former VM (now residing at -ac tree) was when it came to the point where even my system started to become unusable. I am therefore _very_ thankful to L. that he drew the line (who else could _and_ would have done it?), and Andrea of course for his work. I do not think that Riks work is a piece of bs, really not, but it seems to have an inherent complexity that is _very_ hard to handle. It may well be the proof for the "keep it simple"-strategy to deliver the best results.
Anyway we should not see it as a political issue, but a friendly competition. Because in the end, we all want the same: a system we can trust and rely on. There is nothing wrong with having different opinions as long as you can give _some_ proof for it. So if you say current -linus tree does more swapping, please give us some numbers to have a look at.
Regards,
Stephan
also considering more stuff,
someone said that this is more of a political issue, but seeing that I stay out of political issues, Im looking for the better code here. Thats all I care about.
Quoted 34 lines · Patrick McFarland
reading what lang wrote, ive been thinking
Im on the type of machine that swapping the least is most favorable. rik's vm seems that it would be able to swap less, and not swap the wrong things enough of the time. andrea's, if i try to do something major, it swaps like crazy, but I havent tested rik's because I dont trust the rest of the ac tree to mess around with it. Is there any chance of rik's vm being atleast an option to choose, and possibly see what the community wants? Maybe if rik's vm is cleaned up, that 5% of stupidity would go down to the less than 1% we all hope for.
Quoted 20 lines · Linus Torvalds
In article <20011015211216.A1314@localhost>,
Quoted 2 lines · Patrick McFarland
Why is the simple vm system still in place on the linus tree? I would
think the smart vm system in the ac tree would be better suited to .. oh.. say .. everything.
"complex" != "smart".
The benchmarks I've seen says that the simple VM performs better - both in terms of repeatability and in terms of absolute performance. Search this list yourself if you don't believe me.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/Signature · 6 lines
Patrick "Diablo-D3" McFarland || [email protected] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Patrick "Diablo-D3" McFarland || [email protected]
May I ask, why do you think so?
Quoted 17 lines · Robert Love
Quoted 8 lines · Patrick McFarland
Im on the type of machine that swapping the least is most favorable. rik's vm seems that it would be able to swap less, and not swap the wrong things enough of the time. andrea's, if i try to do something major, it swaps like crazy, but I havent tested rik's because I dont trust the rest of the ac tree to mess around with it. Is there any chance of rik's vm being atleast an option to choose, and possibly see what the community wants? Maybe if rik's vm is cleaned up, that 5% of stupidity would go down to the less than 1% we all hope for.
There really is nothing to fear in Alan's tree. Give it a tree. In fact, I view Alan as more stable (or at least, less experimental) at this stage in the game.
Robert Love
Signature · 1 line
Patrick "Diablo-D3" McFarland || [email protected]
I think it was said earlier that we're dealing with definitions of stability and performance.
If you look at stability as being able to depend on an effect given a cause,
then andrea's has been said to be more stable in that sense. If you look at
performance being the amount of different situations that the vm is stable and everything else being a lower priority, then andrea's vm is better performing.
Rik's vm is performance first, meaning it tries to do what's best for each situation and basically the difference is that rik's vm has more variables effecting what happens. This treats everything differently but it means that each situation is dealt with personally instead of trying to blanket each like situation. That basically destroys our previous definition of stability. So we make a new one. Stability here deals with how much the system needs to stop other things for VM things. Of course the not corrupting things and crashing things are implied to both definitions. For instance though, when you swap, that takes time away to write to disk. This can take longer than a complex way of re-arranging pages and removing pages in ram.
It seems like andrea's vm is more tuned for systems that do the same things over and over, like a server. And rik's vm is more tuned for systems that you dont know what is going to be run or is running numerous programs that have no real regularity.
And complex does not mean smart, but it doesn't mean it can't be smart. When you're dealing with something as complex as a VM, using a simplistic approach may just be too limiting in the end and I think many people are seeing that when they say programs are more responsive in alan's kernel and memory usage is more efficient. It doesn't seem very logical to make the VM do B when you do A on a multiprocessing system unless the environment is exactly the same every time you do A because what's good at one time doesn't mean it's good the second or third time you do it either due to memory limitations or other applications requiring different things of the VM.
I'm kind of picturing the two VM's like the two parts of our brain. The brain stem (sometimes called the reptillian brain) and the cerebrum. Your reptillian brain is quite fast at reacting and can make a few decisions on
what to do based on a few specific variables. The cerebrum is slower at
those same tasks but it better manages those tasks, based on many more variables, so that the reaction is not too much or too little so that the next thing that happens is in a better position than what the reptillian brain would have left for it.
Of course being able to do more means you have opened yourself up to more problems. I wont speak for all ac branch users, but i feel that the more complex way of handing memory is a better choice because it's a function of the kernel that demands a complex solution. A simplistic solution is too limited, it would be like reacting from your brain stem and overreacting instead of using your higher logic and taking a more educated reaction.
And that's all the contraversy, deciding if 2.4's VM demands a complex solution that handles each situation uniquely, or it can have a simple solution that handles a wide range fairly good.
Perhaps aiming for a simplistic VM should be the goal of 2.5 from the beginning ( as if it wasn't), that way you can build everything else around it and avoid all this vm trouble that 2.4 has been plagued with since the mid 2.3 days.
and the problem with trying to do the perfect thing in every situation is that you need to predict all the situations so that you have the right response for each one.
it gets even worse when you have a mix of loads (parts of several different basic situations).
this is why a simpler solution that may not be quite as good in several of the simple situations can end up being a winner in the real world (where you almost always have a mix of situations)
also if you have a lot of special cases you need to choose between you can easily end up spending more time selecting the proper mode then you save by being in that mode
one thing that the entire 2.[34] VM hassle has shown is that the VM hackers don't have a good handle on the real world loads along with a way to reasonably duplicate those loads in testing (if anyone had any idea what sort of VM problems 2.4 would have they would have been resolved in the 2.3 series, right?) This makes simple/general solutions better then attempts to define and select a bunch of special purpose solutions.
from the looks of things Rik's VM is getting to the point where it is almost covering enough special cases to be practical, but for the reasons I gave above I have less confidence in it then in the AA VM for the near future. Long term may be a different story however.
One thing that I think is needed for future 2.4/2.5 before much more VM development can be done is some significant instramentation of the existing VM to gather data on real-world loads along with some simulater to be able to recreate them. without this sort of data and tools the best that the VM hackers can do is to test it on the machines and loads they see and hope that they cover enough cases to make the special casing approach work, otherwise we keep running the risk of the same type of problems with the next implementation.
in part this is a chicken-and-egg problem, until the new VM gets extensive real-world testing it won't run into all the corner cases to be able to see if it works well but this also means that when released on the general user base it will break peoples servers. thus the need for better documentation and instramentation of the real world loads (becouse they are frequently NOT the 'best' way of doing things and in some cases they are downright bad ways)
David Lang
Quoted 70 lines · safemode
Date: Tue, 16 Oct 2001 01:08:02 -0400
From: safemode <[email protected]>
To: [email protected]
Subject: Re: VMI think it was said earlier that we're dealing with definitions of stability and performance.
If you look at stability as being able to depend on an effect given a cause,
then andrea's has been said to be more stable in that sense. If you look at
performance being the amount of different situations that the vm is stable and everything else being a lower priority, then andrea's vm is better performing.Rik's vm is performance first, meaning it tries to do what's best for each situation and basically the difference is that rik's vm has more variables effecting what happens. This treats everything differently but it means that each situation is dealt with personally instead of trying to blanket each like situation. That basically destroys our previous definition of stability. So we make a new one. Stability here deals with how much the system needs to stop other things for VM things. Of course the not corrupting things and crashing things are implied to both definitions. For instance though, when you swap, that takes time away to write to disk. This can take longer than a complex way of re-arranging pages and removing pages in ram.
It seems like andrea's vm is more tuned for systems that do the same things over and over, like a server. And rik's vm is more tuned for systems that you dont know what is going to be run or is running numerous programs that have no real regularity.
And complex does not mean smart, but it doesn't mean it can't be smart. When you're dealing with something as complex as a VM, using a simplistic approach may just be too limiting in the end and I think many people are seeing that when they say programs are more responsive in alan's kernel and memory usage is more efficient. It doesn't seem very logical to make the VM do B when you do A on a multiprocessing system unless the environment is exactly the same every time you do A because what's good at one time doesn't mean it's good the second or third time you do it either due to memory limitations or other applications requiring different things of the VM.
I'm kind of picturing the two VM's like the two parts of our brain. The brain stem (sometimes called the reptillian brain) and the cerebrum. Your reptillian brain is quite fast at reacting and can make a few decisions on
what to do based on a few specific variables. The cerebrum is slower at
those same tasks but it better manages those tasks, based on many more variables, so that the reaction is not too much or too little so that the next thing that happens is in a better position than what the reptillian brain would have left for it.Of course being able to do more means you have opened yourself up to more problems. I wont speak for all ac branch users, but i feel that the more complex way of handing memory is a better choice because it's a function of the kernel that demands a complex solution. A simplistic solution is too limited, it would be like reacting from your brain stem and overreacting instead of using your higher logic and taking a more educated reaction.
And that's all the contraversy, deciding if 2.4's VM demands a complex solution that handles each situation uniquely, or it can have a simple solution that handles a wide range fairly good.
Perhaps aiming for a simplistic VM should be the goal of 2.5 from the beginning ( as if it wasn't), that way you can build everything else around it and avoid all this vm trouble that 2.4 has been plagued with since the mid 2.3 days.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 3 lines · David Lang
and the problem with trying to do the perfect thing in every situation is that you need to predict all the situations so that you have the right response for each one.
predicting more situations ...not every situation. Our brains aren't hardcoded with the knowledge of how to deal with everything ...that would be
as impossible as predicting all situations dealing with VM. Instead it
looks at more variables and decides upon that what to do.
Quoted 2 lines
it gets even worse when you have a mix of loads (parts of several different basic situations).
This is where a complex solution can be better than a simple. A mix of loads means you'll have different Kinds of programs....not just different programs, asking the VM for various things. Because they're different kinds of programs they'll benefit from not being treated all under a vague reaction that would be given by a simplistic VM. A simplistic solution may be faster in doing what it does, but in the real world time goes on from that point and what the VM did before effects what is happening later. And overall a simplistic solution may actually hurt the performance of a VM.
Quoted 3 lines
this is why a simpler solution that may not be quite as good in several of the simple situations can end up being a winner in the real world (where you almost always have a mix of situations)
A complex solution (that works), will handle memory in a way that makes what is going to happen next, more efficient. We see this in Rik's vm. This makes overall VM performance much smoother and in most people's opinions, better.
Quoted 3 lines
also if you have a lot of special cases you need to choose between you can easily end up spending more time selecting the proper mode then you save by being in that mode
True, but taking a tiny bit of time to find the right mode may save a lot of time in the long run by having memory allocated in a more efficient way for
new accesses. We see this from the VM being reported "more smooth". The
jerkiness in a vm is caused by having to correct a past operation that probably put too much memory somewhere it wasn't needed and now it needs it so it has to take the time to deallocate it and then allocate it to the new one.
Quoted 6 lines
one thing that the entire 2.[34] VM hassle has shown is that the VM hackers don't have a good handle on the real world loads along with a way to reasonably duplicate those loads in testing (if anyone had any idea what sort of VM problems 2.4 would have they would have been resolved in the 2.3 series, right?) This makes simple/general solutions better then attempts to define and select a bunch of special purpose solutions.
The thing with benchmarks is that you'll always have people saying they're not valid and people saying they are. If you want to package a bunch of static binaries and write a script to time and give all sorts of stats while it runs them in some series or concurrently, go for it. But even if you do and they're all programs most people run daily, people will still say that it's not valid due to compiler optimizations. It's a lose-lose situation
with testing. The only way to do it is go with the majority's reaction to
it on their own systems, there is no fast simulation way that will appease people.
Quoted 4 lines
from the looks of things Rik's VM is getting to the point where it is almost covering enough special cases to be practical, but for the reasons I gave above I have less confidence in it then in the AA VM for the near future. Long term may be a different story however.
Practicality is subjective obviously. It depends on which definitions of
stability and performance you are concerned with. Both are valid in their areas of use but in the end what will determine which gets used is going to be just how much of a hit does Rik's vm do on servers and such by being complex and less predictable. Just because it's less predictable doesn't mean it will hinder a running server in the long run, perhaps it'll be better for a server that stays up for a long time. Only people running it will tell.
Quoted 9 lines
One thing that I think is needed for future 2.4/2.5 before much more VM development can be done is some significant instramentation of the existing VM to gather data on real-world loads along with some simulater to be able to recreate them. without this sort of data and tools the best that the VM hackers can do is to test it on the machines and loads they see and hope that they cover enough cases to make the special casing approach work, otherwise we keep running the risk of the same type of problems with the next implementation.
Quoted 7 lines
in part this is a chicken-and-egg problem, until the new VM gets extensive real-world testing it won't run into all the corner cases to be able to see if it works well but this also means that when released on the general user base it will break peoples servers. thus the need for better documentation and instramentation of the real world loads (becouse they are frequently NOT the 'best' way of doing things and in some cases they are downright bad ways)
People who use the latest 2.4.x kernel aren't running critical servers, rebooting back to their previous non-Rik vm kernel wont be much of an issue. It wont "break" anything, rather just be better or worse performance wise. If you can afford to reboot to upgrade to the latest 2.4.x, you can afford to reboot to move back to your backup kernel.
Makeing a standard way of testing "real world" things is bad. Real world tests are unlimited and thus can be very hard to recreate but with this way you can actually see the "special" cases that become more apparent when more users use the kernel much earlier. This would be the same as your statement above about releasing a bad kernel onto the public as a stable kernel. If you tune to a standard real world test that some people come up with, then you are no longer tuning for the real world since you lose that unpredictability that real users can enter into the equation.
Like i said before, tests are a lose lose situation, if you dont do them you release code unto the world that is well, untested. If you do them, then you're tuning for your test and not the real world and you have people saying that the test is invalid or biased. Well, so far not using a test is out of the question and Both VM's certainly get their round of testing, the contraversy with that is what tests are important in the real world. Figure that out and maybe you'll get somewhere with figuring out which VM is better for everyone. If you manage to convince everyone that those tests are important to the real world, you're probably writing the code and/or a god-like being.
Quoted 74 lines
David Lang
Quoted 71 lines · safemode
Date: Tue, 16 Oct 2001 01:08:02 -0400
From: safemode <[email protected]>
To: [email protected]
Subject: Re: VMI think it was said earlier that we're dealing with definitions of stability and performance.
If you look at stability as being able to depend on an effect given a
cause, then andrea's has been said to be more stable in that sense. If
you look at performance being the amount of different situations that the vm is stable and everything else being a lower priority, then andrea's vm is better performing.Rik's vm is performance first, meaning it tries to do what's best for each situation and basically the difference is that rik's vm has more variables effecting what happens. This treats everything differently but it means that each situation is dealt with personally instead of trying to blanket each like situation. That basically destroys our previous definition of stability. So we make a new one. Stability here deals with how much the system needs to stop other things for VM things. Of course the not corrupting things and crashing things are implied to both definitions. For instance though, when you swap, that takes time away to write to disk. This can take longer than a complex way of re-arranging pages and removing pages in ram.
It seems like andrea's vm is more tuned for systems that do the same things over and over, like a server. And rik's vm is more tuned for systems that you dont know what is going to be run or is running numerous programs that have no real regularity.
And complex does not mean smart, but it doesn't mean it can't be smart. When you're dealing with something as complex as a VM, using a simplistic approach may just be too limiting in the end and I think many people are seeing that when they say programs are more responsive in alan's kernel and memory usage is more efficient. It doesn't seem very logical to make the VM do B when you do A on a multiprocessing system unless the environment is exactly the same every time you do A because what's good at one time doesn't mean it's good the second or third time you do it either due to memory limitations or other applications requiring different things of the VM.
I'm kind of picturing the two VM's like the two parts of our brain. The brain stem (sometimes called the reptillian brain) and the cerebrum. Your reptillian brain is quite fast at reacting and can make a few
decisions on what to do based on a few specific variables. The cerebrum
is slower at those same tasks but it better manages those tasks, based on many more variables, so that the reaction is not too much or too little so that the next thing that happens is in a better position than what the reptillian brain would have left for it.Of course being able to do more means you have opened yourself up to more problems. I wont speak for all ac branch users, but i feel that the more complex way of handing memory is a better choice because it's a function of the kernel that demands a complex solution. A simplistic solution is too limited, it would be like reacting from your brain stem and overreacting instead of using your higher logic and taking a more educated reaction.
And that's all the contraversy, deciding if 2.4's VM demands a complex solution that handles each situation uniquely, or it can have a simple solution that handles a wide range fairly good.
Perhaps aiming for a simplistic VM should be the goal of 2.5 from the beginning ( as if it wasn't), that way you can build everything else around it and avoid all this vm trouble that 2.4 has been plagued with since the mid 2.3 days.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 9 lines · safemode
This is where a complex solution can be better than a simple. A mix of loads means you'll have different Kinds of programs....not just different programs, asking the VM for various things. Because they're different kinds of programs they'll benefit from not being treated all under a vague reaction that would be given by a simplistic VM. A simplistic solution may be faster in doing what it does, but in the real world time goes on from that point and what the VM did before effects what is happening later. And overall a simplistic solution may actually hurt the performance of a VM.
A simplistic solution is more predictable, and therefor easier to write programs for that run well. This is the same principle that makes modern processors fast. We only need to enable any kind of program, not any behavior, becouse that behavior might in fact be harmfull or inefficient.
Quoted 6 lines
People who use the latest 2.4.x kernel aren't running critical servers, rebooting back to their previous non-Rik vm kernel wont be much of an issue. It wont "break" anything, rather just be better or worse performance wise. If you can afford to reboot to upgrade to the latest 2.4.x, you can afford to reboot to move back to your backup kernel.
This sounds more like a pro-Andrea/Linus argument :)
`Allan
Quoted 2 lines · Allan Sandfeld
A simplistic solution is more predictable, and therefor easier to write programs for that run well.
Except for the gimp, I haven't seen any application which actually takes the VM subsystem into account when doing its things, so this balloon doesn't fly.
Quoted 3 lines
This is the same principle that makes modern processors fast. We only need to enable any kind of program, not any behavior, becouse that behavior might in fact be harmfull or inefficient.
When comparing Linux 2.0 and Linux 2.2 you'll see that with the simplistic solution in Linux 2.2 it is MUCH easier to trigger bad behaviour than with Linux 2.0.
You'll also see that it is easier to make a robust VM fast than to make a fast VM robust. The former is hard, the latter is practically impossible.
I readily agree that the initial implementation of the VM for Linux 2.4 had some bad things, but I'm not parting with the idea that a VM should be designed to deal with strange and heavy loads.
regards,
Rik
Signature · 3 lines
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/ (volunteers needed) http://www.surriel.com/ http://distro.conectiva.com/
Quoted 3 lines
Why is the simple vm system still in place on the linus tree? I would think the smart vm system in the ac tree would be better suited to .. oh.. say ..
everything. (The potential for less swapping is_alwaysbetter_)
I've not reached any final conclusions on the VM - there are things that Rik's VM shows up that look like the VM algorithm is right but it triggers other stuff, and there are a couple of hackish bits left in still.
Smart is often good - especially given how slow disk seeks are. But smart is not always best for any algorithm.
Alan
Quoted 4 lines · Linus Torvalds
oh.. say .. everything.
"complex" != "smart".
and almost always
"simple" == "better"
Anuradha
Signature · 4 lines
Debian GNU/Linux (kernel 2.4.12) Absolutum obsoletum. (If it works, it's out of date.) -- Stafford Beer
I used bot VM in many situations and with many different HWs. I came to the conclusion that actually none of the two VMs is suitable for every use.
aa VM deals better because of its design on my web servers, with a non eccessive amount of memory, and with mysql and oracle databases.
When I talk of AA vm i mean the 2.4.13preXaa1 versions.
Unfortunatelly I have also found a problem with
some situations when the VM is higly stressed, but Andrea was very kind to this report, and now I hope it has gone away (will test this afternoon).
aa VM was also good with dualAthlon servers used for montecarlo simulations, but also here, VM was not really stressed, and the system has just 1 GB of RAM.
Rik VM in its later version is dealing better with Ultrasparc64 quadriprocessor with 4 GB RAM. But in this case we are talking of very very stressed system, with hundreds of huge processes, doing a lot of swap in/out, and with 8 GB swap space.
I am just sorry that i have access to this machine just from times to times, when a critical problem appears, but this is a production server.
The reason is the aa VM is more predictable, but rik's one actually seems to be smarter under very very stressed situations.
I do not care which VM is simpler, nor which is faster. I loock for predictability, since this is the most important thing on the servers I am administering. Under a special situation I need something maybe less predictable, but smarter to manage a stressed system.
80%... 5%... I do not care for exact numbers actually, I will care in future, if the situation comes to the point that both VMs will be quite good for everything. anyway it is a good strategy to follow two different way, since they are progressing quite welll together, with competition, and also (I hope) reciprocal help (just to be able to read the code of the other is a good help:) ).
Just now I am sorry I have to deal with this choice for every mission critical server I have. I would like a single VM that is good for everything, but I understand that this is the most difficoult thing to reach, because the casistinc is going to be more and more complex, with technology evolution, and
with time it will be even worse.
Luigi
Quoted 25 lines · Anuradha Ratnaweera
Quoted 4 lines · Linus Torvalds
oh.. say .. everything.
"complex" != "smart".
and almost always
"simple" == "better"Anuradha
--
Debian GNU/Linux (kernel 2.4.12)
Absolutum obsoletum. (If it works, it's out of date.)
-- Stafford Beer-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
In article <[email protected]>
Quoted 12 lines · [email protected] wrote:
I used bot VM in many situations and with many different HWs. I came to the conclusion that actually none of the two VMs is suitable for every use.
aa VM deals better because of its design on my web servers, with a non eccessive amount of memory, and with mysql and oracle databases.I do not care which VM is simpler, nor which is faster. I loock for predictability, since this is the most important thing on the servers I am administering. Under a special situation I need something maybe less predictable, but smarter to manage a stressed system.
Quoted 6 lines
80%... 5%... I do not care for exact numbers actually, I will care in future, if the situation comes to the point that both VMs will be quite good for everything. anyway it is a good strategy to follow two different way, since they are progressing quite welll together, with competition, and also (I hope) reciprocal help (just to be able to read the code of the other is a good help:) ).
Very well said. And I might add that some input from people with small desktop machines might be useful to the developers, since I doubt they are running small slow machines. While I wouldn't compromise big memory performance (much) for small, one beauty of Linux is that it will run well on small machines.
Of course it may be that some other VM will prove to be bnetter than either, but hopefully not until 2.5. I'd still like to see VM in a module and then everyone could play with their pet theory;-)
Signature · 4 lines
bill davidsen <[email protected]> "If I were a diplomat, in the best case I'd go hungry. In the worst case, people would die." -- Robert Lipe
Quoted 2 lines · Luigi Genoni
The reason is the aa VM is more predictable, but rik's one actually seems to be smarter under very very stressed situations.
This is a different approach to the situation. Most of the time in the early 2.4 kernels we were much too busy to stop machines from crashing to care about performance.
Only in more recent -ac kernels have I actually had time to look at performance and it seems to be relatively easy to get the VM to perform better.
Andrea seems to have optimised his VM for performance under low to medium loads from the beginning ... but in Linux 2.2 we've seen how impossible it is to tune such a simplistic VM to not fall apart under very high loads, so I won't be going that way ;)
regards,
Rik
Signature · 3 lines
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/ (volunteers needed) http://www.surriel.com/ http://distro.conectiva.com/
Quoted 24 lines · Luigi Genoni
I used bot VM in many situations and with many different HWs. I came to the conclusion that actually none of the two VMs is suitable for every use.
aa VM deals better because of its design on my web servers, with a non eccessive amount of memory, and with mysql and oracle databases.When I talk of AA vm i mean the 2.4.13preXaa1 versions.
Unfortunatelly I have also found a problem with
some situations when the VM is higly stressed, but Andrea was very kind to this report, and now I hope it has gone away (will test this afternoon).aa VM was also good with dualAthlon servers used for montecarlo simulations, but also here, VM was not really stressed, and the system has just 1 GB of RAM.
Rik VM in its later version is dealing better with Ultrasparc64 quadriprocessor with 4 GB RAM. But in this case we are talking of very very stressed system, with hundreds of huge processes, doing a lot of swap in/out, and with 8 GB swap space.
I am just sorry that i have access to this machine just from times to times, when a critical problem appears, but this is a production server.
Quoted 7 lines
The reason is the aa VM is more predictable, but rik's one actually seems to be smarter under very very stressed situations.
I do not care which VM is simpler, nor which is faster. I loock for predictability, since this is the most important thing on the servers I am administering. Under a special situation I need something maybe less predictable, but smarter to manage a stressed system.
OK so let me bite now... If you look for predictability, then see:
Rik's page aging mechanism based VM went in short before 2.4.0 time. Several months ago... Conceptually it was supposed to be a clone of FreeBSD's memmory management system. However Rik overdesigned it entierly. He inventen useless and far too many "tuning knobs". And everybody should known that optimizations get really hard with more then very few parameters. (Becouse even the math behind it get's trully
complicated ;-) The situation until recently was
entierly inacceptable becouse the inadequacies of the VM where blatant. No ORACLE running, no X session on lower memmory and so on and so
on. Maybe by now he reached some level of stability, (by trail and error if
you ask me), but please see that it only took 2 kernel release cycles until Andreas efforts showed fruits at least comparable (and in my oppinnion much suprerior) to what can be currently seen in the ac series in regard
of memmory management. This still holds even if you take into account that Linus and AA both went a bit wild with the IO system redesign.
Don't missunderstand me please I appreciate those changes as well becouse
I see that they are finally addressing block device handling problems I'm complaining about since 2 years regularly here and which are artefact from the youngest days of Linux.
So if you look at this history and see what's going on the conculsion is easly found: The chances are indeed very high that the behaviour of the Linus tree is much more predictable ;-). (In the VM context of course.)
Quoted 1 line · David Lang
before switching my production machines from 2.4.5 to a newer kernel.
running such an old kernel does not give a fair comparison. Personally I've found the /current/ ac VM to be stable and give slightly smoother feel than the linus tree VM.
regards
john
Signature · 2 lines
"I hear you have four hundred and eighty six PCs for sale ?" - Some Fool
Quoted 7 lines · John Levon
Quoted 1 line · David Lang
before switching my production machines from 2.4.5 to a newer kernel.
running such an old kernel does not give a fair comparison. Personally I've found the /current/ ac VM to be stable and give slightly smoother feel than the linus tree VM.
Are you comparing with post 2.4.10 Linus kernels?
/Tobias
I'm not doing a comparison of the VM in 2.4.5 and the newer kernels, I'm saying that based on the reports I have been seeing here I have not been willing to risk moving to a newer kernel becouse I couldn't trust the VM not to give me problems on my production boxes. the 2.4.5 has it's problems, but they are survivable (except on one box which I had to remove)
it's not a direct comparison of the kernels, it's a matter of being able to trust that the new kernel won't hang me out to dry under load. (and I am one of the many people out here who doesn't have the ability to simulate the load before going into production)
David Lang
Quoted 25 lines · John Levon
Date: Tue, 16 Oct 2001 13:28:51 +0100
From: John Levon <[email protected]>
To: [email protected]
Subject: Re: VMQuoted 1 line · David Lang
before switching my production machines from 2.4.5 to a newer kernel.
running such an old kernel does not give a fair comparison. Personally I've found the /current/ ac VM to be stable and give slightly smoother feel than the linus tree VM.
regards
john--
"I hear you have four hundred and eighty six PCs for sale ?"
- Some Fool
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 6 lines · Patrick McFarland
reading what lang wrote, ive been thinking
Im on the type of machine that swapping the least is most favorable. rik's vm seems that it would be able to swap less, and not swap the wrong things enough of the time. andrea's, if i try to do something major, it swaps like crazy, but I havent tested rik's because I dont
strance if something it should swap less. It may look less responsive under swap but that's mostly because we swap less and we drop more cache instead.
Infact if you test 2.4.13pre3aa1 it should swap more and also be more responsive under swap.
Andrea
Quoted 11 lines · Stephan von Krawczynski
Quoted 3 lines · On Mon, 15 Oct 2001 23:08:38 -0400 Patrick McFarland <[email protected]> wrote:
reading what lang wrote, ive been thinking
Im on the type of machine that swapping the least is most favorable. rik's vm
seems that it would be able to swap less, and not swap the wrong things enough of the time.
Well, I cannot really comment on who does what based on the code. But I can see the results on my machine(s). And what I see is that the current linus-tree does not swap at all in my environment. Maybe one could say that 1 GB of RAM is
I was also surprised that mainline was swapping more, it shouldn't really be the case. Infact in 2.4.13pre3aa1 I'm using shrink_cache to probe when it's time to start paging, instead of waiting shrink_cache to fail, this to avoid being too aggressive against the cache. So with those recent changes it should start swapping earlier and it should swap more. But if it's now swapping too much it will be very easy to turn it down the swap rate as worse with a few liner that removes such cache-probe early-swap logic.
Andrea
Quoted 8 lines · Andrea Arcangeli
I was also surprised that mainline was swapping more, it shouldn't really be the case. Infact in 2.4.13pre3aa1 I'm using
shrink_cacheto probe when it's time to start paging, instead of waitingshrink_cacheto fail, this to avoid being too aggressive against the cache. So with those recent changes it should start swapping earlier and it should swap more. But if it's now swapping too much it will be very easy to turn it down the swap rate as worse with a few liner that removes such cache-probe early-swap logic.
This is exactly why I am so religious about page aging on objects which are being used. With just the referenced bit you'll end up almost needing code tweaks to make the system behave well under various different loads.
The VM just doesn't have the information it needs to determine what to do...
regards,
Rik
Signature · 3 lines
DMCA, SSSCA, W3C? Who cares? http://thefreeworld.net/ (volunteers needed) http://www.surriel.com/ http://distro.conectiva.com/
Quoted 2 lines · Rik van Riel
The VM just doesn't have the information it needs to determine what to do...
additional page aging cannot make it different as far I can tell. The twekaing I'm speaking about is a number. After probing the cache and after getting many faliures I need to choose when it's time to start the pagetable scanning. Additional bit of aging can only influence the number of faliures, I cannot see how can it help to know when to start the pagetable scanning. It's a _ratio_ between the faliures and the size of the scan that tells me when it's the time. You need the same logic too somewhere in -ac vm. Now if I turn the ratio very high the cache will shrink more before we start pagetable scanning. If I make it low we'll swapout very easily. This ratio doesn't need to be perfect, it will never trigger anyways most of the time, but it must be a sane number, and it can make some difference during swapout.
Andrea