Two naive questions and a suggestion
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
2) Are there plans for implementing the swapping of whole processes a la BSD?
Suggestion: Given that the requiremnts for a workstation (quick response) are different than for a server (high throughput) it could make sense to allow the user either use /proc for selecting the VM policy or have a form of loadable VM manager. Or select it at compile time.
Signature · 5 lines
Jean Francois Martinez -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 2 lines · On 19 Nov 1998 [email protected] wrote:
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
Not yet, I really should be working on that (the code seems to have stabilized now)...
Quoted 2 lines
2) Are there plans for implementing the swapping of whole processes a la BSD?
Yes, there are plans. The plans are quite detailed too, but I think I haven't put them up on my home page yet.
Quoted 5 lines
Suggestion: Given that the requiremnts for a workstation (quick response) are different than for a server (high throughput) it could make sense to allow the user either use /proc for selecting the VM policy or have a form of loadable VM manager. Or select it at compile time.
There are quite a lot of things you can tune in /proc, I don't know if you have read the documentation, but if you start trying things you'll be amazed hom much you can change the system's behaviour with the existing controls.
Btw, since you are so enthusiastic about documentation, would you be willing to help me write it?
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 6 lines · On 19 Nov 1998 [email protected] wrote:
Quoted 3 lines
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
Quoted 9 lines
Not yet, I really should be working on that (the code seems to have stabilized now)...
Quoted 2 lines
2) Are there plans for implementing the swapping of whole processes a la BSD?
Yes, there are plans. The plans are quite detailed too, but I think I haven't put them up on my home page yet.
This will close the gap between Linux and *BSDs at high loads. It will also close the mouthes of some BSD people who talk loudly about what areas where BSD is superior and carefully forget SMP, Elf or modules to name just a few areas where Linux got it first.
Quoted 6 lines
Quoted 5 lines
Suggestion: Given that the requiremnts for a workstation (quick response) are different than for a server (high throughput) it could make sense to allow the user either use /proc for selecting the VM policy or have a form of loadable VM manager. Or select it at compile time.
Quoted 6 lines
There are quite a lot of things you can tune in /proc, I don't know if you have read the documentation, but if you start trying things you'll be amazed hom much you can change the system's behaviour with the existing controls.
I have read a bit about them but sometimes changing the algorythm is the right thing to do.
Quoted 3 lines
Btw, since you are so enthusiastic about documentation, would you be willing to help me write it?
I could try to help you but it will be limited help. I already work on a project of "Linux for normal people" and I also wanted to write an article about optimizing a Linux box. The goal is to smash the myth about kernel compiling. Why? Because in 95 my brother in law needed a computer for his thesis in Spanish litterature. I remebered kernel compiling and I led him to Apple Expo. That day one thing was
clear: Linux will never reach world domination as long as litterature professors cannot use it and as long as kernel compiling be necessary or even recommended then Linux will be off limits for litterature professors.
So I scanned the source code in 2.0.34 and found the unsignificant differences between code compiled for Pentiums and 386. Then I compiled the Byte benchmark using the same compile flags used for the 386 kernels and the ones for Pentiums and PPros. Difference in speed was < 2 % both on a real Pentium and on a K6. So much for "it will allow you to tune to the processor".
About memory savings. First of all in 98 distributors shipping crippled kernels should be shot: modular 2.0 has been around for over two years. Also modularity has reduced the memory savings you get from recompiling the kernel (if the distributor did a good job) qwhile machines got bigger: over 1.5 Megs saved on an 8 Meg box are significant (1.2.13 in 95), 500K on a 32 Meg box are a triffle (2.0 in 98). This is not entirely true: you can write pathological programs where a single page means the difference between blinding speed and hours swapping. Also the significant number is the increase in memory you lack: a 500K deficit becoming 1 Meg. Consider also disk
bandwidth: being 16 megs short on a 32 Megs is much worse than 4 Meg box on a 8 Meg because you need much more time to push 16 Megs to the disk. On the other hand proceses will have to spend more time analyzing a big array on a big box than a small one in a small box (processor speed being equal) and this plays in the side of the 32 Megs being 16 Megs short for normal, non pathological programs. Finally there is the question of probability: 500K is under 2% on a 32 Meg box so there is a good chance that when programs need more memory than what you have they miss the mark by 20 or 30% and rarely fall just straight on the 500K zone.
Needs refining but indulge with the fact I am writing at 2am.
A (not to be published) conclusion is: "Kernel compiling is a thing performed only by idiots and kernel hackers". I am not a kernel hacker and I have performed over 2 hundred of them. :-)
Perhaps we could help one another for our docs/articles.
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]'s
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"jfm2" == jfm2 <[email protected]> writes:
jfm2> A (not to be published) conclusion is: "Kernel compiling is a thing jfm2> performed only by idiots and kernel hackers". I am not a kernel jfm2> hacker and I have performed over 2 hundred of them. :-)
No.
As far as functionality I don't trust a linux box that doesn't have it's standard hardware drivers, comm port, floppy disk etc compiled in. A modular kernel seems to work well for protocol layers however.
An important advantage of linux is what you can do if something isn't working automatically.
With Windows you have 2 possibilities.
1) Either something works automatically
2) Soemthing doesn't work.
With Linux you have 3 posibilities.
1) Something works automatically. (We need more in this category).
2) Something with research and looking around can be made to work. (The ability to compile a kernel is an advantage here)
3) Something doesn't work. (Linux has much less in this category than any other OS)
With the memory management system. There are tuning paramenters. But generally resorting to them is dropping down to case 2. And in most cases with 2.0 and probably also with 2.2 the memory management system should be a case of it works automatically.
Eric
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"jfm2" == jfm2 <[email protected]> writes:
jfm2> Say the Web or database server can be deemed important enough for it jfm2> not being killed just because some dim witt is playing with the GIMP jfm2> at the console and the GIMP has allocated 80 Megs.
jfm2> More reallistically, it can happen that the X server is killed jfm2> (-9) due to the misbeahviour of a user program and you get jfm2> trapped with a useless console. Very diificult to recover. Specially jfm2> if you consider inetd could have been killed too, so no telnetting.
jfm2> You can also find half of your daemons, are gone. That is no mail, no jfm2> printing, no nothing.
initd is never killed. Won't & can't be killed.
initd should be configured to restart all of your important daemons if they go down.
Currently most unix systems ( I don't think i'ts linux specific) are misconfigured so they don't automatically restart their important daemons if they go down.
jfm2> In situation like those above I would like Linux supported a concept jfm2> like guaranteed processses: if VM is exhausted by one of them then try jfm2> to get memory by killing non guaranteed processes and only kill the jfm2> original one if all reamining survivors are guaranteed ones. jfm2> It would be better for mission critical tasks.
Some. But it would be simple and much healthier for tasks that can be down for a little bit to have initd restart the processes after they
go down. That allows for other cases when the important system
daemons goes down, is more robust, and doesn't require kernel changes.
Eric
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 12 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
"jfm2" == jfm2 <[email protected]> writes:
jfm2> Say the Web or database server can be deemed important enough for it jfm2> not being killed just because some dim witt is playing with the GIMP jfm2> at the console and the GIMP has allocated 80 Megs.
jfm2> More reallistically, it can happen that the X server is killed jfm2> (-9) due to the misbeahviour of a user program and you get jfm2> trapped with a useless console. Very diificult to recover. Specially jfm2> if you consider inetd could have been killed too, so no telnetting.
Quoted 7 lines
jfm2> You can also find half of your daemons, are gone. That is no mail, no jfm2> printing, no nothing.
initd is never killed. Won't & can't be killed.
initd should be configured to restart all of your important daemons if they go down.
This does not solve the problem. To begin with after an unclean shutdown a database server spends time rolling back uncommitted transactions and possibly writing somye comitted ones to the database from its journals. Users could prefer a database who doesn't go down in the first place.
Second: the 80 Megs GIMP is still there so when init restarts the database, the databse tries to allocate memory and it crashes again.
Third: A process can crash because it is misconfigured or a file is corrupted. And crash again if you restart it. It si not Init's job to do things like try five times and use a pager interface to send a message to the admin in case there is a sixth crash.
It could be considered that "guaranteed" processes is not a good idea but using Init is not the way to address the problem.
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 2 lines · On 19 Nov 1998 00:20:37 -0000, [email protected] said:
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
The source code. :)
Quoted 2 lines
2) Are there plans for implementing the swapping of whole processes a la BSD?
Not exactly, but there are substantial plans for other related changes. In particular, most of the benefits of BSD-style swapping can be achieved through swapping of page tables, dynamic RSS limits and streaming swapout, all of which are on the slate for 2.3.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 11 lines
Hi,
Quoted 2 lines · On 19 Nov 1998 00:20:37 -0000, [email protected] said:
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
The source code. :)
I knew about it. :) And this is not an URL :)
Quoted 8 lines
Quoted 2 lines
2) Are there plans for implementing the swapping of whole processes a la BSD?
Not exactly, but there are substantial plans for other related changes. In particular, most of the benefits of BSD-style swapping can be achieved through swapping of page tables, dynamic RSS limits and streaming swapout, all of which are on the slate for 2.3.
The problem is: will you be able to manage the following situation?
Two processes running in an 8 Meg box. Both will page fault every ms if you give them 4 Megs (they are scanning large arrays so no locality), a page fault will take 20 ms to handle. That means only 5% of the CPU time is used, remainder is spent waiting for page being brought from disk or pushing a page of the other process out of memory. And both of these processes would run like hell (no page fault) given 6 Megs of memory.
Only solution I see is stop one of them (short of adding memory :) and let the other one make some progress. That is swapping. Of course swapping can be undesiarable in work stations and that is the reason I suggested user control about MM policy be it by recompiling, by /proc or by module insertion.
In 96 I asked for that same feature, gave the same example (same numbers :-) and Alan Cox agreed but told Linux was not used under heavy loads. That means we are in a catch 22 situation: Linux not used for heavy loads because it does not handle them well and the necessary feaatures not implemented because it is not used in such situations.
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM. Problem is: it was a normal user process who had allocatedc most of it and in addition that daemon could be important enough it is better to kill anything else, so it would be useful to give some privilege to root processes here.
I think this ends my Christmas wish list. :)
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 46 lines · On 23 Nov 1998 [email protected] wrote:
Quoted 11 lines
Hi,
Quoted 2 lines · On 19 Nov 1998 00:20:37 -0000, [email protected] said:
1) Is there any text describing memory management in 2.1? (Forgive me if I missed an obvious URL)
The source code. :)
I knew about it. :) And this is not an URL :)
Quoted 8 lines
Quoted 2 lines
2) Are there plans for implementing the swapping of whole processes a la BSD?
Not exactly, but there are substantial plans for other related changes. In particular, most of the benefits of BSD-style swapping can be achieved through swapping of page tables, dynamic RSS limits and streaming swapout, all of which are on the slate for 2.3.
The problem is: will you be able to manage the following situation?
Two processes running in an 8 Meg box. Both will page fault every ms if you give them 4 Megs (they are scanning large arrays so no locality), a page fault will take 20 ms to handle. That means only 5% of the CPU time is used, remainder is spent waiting for page being brought from disk or pushing a page of the other process out of memory. And both of these processes would run like hell (no page fault) given 6 Megs of memory.
Only solution I see is stop one of them (short of adding memory :) and let the other one make some progress. That is swapping. Of course swapping can be undesiarable in work stations and that is the reason I suggested user control about MM policy be it by recompiling, by /proc or by module insertion.
In 96 I asked for that same feature, gave the same example (same numbers :-) and Alan Cox agreed but told Linux was not used under heavy loads. That means we are in a catch 22 situation: Linux not used for heavy loads because it does not handle them well and the necessary feaatures not implemented because it is not used in such situations.
Quoted 7 lines
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM. Problem is: it was a normal user process who had allocatedc most of it and in addition that daemon could be important enough it is better to kill anything else, so it would be useful to give some privilege to root processes here.
I think this ends my Christmas wish list. :)
what about this solution: write a small program the monitors for programs (via /proc) that swap a lot. If this happens make them get "wide slice" time - i.e. send SIGSTOP to everybody and SIGRESUME/SIGSTOP with interval 0.5-1 sec. This would have the concurency you want and will also eliminate the swapping (mostly). Since you need this heavy load anyway I don't think your programs will complain about long delays - for all they know this time is taken up swapping.
And this being a userspace program will allow you to fine tune this as much as you want, all the way to OOM/OOT killer that will pop-up a nice box on the terminal and ask what you want to do.
Vladimir Dergachev
Quoted 11 lines
Signature · 9 lines
-- Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 9 lines · On 23 Nov 1998 21:59:33 -0000, [email protected] said:
The problem is: will you be able to manage the following situation?
Two processes running in an 8 Meg box. Both will page fault every ms if you give them 4 Megs (they are scanning large arrays so no locality), a page fault will take 20 ms to handle. That means only 5% of the CPU time is used, remainder is spent waiting for page being brought from disk or pushing a page of the other process out of memory. And both of these processes would run like hell (no page fault) given 6 Megs of memory.
These days, most people agree that in this situation your box is simply misconfigured for the load. :) Seriously, requirements have changed enormously since swapping was first implemented.
Quoted 2 lines
Only solution I see is stop one of them (short of adding memory :) and let the other one make some progress. That is swapping.
No it is not. That is scheduling. Swapping is a very precise term used to define a mechanism by which we suspend a process and stream all of its internal state to disk, including page tables and so on. There's no reason why we can't do a temporary schedule trick to deal with this in
Linux: it's still not true swapping.
Quoted 5 lines
In 96 I asked for that same feature, gave the same example (same numbers :-) and Alan Cox agreed but told Linux was not used under heavy loads. That means we are in a catch 22 situation: Linux not used for heavy loads because it does not handle them well and the necessary feaatures not implemented because it is not used in such situations.
Linux is used under very heavy load, actually.
Quoted 2 lines
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM.
Same on any BSD. Once virtual memory is full, any new memory allocations must fail. It doesn't matter whether that allocation comes from a user process or a daemon: if there is no more virtual memory then the process will get a NULL back from malloc. If a daemon dies as a result of that, the death will happen on any Unix system.
Quoted 4 lines
Problem is: it was a normal user process who had allocatedc most of it and in addition that daemon could be important enough it is better to kill anything else, so it would be useful to give some privilege to root processes here.
No. It's not an issue of the operating system killing processes. It is an issue of the O/S failing a request for new memory, and a process exit()ing as a result of that failed malloc. The process is voluntarily exiting, as far as the kernel is concerned.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 15 lines
Quoted 10 lines
The problem is: will you be able to manage the following situation?
Two processes running in an 8 Meg box. Both will page fault every ms if you give them 4 Megs (they are scanning large arrays so no locality), a page fault will take 20 ms to handle. That means only 5% of the CPU time is used, remainder is spent waiting for page being brought from disk or pushing a page of the other process out of memory. And both of these processes would run like hell (no page fault) given 6 Megs of memory.
These days, most people agree that in this situation your box is simply misconfigured for the load. :) Seriously, requirements have changed enormously since swapping was first implemented.
Quoted 9 lines
Quoted 2 lines
Only solution I see is stop one of them (short of adding memory :) and let the other one make some progress. That is swapping.
No it is not. That is scheduling. Swapping is a very precise term used to define a mechanism by which we suspend a process and stream all of its internal state to disk, including page tables and so on. There's no reason why we can't do a temporary schedule trick to deal with this in
Linux: it's still not true swapping.
Agreed, the important feature is the stopping one of the processes when critically short of memory. Swapping is only a trick for getting more bandwidth at the expenses of pushing in an out of memory a greater amount of process space so there is no proof it is faster than letting other processes steal memory page by page from the now stopped process.
Quoted 8 lines
Quoted 5 lines
In 96 I asked for that same feature, gave the same example (same numbers :-) and Alan Cox agreed but told Linux was not used under heavy loads. That means we are in a catch 22 situation: Linux not used for heavy loads because it does not handle them well and the necessary feaatures not implemented because it is not used in such situations.
Linux is used under very heavy load, actually.
BSD and Solaris partisans are still boasting about how much better those systems are at heavy loads. I agree boasting tends to survive to the situation who originated it.
Quoted 9 lines
Quoted 2 lines
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM.
Same on any BSD. Once virtual memory is full, any new memory allocations must fail. It doesn't matter whether that allocation comes from a user process or a daemon: if there is no more virtual memory then the process will get a NULL back from malloc. If a daemon dies as a result of that, the death will happen on any Unix system.
Say the Web or database server can be deemed important enough for it not being killed just because some dim witt is playing with the GIMP at the console and the GIMP has allocated 80 Megs.
More reallistically, it can happen that the X server is killed (-9) due to the misbeahviour of a user program and you get trapped with a useless console. Very diificult to recover. Specially if you consider inetd could have been killed too, so no telnetting.
You can also find half of your daemons, are gone. That is no mail, no printing, no nothing.
Quoted 10 lines
Quoted 4 lines
Problem is: it was a normal user process who had allocatedc most of it and in addition that daemon could be important enough it is better to kill anything else, so it would be useful to give some privilege to root processes here.
No. It's not an issue of the operating system killing processes. It is an issue of the O/S failing a request for new memory, and a process exit()ing as a result of that failed malloc. The process is voluntarily exiting, as far as the kernel is concerned.
In situation like those above I would like Linux supported a concept like guaranteed processses: if VM is exhausted by one of them then try to get memory by killing non guaranteed processes and only kill the original one if all reamining survivors are guaranteed ones. It would be better for mission critical tasks.
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 6 lines · On 24 Nov 1998 [email protected] wrote:
Agreed, the important feature is the stopping one of the processes when critically short of memory. Swapping is only a trick for getting more bandwidth at the expenses of pushing in an out of memory a greater amount of process space so there is no proof it is faster than letting other processes steal memory page by page from the now stopped process.
When the mythical swapin readahead will be merged, we can gain some ungodly amount of speed almost for free. I don't know if we'll ever implement the scheduling tricks...
I do have a few ideas for the scheduling stuff though, with RSS limits (we can safely implement those when the swap cache trick is implemented) and the keeping of a few statistics, we will be able to implement the swapping tricks.
Without swapin readahead, we'll be unable to implement them properly however :(
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM.
Same on any BSD.
Say the Web or database server can be deemed important enough for it not being killed just because some dim witt is playing with the GIMP at the console and the GIMP has allocated 80 Megs.
I sounds remarkably like you want my Out Of Memory killer patch. This patch tries to remove the randomness in killing a process when you're OOM by carefully selecting a process based on a lot of different factors (size, age, CPU used, suid, root, IOPL, etc).
It needs to be cleaned up, ported to 2.1.129 and improved a little bit though... After that it should be ready for inclusion in the kernel.
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 3 lines · Rik van Riel
When the mythical swapin readahead will be merged, we can gain some ungodly amount of speed almost for free. I don't know if we'll ever implement the scheduling tricks...
Agreed: usage patterns these days are very different. We simply don't expect to run parallel massive processes whose combined working sets exceed physical memory these days. Making sure that we don't thrash to death is still an important point, but we can achieve that by guaranteeing processes a minimum rss quota (so that only those processes exceeding that quota compete for the remaining physical memory).
Quoted 4 lines
I do have a few ideas for the scheduling stuff though, with RSS limits (we can safely implement those when the swap cache trick is implemented) and the keeping of a few statistics, we will be able to implement the swapping tricks.
Rick, get real: when will you work out how the VM works? We can safely implement RSS limits *today*, and have been able to since 2.1.89. <grin> It's just a matter of doing a vmscan on the current process whenever it exceeds its own RSS limit. The mechanism is all there.
Quoted 2 lines
Without swapin readahead, we'll be unable to implement them properly however :(
No, we don't need readahead (although the swap cache itself already includes all of the necessary mechanism: rw_swap_page(READ, nowait) will do it). The only extra functionality we might want is extra control over when we write swap-cached pages: right now, all dirty pages need to be in the RSS, and we write them to disk when we evict them to the swap cache. Thus, only clean pages can be in the swap cache. If we want to support processes with a dirty working set > RSS, we'd need to extend this.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 8 lines · Stephen C. Tweedie
Quoted 4 lines · Rik van Riel
I do have a few ideas for the scheduling stuff though, with RSS limits (we can safely implement those when the swap cache trick is implemented) and the keeping of a few statistics, we will be able to implement the swapping tricks.
Quoted 5 lines
Rick, get real: when will you work out how the VM works? We can safely implement RSS limits *today*, and have been able to since 2.1.89. <grin> It's just a matter of doing a vmscan on the current process whenever it exceeds its own RSS limit. The mechanism is all there.
If we tried to implement RSS limits now, it would mean that the large task(s) we limited would be continuously thrashing and keep the I/O subsystem busy -- this impacts the rest of the system a lot.
With the new scheme, we can implement the RSS limit, but the truly busily used pages would simply stay inside the swap cache, freeing up I/O bandwidth (at the cost of some memory) for the rest of the system.
I think that with the new scheme the balancing will be so much better that we can implement RSS limits without a negative impact on the rest of the system. With the current VM system RSS limits would probably hamper the performance the rest of the system gets.
We might want to perform the scheduling tricks for over-RSS processes however. Without swap readahead I really don't see any way we could run them without keeping back the rest of the system too much...
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 9 lines · Rik van Riel
Quoted 3 lines · Stephen C. Tweedie
Rick, get real: when will you work out how the VM works? We can safely implement RSS limits *today*, and have been able to since 2.1.89. <grin>
If we tried to implement RSS limits now, it would mean that the large task(s) we limited would be continuously thrashing and keep the I/O subsystem busy -- this impacts the rest of the system a lot.
WRONG. We can very very easily unlink pages from a process's pte (hence reducing the process's RSS) without removing that page from memory. It's trivial. We do it all the time. We can do it both for memory-mapped files and for anonymous pages. In the latest 2.1.130 prepatch, this is in fact the *preferred* way of swapping. This mechanism is fundamental to the way we maintain page sharing of swapped COW pages.
The only thing we cannot do is unlink dirty pages (for swap, that means pages which have been modified since we last paged the swap back into memory). We have to write them back before we unlink. That does not mean that we have to throw the data away: as long as the copy on disk is uptodate, we can have as much of a process's address space as we want in the page cache or swap cache without it being mapped in the process' address space and without it counting as task RSS.
Today, such an RSS limit would NOT thrash the IO: it would just cause minor page faults as we relink the cached page back into the page tables. All of that functionality exists today.
Rik, you should probably try to work out how try_to_swap_out() actually works one of these days. You'll find it does a lot of neat stuff you seem to be unaware of! We are really a lot closer to having a proper unified page handling mechanism than you think. The handling of dirty pages is pretty much the only missing part of the mechanism right now. Even that is not necessarily a bad thing: there are good performance reasons why we might want the swap cache to contain only clean pages:
for example, it makes it easier to guarantee that those pages can be reclaimed for another use at short notice.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 8 lines · Stephen C. Tweedie
Quoted 4 lines · Rik van Riel
If we tried to implement RSS limits now, it would mean that the large task(s) we limited would be continuously thrashing and keep the I/O subsystem busy -- this impacts the rest of the system a lot.
Quoted 5 lines
WRONG. We can very very easily unlink pages from a process's pte (hence reducing the process's RSS) without removing that page from memory. It's trivial. We do it all the time. Rik, you should probably try to work out how
try_to_swap_out()actually works one of these days.
I just looked in mm/vmscan.c of kernel version 2.1.129, and line 173, 191 and 205 feature a prominent:
free_page_and_swap_cache(page);
Quoted 3 lines
We are really a lot closer to having a proper unified page handling mechanism than you think. The handling of dirty pages is pretty much the only missing part of the mechanism right now.
I know how close we are. I think I posted an assesment on what to do and what to leave yesterday :)) The most essential things can probably be coded in a day or two, if we want to.
Oh, one question. Can we attach a swap page to the swap cache while there's no program using it? This way we can implement a very primitive swapin readahead right now, improving the algorithm as we go along...
Quoted 4 lines
Even that is not necessarily a bad thing: there are good performance reasons why we might want the swap cache to contain only clean
pages: for example, it makes it easier to guarantee that those pages can be reclaimed for another use at short notice.
IMHO it would be a big loss to have dirty pages in the swap cache. Writing out swap pages is cheap since we do proper I/O clustering, not writing them out immediately will result in them being written out in the order that shrink_mmap() comes across them, which is a suboptimal way for when we want to read the pages back.
Besides, having a large/huge clean swap cache means that we can very easily free up memory when we need to, this is essential for NFS buffers, networking stuff, etc.
If we keep a quota of 20% of memory in buffers and unmapped cache, we can also do away with a buffer for the 8 and 16kB area's. We can always find some contiguous area in swap/page cache that we can free...
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 9 lines · Rik van Riel
Quoted 5 lines
WRONG. We can very very easily unlink pages from a process's pte (hence reducing the process's RSS) without removing that page from memory. It's trivial. We do it all the time. Rik, you should probably try to work out how
try_to_swap_out()actually works one of these days.I just looked in
mm/vmscan.cof kernel version 2.1.129, and line 173, 191 and 205 feature a prominent:free_page_and_swap_cache(page);
It is not there in 2.1.130-pre3, however. :) That misses the point, though. The point is that it is trivial to remove these mappings without freeing the swap cache, and the code you point to confirms this:
vmscan actually has to go to _extra_ trouble to free the underlying cache if that is wanted (the shared page case is the same, hence the unuse_page call at the end of try_to_swap_out() (also removed in 2.1.130-3). The default action of the free_page alone removes the mapping but not the cache entry, and the functionality of leaving the cache present is already there.
Quoted 4 lines
Oh, one question. Can we attach a swap page to the swap cache while there's no program using it? This way we can implement a very primitive swapin readahead right now, improving the algorithm as we go along...
Yes, rw_swap_page(READ, nowait) does exactly that: it primes the swap cache asynchronously but does not map it anywhere. It should be completely safe right now: the normal swap read is just a special case of this.
Quoted 7 lines
IMHO it would be a big loss to have dirty pages in the swap cache. Writing out swap pages is cheap since we do proper I/O clustering ...
Besides, having a large/huge clean swap cache means that we can very easily free up memory when we need to, this is essential for NFS buffers, networking stuff, etc.
Yep, absolutely: agreed on both counts. This is exactly how 2.1.130-3 works!
Quoted 4 lines
If we keep a quota of 20% of memory in buffers and unmapped cache, we can also do away with a buffer for the 8 and 16kB area's. We can always find some contiguous area in swap/page cache that we can free...
That will kill performance if you have a large simulation which has a legitimate need to keep 90% of physical memory full of anonymous pages. I'd rather do without that 20% magic limit if we can. The only special limit we really need is to make sure that kswapd keeps far enough in advance of interrupt memory load that the free list doesn't empty.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 5 lines · Stephen C. Tweedie
<[email protected]> said:
It is not there in 2.1.130-pre3, however. :) That misses the point, though. The point is that it is trivial to remove these mappings without freeing the swap cache, and the code you point to confirms this:
OK, point taken. {:-)
Quoted 9 lines
Quoted 4 lines
Oh, one question. Can we attach a swap page to the swap cache while there's no program using it? This way we can implement a very primitive swapin readahead right now, improving the algorithm as we go along...
Yes,
rw_swap_page(READ, nowait)does exactly that: it primes the swap cache asynchronously but does not map it anywhere. It should be completely safe right now: the normal swap read is just a special case of this.
Then I think it's time to do swapin readahead on the entire SWAP_CLUSTER (or just from the point where we faulted) on a dumb-and-dumber basis, awaiting a good readahead scheme. Of course it will need to be sysctl tuneable :)
The reason I propose this dumb scheme is because we can read one SWAP_CLUSTER_MAX sized chunk in one sweep without having to move the disks head... Plus Linus might actually accept a change like this :)
Quoted 10 lines
Quoted 4 lines
If we keep a quota of 20% of memory in buffers and unmapped cache, we can also do away with a buffer for the 8 and 16kB area's. We can always find some contiguous area in swap/page cache that we can free...
That will kill performance if you have a large simulation which has a legitimate need to keep 90% of physical memory full of anonymous pages. I'd rather do without that 20% magic limit if we can. The only special limit we really need is to make sure that kswapd keeps far enough in advance of interrupt memory load that the free list doesn't empty.
OK, then we should let the kernel calculate the limit itself based on the number of soft faults, swapout pressure, memory pressure and process priority.
We can also use stats like this to temporarily suspend very large processes when we've got multiple processes with:
(p->vm_mm->rss + p->dec_flt) > RSS_THRASH_LIMIT, wherep->dec_flt is a floating average and the RSS limit is calculated dynamically as well... I know this could be a slightly expensive trick, but we can easily make that sysctl tuneable as well.
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 5 lines · Rik van Riel
Then I think it's time to do swapin readahead on the entire
SWAP_CLUSTER(or just from the point where we faulted) on a dumb-and-dumber basis, awaiting a good readahead scheme. Of course it will need to be sysctl tuneable :)
Yep, although I'm not sure that reading a whole SWAP_CLUSTER would be a good idea. Contrary to popular belief, disks are still quite slow at sequential data transfer. Non-sequential IO is obviously enormously slower still, but doing readahead on a whole SWAP_CLUSTER (128k) is definitely _not_ free. It will increase the VM latency enormously if we start reading in a lot of unnecessary data.
On the other hand, swap readahead is sufficiently trivial to code that experimenting with good values is not hard. Normal pagein already does a one-block readahead, and doing this in swap would be pretty easy.
The biggest problem with swap readahead is that there is very little guarantee that the next page in any one swap partition is related to the current page: the way we select pages for swapout makes it quite likely that bits of different processes may intermix, and swap partitions can also get fragmented over time. To really benefit from swap readahead, we would also want improved swap clustering which tried to keep a logical association between adjacent physical pages, in the same way that the filesystem does. Right now, the swap clustering is great for output performance but doesn't necessarily lead to disk layouts which are good for swaping.
Quoted 1 line
Plus Linus might actually accept a change like this :)
If it is tunable, then it is so easy that he might well, yes.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 9 lines · Stephen C. Tweedie
Quoted 2 lines · Rik van Riel
Then I think it's time to do swapin readahead on the entire
SWAP_CLUSTERYep, although I'm not sure that reading a whole
SWAP_CLUSTERwould be a good idea. Contrary to popular belief, disks are still quite slow at sequential data transfer.
I have a better idea for a default limit:
swap_stream.max = num_physpages >> 9; if (swap_stream.max > SWAP_CLUSTER_MAX) swap_stream.max = SWAP_CLUSTER_MAX; swap_stream.enabled = 0;
Quoted 4 lines
Non-sequential IO is obviously enormously slower still, but doing readahead on a whole
SWAP_CLUSTER(128k) is definitely _not_ free. It will increase the VM latency enormously if we start reading in a lot of unnecessary data.
We could simply increase the readahead if we were more than 50% succesful (ie. 80% of swap requests can be satisfied from the swap cache) and decrease it if we drop below 40% (or less than 50% of swap requests can be serviced from the swap cache).
One thing that helps us enormously is the way kswapd pages out stuff. If pages (within a process) have the same kind of usage pattern and are near eachother, they will be swapped out together. Now since they have the same usage pattern, it is likely that they are needed together as well.
Especially without page aging we are likely to store adjecant pages next to eachother in swap.
Later on (when the simple code has been proven to work and Linus doesn't pay attention) we can introduce a really intelligent swapin readahead mechanism that will make Linux rock :)
It's just that we need something simple now because Linus wants the kernel to stay relatively unchanged at the moment...
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Quoted 5 lines · Rik van Riel
We could simply increase the readahead if we were more than 50% succesful (ie. 80% of swap requests can be satisfied from the swap cache) and decrease it if we drop below 40% (or less than 50% of swap requests can be serviced from the swap cache).
Yes --- do a patch, show us some benchmarks! We could make a big difference with this.
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 19 lines
Without swapin readahead, we'll be unable to implement them properly however :(
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
And now we are at it: in 2.0 I found a deamon can be killed by the system if it runs out of VM.
Same on any BSD.
Say the Web or database server can be deemed important enough for it not being killed just because some dim witt is playing with the GIMP at the console and the GIMP has allocated 80 Megs.
I sounds remarkably like you want my Out Of Memory killer patch. This patch tries to remove the randomness in killing a process when you're OOM by carefully selecting a process based on a lot of different factors (size, age, CPU used, suid, root, IOPL, etc).
Quoted 4 lines
It needs to be cleaned up, ported to 2.1.129 and improved a little bit though... After that it should be ready for inclusion in the kernel.
Your scheme is (IMHO) far too complicated and (IMHO) falls short. The problem is that the kernel has no way to know what is the really important process in the box. For instance you can have a database server running as normal user and that be considered far more important the X server (setuid root) whose only real goal is to allow a user friendly UI for administering the database.
Why not simply allow a root-owned process declare itself (and the program it will exec into) as "guaranteed"? Only a human can know what is important and what is unimportant in a box so it should be a human who, by the way of starting a program throuh a "guaranteer", has the final word on what should be protected
Allow an option for having this priviliege extended to descendents of the process given some database programs start special daemons for other tasks and will not run without them. Or a box used as a mail server using qmail: qmail starts sub-servers each one for a different task.
Of course this is only a suugestion for a mechanism but the important is allowing a human to have the final word.
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 9 lines · On 25 Nov 1998 [email protected] wrote:
Quoted 5 lines
I sounds remarkably like you want my Out Of Memory killer patch. This patch tries to remove the randomness in killing a process when you're OOM by carefully selecting a process based on a lot of different factors (size, age, CPU used, suid, root, IOPL, etc).
Your scheme is (IMHO) far too complicated and (IMHO) falls short. The problem is that the kernel has no way to know what is the really important process in the box.
In my (and other people's) experience, an educated guess is better than a random kill. Furthermore it is not possible to get out of the OOM situation without killing one or more processes, so we want to limit:
- the number of processes we kill (reducing the chance of killing something important)
- the CPU time 'lost' when we kill something (so we don't have to run that simulation for two weeks again)
- the risk of killing something important and stable, we try to avoid this by giving less hitpoints to older processes (which presumably are stable and take a long time to 'recreate' the state in which they are now)
- the amount of work lost -- killing new processes that haven't used much CPU is a way of doing this
- the probability of the machine hanging -- don't kill IOPL programs and limit the points for old daemons and root/suid stuff
Granted, we can never make a perfect guess. It will be a lot better than a more or less random kill, however.
The large simulation that's taking 70% of your RAM and has run for 2 weeks is the most likely victim under our current scheme, but with my killer code it's priority will be far less that that of a newly-started and exploded GIMP or Netscape...
Quoted 2 lines
Why not simply allow a root-owned process declare itself (and the program it will exec into) as "guaranteed"?
If the guaranteed program explodes it will kill the machine. Even for single-purpose machines this will be bad since it will increase the downtime with a reboot&fsck cycle instead of just a program restart.
Quoted 2 lines
Or a box used as a mail server using qmail: qmail starts sub-servers each one for a different task.
The children are younger and will be killed first. Starting the master server from init will make sure that it is restarted in the case of a real emergency or fluke.
Quoted 2 lines
Of course this is only a suugestion for a mechanism but the important is allowing a human to have the final word.
What? You have a person sitting around keeping an eye on your mailserver 24x7? Usually the most important servers are tucked away in a closet and crash at 03:40 AM when the sysadmin is in bed 20 miles away...
The kernel is there to prevent Murphy from taking over :)
cheers,
Rik -- slowly getting used to dvorak kbd layout...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 31 lines · On 25 Nov 1998 [email protected] wrote:
Quoted 10 lines
Quoted 6 lines
I sounds remarkably like you want my Out Of Memory killer patch. This patch tries to remove the randomness in killing a process when you're OOM by carefully selecting a process based on a lot of different factors (size, age, CPU used, suid, root, IOPL, etc).
Your scheme is (IMHO) far too complicated and (IMHO) falls short. The problem is that the kernel has no way to know what is the really important process in the box.
In my (and other people's) experience, an educated guess is better than a random kill. Furthermore it is not possible to get out of the OOM situation without killing one or more processes, so we want to limit:
- the number of processes we kill (reducing the chance of killing something important)
- the CPU time 'lost' when we kill something (so we don't have to run that simulation for two weeks again)
- the risk of killing something important and stable, we try to avoid this by giving less hitpoints to older processes (which presumably are stable and take a long time to 'recreate' the state in which they are now)
- the amount of work lost -- killing new processes that haven't used much CPU is a way of doing this
- the probability of the machine hanging -- don't kill IOPL programs and limit the points for old daemons and root/suid stuff
Quoted 9 lines
Granted, we can never make a perfect guess. It will be a lot better than a more or less random kill, however.
The large simulation that's taking 70% of your RAM and has run for 2 weeks is the most likely victim under our current scheme, but with my killer code it's priority will be far less that that of a newly-started and exploded GIMP or Netscape...
My idea was:
-VM exhausted and process allocating is a normal process then kill
process.
-VM exhausted and process is a guaranteed one then kill a non guaranteed process.
-VM exhausted, process is guaranteed but only remaining processes are
guaranteed ones. Kill allocated process.
Of course INIT is guaranteed.
Quoted 8 lines
Quoted 2 lines
Why not simply allow a root-owned process declare itself (and the program it will exec into) as "guaranteed"?
If the guaranteed program explodes it will kill the machine. Even for single-purpose machines this will be bad since it will increase the downtime with a reboot&fsck cycle instead of just a program restart.
Nope see higher. The guaranteed program would be killed once "unimportant" processes have been killed. The goal is not to allow impunity to guaranteed programs but to protect an important program against possible misbehaviour of other programs: a misbehaving process who has allocated all the VM except 1 page and then our database server tries to allocate two more pages.
Quoted 7 lines
Quoted 2 lines
Or a box used as a mail server using qmail: qmail starts sub-servers each one for a different task.
The children are younger and will be killed first. Starting the master server from init will make sure that it is restarted in the case of a real emergency or fluke.
Quoted 8 lines
Quoted 2 lines
Of course this is only a suugestion for a mechanism but the important is allowing a human to have the final word.
What? You have a person sitting around keeping an eye on your mailserver 24x7? Usually the most important servers are tucked away in a closet and crash at 03:40 AM when the sysadmin is in bed 20 miles away...
No. The sysadmin uses emacs at normal hours to edit a file telling what are the important processes. Now it is to you to find a scheme in order the sysadmin's wishes are communicated to the kernel. :-)
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Hi,
Review
Quoted 8 lines · On 26 Nov 1998 19:59:42 -0000, [email protected] said:
My idea was:
(quoted diff) +0
-2-VM exhausted and process allocating is a normal process then kill process. -VM exhausted and process is a guaranteed one then kill a non guaranteed process. -VM exhausted, process is guaranteed but only remaining processes are guaranteed ones. Kill allocated process.
But the _whole_ problem is that we do not necessarily go around killing processes. We just fail requests for new allocations. In that case we still have not run out of memory yet, but a daemon may have died. It is simply not possible to guarantee all of the future memory allocations which a process might make!
--Stephen
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 12 lines
Date: Fri, 27 Nov 1998 17:45:55 GMT
From: "Stephen C. Tweedie" <[email protected]>
Content-Type: text/plain; charset=us-ascii
Cc: [email protected], [email protected], [email protected]
X-UIDL:62f6721511a1878f885583dcf30990c3Hi,
Quoted 1 line · On 26 Nov 1998 19:59:42 -0000, [email protected] said:
My idea was:
Quoted 13 lines
Review
Quoted 6 lines
(quoted diff) +0
-2-VM exhausted and process allocating is a normal process then kill process. -VM exhausted and process is a guaranteed one then kill a non guaranteed process. -VM exhausted, process is guaranteed but only remaining processes are guaranteed ones. Kill allocated process.But the _whole_ problem is that we do not necessarily go around killing processes. We just fail requests for new allocations. In that case we still have not run out of memory yet, but a daemon may have died. It is simply not possible to guarantee all of the future memory allocations which a process might make!
The word "guaranteed" was an unfortunate one. "Protected" would have been better.
As a user I feel there are processes more equal than others and I find unfortunate one of them is killed when it tries to grow its stack (SIGKILL so no recovering) and it is unable to do so due to mibehaviour of an unimportant process. I think they should be protected and that it is the sysadmin and not a heuristic who should define what is important and what is not in a box. We cannot guarantee the success of a memory allocation but we can make mission critical software motre robust.
But if you think the idea is bad we can kill this thread.
Signature · 8 lines
Jean Francois Martinez Project Independence: Linux for the Masses http://www.independence.seul.org -- This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 5 lines · On Tue, Nov 24, 1998 at 09:44:32PM -0000, [email protected] wrote:
In situation like those above I would like Linux supported a concept like guaranteed processses: if VM is exhausted by one of them then try to get memory by killing non guaranteed processes and only kill the original one if all reamining survivors are guaranteed ones. It would be better for mission critical tasks.
Long time ago I suggested to make it configurable whether a process gets memory which might be overcommited or not. This leaves malloc(x) == NULL to deal with and that's a userland problem anyway.
Ralf
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]
Quoted 11 lines · On Wed, 25 Nov 1998 [email protected] wrote:
Quoted 5 lines · On Tue, Nov 24, 1998 at 09:44:32PM -0000, [email protected] wrote:
In situation like those above I would like Linux supported a concept like guaranteed processses: if VM is exhausted by one of them then try to get memory by killing non guaranteed processes and only kill the original one if all reamining survivors are guaranteed ones. It would be better for mission critical tasks.
Long time ago I suggested to make it configurable whether a process gets memory which might be overcommited or not. This leaves malloc(x) == NULL to deal with and that's a userland problem anyway.
Then what would you do when your 250MB non-overcommitting program needs to do a fork() in order to call /usr/bin/lpr?
Install an extra 250MB of swap? I don't think so :) These are the situations where sane people want overcommit.
regards,
Rik -- who actually has 250MB of extra swap...
+-------------------------------------------------------------------+ | Linux memory management tour guide. [email protected] | | Scouting Vries cubscout leader. http://www.phys.uu.nl/~riel/ | +-------------------------------------------------------------------+
--
This is a majordomo managed list. To unsubscribe, send a message with the body 'unsubscribe linux-mm me@address' to: [email protected]