Kernel cached memory
Hi all
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory,
the problem is: when this aplications exits, the free memory of the server is still very low(accroding to the output of "top"), and from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
Sincerely,
Ashley
------------------------------------------------------
AlChip Technologies, Limited
26F, Huamin Empire Plaza, Suite B-H,
726 West YanAn Road,
Shanghai, China 200050
Tel: 86-21-5237-8800 ext.220
Fax: 86-21-5237-8880
http://www.alchip.com
------------------------------------------------------
Quoted 7 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory, the problem is: when this aplications exits, the free memory of the server is still verylow(accroding to the output of "top"), and
from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
Free memory is bad, it means the memory doesn't have a proper use. Cached memory will be freed automatically when the kernel needs memory for other (more important) things.
Erik
Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 -- | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
Quoted 3 lines · Ashley
from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
You don't want that. Kernel will free cached memory when apps need it. When there's a lot of free memory, linux always tries to use it for cache - because caches speed up things, and it's silly to to leave memory free if you can use it for caching.
Quoted 7 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory, the problem is: when this aplications exits, the free memory of the server is still verylow(accroding to the output of "top"), and
from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
It's a very - very - very old and bad logic (at least nowdays) from the stone age to free up memory. Memory is for using ... If you have memory why don't you want to use? For an actively used system, memory should be used as much as possible to maximize the performance. The only problem when kernel does not want to "give back" used memory for eg caching for an application though but it's another problem ...
Anyway, want to have 'free memory' is a thing like having dozens of cars in your garage which don't want to be used ...
Signature · 1 line
- Gábor
Quoted 3 lines · Gábor Lénárt
Anyway, want to have 'free memory' is a thing like having dozens of cars in your garage which don't want to be used ...
Really? I thought it was good to leave some memory free to speed up application startup, so we don't have to evict a bunch of pages first.
Lee
Wouldn't having (practically) all your memory used for cache slow down starting a new program? First it would have to free up that space, and then put stuff in that space, taking potentially twice as long. I think there should be a system call for freeing cached memory, for those that do want to do it.
-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of Erik Mouw
Sent: Friday, July 22, 2005 5:58 AM
To: Ashley
Cc: [email protected]
Subject: Re: Kernel cached memory
Quoted 3 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory, the problem is: when this aplications exits, the free memory of the
server
Quoted 2 lines
is still very
low(accroding to the output of "top"), and from the output of command "free", I can see that many GB memory was
cached
Quoted 2 lines
by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
Free memory is bad, it means the memory doesn't have a proper use. Cached memory will be freed automatically when the kernel needs memory for other (more important) things.
Erik
--
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 -- | Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Am Freitag, 22. Juli 2005 19:58 schrieb Lee Revell:
Quoted 7 lines · Gábor Lénárt
Quoted 3 lines
Anyway, want to have 'free memory' is a thing like having dozens of cars in your garage which don't want to be used ...
Really? I thought it was good to leave some memory free to speed up application startup, so we don't have to evict a bunch of pages first.
Eviction is cheap when the pages are clean.
Regards
Oliver
Quoted 5 lines · John Pearson
Wouldn't having (practically) all your memory used for cache slow down starting a new program? First it would have to free up that space, and then put stuff in that space, taking potentially twice as long. I think there should be a system call for freeing cached memory, for those that do want to do it.
I think this one is good enough:
#include <stdlib.h>
int main() {
void *p;
unsigned size = 1<<20;
unsigned long total=0;
while(size) {
p = malloc(size);
if(!p) size>>=1;
else {
memset(p, 0x77, size);
total+=size;
printf("Allocated %9u bytes, %12lu total\n",size,total);
}
}
return 0;
}You may want to adapt it so that it takes an argument now many megabytes to eat before it dies.
--
vda
Quoted 11 lines · Gábor Lénárt
Quoted 5 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory,
the problem is: when this aplications exits, the free memory of the server is still verylow(accroding to the output of "top"), and from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.It's a very - very - very old and bad logic (at least nowdays) from the stone age to free up memory.
It's very Microsoft to claim that the OS always knows best, and not let the user tune the system the way they want it tuned. And if that means to leave a bunch of free memory for absolute fastest availability, the admin should have that option.
Quoted 5 lines
Memory is for using ... If you have memory why
don't you want to use? For an actively used system, memory should be used as much as possible to maximize the performance. The only problem when kernel does not want to "give back" used memory for eg caching for an application though but it's another problem ...
No, that's the problem he's trying to address.
Quoted 4 lines
Anyway, want to have 'free memory' is a thing like having dozens of cars in your garage which don't want to be used ...
Which wait to be used when needed, rather than after someone cleans a bunch of old junk out of them and scurries around the garage looking for the right car to let you use.
Just because default operation works well for you, kindly don't try to convice us that there are no cases when the default operation is NOT optimal. And IMHO Linux is *way* too willing to evicy clean pages of my programs to use as disk buffer, so that when system memory is full I pay the overhead of TWO disk i/o's, one to finally write the data to the disk and one to read my program back in. If free software is about choice, I wish there was more in the area of how memory is used.
Signature · 3 lines
-bill davidsen ([email protected]) "The secret to procrastination is to put things off until the last possible moment - but no longer" -me
Quoted 7 lines · Bill Davidsen
Just because default operation works well for you, kindly don't try to convice us that there are no cases when the default operation is NOT optimal. And IMHO Linux is *way* too willing to evicy clean pages of my programs to use as disk buffer, so that when system memory is full I pay the overhead of TWO disk i/o's, one to finally write the data to the disk and one to read my program back in. If free software is about choice, I wish there was more in the area of how memory is used.
You must be more specific here; cached memory isn't always dirty. Most of the times the pages used for cache are clean and can be evicted quite fast and of course without writting anything to the disk.
If you want to tune the balance between "pages used for programs" vs "pages used for cache" that's another problem, but I can't find a reason why kernel shouldn't cache things when free memory is available. And certainly that's not what Ashley was asking.
Quoted 6 lines · On Mon, 25 Jul 2005 12:47:50 -0400 Bill Davidsen <[email protected]> wrote:
And IMHO Linux is *way* too willing to evicy clean pages of my programs to use as disk buffer, so that when system memory is full I pay the overhead of TWO disk i/o's, one to finally write the data to the disk and one to read my program back in. If free software is about choice, I wish there was more in the area of how memory is used.
isn't this tuned enough by "/proc/sys/vm/swappiness" ?
Signature · 2 lines
Paolo Ornati Linux 2.6.13-rc3 on x86_64
Quoted 13 lines · Paolo Ornati
Quoted 7 lines · On Mon, 25 Jul 2005 12:47:50 -0400 Bill Davidsen <[email protected]> wrote:
And IMHO Linux is *way* too willing to evicy clean pages of my programs to use as disk buffer, so that when system memory is full I pay the overhead of TWO disk i/o's, one to finally write the data to
the disk and one to read my program back in. If free software is
about choice, I wish there was more in the area of how memory is
used.
Quoted 4 lines
isn't this tuned enough by "
/proc/sys/vm/swappiness" ?
Let me generate some data points for discussion. But the general answer is no, I just want to try to have some numbers to discuss. I also want to go back to some 2.4.xx-aa kernels, Andrea had some very nice things in his bdflush code, and he was kind enough to explain to me how to tune them so I avoided the worst case events.
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979
Quoted 12 lines · Bill Davidsen
Quoted 11 lines · Gábor Lénárt
Quoted 7 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server
is used to run applications which will comsume huge memory, the problem is: when this aplications exits, the free memory of the server is still verylow(accroding to the output of "top"), and from the output of command "free", I can see that many GB memory was cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
Quoted 7 lines
Quoted 2 lines
It's a very - very - very old and bad logic (at least nowdays) from the stone age to free up memory.
It's very Microsoft to claim that the OS always knows best, and not let the user tune the system the way they want it tuned. And if that means to leave a bunch of free memory for absolute fastest availability, the admin should have that option.
Sure, sorry if my comment can be treated in this way ... I mean surprising amount of people I've met criticised Linux (well, some years ago when DOS was popular) that he/she want to see that 'free memory' field reported eg by 'top' should be the maximum all the time ... I mean this way: this is the behaviour which is quite wrong, I've written about this.
Sure, because of my not too good English, I may have missed the real meaning of the mail, sorry about it!
Signature · 1 line
- Gábor
Quoted 29 lines · Gábor Lénárt
Quoted 18 lines · Bill Davidsen
Quoted 15 lines · Gábor Lénárt
Quoted 11 lines · Ashley
I've a server with 2 Operton 64bit CPU and 12G memory, and this server is used to run applications which will comsume huge memory, the problem is: when this aplications exits, the free memory of the server is still very
low(accroding to the output of "top"), andfrom the output of command "free", I can see that many GB memory was
cached by kernel. Does anyone know how to free the kernel cached memory? thanks in advance.
Quoted 7 lines
Quoted 4 lines
It's a very - very - very old and bad logic (at least nowdays) from the stone age to free up memory.
It's very Microsoft to claim that the OS always knows best, and not let the user tune the system the way they want it tuned. And if that means to leave a bunch of free memory for absolute fastest availability, the admin should have that option.
Quoted 9 lines
Sure, sorry if my comment can be treated in this way ... I mean surprising amount of people I've met criticised Linux (well, some years ago when DOS was popular) that he/she want to see that 'free memory' field reported eg by 'top' should be the maximum all the time ... I mean this way: this is the behaviour which is quite wrong, I've written about this.
Sure, because of my not too good English, I may have missed the real meaning of the mail, sorry about it!
Well, I thought I understood "from the stone age" but I may have taken it slightly too literally. But I really would like to have more control over Linux memory use, because it does cause bad behaviour at times. If I have 4GB of RAM, I'd like to set 200MB or so aside for programs, and never page out the window I'm going to uncover later. Likewise when I write a DVD image, I would like to avoid buffering a few GB without i/o and then driving the disk totally busy while it gets written out (after it has pushed out things I will use again).
The old 2.4.x-aa kernels had some tunables to make the kernel aggressive about writing pages to disk quickly, and I haven't been able to match that behaviour without patches in 2.6. I may be missing a tunable, but swappiness doesn't seem to be the one I want. I have a patch I'm playing with, but it's not ready for prime time, and is probably counter to the current philosophy of memory management.
Thanks for clarifying.
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979