Skip to contentSkip to conversation

Slab allocator . . . cache? WTF is it?

DISCUSSION john moser 5 participants 6 messages Dec 30, 2003 – Dec 31, 2003

lore
Share & tools
lore

john moser

author
#0 lore raw
Mem:    775616k total,   747740k used,    27876k free,    96584k buffers
Swap:   250480k total,    71340k used,   179140k free,   298852k cached

This is somewhat distressing. Last time this happened, I started opening every program I had (including every OpenOffice.org product and every browser I had installed), and the "cached" value dropped quickly.

I'm wondering, what IS cache? It seems to increase even when swap is not used, and sometimes when there's no swap partition enabled. It also seems to cause me to run into swap when I have ample ram available, assuming that cache is just some sort of cache that is copied from and mirrors another portion of ram for some sort of speed increase. It's wasteful to me, and I want to more understand its implimentation and its purpose, and in all honesty limit its impact if possible. I got this RAM upgrade because I was using about 676M of RAM total, including swap, at peak; and now I find myself using 820M RAM at peak and about 750-800 continually.

As always, i'm not on the list, so I'd prefer to be CC'd replies.

_____________________________________________________________
Linux.Net -->Open Source to everyone
Powered by Linare Corporation
http://www.linare.com/

#1 lore raw

Hi,
I'm kind of a kernel-dev newbie but I'll try to explain my understanding in BFU-friendly way.
The cache you're seeing here is a memory used to store IO requests and data when you request some operation on IO. For example if you are writing to hard drive then (if your partition is not mounted with sync option or its not some special case) the data you are writing are put in the cache and the disk starts writing. And because your memory is fast, let's say it can do a 500MiB/s, then your write is completed in 0.1sec and your program can continue doing other stuff. Data you requested to write to the drive will get written as soon as it is possible (or with a small amounts of data as soon as some timer reaches zero - this is to provide). But because your disk is able to write only let's say a 20MiB/s and you are writing a 50MiB file the write without using cache would take about 2.5 seconds before your program can continue. So you are having a 25 times faster write at user level with cache than without. You could probably see it even better when reading lot of small files (ie. grepping a lot of text files), where the drive would have to seek for each request, but kernel can read ahead more data than was actually requested guessing that you will request them later anyway and put them in cache. And when you request them you get them at the speed of your memory. So cache has (almost) nothing to do with swap and its on the same line as swap just because it fits better:P. Kernel is trying to be smart about swap and cache so it will swap out programs out of memory when you are really not using them to be able to use the memory for cache that can really speed up your system.

Well I believe this is correct, if not please somebody be so kind and correct me before beating me with a kernel tarball :).

Tomas Zvala

Quoted 31 lines · john moser
Mem:    775616k total,   747740k used,    27876k free,    96584k buffers
Swap:   250480k total,    71340k used,   179140k free,   298852k cached

This is somewhat distressing. Last time this happened, I started opening every program I had (including every OpenOffice.org product and every browser I had installed), and the "cached" value dropped quickly.

I'm wondering, what IS cache? It seems to increase even when swap is not used, and sometimes when there's no swap partition enabled. It also seems to cause me to run into swap when I have ample ram available, assuming that cache is just some sort of cache that is copied from and mirrors another portion of ram for some sort of speed increase. It's wasteful to me, and I want to more understand its implimentation and its purpose, and in all honesty limit its impact if possible. I got this RAM upgrade because I was using about 676M of RAM total, including swap, at peak; and now I find myself using 820M RAM at peak and about 750-800 continually.

As always, i'm not on the list, so I'd prefer to be CC'd replies.

_____________________________________________________________
Linux.Net -->Open Source to everyone
Powered by Linare Corporation
http://www.linare.com/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
In reply to john moser
Quoted 8 lines · john moser

I'm wondering, what IS cache? It seems to increase even when swap is not used, and sometimes when there's no swap partition enabled. It also seems to cause me to run into swap when I have ample ram available, assuming that cache is just some sort of cache that is copied from and mirrors another portion of ram for some sort of speed increase. It's wasteful to me, and I want to more understand its implimentation and its purpose, and in all honesty limit its impact if possible.

Short answer: cache is the use of memory to store what would otherwise have to be read from disk, like parts of frequently used files etc. Linux will trade off cache against other RAM needs like program memory on an ongoing basis, and (almost always) does a really good job of it.

Don't be alarmed about going into swap somewhat in normal operation--Linux is storing less important data on the disk to make more room in RAM for "hot" data. Obviously, if you have hundreds of megs in swap or it grows quickly, you may need to look at it.

Quoted 3 lines

I got this RAM upgrade because I was using about 676M of RAM total, including swap, at peak; and now I find myself using 820M RAM at peak and about 750-800 continually.

That means the kernel is doing exactly what it's supposed to do. If memory isn't needed by programs and isn't used for cache, it just goes to waste, so Linux tries to use it for one or the other. :)

-Doug

#3 lore raw
In reply to john moser
Quoted 9 lines
Mem:    775616k total,   747740k used,    27876k free,    96584k buffers
Swap:   250480k total,    71340k used,   179140k free,   298852k cached

This is somewhat distressing. Last time this happened, I started opening every
program I had (including every OpenOffice.org product and every browser I had
installed), and the "cached" value dropped quickly.

So, as you've seen, when the kernel needs memory, it can easily throw its
cached data away and gain as much free memory as it needs.

Quoted 5 lines

I'm wondering, what IS cache? It seems to increase even when swap is not used,
and sometimes when there's no swap partition enabled. It also seems to cause
me to run into swap when I have ample ram available,

All true, all good. No matter how much RAM you have available, odds are you
are moving more dat to and from disk then you have RAM. So it sometimes makes sense to swap stuff to make room for more cache.

Quoted 5 lines

assuming
that cache is just
some sort of cache that is copied from and mirrors another portion of ram for
some sort of speed increase.

More typically, it contains copies of data that was written to or read from
disk in case it is requested (again).

Quoted 1 line

It's wasteful to me,

Huh? Why do you say that?

Quoted 8 lines

and I want to
more understand
its implimentation and its purpose, and in all honesty limit its impact if possible.
I got this RAM upgrade because I was using about 676M of RAM total, including swap,
at peak; and now I find myself using 820M RAM at peak and about 750-800 continually.

If you don't want the computer to use the RAM, take the RAM out of the
computer. The computer will use all of the RAM you give it. Why shouldn't it?

DS

#4 lore raw
In reply to john moser
Quoted 5 lines · john moser

I'm wondering, what IS cache? It seems to increase even when swap is not used, and sometimes when there's no swap partition enabled. It also seems to cause me to run into swap when I have ample ram available, assuming that cache is just some sort of cache that is copied from and mirrors another portion of ram for some sort of speed increase.

First of all, does it swap out and stay swapped out, or does it swap in and out constantly?

What kernel version are you running?

The cache, or pagecache is at the core of Linux' memory management. When you start an application (or process) it first maps the file into the cache, and then executes it. So the cached value also counts the parts of the files used recently. It gets much more complicated, but you're invited to learn more about it. I'm sure someone else can give you a few URLs with more details.