Disappearing RAM
There appears to be quite a difference between the memory usage reported by top
(immediately after boot and minus all user processes) and that reported by the kernel
during boot. Can anyone give me any hints as to why this might happen?
I am assuming that if I subtract all the memory in use by processes listed by top from all
the memory in use that this equals the amount of memory used by the kernel.
These numbers do not add up.
A related question: In the 2.2 kernels, does the kernel pre-allocate any amount of memory
for modules?
Or is everything I'm seeing related to VM bugs in 2.2.<19 ?
Signature · 3 lines
-o) j o h n e w e b e r / \ aspiring computer scientist & lover of pengiuns _\_v
Quoted 7 lines · John Weber
There appears to be quite a difference between the memory usage reported by top
(immediately after boot and minus all user processes) and that reported by the kernel
during boot. Can anyone give me any hints as to why this might happen?
Quoted 5 lines
I am assuming that if I subtract all the memory in use by processes listed by top from all
the memory in use that this equals the amount of memory used by the kernel.
These numbers do not add up.
The view given by top is inaccurate because it has no way to detect shared pages properly. This means for example that when you have glibc mapped into all your N processes it counted N times its size, even though it only exists once in memory. Worse it gets when you work with threaded programs; they share their memory completely, but top/ps has no way to detect this and counts the single shared process space for every process (I have an experimental patch to fix the later problem)
In addition there are quite a bit of internal data structures in the kernel which never appear in top, but eat memory. Most prominent of that is the page table itself, which eats a few percent of your ram for a management structure for every page (in 2.2 on 32bit ~40 bytes every 4K of memory; in 2.4 it is much worse)
Quoted 3 lines
A related question: In the 2.2 kernels, does the kernel pre-allocate any amount of memory
for modules?
No.
-Andi
Quoted 13 lines · John Weber
There appears to be quite a difference between the memory usage reported by top
(immediately after boot and minus all user processes) and that reported by the kernel
during boot. Can anyone give me any hints as to why this might happen?I am assuming that if I subtract all the memory in use by processes listed by top from all
the memory in use that this equals the amount of memory used by the kernel.
These numbers do not add up.
Quoted 5 lines
A related question: In the 2.2 kernels, does the kernel pre-allocate any amount of memory
for modules?Or is everything I'm seeing related to VM bugs in 2.2.<19 ?
Not VM bugs. There may be a bug (or two), but this is not it. Linux tries to use ALL the memory that it finds. This is to speed I/O and other operations. If you do ls -R /, you will see that a lot of memory is now allocated to the directory cache. Basically, any time you perform some I/O Linux tries to keep the result of the last I/O operation in memory just in case you need that data again.
In fact, when disk data are modified, the only time it's guaranteed to have been all written to the disk is when the disk is umounted during shutdown, unless the disk is mounted in a special way.
When processes need RAM, these caches are reused on, the last time I checked, a least-used basis. So, if a process needs memory, some pages from the cache that have not been used for awhile are torn down (possibly written, if modified) and its RAM is used for process RAM.
When modules are installed, the kernel dynamically allocates the RAM for the module. There are three main types of memory that the kernel can allocate; (1) normal virtual RAM; (2) non-paged virtual RAM (for stuff that must be present during an interrupt), and; (3) physical RAM (for DMA buffers). All such RAM is accessed by its virtual address in the kernel so, once allocated, there are no special coding provisions required. See /usr/include/linux/mm.h for details.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
"Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk.