Is anonymous memory part of the page cache on Linux?
Hello!
Hopefully I'm asking here in the right place and don't disturb.
Is anonymous memory - i.e. program heap and stack - part of the page cache on Linux? The documentation[1] of the kernel does not state that. But the Wikipedia entry about page cache contains a graphic[2] (look at the top right) which gives me the impression that 'malloc()' allocates dynamic memory within the page cache. Is it possible that the page cache serves as general foundation for memory management on Linux? Taking into account that mmap() also allows for anonymous mappings this could fit together?
I've asked this originally on Stackoverflow[3].
Thank you
Peter
[1] https://www.kernel.org/doc/html/latest/admin-guide/mm/concepts.html#anonymous-memory
[2] https://en.wikipedia.org/wiki/Page_cache#/media/File:The_Linux_Storage_Stack_Diagram.svg
[3] https://stackoverflow.com/q/66587345/1054324
Quoted 8 lines · Peter Weber
Hello!
Hopefully I'm asking here in the right place and don't disturb.Is anonymous memory - i.e. program heap and stack - part of the page cache on Linux? The documentation[1] of the kernel does not state that. But the Wikipedia entry about page cache contains a graphic[2] (look at the top right) which gives me the impression that '
malloc()' allocates dynamic memory within the page cache.
The wikipedia diagram is wrong. Anonymous memory is not handled by the page cache. Anonymous pages enter the storage stack via swap; they are found in the page tables, sent to the swap cache and then written to swap devices or swap files. Filesystems may get involved at that point, but not always.
There are other weird things in the wikipedia diagram, like Direct I/O being seemingly detached from applications, and not appearing to pass through the VFS.
Thank you Matthew!
Am 2021-03-12 16:15, schrieb Matthew Wilcox:
Quoted 2 lines
The wikipedia diagram is wrong. Anonymous memory is not handled by the page cache.
Is it roughly right to say, that the virtual memory uses page tables to handle anonymous memory?
Quoted 4 lines
Anonymous pages enter the storage stack via swap; they are found in the page tables, sent to the swap cache and then written to swap devices or swap files. Filesystems may get involved at that point, but not always.
And the page cache doesn't handle anonymous memory - even when it is swapped?
Quoted 3 lines
There are other weird things in the wikipedia diagram, like Direct I/O being seemingly detached from applications, and not appearing to pass through the VFS.
I'm not an expert. Maybe I should add at least note about that on Wikipedia?
Quoted 9 lines · Peter Weber
Thank you Matthew!
Am 2021-03-12 16:15, schrieb Matthew Wilcox:
Quoted 2 lines
The wikipedia diagram is wrong. Anonymous memory is not handled by the page cache.
Is it roughly right to say, that the virtual memory uses page tables to handle anonymous memory?
I tend to deal with page cache and not anonymous memory, so I'm not sure exactly how someone who's an expert would phrase it. Anonymous pages are "handled" in a number of different ways -- they can be found on LRU lists and they can be found through the page tables. It's all a bit ad-hoc as far as I can tell ;-)
Quoted 7 lines
Quoted 4 lines
Anonymous pages enter the storage stack via swap; they are found in the page tables, sent to the swap cache and then written to swap devices or swap files. Filesystems may get involved at that point, but not always.
And the page cache doesn't handle anonymous memory - even when it is swapped?
Right. There's a swap cache, but that's not the same thing as the page cache.
Quoted 5 lines
Quoted 3 lines
There are other weird things in the wikipedia diagram, like Direct I/O being seemingly detached from applications, and not appearing to pass through the VFS.
I'm not an expert. Maybe I should add at least note about that on Wikipedia?
Maybe!
Quoted 11 lines · Matthew Wilcox
Quoted 9 lines · Peter Weber
Thank you Matthew!
Am 2021-03-12 16:15, schrieb Matthew Wilcox:
Quoted 2 lines
The wikipedia diagram is wrong. Anonymous memory is not handled by the page cache.
Is it roughly right to say, that the virtual memory uses page tables to handle anonymous memory?
You'll have to distinguish between private and shared anonymous memory.
"private anonymous memory" -- mmap(MAP_ANONYMOUS | MAP_PRIVATE) -- is usually completely managed using the process page tables. The only way to get such pages mapped into another process is via fork(), whereby page table are copied and COW (Copy On Write) applies.
Once dereferenced from all page tables, there are no other references anymore; memory is handed back to the buddy as free memory, from where it can be reused for other purposes. Of course, there are case where there might be other references being taken (i.e., GUP, direct I/O ...), but let's ignore that for now.
The essence for "private anonymous memory" is, that there is no other way to get access to that memory besides the page tables. If you MADV_DONTNEED virtual memory regions to zap the page table entries and later reaccess the same virtual memory locations, always fresh memory will be populated.
"shared anonymous memory" -- mmap(MAP_ANONYMOUS | MAP_SHARED) -- is a little different and involves the page cache (it's pretty much shmem without a user-visible fd). The only way to get such pages mapped into another process is similarly via fork() (AFAIU), whereby page table are copied and COW does _not_ apply.
Once dereferenced from all page tables, there is still a reference to the page from the page cache. Only once evicted from the page cache, the memory can be freed up.
The essence for "shared anonymous memory" is, that there are ways to get access to that memory besides the page tables. If you MADV_DONTNEED such virtual memory locations to zap the page table entries and later reaccess the same virtual memory location again, the previous page will be re-instantiated from the page cache. To evict the pages from the page cache you would need MADV_REMOVE.
Signature · 3 lines
Thanks, David / dhildenb