mm: PG_reserved cleanups and documentation

The VDSO is part of the kernel image and therefore the struct pages are marked as reserved during boot.

As we install a special mapping, the actual struct pages will never be exposed to MM via the page tables. We can therefore leave the pages marked as reserved.

Cc Palmer Dabbelt <[email protected]>

Cc Albert Ou <[email protected]>

Cc Tobias Klauser <[email protected]>

Cc Andrew Morton <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Matthew Wilcox <[email protected]>

Signed-off-by David Hildenbrand <[email protected]>

---
 arch/riscv/kernel/vdso.c | 1 -
 1 file changed, 1 deletion(-)

arch/riscv/kernel/vdso.c +0 -0

index 582cb153eb24..0cd044122234 100644

This will already be done by free_reserved_page().

The usage of PG_reserved and how PG_reserved pages are to be treated is burried deep down in different parts of the kernel. Let's shine some light onto these details by documenting (most?) current users and expected behavior.

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

Cc Andrew Morton <[email protected]>

Cc Stephen Rothwell <[email protected]>

Cc Pavel Tatashin <[email protected]>

Cc Michal Hocko <[email protected]>

Cc Alexander Duyck <[email protected]>

Cc Matthew Wilcox <[email protected]>

Cc Anthony Yznaga <[email protected]>

Cc Miles Chen <[email protected]>

Cc [email protected]

Cc Dan Williams <[email protected]>

Signed-off-by David Hildenbrand <[email protected]>

---
 include/linux/page-flags.h | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

include/linux/page-flags.h +16 -2

index 68b8495e2fbc..112526f5ba61 100644
@@ -17,8 +17,22 @@
 /*
  * Various page->flags bits:
  *
- * PG_reserved is set for special pages, which can never be swapped out. Some
- * of them might not even exist...
+ * PG_reserved is set for special pages. The "struct page" of such a page
+ * should in general not be touched (e.g. set dirty) except by their owner.
+ * Pages marked as PG_reserved include:
+ * - Kernel image (including vDSO) and similar (e.g. BIOS, initrd)
+ * - Pages allocated early during boot (bootmem, memblock)
+ * - Zero pages
+ * - Pages that have been associated with a zone but are not available for
+ *   the page allocator (e.g. excluded via online_page_callback())
+ * - Pages to exclude from the hibernation image (e.g. loaded kexec images)
+ * - MMIO pages (communicate with a device, special caching strategy needed)
+ * - MCA pages on ia64 (pages with memory errors)
+ * - Device memory (e.g. PMEM, DAX, HMM)
+ * Some architectures don't allow to ioremap pages that are not marked
+ * PG_reserved (as they might be in use by somebody else who does not respect
+ * the caching strategy). Consequently, PG_reserved for a page mapped into
+ * user space can indicate the zero page, the vDSO, MMIO pages or device memory.
  *
  * The PG_private bitflag is set on pagecache pages if they contain filesystem
  * specific data (which is normally at page->private). It can be used by

Signature · 1 line

2.17.2

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

+++ b/include/linux/page-flags.h

include/linux/page-flags.h +6 -2

@@ -17,8 +17,22 @@
 /*
  * Various page->flags bits:
  *
- * PG_reserved is set for special pages, which can never be swapped out. Some
- * of them might not even exist...
+ * PG_reserved is set for special pages. The "struct page" of such a page
+ * should in general not be touched (e.g. set dirty) except by their owner.
+ * Pages marked as PG_reserved include:
+ * - Kernel image (including vDSO) and similar (e.g. BIOS, initrd)
+ * - Pages allocated early during boot (bootmem, memblock)
+ * - Zero pages

Quoted 5 lines · David Hildenbrand

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

I think you're confused by the meaning of "some of them might not even exist". What this means is that there might not be memory there; maybe writes to that memory will be discarded, or maybe they'll cause a machine check. Maybe reads will return ~0, or 0, or cause a machine check. We just don't know what's there, and we shouldn't try touching the memory.

Quoted 14 lines

+++ b/include/linux/page-flags.h

include/linux/page-flags.h +6 -2

@@ -17,8 +17,22 @@
 /*
  * Various page->flags bits:
  *
- * PG_reserved is set for special pages, which can never be swapped out. Some
- * of them might not even exist...
+ * PG_reserved is set for special pages. The "struct page" of such a page
+ * should in general not be touched (e.g. set dirty) except by their owner.
+ * Pages marked as PG_reserved include:
+ * - Kernel image (including vDSO) and similar (e.g. BIOS, initrd)
+ * - Pages allocated early during boot (bootmem, memblock)
+ * - Zero pages

include/linux/page-flags.h line 35

Quoted 10 lines

include/linux/page-flags.h +10 -0

+ * - Pages that have been associated with a zone but are not available for
+ *   the page allocator (e.g. excluded via online_page_callback())
+ * - Pages to exclude from the hibernation image (e.g. loaded kexec images)
+ * - MMIO pages (communicate with a device, special caching strategy needed)
+ * - MCA pages on ia64 (pages with memory errors)
+ * - Device memory (e.g. PMEM, DAX, HMM)
+ * Some architectures don't allow to ioremap pages that are not marked
+ * PG_reserved (as they might be in use by somebody else who does not respect
+ * the caching strategy). Consequently, PG_reserved for a page mapped into
+ * user space can indicate the zero page, the vDSO, MMIO pages or device memory.

So maybe just add one more option to the list.

Quoted 7 lines · Matthew Wilcox

Quoted 5 lines · David Hildenbrand

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

Quoted 5 lines

I think you're confused by the meaning of "some of them might not even exist". What this means is that there might not be memory there; maybe writes to that memory will be discarded, or maybe they'll cause a machine check. Maybe reads will return ~0, or 0, or cause a machine check. We just don't know what's there, and we shouldn't try touching the memory.

If there are users, let's document it. And I need more details for that :)

1. machine check: if there is a HW error, we set PG_hwpoison (except ia64 MCA, see the list)

2. Writes to that memory will be discarded

Who is the user of that? When will we have such pages right now?

3. Reads will return ~0, / 0?

I think this is a special case of e.g. x86? But where do we have that, are there any user?

Quoted 19 lines · David Hildenbrand

Quoted 7 lines · Matthew Wilcox

Quoted 5 lines · David Hildenbrand

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

Quoted 5 lines

I think you're confused by the meaning of "some of them might not even exist". What this means is that there might not be memory there; maybe writes to that memory will be discarded, or maybe they'll cause a machine check. Maybe reads will return ~0, or 0, or cause a machine check. We just don't know what's there, and we shouldn't try touching the memory.

If there are users, let's document it. And I need more details for that :)

1. machine check: if there is a HW error, we set PG_hwpoison (except ia64 MCA, see the list)

Quoted 8 lines

2. Writes to that memory will be discarded

Who is the user of that? When will we have such pages right now?

3. Reads will return ~0, / 0?

I think this is a special case of e.g. x86? But where do we have that, are there any user?

When there are gaps in the physical memory. As in, if you put that physical address on the bus (or in a packet), no device will respond to it. Look:

00000000-00000fff : Reserved
00001000-00057fff : System RAM
00058000-00058fff : Reserved
00059000-0009dfff : System RAM
0009e000-000fffff : Reserved

Those examples I gave are examples of how various different architectures respond to "no device responded to this memory access".

Quoted 33 lines · Matthew Wilcox

Quoted 19 lines · David Hildenbrand

Quoted 7 lines · Matthew Wilcox

Quoted 5 lines · David Hildenbrand

I don't see a reason why we have to document "Some of them might not even exist". If there is a user, we should document it. E.g. for balloon drivers we now use PG_offline to indicate that a page might currently not be backed by memory in the hypervisor. And that is independent from PG_reserved.

Quoted 5 lines

I think you're confused by the meaning of "some of them might not even exist". What this means is that there might not be memory there; maybe writes to that memory will be discarded, or maybe they'll cause a machine check. Maybe reads will return ~0, or 0, or cause a machine check. We just don't know what's there, and we shouldn't try touching the memory.

If there are users, let's document it. And I need more details for that :)

1. machine check: if there is a HW error, we set PG_hwpoison (except ia64 MCA, see the list)

Quoted 8 lines

2. Writes to that memory will be discarded

Who is the user of that? When will we have such pages right now?

3. Reads will return ~0, / 0?

I think this is a special case of e.g. x86? But where do we have that, are there any user?

When there are gaps in the physical memory. As in, if you put that physical address on the bus (or in a packet), no device will respond to it. Look:

Quoted 9 lines

00000000-00000fff : Reserved
00001000-00057fff : System RAM
00058000-00058fff : Reserved
00059000-0009dfff : System RAM
0009e000-000fffff : Reserved

Those examples I gave are examples of how various different architectures respond to "no device responded to this memory access".

Okay, so for this memory we will have
a) vmmaps
b) Memory block devices
c) A sections that is online

So essentially "Gaps in physical memory" which is part of a online section.

This might be a candidate for PG_offline as well.

Thanks for the info, I'll try to find out how such things are handled. In general I assume this memory has to be readable, because otherwise kdump and friends would crash already when trying to dump?

Most notably, for device memory we can hopefully soon stop setting it PG_reserved

On Wed 05-12-18 13:28:44, David Hildenbrand wrote:
[...]

Quoted 2 lines

Most notably, for device memory we can hopefully soon stop setting it PG_reserved

I am busy as hell so I am not likely to look at specific patche this week. But could you be more specific why we need to get rid of other PG_reserved users before we can do so for device memory?

I am all for removing relicts because they just confuse people but I fail to see any relation here.