Skip to contentSkip to conversation

mm: Add PM_THP to /proc/pid/pagemap

PATCH Superseded Mina Almasry 2 participants 4 messages Nov 4, 2021 – Nov 7, 2021

Subsystems (5)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm: Add PM_THP to /proc/pid/pagemap

Add PM_THP to allow userspace to detect whether a given virt address is currently mapped by a hugepage or not.

Example use case is a process requesting hugepages from the kernel (via a huge tmpfs mount for example), for a performance critical region of memory. The userspace may want to query whether the kernel is actually backing this memory by hugepages or not.

Tested manually by adding logging into transhuge-stress.

Cc David Rientjes [email protected]
Cc Paul E. McKenney <[email protected]>
Cc Yu Zhao <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Peter Xu <[email protected]>
Cc Ivan Teterevkov <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Florian Schmidt <[email protected]>
---
 fs/proc/task_mmu.c                            |  5 +++++
 tools/testing/selftests/vm/transhuge-stress.c | 21 +++++++++++++++----
 2 files changed, 22 insertions(+), 4 deletions(-)
fs/proc/task_mmu.c +5 -0
index ad667dbc96f5c..9847514937fc7 100644
@@ -1302,6 +1302,7 @@ struct pagemapread {
 #define PM_SOFT_DIRTY		BIT_ULL(55)
 #define PM_MMAP_EXCLUSIVE	BIT_ULL(56)
 #define PM_UFFD_WP		BIT_ULL(57)
+#define PM_THP			BIT_ULL(58)
 #define PM_FILE			BIT_ULL(61)
 #define PM_SWAP			BIT_ULL(62)
 #define PM_PRESENT		BIT_ULL(63)
@@ -1396,6 +1397,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm,
 		flags |= PM_FILE;
 	if (page && page_mapcount(page) == 1)
 		flags |= PM_MMAP_EXCLUSIVE;
+	if (page && PageTransCompound(page))
+		flags |= PM_THP;
 	if (vma->vm_flags & VM_SOFTDIRTY)
 		flags |= PM_SOFT_DIRTY;
 
@@ -1456,6 +1459,8 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,
 
 		if (page && page_mapcount(page) == 1)
 			flags |= PM_MMAP_EXCLUSIVE;
+		if (page && PageTransCompound(page))
+			flags |= PM_THP;
 
 		for (; addr != end; addr += PAGE_SIZE) {
 			pagemap_entry_t pme = make_pme(frame, flags);
tools/testing/selftests/vm/transhuge-stress.c +17 -4
index fd7f1b4a96f94..7dce18981fff5 100644
@@ -16,6 +16,12 @@
 #include <string.h>
 #include <sys/mman.h>
 
+/*
+ * We can use /proc/pid/pagemap to detect whether the kernel was able to find
+ * hugepages or no. This can be very noisy, so is disabled by default.
+ */
+#define NO_DETECT_HUGEPAGES
+
 #define PAGE_SHIFT 12
 #define HPAGE_SHIFT 21
 
@@ -23,6 +29,7 @@
 #define HPAGE_SIZE (1 << HPAGE_SHIFT)
 
 #define PAGEMAP_PRESENT(ent)	(((ent) & (1ull << 63)) != 0)
+#define PAGEMAP_THP(ent)	(((ent) & (1ull << 58)) != 0)
 #define PAGEMAP_PFN(ent)	((ent) & ((1ull << 55) - 1))
 
 int pagemap_fd;
@@ -47,10 +54,16 @@ int64_t allocate_transhuge(void *ptr)
 			(uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))
 		err(2, "read pagemap");
 
-	if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1]) &&
-	    PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
-	    !(PAGEMAP_PFN(ent[0]) & ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
-		return PAGEMAP_PFN(ent[0]);
+	if (PAGEMAP_PRESENT(ent[0]) && PAGEMAP_PRESENT(ent[1])) {
+#ifndef NO_DETECT_HUGEPAGES
+		if (!PAGEMAP_THP(ent[0]))
+			fprintf(stderr, "WARNING: detected non THP page\n");
+#endif
+		if (PAGEMAP_PFN(ent[0]) + 1 == PAGEMAP_PFN(ent[1]) &&
+		    !(PAGEMAP_PFN(ent[0]) &
+		      ((1 << (HPAGE_SHIFT - PAGE_SHIFT)) - 1)))
+			return PAGEMAP_PFN(ent[0]);
+	}
 
 	return -1;
 }

--
2.34.0.rc0.344.g81b53c2807-goog

#1 lore raw
Quoted 2 lines · Mina Almasry

Add PM_THP to allow userspace to detect whether a given virt address is currently mapped by a hugepage or not.

Well, no, that's not what that means.

fs/proc/task_mmu.c static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm, · line 1401

Quoted 6 lines
fs/proc/task_mmu.c +2 -0
@@ -1396,6 +1397,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm,
 		flags |= PM_FILE;
 	if (page && page_mapcount(page) == 1)
 		flags |= PM_MMAP_EXCLUSIVE;
+	if (page && PageTransCompound(page))
+		flags |= PM_THP;

All that PageTransCompound() does is call PageCompound(). It doesn't tell you if the underlying allocation is PMD sized, nor properly aligned.

And you didn't answer my question about whether you want information about whether a large page is being used that's not quite as large as a PMD.

#2 lore raw
Quoted 7 lines · Matthew Wilcox
Quoted 2 lines · Mina Almasry

Add PM_THP to allow userspace to detect whether a given virt address is currently mapped by a hugepage or not.

Well, no, that's not what that means.

Sorry, that was the intention, but I didn't implement the intention correctly.

Quoted 10 lines

fs/proc/task_mmu.c static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm, · line 1401

Quoted 6 lines
fs/proc/task_mmu.c +2 -0
@@ -1396,6 +1397,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm,
              flags |= PM_FILE;
      if (page && page_mapcount(page) == 1)
              flags |= PM_MMAP_EXCLUSIVE;
+     if (page && PageTransCompound(page))
+             flags |= PM_THP;

All that PageTransCompound() does is call PageCompound(). It doesn't tell you if the underlying allocation is PMD sized, nor properly aligned.

Quoted 3 lines

And you didn't answer my question about whether you want information about whether a large page is being used that's not quite as large as a PMD.

Sorry, I thought the implementation would make it clear but I didn't do that correctly. Right now and for the foreseeable future what I want to know is whether the page is mapped by a PMD. All the below work for me:

1. Flag is set if the page is either a PMD size THP page.
2. Flag is set if the page is either a PMD size THP page or PMD size hugetlbfs page.
3. Flag is set if the page is either a PMD size THP page or PMD size hugetlbfs page or contig PTE size hugetlbfs page.

I prefer #2 and I think it's maybe most extensible for future use cases that 1 flag tells whether the page is PMD hugepage and another flag is a large cont PTE page.

#3 lore raw
Quoted 22 lines · Mina Almasry
Quoted 7 lines · Matthew Wilcox
Quoted 2 lines · Mina Almasry

Add PM_THP to allow userspace to detect whether a given virt address is currently mapped by a hugepage or not.

Well, no, that's not what that means.

Sorry, that was the intention, but I didn't implement the intention correctly.

Quoted 10 lines

fs/proc/task_mmu.c static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm, · line 1401

Quoted 6 lines
fs/proc/task_mmu.c +2 -0
@@ -1396,6 +1397,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pagemapread *pm,
              flags |= PM_FILE;
      if (page && page_mapcount(page) == 1)
              flags |= PM_MMAP_EXCLUSIVE;
+     if (page && PageTransCompound(page))
+             flags |= PM_THP;

All that PageTransCompound() does is call PageCompound(). It doesn't tell you if the underlying allocation is PMD sized, nor properly aligned.

Sorry Matthew again for getting this check wrong. After taking a deeper look, you're completely correct. My check was returning true on all compound pages without regard to whether they are actually THP, or whether they're mapped at the PMD level.

I've renamed the flag from PM_THP to PM_HUGE_THP_MAPPING to be more accurate, and it looks to me like the correct check is if we're in pagemap_pmd_range() and the underlying page is_transparent_huegpage(), then we set the flag.

I'm about to upload v3 with this new check; please take another look. Thank you for catching this.

Quoted 18 lines
Quoted 3 lines

And you didn't answer my question about whether you want information about whether a large page is being used that's not quite as large as a PMD.

Sorry, I thought the implementation would make it clear but I didn't do that correctly. Right now and for the foreseeable future what I want to know is whether the page is mapped by a PMD. All the below work for me:

1. Flag is set if the page is either a PMD size THP page.
2. Flag is set if the page is either a PMD size THP page or PMD size hugetlbfs page.
3. Flag is set if the page is either a PMD size THP page or PMD size hugetlbfs page or contig PTE size hugetlbfs page.

I prefer #2 and I think it's maybe most extensible for future use cases that 1 flag tells whether the page is PMD hugepage and another flag is a large cont PTE page.