mm: Add PM_HUGE_THP_MAPPING to /proc/pid/pagemap
Subsystems (7)
PATCH v5 1/1 mm: Add PM_HUGE_THP_MAPPING to /proc/pid/pagemap
Add PM_HUGE_THP MAPPING to allow userspace to detect whether a given virt address is currently mapped by a transparent huge page or not. Example use case is a process requesting THPs from the kernel (via a huge tmpfs mount for example), for a performance critical region of memory. The userspace may want to query whether the kernel is actually backing this memory by hugepages or not.
PM_HUGE_THP_MAPPING bit is set if the virt address is mapped at the PMD level and the underlying page is a transparent huge page.
A few options were considered:
1. Add /proc/pid/pageflags that exports the same info as /proc/kpageflags. This is not appropriate because many kpageflags are inappropriate to expose to userspace processes.
2. Simply get this info from the existing /proc/pid/smaps interface. There are a couple of issues with that:
1. /proc/pid/smaps output is human readable and unfriendly to programatically parse.
2. /proc/pid/smaps is slow. The cost of reading /proc/pid/smaps into userspace buffers is about ~800us per call, and this doesn't include parsing the output to get the information you need. The cost of querying 1 virt address in /proc/pid/pagemaps however is around 5-7us.
Tested manually by adding logging into transhuge-stress, and by allocating THP and querying the PM_HUGE_THP_MAPPING flag at those virtual addresses.
---
Changes in v5:
- Added justification for this interface in the commit message!
Changes in v4:
- Removed unnecessary moving of flags variable declaration
Changes in v3:
- Renamed PM_THP to PM_HUGE_THP_MAPPING
- Fixed checks to set PM_HUGE_THP_MAPPING
- Added PM_HUGE_THP_MAPPING docs
--- Documentation/admin-guide/mm/pagemap.rst | 3 ++- fs/proc/task_mmu.c | 3 +++ tools/testing/selftests/vm/transhuge-stress.c | 21 +++++++++++++++---- 3 files changed, 22 insertions(+), 5 deletions(-)
Documentation/admin-guide/mm/pagemap.rst +2 -1
@@ -23,7 +23,8 @@ There are four components to pagemap: * Bit 56 page exclusively mapped (since 4.2) * Bit 57 pte is uffd-wp write-protected (since 5.13) (see :ref:`Documentation/admin-guide/mm/userfaultfd.rst <userfaultfd>`) - * Bits 57-60 zero + * Bit 58 page is a huge (PMD size) THP mapping + * Bits 59-60 zero * Bit 61 page is file-page or shared-anon (since 3.5) * Bit 62 page swapped * Bit 63 page present
fs/proc/task_mmu.c +3 -0
@@ -1302,6 +1302,7 @@ struct pagemapread {#define PM_SOFT_DIRTY BIT_ULL(55)#define PM_MMAP_EXCLUSIVE BIT_ULL(56)#define PM_UFFD_WP BIT_ULL(57) +#define PM_HUGE_THP_MAPPING BIT_ULL(58)#define PM_FILE BIT_ULL(61)#define PM_SWAP BIT_ULL(62)#define PM_PRESENT BIT_ULL(63)
@@ -1456,6 +1457,8 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,if (page &&page_mapcount (page) ==1 ) flags |=PM_MMAP_EXCLUSIVE ; +if (page &&is_transparent_hugepage (page)) + flags |=PM_HUGE_THP_MAPPING ;for (; addr != end; addr +=PAGE_SIZE ) {pagemap_entry_t pme =make_pme (frame, flags);
tools/testing/selftests/vm/transhuge-stress.c +17 -4
@@ -16,6 +16,12 @@#include <string.h> #include <sys/mman.h> +/* +* We can use /proc/pid/pagemap to detect whether the kernel was able to find +* hugepages or no. This can be very noisy, so is disabled by default. +*/ +#define NO_DETECT_HUGEPAGES +#define PAGE_SHIFT 12#define HPAGE_SHIFT 21
@@ -23,6 +29,7 @@#define HPAGE_SIZE (1 << HPAGE_SHIFT)#define PAGEMAP_PRESENT (ent) (((ent) & (1ull << 63)) != 0) +#define PAGEMAP_THP (ent) (((ent) & (1ull << 58)) != 0)#define PAGEMAP_PFN (ent) ((ent) & ((1ull << 55) - 1))int pagemap_fd;
@@ -47,10 +54,16 @@ int64_t allocate_transhuge(void *ptr) (uintptr_t)ptr >> (PAGE_SHIFT - 3)) != sizeof(ent))err (2 ,"read pagemap" ); -if (PAGEMAP_PRESENT (ent[0 ]) &&PAGEMAP_PRESENT (ent[1 ]) && -PAGEMAP_PFN (ent[0 ]) +1 ==PAGEMAP_PFN (ent[1 ]) && - !(PAGEMAP_PFN (ent[0 ]) & ((1 << (HPAGE_SHIFT -PAGE_SHIFT )) -1 ))) -return PAGEMAP_PFN (ent[0 ]); +if (PAGEMAP_PRESENT (ent[0 ]) &&PAGEMAP_PRESENT (ent[1 ])) { +#ifndef NO_DETECT_HUGEPAGES +if (!PAGEMAP_THP (ent[0 ])) +fprintf (stderr,"WARNING: detected non THP page\n" ); +#endif +if (PAGEMAP_PFN (ent[0 ]) +1 ==PAGEMAP_PFN (ent[1 ]) && + !(PAGEMAP_PFN (ent[0 ]) & + ((1 << (HPAGE_SHIFT -PAGE_SHIFT )) -1 ))) +return PAGEMAP_PFN (ent[0 ]); + }return -1 ; }
--
2.34.0.rc0.344.g81b53c2807-goog
Andrew Morton
maintainerQuoted 37 lines · On Wed, 10 Nov 2021 14:11:20 -0800 Mina Almasry <[email protected]> wrote:
Add
PM_HUGE_THPMAPPING to allow userspace to detect whether a given virt address is currently mapped by a transparent huge page or not. Example use case is a process requesting THPs from the kernel (via a huge tmpfs mount for example), for a performance critical region of memory. The userspace may want to query whether the kernel is actually backing this memory by hugepages or not.
PM_HUGE_THP_MAPPINGbit is set if the virt address is mapped at the PMD level and the underlying page is a transparent huge page.A few options were considered:
1. Add/proc/pid/pageflagsthat exports the same info as/proc/kpageflags. This is not appropriate because many kpageflags are inappropriate to expose to userspace processes.
2. Simply get this info from the existing/proc/pid/smapsinterface. There are a couple of issues with that:
1./proc/pid/smapsoutput is human readable and unfriendly to programatically parse.
2./proc/pid/smapsis slow. The cost of reading/proc/pid/smapsinto userspace buffers is about ~800us per call, and this doesn't include parsing the output to get the information you need. The cost of querying 1 virt address in/proc/pid/pagemapshowever is around 5-7us.Tested manually by adding logging into transhuge-stress, and by allocating THP and querying the
PM_HUGE_THP_MAPPINGflag at those virtual addresses.tools/testing/selftests/vm/transhuge-stress.c +3
-0@@ -16,6 +16,12 @@#include <string.h> #include <sys/mman.h> +/* +* We can use /proc/pid/pagemap to detect whether the kernel was able to find +* hugepages or no. This can be very noisy, so is disabled by default.
tools/testing/selftests/vm/transhuge-stress.c
Quoted 10 lines
tools/testing/selftests/vm/transhuge-stress.c +7
-0+*/ +#define NO_DETECT_HUGEPAGES + +#ifndef NO_DETECT_HUGEPAGES +if (!PAGEMAP_THP (ent[0 ])) +fprintf (stderr,"WARNING: detected non THP page\n" ); +#endif
This looks like a developer thing. Is there any point in leaving it in the mainline code?