mm: thp: use generic THP migration for NUMA hinting fault
Subsystems (5)
v3 cover mm: thp: use generic THP migration for NUMA hinting fault
Changelog
---------
v2 --> v3:
* Made orig_pte and orig_pmd a union per Mel (patch 1/7).
* Renamed pmd and oldpmd in patch 3/7 per Huang Ying.
* Used total_mapcount() instead of page_mapcount() in patch 6/7.
* Collected ack tags from Mel.
* Rebased to linux-next-20210513.
v1 --> v2:
* Adopted the suggestion from Gerald Schaefer to skip huge PMD for S390 for now.
* Used PageTransHuge to distinguish base page or THP instead of a new parameter for migrate_misplaced_page() per Huang Ying.
* Restored PMD lazily to avoid unnecessary TLB shootdown per Huang Ying.
* Skipped shared THP.
* Updated counters correctly.
* Rebased to linux-next (next-20210412).
When the THP NUMA fault support was added THP migration was not supported yet. So the ad hoc THP migration was implemented in NUMA fault handling. Since v4.14 THP migration has been supported so it doesn't make too much sense to still keep another THP migration implementation rather than using the generic migration code. It is definitely a maintenance burden to keep two THP migration implementation for different code paths and it is more error prone. Using the generic THP migration implementation allows us remove the duplicate code and some hacks needed by the old ad hoc implementation.
A quick grep shows x86_64, PowerPC (book3s), ARM64 ans S390 support both THP and NUMA balancing. The most of them support THP migration except for S390. Zi Yan tried to add THP migration support for S390 before but it was not accepted due to the design of S390 PMD. For the discussion, please see:
https://lkml.org/lkml/2018/4/27/953.
Per the discussion with Gerald Schaefer in v1 it is acceptible to skip huge PMD for S390 for now.
I saw there were some hacks about gup from git history, but I didn't figure out if they have been removed or not since I just found FOLL_NUMA code in the current gup implementation and they seems useful.
Patch #1 ~ #2 are preparation patches.
Patch #3 is the real meat.
Patch #4 ~ #6 keep consistent counters and behaviors with before. Patch #7 skips change huge PMD to prot_none if thp migration is not supported.
Test
----
Did some tests to measure the latency of do_huge_pmd_numa_page. The test VM has 80 vcpus and 64G memory. The test would create 2 processes to consume 128G memory together which would incur memory pressure to cause THP splits. And it also creates 80 processes to hog cpu, and the memory consumer processes are bound to different nodes periodically in order to increase NUMA faults.
The below test script is used:
echo 3 > /proc/sys/vm/drop_caches
# Run stress-ng for 24 hours
./stress-ng/stress-ng --vm 2 --vm-bytes 64G --timeout 24h & PID=$!
./stress-ng/stress-ng --cpu $NR_CPUS --timeout 24h &
# Wait for vm stressors forked
sleep 5
PID_1=pgrep -P $PID | awk 'NR == 1'
PID_2=pgrep -P $PID | awk 'NR == 2'
JOB1=pgrep -P $PID_1
JOB2=pgrep -P $PID_2
# Bind load jobs to different nodes periodically to force generate
# cross node memory access
while [ -d "/proc/$PID" ]
do
taskset -apc 8 $JOB1
taskset -apc 8 $JOB2
sleep 300
taskset -apc 58 $JOB1
taskset -apc 58 $JOB2
sleep 300
doneWith the above test the histogram of latency of do_huge_pmd_numa_page is as shown below. Since the number of do_huge_pmd_numa_page varies drastically for each run (should be due to scheduler), so I converted the raw number to percentage.
patched base
@us[stress-ng]:
[0] 3.57% 0.16% [1] 55.68% 18.36% [2, 4) 10.46% 40.44% [4, 8) 7.26% 17.82% [8, 16) 21.12% 13.41% [16, 32) 1.06% 4.27% [32, 64) 0.56% 4.07% [64, 128) 0.16% 0.35% [128, 256) < 0.1% < 0.1% [256, 512) < 0.1% < 0.1% [512, 1K) < 0.1% < 0.1% [1K, 2K) < 0.1% < 0.1% [2K, 4K) < 0.1% < 0.1% [4K, 8K) < 0.1% < 0.1% [8K, 16K) < 0.1% < 0.1% [16K, 32K) < 0.1% < 0.1% [32K, 64K) < 0.1% < 0.1%
Per the result, patched kernel is even slightly better than the base kernel. I think this is because the lock contention against THP split is less than base kernel due to the refactor.
To exclude the affect from THP split, I also did test w/o memory pressure. No obvious regression is spotted. The below is the test result *w/o* memory pressure.
patched base
@us[stress-ng]:
[0] 7.97% 18.4% [1] 69.63% 58.24% [2, 4) 4.18% 2.63% [4, 8) 0.22% 0.17% [8, 16) 1.03% 0.92% [16, 32) 0.14% < 0.1% [32, 64) < 0.1% < 0.1% [64, 128) < 0.1% < 0.1% [128, 256) < 0.1% < 0.1% [256, 512) 0.45% 1.19% [512, 1K) 15.45% 17.27% [1K, 2K) < 0.1% < 0.1% [2K, 4K) < 0.1% < 0.1% [4K, 8K) < 0.1% < 0.1% [8K, 16K) 0.86% 0.88% [16K, 32K) < 0.1% 0.15% [32K, 64K) < 0.1% < 0.1% [64K, 128K) < 0.1% < 0.1% [128K, 256K) < 0.1% < 0.1%
The series also survived a series of tests that exercise NUMA balancing migrations by Mel.
Yang Shi (7):
mm: memory: add orig_pmd to struct vm_fault
mm: memory: make numa_migrate_prep() non-static
mm: thp: refactor NUMA fault handling
mm: migrate: account THP NUMA migration counters correctly
mm: migrate: don't split THP for misplaced NUMA page
mm: migrate: check mapcount for THP instead of ref count
mm: thp: skip make PMD PROT_NONE if THP migration is not supported
include/linux/huge_mm.h | 9 ++--- include/linux/migrate.h | 23 ----------- include/linux/mm.h | 3 ++ mm/huge_memory.c | 156 +++++++++++++++++++++++++----------------------------------------------- mm/internal.h | 21 ++-------- mm/memory.c | 31 +++++++-------- mm/migrate.c | 204 +++++++++++++++++++++-------------------------------------------------------------------------- 7 files changed, 123 insertions(+), 324 deletions(-)
PATCH v3 1/7 mm: memory: add orig_pmd to struct vm_fault
Add orig_pmd to struct vm_fault so the "orig_pmd" parameter used by huge page fault could be removed, just like its PTE counterpart does.
--- include/linux/huge_mm.h | 9 ++++----- include/linux/mm.h | 7 ++++++- mm/huge_memory.c | 9 ++++++--- mm/memory.c | 26 +++++++++++++------------- 4 files changed, 29 insertions(+), 22 deletions(-)
include/linux/huge_mm.h +4 -5
@@ -11,7 +11,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf);int copy_huge_pmd (struct mm_struct *dst_mm,struct mm_struct *src_mm,pmd_t *dst_pmd,pmd_t *src_pmd,unsigned long addr,struct vm_area_struct *dst_vma,struct vm_area_struct *src_vma); -void huge_pmd_set_accessed (struct vm_fault *vmf,pmd_t orig_pmd); +void huge_pmd_set_accessed (struct vm_fault *vmf);int copy_huge_pud (struct mm_struct *dst_mm,struct mm_struct *src_mm,pud_t *dst_pud,pud_t *src_pud,unsigned long addr,struct vm_area_struct *vma);
@@ -24,7 +24,7 @@ static inline void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud) }#endif -vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf,pmd_t orig_pmd); +vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf);struct page *follow_trans_huge_pmd (struct vm_area_struct *vma,unsigned long addr,pmd_t *pmd,unsigned int flags);
@@ -283,7 +283,7 @@ struct page *follow_devmap_pmd(struct vm_area_struct *vma, unsigned long addr,struct page *follow_devmap_pud (struct vm_area_struct *vma,unsigned long addr,pud_t *pud,int flags,struct dev_pagemap **pgmap); -vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf,pmd_t orig_pmd); +vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf);extern struct page *huge_zero_page;
@@ -429,8 +429,7 @@ static inline spinlock_t *pud_trans_huge_lock(pud_t *pud,return NULL ; } -static inline vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf, -pmd_t orig_pmd) +static inline vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf) {return 0 ; }
include/linux/mm.h +6 -1
@@ -550,7 +550,12 @@ struct vm_fault {pud_t *pud;/* Pointer to pud entry matching * the 'address' */ -pte_t orig_pte;/* Value of PTE at the time of fault */ +union { +pte_t orig_pte;/* Value of PTE at the time of fault */ +pmd_t orig_pmd;/* Value of PMD at the time of fault, +* used by PMD fault only. +*/ + };struct page *cow_page;/* Page handler may use for COW fault */ struct page *page;/* ->fault handlers should return a
mm/huge_memory.c +6 -3
@@ -1245,11 +1245,12 @@ void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud) }#endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ -void huge_pmd_set_accessed (struct vm_fault *vmf,pmd_t orig_pmd) +void huge_pmd_set_accessed (struct vm_fault *vmf) {pmd_t entry;unsigned long haddr;bool write = vmf->flags &FAULT_FLAG_WRITE ; +pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl =pmd_lock (vmf->vma->vm_mm, vmf->pmd);if (unlikely (!pmd_same (*vmf->pmd, orig_pmd)))
@@ -1266,11 +1267,12 @@ void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd)spin_unlock (vmf->ptl); } -vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf,pmd_t orig_pmd) +vm_fault_t do_huge_pmd_wp_page (struct vm_fault *vmf) {struct vm_area_struct *vma = vmf->vma;struct page *page;unsigned long haddr = vmf->address &HPAGE_PMD_MASK ; +pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl =pmd_lockptr (vma->vm_mm, vmf->pmd);VM_BUG_ON_VMA (!vma->anon_vma, vma);
@@ -1406,9 +1408,10 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, }/* NUMA hinting page fault entry point for trans huge pmds */ -vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf,pmd_t pmd) +vm_fault_t do_huge_pmd_numa_page (struct vm_fault *vmf) {struct vm_area_struct *vma = vmf->vma; +pmd_t pmd = vmf->orig_pmd;struct anon_vma *anon_vma =NULL ;struct page *page;unsigned long haddr = vmf->address &HPAGE_PMD_MASK ;
mm/memory.c +13 -13
@@ -4252,12 +4252,12 @@ static inline vm_fault_t create_huge_pmd(struct vm_fault *vmf) }/* `inline' is required to avoid gcc 4.1.2 build error */ -static inline vm_fault_t wp_huge_pmd (struct vm_fault *vmf,pmd_t orig_pmd) +static inline vm_fault_t wp_huge_pmd (struct vm_fault *vmf) {if (vma_is_anonymous (vmf->vma)) { -if (userfaultfd_huge_pmd_wp (vmf->vma, orig_pmd)) +if (userfaultfd_huge_pmd_wp (vmf->vma, vmf->orig_pmd))return handle_userfault (vmf,VM_UFFD_WP ); -return do_huge_pmd_wp_page (vmf, orig_pmd); +return do_huge_pmd_wp_page (vmf); }if (vmf->vma->vm_ops->huge_fault) {vm_fault_t ret = vmf->vma->vm_ops->huge_fault (vmf,PE_SIZE_PMD );
@@ -4484,26 +4484,26 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma,if (!(ret &VM_FAULT_FALLBACK ))return ret; }else { -pmd_t orig_pmd = *vmf.pmd; + vmf.orig_pmd = *vmf.pmd;barrier (); -if (unlikely (is_swap_pmd (orig_pmd))) { +if (unlikely (is_swap_pmd (vmf.orig_pmd))) {VM_BUG_ON (thp_migration_supported () && - !is_pmd_migration_entry (orig_pmd)); -if (is_pmd_migration_entry (orig_pmd)) + !is_pmd_migration_entry (vmf.orig_pmd)); +if (is_pmd_migration_entry (vmf.orig_pmd))pmd_migration_entry_wait (mm, vmf.pmd);return 0 ; } -if (pmd_trans_huge (orig_pmd) ||pmd_devmap (orig_pmd)) { -if (pmd_protnone (orig_pmd) &&vma_is_accessible (vma)) -return do_huge_pmd_numa_page (&vmf, orig_pmd); +if (pmd_trans_huge (vmf.orig_pmd) ||pmd_devmap (vmf.orig_pmd)) { +if (pmd_protnone (vmf.orig_pmd) &&vma_is_accessible (vma)) +return do_huge_pmd_numa_page (&vmf); -if (dirty && !pmd_write (orig_pmd)) { - ret =wp_huge_pmd (&vmf, orig_pmd); +if (dirty && !pmd_write (vmf.orig_pmd)) { + ret =wp_huge_pmd (&vmf);if (!(ret &VM_FAULT_FALLBACK ))return ret; }else { -huge_pmd_set_accessed (&vmf, orig_pmd); +huge_pmd_set_accessed (&vmf);return 0 ; } }
Signature · 1 line
2.26.2
Re: [v3 PATCH 1/7] mm: memory: add orig_pmd to struct vm_fault
Quoted 4 lines · Yang Shi
Add
orig_pmdto structvm_faultso the "orig_pmd" parameter used by huge page fault could be removed, just like its PTE counterpart does.Signed-off-by Yang Shi <[email protected]>
Signature · 2 lines
Mel Gorman SUSE Labs
PATCH v3 2/7 mm: memory: make numa_migrate_prep() non-static
In reply to Yang ShiThe numa_migrate_prep() will be used by huge NUMA fault as well in the following patch, make it non-static.
--- mm/internal.h | 3 +++ mm/memory.c | 5 ++--- 2 files changed, 5 insertions(+), 3 deletions(-)
mm/internal.h +3 -0
@@ -653,4 +653,7 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end,void vunmap_range_noflush (unsigned long start,unsigned long end); +int numa_migrate_prep (struct page *page,struct vm_area_struct *vma, +unsigned long addr,int page_nid,int *flags); +#endif /* __MM_INTERNAL_H */
mm/memory.c +2 -3
@@ -4129,9 +4129,8 @@ static vm_fault_t do_fault(struct vm_fault *vmf)return ret; } -static int numa_migrate_prep (struct page *page,struct vm_area_struct *vma, -unsigned long addr,int page_nid, -int *flags) +int numa_migrate_prep (struct page *page,struct vm_area_struct *vma, +unsigned long addr,int page_nid,int *flags) {get_page (page);
Signature · 1 line
2.26.2
PATCH v3 3/7 mm: thp: refactor NUMA fault handling
In reply to Yang ShiWhen the THP NUMA fault support was added THP migration was not supported yet. So the ad hoc THP migration was implemented in NUMA fault handling. Since v4.14 THP migration has been supported so it doesn't make too much sense to still keep another THP migration implementation rather than using the generic migration code.
This patch reworked the NUMA fault handling to use generic migration implementation to migrate misplaced page. There is no functional change.
After the refactor the flow of NUMA fault handling looks just like its PTE counterpart:
Acquire ptl
Prepare for migration (elevate page refcount)
Release ptl
Isolate page from lru and elevate page refcount
Migrate the misplaced THP
If migration is failed just restore the old normal PMD.
In the old code anon_vma lock was needed to serialize THP migration against THP split, but since then the THP code has been reworked a lot, it seems anon_vma lock is not required anymore to avoid the race.
The page refcount elevation when holding ptl should prevent from THP split.
Use migrate_misplaced_page() for both base page and THP NUMA hinting fault and remove all the dead and duplicate code.
--- include/linux/migrate.h | 23 ------ mm/huge_memory.c | 147 +++++++++++---------------------- mm/internal.h | 18 ---- mm/migrate.c | 177 ++++++++-------------------------------- 4 files changed, 79 insertions(+), 286 deletions(-)
include/linux/migrate.h +0 -23
@@ -95,14 +95,9 @@ static inline void __ClearPageMovable(struct page *page)#endif #ifdef CONFIG_NUMA_BALANCING -extern bool pmd_trans_migrating (pmd_t pmd);extern int migrate_misplaced_page (struct page *page,struct vm_area_struct *vma,int node);#else -static inline bool pmd_trans_migrating (pmd_t pmd) -{ -return false; -}static inline int migrate_misplaced_page (struct page *page,struct vm_area_struct *vma,int node) {
@@ -110,24 +105,6 @@ static inline int migrate_misplaced_page(struct page *page, }#endif /* CONFIG_NUMA_BALANCING */ -#if defined(CONFIG_NUMA_BALANCING ) && defined(CONFIG_TRANSPARENT_HUGEPAGE ) -extern int migrate_misplaced_transhuge_page (struct mm_struct *mm, -struct vm_area_struct *vma, -pmd_t *pmd,pmd_t entry, -unsigned long address, -struct page *page,int node); -#else -static inline int migrate_misplaced_transhuge_page (struct mm_struct *mm, -struct vm_area_struct *vma, -pmd_t *pmd,pmd_t entry, -unsigned long address, -struct page *page,int node) -{ -return -EAGAIN ; -} -#endif /* CONFIG_NUMA_BALANCING && CONFIG_TRANSPARENT_HUGEPAGE*/ - -#ifdef CONFIG_MIGRATION /*
mm/huge_memory.c +46 -101
183 lines not on this page · raw message
mm/internal.h +0 -18
@@ -375,23 +375,6 @@ extern int mlock_future_check(struct mm_struct *mm, unsigned long flags,*/ extern void clear_page_mlock (struct page *page); -/* -* mlock_migrate_page - called only from migrate_misplaced_transhuge_page() -* (because that does not go through the full procedure of migration ptes): -* to migrate the Mlocked page flag; update statistics. -*/ -static inline void mlock_migrate_page (struct page *newpage,struct page *page) -{ -if (TestClearPageMlocked (page)) { -int nr_pages =thp_nr_pages (page); - -/* Holding pmd lock, no change in irq context: __mod is safe */ -__mod_zone_page_state (page_zone (page),NR_MLOCK , -nr_pages); -SetPageMlocked (newpage); -__mod_zone_page_state (page_zone (newpage),NR_MLOCK , nr_pages); - } -} -extern pmd_t maybe_pmd_mkwrite (pmd_t pmd,struct vm_area_struct *vma);/*
@@ -442,7 +425,6 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,#else /* !CONFIG_MMU */ static inline void clear_page_mlock (struct page *page) { }static inline void mlock_vma_page (struct page *page) { } -static inline void mlock_migrate_page (struct page *new,struct page *old) { }static inline void vunmap_range_noflush (unsigned long start,unsigned long end) { }
mm/migrate.c +33 -144
207 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH v3 4/7 mm: migrate: account THP NUMA migration counters correctly
In reply to Yang ShiNow both base page and THP NUMA migration is done via migrate_misplaced_page(), keep the counters correctly for THP.
--- mm/migrate.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
mm/migrate.c +4 -3
@@ -2114,6 +2114,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,LIST_HEAD (migratepages);new_page_t *new;bool compound; +unsigned int nr_pages =thp_nr_pages (page);/* * PTE mapped THP or HugeTLB page can't reach here so the page could
@@ -2152,13 +2153,13 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,if (nr_remaining) {if (!list_empty (&migratepages)) {list_del (&page->lru); -dec_node_page_state (page,NR_ISOLATED_ANON + -page_is_file_lru (page)); +mod_node_page_state (page_pgdat (page),NR_ISOLATED_ANON + +page_is_file_lru (page), -nr_pages);putback_lru_page (page); } isolated =0 ; }else -count_vm_numa_event (NUMA_PAGE_MIGRATE ); +count_vm_numa_events (NUMA_PAGE_MIGRATE , nr_pages);BUG_ON (!list_empty (&migratepages));return isolated;
Signature · 1 line
2.26.2
PATCH v3 5/7 mm: migrate: don't split THP for misplaced NUMA page
In reply to Yang ShiThe old behavior didn't split THP if migration is failed due to lack of memory on the target node. But the THP migration does split THP, so keep the old behavior for misplaced NUMA page migration.
--- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
mm/migrate.c +3 -1
@@ -1420,6 +1420,7 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,int swapwrite = current->flags &PF_SWAPWRITE ;int rc, nr_subpages;LIST_HEAD (ret_pages); +bool nosplit = (reason ==MR_NUMA_MISPLACED );trace_mm_migrate_pages_start (mode, reason);
@@ -1491,8 +1492,9 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,/* * When memory is low, don't bother to try to migrate * other pages, just exit. +* THP NUMA faulting doesn't split THP to retry. */ -if (is_thp) { +if (is_thp && !nosplit) {if (!try_split_thp (page, &page2, from)) { nr_thp_split++; goto retry;
Signature · 1 line
2.26.2
PATCH v3 6/7 mm: migrate: check mapcount for THP instead of refcount
In reply to Yang ShiThe generic migration path will check refcount, so no need check refcount here. But the old code actually prevents from migrating shared THP (mapped by multiple processes), so bail out early if mapcount is > 1 to keep the behavior.
--- mm/migrate.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
mm/migrate.c +4 -12
@@ -2070,6 +2070,10 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)VM_BUG_ON_PAGE (compound_order (page) && !PageTransHuge (page), page); +/* Do not migrate THP mapped by multiple processes */ +if (PageTransHuge (page) &&total_mapcount (page) >1 ) +return 0 ; +/* Avoid migrating to a node that is nearly full */ if (!migrate_balanced_pgdat (pgdat,compound_nr (page)))return 0 ;
@@ -2077,18 +2081,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)if (isolate_lru_page (page))return 0 ; -/* -* migrate_misplaced_transhuge_page() skips page migration's usual -* check on page_count(), so we must do it here, now that the page -* has been isolated: a GUP pin, or any other pin, prevents migration. -* The expected page count is 3: 1 for page's mapcount and 1 for the -* caller's pin and 1 for the reference taken by isolate_lru_page(). -*/ -if (PageTransHuge (page) &&page_count (page) !=3 ) { -putback_lru_page (page); -return 0 ; - } - page_lru =page_is_file_lru (page);mod_node_page_state (page_pgdat (page),NR_ISOLATED_ANON + page_lru,thp_nr_pages (page));
Signature · 1 line
2.26.2
PATCH v3 7/7 mm: thp: skip make PMD PROT_NONE if THP migration is not supported
In reply to Yang ShiA quick grep shows x86_64, PowerPC (book3s), ARM64 and S390 support both NUMA balancing and THP. But S390 doesn't support THP migration so NUMA balancing actually can't migrate any misplaced pages.
Skip make PMD PROT_NONE for such case otherwise CPU cycles may be wasted by pointless NUMA hinting faults on S390.
--- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+)
mm/huge_memory.c +4 -0
@@ -1735,6 +1735,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,* Returns * - 0 if PMD could not be locked * - 1 if PMD was locked but protections unchanged and TLB flush unnecessary +* or if prot_numa but THP migration is not supported * - HPAGE_PMD_NR if protections changed and TLB flush necessary */ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,
@@ -1749,6 +1750,9 @@ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd,bool uffd_wp = cp_flags &MM_CP_UFFD_WP ;bool uffd_wp_resolve = cp_flags &MM_CP_UFFD_WP_RESOLVE ; +if (prot_numa && !thp_migration_supported ()) +return 1 ; + ptl =__pmd_trans_huge_lock (pmd, vma);if (!ptl)return 0 ;
Signature · 1 line
2.26.2