mm: pvmw: check possible huge PMD map by transhuge_vma_suitable()
Subsystems (3)
PATCH v2 1/1 mm: pvmw: check possible huge PMD map by transhuge_vma_suitable()
IIUC PVMW checks if the vma is possibly huge PMD mapped by transparent_hugepage_active() and "pvmw->nr_pages >= HPAGE_PMD_NR".
Actually pvmw->nr_pages is returned by compound_nr() or folio_nr_pages(), so the page should be THP as long as "pvmw->nr_pages
Quoted 1 line
=
HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMA
in the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The transhuge_vma_suitable() is used to do such check, so replace transparent_hugepage_active() to it, which is too heavy and overkilling.
---
v2: * Fixed build error for !CONFIG_TRANSPARENT_HUGEPAGE
* Removed fixes tag per Willy
include/linux/huge_mm.h | 8 ++++++-- mm/page_vma_mapped.c | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-)
include/linux/huge_mm.h +6 -2
@@ -117,8 +117,10 @@ extern struct kobj_attribute shmem_enabled_attr;extern unsigned long transparent_hugepage_flags; static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) + unsigned long addr) { + unsigned long haddr; + /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
@@ -126,6 +128,8 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,return false; } + haddr = addr &HPAGE_PMD_MASK ; +if (haddr < vma->vm_start || haddr +HPAGE_PMD_SIZE > vma->vm_end)return false;return true;
@@ -328,7 +332,7 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) }static inline bool transhuge_vma_suitable (struct vm_area_struct *vma, -unsigned long haddr) +unsigned long addr) {return false; }
mm/page_vma_mapped.c +1 -1
@@ -243,7 +243,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount(). */ if ((pvmw->flags &PVMW_SYNC ) && -transparent_hugepage_active (vma) && +transhuge_vma_suitable (vma, pvmw->address) && (pvmw->nr_pages >=HPAGE_PMD_NR )) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd);
Signature · 1 line
2.26.3
Andrew Morton
maintainerQuoted 10 lines · On Fri, 13 May 2022 12:17:05 -0700 Yang Shi <[email protected]> wrote:
IIUC PVMW checks if the vma is possibly huge PMD mapped by
transparent_hugepage_active()and "pvmw->nr_pages>=HPAGE_PMD_NR".Actually
pvmw->nr_pagesis returned bycompound_nr()orfolio_nr_pages(), so the page should be THP as long as "pvmw->nr_pagesQuoted 1 line
=
HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMAin the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The
transhuge_vma_suitable()is used to do such check, so replacetransparent_hugepage_active()to it, which is too heavy and overkilling.
I messed with the changelog a bit. The function is called page_vma_mapped_walk(), so let's call it that.
This patch has been in the trees since May 12, which isn't terribly long. Does anyone feel up to a reviewed-by?
Thanks.
From: Yang Shi <[email protected]>
Subject: mm/page_vma_mapped.c: check possible huge PMD map with transhuge_vma_suitable()
Date: Fri, 13 May 2022 12:17:05 -0700
IIUC page_vma_mapped_walk() checks if the vma is possibly huge PMD mapped with transparent_hugepage_active() and "pvmw->nr_pages >= HPAGE_PMD_NR".
Actually pvmw->nr_pages is returned by compound_nr() or folio_nr_pages(), so the page should be THP as long as "pvmw->nr_pages >= HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMA in the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The transhuge_vma_suitable() is used to do such check, so replace transparent_hugepage_active() to it, which is too heavy and overkilling.
---
include/linux/huge_mm.h | 8 ++++++-- mm/page_vma_mapped.c | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-)
include/linux/huge_mm.h renamed from include/linux/huge_mm.h~mm-pvmw-check-possible-huge-pmd-map-by-transhuge_vma_suitable +6 -2
@@ -117,8 +117,10 @@ extern struct kobj_attribute shmem_enablextern unsigned long transparent_hugepage_flags; static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) + unsigned long addr) { + unsigned long haddr; + /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
@@ -126,6 +128,8 @@ static inline bool transhuge_vma_suitablreturn false; } + haddr = addr &HPAGE_PMD_MASK ; +if (haddr < vma->vm_start || haddr +HPAGE_PMD_SIZE > vma->vm_end)return false;return true;
@@ -342,7 +346,7 @@ static inline bool transparent_hugepage_ }static inline bool transhuge_vma_suitable (struct vm_area_struct *vma, -unsigned long haddr) +unsigned long addr) {return false; }
mm/page_vma_mapped.c renamed from mm/page_vma_mapped.c~mm-pvmw-check-possible-huge-pmd-map-by-transhuge_vma_suitable +1 -1
@@ -243,7 +243,7 @@ restart:* cleared *pmd but not decremented compound_mapcount(). */ if ((pvmw->flags &PVMW_SYNC ) && -transparent_hugepage_active (vma) && +transhuge_vma_suitable (vma, pvmw->address) && (pvmw->nr_pages >=HPAGE_PMD_NR )) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd);
_
Quoted 61 lines · Yang Shi
IIUC PVMW checks if the vma is possibly huge PMD mapped by
transparent_hugepage_active()and "pvmw->nr_pages>=HPAGE_PMD_NR".Actually
pvmw->nr_pagesis returned bycompound_nr()orfolio_nr_pages(), so the page should be THP as long as "pvmw->nr_pagesQuoted 1 line
=
HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMAin the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The
transhuge_vma_suitable()is used to do such check, so replacetransparent_hugepage_active()to it, which is too heavy and overkilling.Cc Matthew Wilcox (Oracle) <[email protected]>Cc Muchun Song <[email protected]>Signed-off-by Yang Shi <[email protected]>---
v2: * Fixed build error for !CONFIG_TRANSPARENT_HUGEPAGE
* Removed fixes tag per Willyinclude/linux/huge_mm.h | 8 ++++++-- mm/page_vma_mapped.c | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-)include/linux/huge_mm.h +6
-2@@ -117,8 +117,10 @@ extern struct kobj_attribute shmem_enabled_attr;extern unsigned long transparent_hugepage_flags; static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) + unsigned long addr) { + unsigned long haddr; + /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,@@ -126,6 +128,8 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,return false; } + haddr = addr &HPAGE_PMD_MASK ; +if (haddr < vma->vm_start || haddr +HPAGE_PMD_SIZE > vma->vm_end)return false;return true;@@ -328,7 +332,7 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) }static inline bool transhuge_vma_suitable (struct vm_area_struct *vma, -unsigned long haddr) +unsigned long addr) {return false; }mm/page_vma_mapped.c +0
-0
mm/page_vma_mapped.c bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) · line 246
Quoted 5 lines
mm/page_vma_mapped.c +1
-1@@ -243,7 +243,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount(). */ if ((pvmw->flags & PVMW_SYNC) && - transparent_hugepage_active(vma) && + transhuge_vma_suitable(vma, pvmw->address) &&
How about the following diff? Then we do not need to change transhuge_vma_suitable(). All the users of transhuge_vma_suitable() are already do the alignment by themselves.
Thanks.
mm/page_vma_mapped.c +2 -1
@@ -243,7 +243,8 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount(). */ if ((pvmw->flags &PVMW_SYNC ) && -transparent_hugepage_active (vma) && +IS_ENABLED (CONFIG_TRANSPARENT_HUGEPAGE ) && +transhuge_vma_suitable (vma, pvmw->address &HPAGE_PMD_MASK ) && (pvmw->nr_pages >=HPAGE_PMD_NR )) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd);
Quoted 7 lines
mm/page_vma_mapped.c +0
-0(pvmw->nr_pages >= HPAGE_PMD_NR)) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd);Signature · 1 line
2.26.3
Quoted 64 lines · Muchun Song
Quoted 62 lines · Yang Shi
IIUC PVMW checks if the vma is possibly huge PMD mapped by
transparent_hugepage_active()and "pvmw->nr_pages>=HPAGE_PMD_NR".Actually
pvmw->nr_pagesis returned bycompound_nr()orfolio_nr_pages(), so the page should be THP as long as "pvmw->nr_pagesQuoted 1 line
=
HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMAin the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The
transhuge_vma_suitable()is used to do such check, so replacetransparent_hugepage_active()to it, which is too heavy and overkilling.Cc Matthew Wilcox (Oracle) <[email protected]>Cc Muchun Song <[email protected]>Signed-off-by Yang Shi <[email protected]>---
v2: * Fixed build error for !CONFIG_TRANSPARENT_HUGEPAGE
* Removed fixes tag per Willyinclude/linux/huge_mm.h | 8 ++++++-- mm/page_vma_mapped.c | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-)include/linux/huge_mm.h +6
-2@@ -117,8 +117,10 @@ extern struct kobj_attribute shmem_enabled_attr;extern unsigned long transparent_hugepage_flags; static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) + unsigned long addr) { + unsigned long haddr; + /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,@@ -126,6 +128,8 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,return false; } + haddr = addr &HPAGE_PMD_MASK ; +if (haddr < vma->vm_start || haddr +HPAGE_PMD_SIZE > vma->vm_end)return false;return true;@@ -328,7 +332,7 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) }static inline bool transhuge_vma_suitable (struct vm_area_struct *vma, -unsigned long haddr) +unsigned long addr) {return false; }mm/page_vma_mapped.c +0
-0@@ -243,7 +243,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount().
Quoted 8 lines
mm/page_vma_mapped.c bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
Quoted 4 lines
mm/page_vma_mapped.c +1
-1*/ if ((pvmw->flags & PVMW_SYNC) && - transparent_hugepage_active(vma) && + transhuge_vma_suitable(vma, pvmw->address) &&How about the following diff? Then we do not need to change
transhuge_vma_suitable(). All the users oftranshuge_vma_suitable()are already do the alignment by themselves.
Thanks for the suggestion. But TBH I don't think this is a better way. I did think about this before proposing v2, but I don't prefer to pollute the code with IS_ENABLED(CONFIG_xxx) since the definition of transhuge_vma_suitable() is already protected by #ifdef. Rounding the address in transhuge_vma_suitable() seems neater and more readable to me IMHO.
Some callers of transhuge_vma_suitable() do round the address before calling it, but the rounded address is used by other codes in the callers too.
Quoted 24 lines
Thanks.
mm/page_vma_mapped.c +2
-1@@ -243,7 +243,8 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount(). */ if ((pvmw->flags &PVMW_SYNC ) && -transparent_hugepage_active (vma) && +IS_ENABLED (CONFIG_TRANSPARENT_HUGEPAGE ) && +transhuge_vma_suitable (vma, pvmw->address &HPAGE_PMD_MASK ) && (pvmw->nr_pages >=HPAGE_PMD_NR )) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd);Quoted 7 lines
mm/page_vma_mapped.c +0
-1(pvmw->nr_pages >= HPAGE_PMD_NR)) {spinlock_t *ptl =pmd_lock (mm, pvmw->pmd); --2.26.3
Muchun Song
Reviewed-byQuoted 81 lines · Yang Shi
Quoted 64 lines · Muchun Song
Quoted 62 lines · Yang Shi
IIUC PVMW checks if the vma is possibly huge PMD mapped by
transparent_hugepage_active()and "pvmw->nr_pages>=HPAGE_PMD_NR".Actually
pvmw->nr_pagesis returned bycompound_nr()orfolio_nr_pages(), so the page should be THP as long as "pvmw->nr_pagesQuoted 1 line
=
HPAGE_PMD_NR". And it is guaranteed THP is allocated for valid VMAin the first place. But it may be not PMD mapped if the VMA is file VMA and it is not properly aligned. The
transhuge_vma_suitable()is used to do such check, so replacetransparent_hugepage_active()to it, which is too heavy and overkilling.Cc Matthew Wilcox (Oracle) <[email protected]>Cc Muchun Song <[email protected]>Signed-off-by Yang Shi <[email protected]>---
v2: * Fixed build error for !CONFIG_TRANSPARENT_HUGEPAGE
* Removed fixes tag per Willyinclude/linux/huge_mm.h | 8 ++++++-- mm/page_vma_mapped.c | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-)include/linux/huge_mm.h +6
-2@@ -117,8 +117,10 @@ extern struct kobj_attribute shmem_enabled_attr;extern unsigned long transparent_hugepage_flags; static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) + unsigned long addr) { + unsigned long haddr; + /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,@@ -126,6 +128,8 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,return false; } + haddr = addr &HPAGE_PMD_MASK ; +if (haddr < vma->vm_start || haddr +HPAGE_PMD_SIZE > vma->vm_end)return false;return true;@@ -328,7 +332,7 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) }static inline bool transhuge_vma_suitable (struct vm_area_struct *vma, -unsigned long haddr) +unsigned long addr) {return false; }mm/page_vma_mapped.c +0
-0@@ -243,7 +243,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)* cleared *pmd but not decremented compound_mapcount(). Quoted 8 lines
mm/page_vma_mapped.c bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
Quoted 4 lines
mm/page_vma_mapped.c +1
-1*/ if ((pvmw->flags & PVMW_SYNC) && - transparent_hugepage_active(vma) && + transhuge_vma_suitable(vma, pvmw->address) &&How about the following diff? Then we do not need to change
transhuge_vma_suitable(). All the users oftranshuge_vma_suitable()are already do the alignment by themselves.Thanks for the suggestion. But TBH I don't think this is a better way. I did think about this before proposing v2, but I don't prefer to pollute the code with
IS_ENABLED(CONFIG_xxx)since the definition oftranshuge_vma_suitable()is already protected by #ifdef. Rounding the address intranshuge_vma_suitable()seems neater and more readable to me IMHO.
Quoted 4 lines
Some callers of
transhuge_vma_suitable()do round the address before calling it, but the rounded address is used by other codes in the callers too.
All right.
Thanks.