Fix incorrect access of hugetlb pte entries
Subsystems (7)
v2 cover Fix incorrect access of hugetlb pte entries
There are various places which use ptep_get() to get the pte entry corresponding to a hugetlb folio. Some arches (like s390) have special handling to compute the pteval, so they provide huge_ptep_get(). Use this helper consistently.
Additionally, some code paths may provide huge_ptep_get with an unaligned address. This is a problem on arm64 (I checked other arches and it looks fine for them), which is fixed in patch 1. The fix is made to be
backport-friendly: the cleaner fix would be to perhaps pass the hstate to huge_ptep_get() - that is wider churn and we can do that later.
---v1->v2:
- Add patch 1
- Mention s390 in cover letter
Dev Jain (6):
arm64: make huge_ptep_get handled unaligned addresses mm/rmap: use huge_ptep_get() in try_to_unmap_one() mm/rmap: use huge_ptep_get() in try_to_migrate_one() mm/migrate: use huge_ptep_get() in remove_migration_pte() mm/page_vma_mapped: use huge_ptep_get() for hugetlb mm/mprotect: use huge_ptep_get() for hugetlb
arch/arm64/mm/hugetlbpage.c | 2 +- include/linux/hugetlb.h | 3 +++ mm/migrate.c | 6 +++++- mm/mprotect.c | 8 +++++++- mm/page_vma_mapped.c | 8 +++++++- mm/rmap.c | 32 ++++++++++++++++++++------------ 6 files changed, 43 insertions(+), 16 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/6 arm64: make huge_ptep_get handled unaligned addresses
huge_ptep_get() can be handed a virtual address pointing to the middle of a contpmd/contpte mapped hugetlb folio (examples of callers are pagemap_hugetlb_range, page_mapped_in_vma).
The arm64 helper rewalks the pgtables in find_num_contig to answer whether the huge pte we have maps a contpmd or a contpte hugetlb folio, and returns CONT_PMDS or CONT_PTES, so that it can collect a/d bits over the contiguous ptes. We can falsely return CONT_PTES instead of CONT_PMDS if the addr is not aligned.
Fix this by aligning the pmdp pointer down to a contpmd base before checking equality with the passed huge pte pointer, to correctly answer whether the huge pte is the base of a contpmd block.
--- arch/arm64/mm/hugetlbpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/mm/hugetlbpage.c +1 -1
@@ -87,7 +87,7 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, p4dp =p4d_offset (pgdp, addr); pudp =pud_offset (p4dp, addr); pmdp =pmd_offset (pudp, addr); -if ((pte_t *)pmdp == ptep) { +if ((pte_t *)PTR_ALIGN_DOWN (pmdp,sizeof (*pmdp) *CONT_PMDS ) == ptep) { *pgsize =PMD_SIZE ;return CONT_PMDS ; }
Signature · 1 line
2.43.0
Re: [PATCH v2 1/6] arm64: make huge_ptep_get handled unaligned addresses
Quoted 26 lines · Dev Jain
huge_ptep_get()can be handed a virtual address pointing to the middle of a contpmd/contpte mapped hugetlb folio (examples of callers arepagemap_hugetlb_range,page_mapped_in_vma).The arm64 helper rewalks the pgtables in
find_num_contigto answer whether the huge pte we have maps a contpmd or a contpte hugetlb folio, and returnsCONT_PMDSorCONT_PTES, so that it can collect a/d bits over the contiguous ptes. We can falsely returnCONT_PTESinstead ofCONT_PMDSif the addr is not aligned.Fix this by aligning the pmdp pointer down to a contpmd base before checking equality with the passed huge pte pointer, to correctly answer whether the huge pte is the base of a contpmd block.
Fixes 29cb80519689 ("arm64: hugetlb: Cleanup huge_pte size discovery mechanisms")Signed-off-by Dev Jain <[email protected]>--- arch/arm64/mm/hugetlbpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/arm64/mm/hugetlbpage.c +0
-0
arch/arm64/mm/hugetlbpage.c static int find_num_contig(struct mm_struct *mm, unsigned long addr, · line 93
Quoted 8 lines
arch/arm64/mm/hugetlbpage.c +1
-1@@ -87,7 +87,7 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, p4dp =p4d_offset (pgdp, addr); pudp =pud_offset (p4dp, addr); pmdp =pmd_offset (pudp, addr); -if ((pte_t *)pmdp == ptep) { +if ((pte_t *)PTR_ALIGN_DOWN (pmdp,sizeof (*pmdp) *CONT_PMDS ) == ptep) { *pgsize =PMD_SIZE ;return CONT_PMDS ; }
Signature · 3 lines
Cheers, David
PATCH v2 2/6 mm/rmap: use huge_ptep_get() in try_to_unmap_one()
In reply to Dev Jaintry_to_unmap_one() handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that case page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it with ptep_get().
On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave.
It is not clear whether this has a trivially visible effect to userspace.
Just use huge_ptep_get() for dereferencing a huge pte pointer.
--- include/linux/hugetlb.h | 3 +++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-)
include/linux/hugetlb.h +3 -0
@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get (struct mm_struct *mm,unsigned long addr, +pte_t *ptep); +static inline pte_t huge_ptep_clear_flush (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {
mm/rmap.c +10 -6
@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ - pteval =ptep_get (pvmw.pte); + address = pvmw.address; +if (folio_test_hugetlb (folio)) { + pteval =huge_ptep_get (mm, address, pvmw.pte); + }else { +/* +* Handle PFN swap PTEs, such as device-exclusive ones, +* that actually map pages. +*/ + pteval =ptep_get (pvmw.pte); + }if (likely (pte_present (pteval))) { pfn =pte_pfn (pteval); }else {
@@ -2110,7 +2115,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, } subpage =folio_page (folio, pfn -folio_pfn (folio)); - address = pvmw.address; anon_exclusive =folio_test_anon (folio) &&PageAnonExclusive (subpage);
Signature · 1 line
2.43.0
Muchun Song
maintainerRe: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 27 lines · Dev Jain
try_to_unmap_one()handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that casepage_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Just use
huge_ptep_get()for dereferencing a huge pte pointer.Fixes c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")Reported-by David Hildenbrand <[email protected]>Signed-off-by Dev Jain <[email protected]>--- include/linux/hugetlb.h | 3 +++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-)include/linux/hugetlb.h +0
-0
include/linux/hugetlb.h static inline void hugetlb_count_sub(long l, struct mm_struct *mm) · line 1266
Quoted 6 lines
include/linux/hugetlb.h +3
-0@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get (struct mm_struct *mm,unsigned long addr, +pte_t *ptep); +
Maybe I didn't express my thoughts clearly in the first version, let me explain in more detail.
We should define this stub as a no-op for !CONFIG_HUGETLB_PAGE (like set_huge_pte_at, that is why I mentioned 5d4af6195c87c6 for your reference in your previous version). Currently, you've added a declaration, but the function itself doesn't actually exist, which seems quite strange to me.
Muchun,
Thanks.
Quoted 37 lines
include/linux/hugetlb.h +0
-0static inline pte_t huge_ptep_clear_flush (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {mm/rmap.c +10
-6@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ - pteval =ptep_get (pvmw.pte); + address = pvmw.address; +if (folio_test_hugetlb (folio)) { + pteval =huge_ptep_get (mm, address, pvmw.pte); + }else { +/* +* Handle PFN swap PTEs, such as device-exclusive ones, +* that actually map pages. +*/ + pteval =ptep_get (pvmw.pte); + }if (likely (pte_present (pteval))) { pfn =pte_pfn (pteval); }else {@@ -2110,7 +2115,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, } subpage =folio_page (folio, pfn -folio_pfn (folio)); - address = pvmw.address; anon_exclusive =folio_test_anon (folio) &&PageAnonExclusive (subpage);
Re: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 39 lines · Muchun Song
Quoted 27 lines · Dev Jain
try_to_unmap_one()handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that casepage_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Just use
huge_ptep_get()for dereferencing a huge pte pointer.Fixes c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")Reported-by David Hildenbrand <[email protected]>Signed-off-by Dev Jain <[email protected]>---
include/linux/hugetlb.h| 3 +++
mm/rmap.c | 16 ++++++++++------2 files changed, 13 insertions(+), 6 deletions(-)include/linux/hugetlb.h +0
-0include/linux/hugetlb.h static inline void hugetlb_count_sub(long l, struct mm_struct *mm) · line 1265
Quoted 5 lines
include/linux/hugetlb.h +2
-0@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, + pte_t *ptep); +Maybe I didn't express my thoughts clearly in the first version, let me explain in more detail.
Quoted 4 lines
We should define this stub as a no-op for !CONFIG_HUGETLB_PAGE (like
set_huge_pte_at, that is why I mentioned5d4af6195c87c6for your reference in your previous version). Currently, you've added a declaration, but the function itself doesn't actually exist, which seems quite strange to me.
https://lore.kernel.org/all/[email protected]/
David suggested this. Honestly I quite like David's suggestion, what do you think?
Quoted 39 lines
Muchun,
Thanks.Quoted 35 lines
include/linux/hugetlb.h +0
-0static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) {mm/rmap.c +0
-0@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, /* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO(!pvmw.pte, folio);- /*
- * Handle PFN swap PTEs, such as device-exclusive ones, that - * actually map pages.
- */
- pteval =ptep_get(pvmw.pte);
+ address = pvmw.address;
+ if (folio_test_hugetlb(folio)) {
+ pteval =huge_ptep_get(mm, address, pvmw.pte); + } else {
+ /*
+ * Handle PFN swap PTEs, such as device-exclusive ones, + * that actually map pages.
+ */
+ pteval =ptep_get(pvmw.pte);
+ }
if (likely(pte_present(pteval))) {
pfn =pte_pfn(pteval);
} else {mm/rmap.c +0
-1@@ -2110,7 +2115,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, } subpage = folio_page(folio, pfn - folio_pfn(folio)); - address = pvmw.address;
anon_exclusive=folio_test_anon(folio)&& PageAnonExclusive(subpage);
Muchun Song
maintainerRe: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 51 lines · Dev Jain
Quoted 39 lines · Muchun Song
Quoted 27 lines · Dev Jain
try_to_unmap_one()handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that casepage_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Just use
huge_ptep_get()for dereferencing a huge pte pointer.Fixes c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")Reported-by David Hildenbrand <[email protected]>Signed-off-by Dev Jain <[email protected]>--- include/linux/hugetlb.h | 3 +++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-)include/linux/hugetlb.h +0
-0include/linux/hugetlb.h static inline void hugetlb_count_sub(long l, struct mm_struct *mm) · line 1265
Quoted 5 lines
include/linux/hugetlb.h +2
-0@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, + pte_t *ptep); +Maybe I didn't express my thoughts clearly in the first version, let me explain in more detail.
Quoted 4 lines
We should define this stub as a no-op for !CONFIG_HUGETLB_PAGE (like
set_huge_pte_at, that is why I mentioned5d4af6195c87c6for your reference in your previous version). Currently, you've added a declaration, but the function itself doesn't actually exist, which seems quite strange to me.
Quoted 4 lines
https://lore.kernel.org/all/[email protected]/
David suggested this. Honestly I quite like David's suggestion, what do you think?
Thanks for pointing that out, I missed it earlier. That said, looking at hugetlb.h, it already contains quite a few no-op stubs. To keep things consistent, I'd personally prefer a stub here. Since David suggested this, I’d love to hear his thoughts on this as well.
Muchun,
Thanks
Quoted 39 lines
Quoted 39 lines
Muchun,
Thanks.Quoted 34 lines
include/linux/hugetlb.h +0
-0static inline pte_t huge_ptep_clear_flush (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {mm/rmap.c +10
-5@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ - pteval =ptep_get (pvmw.pte); +address = pvmw.address; +if (folio_test_hugetlb(folio)) { +pteval = huge_ptep_get(mm, address, pvmw.pte); +} else { +/* +* Handle PFN swap PTEs, such as device-exclusive ones, +* that actually map pages. +*/ + pteval =ptep_get (pvmw.pte); + }if (likely (pte_present (pteval))) { pfn =pte_pfn (pteval); }else {@@ -2110,7 +2115,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, } subpage =folio_page (folio, pfn -folio_pfn (folio)); - address = pvmw.address; anon_exclusive =folio_test_anon (folio) &&PageAnonExclusive (subpage);
David Hildenbrand (Arm)
maintainerRe: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 20 lines · Muchun Song
Quoted 20 lines · Dev Jain
Quoted 10 lines · Muchun Song
Maybe I didn't express my thoughts clearly in the first version, let me explain in more detail.
We should define this stub as a no-op for !CONFIG_HUGETLB_PAGE (like
set_huge_pte_at, that is why I mentioned5d4af6195c87c6for your reference in your previous version). Currently, you've added a declaration, but the function itself doesn't actually exist, which seems quite strange to me.
Quoted 7 lines
Quoted 2 lines
David suggested this. Honestly I quite like David's suggestion, what do you think?
Thanks for pointing that out, I missed it earlier. That said, looking at
hugetlb.h, it already contains quite a few no-op stubs. To keep things consistent, I'd personally prefer a stub here. Since David suggested this, I’d love to hear his thoughts on this as well.
It's the shortest possible way to enforce (through the linker!) that you function is not called from wrong context.
The only alternative is using a stub with a BUILD_BUG() in it. Which will 4 5 LOC :)
Signature · 3 lines
Cheers, David
David Hildenbrand (Arm)
maintainerRe: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
In reply to Dev JainQuoted 27 lines · Dev Jain
try_to_unmap_one()handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that casepage_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Just use
huge_ptep_get()for dereferencing a huge pte pointer.Fixes c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")Reported-by David Hildenbrand <[email protected]>Signed-off-by Dev Jain <[email protected]>--- include/linux/hugetlb.h | 3 +++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-)include/linux/hugetlb.h +0
-0
include/linux/hugetlb.h static inline void hugetlb_count_sub(long l, struct mm_struct *mm) · line 1265
Quoted 5 lines
include/linux/hugetlb.h +2
-0@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get (struct mm_struct *mm,unsigned long addr, +pte_t *ptep);
Two tabs, or just in a single line.
If others prefer a stub, I don't care. This here is shortest to let the linker bail out.
Quoted 9 lines
include/linux/hugetlb.h +1
-0+static inline pte_t huge_ptep_clear_flush (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {mm/rmap.c +0
-0
mm/rmap.c static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, · line 2101
Quoted 7 lines
mm/rmap.c +0
-4@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/
That comment now actually belongs above the pte_present() check below.
Signature · 3 lines
Cheers, David
Re: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 51 lines · David Hildenbrand (Arm)
Quoted 27 lines · Dev Jain
try_to_unmap_one()handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that casepage_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Just use
huge_ptep_get()for dereferencing a huge pte pointer.Fixes c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()")Reported-by David Hildenbrand <[email protected]>Signed-off-by Dev Jain <[email protected]>--- include/linux/hugetlb.h | 3 +++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 13 insertions(+), 6 deletions(-)include/linux/hugetlb.h +0
-0include/linux/hugetlb.h static inline void hugetlb_count_sub(long l, struct mm_struct *mm) · line 1265
Quoted 5 lines
include/linux/hugetlb.h +2
-0@@ -1261,6 +1261,9 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get (struct mm_struct *mm,unsigned long addr, +pte_t *ptep);Two tabs, or just in a single line.
If others prefer a stub, I don't care. This here is shortest to let the linker bail out.
Quoted 12 lines
include/linux/hugetlb.h +1
-0+static inline pte_t huge_ptep_clear_flush (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {mm/rmap.c +0
-0@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio);
Quoted 6 lines
mm/rmap.c static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, · line 2101
Quoted 4 lines
mm/rmap.c +0
-4-/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ That comment now actually belongs above the
pte_present()check below.
Oops I mindlessly just copied the comment.
I will then rather put it in the else block which gets the pfn from the softleaf, that is the most appropriate.
Quoted 2 lines
David Hildenbrand (Arm)
maintainerRe: [PATCH v2 2/6] mm/rmap: use huge_ptep_get() in try_to_unmap_one()
Quoted 16 lines
Quoted 9 lines
Quoted 9 lines
mm/rmap.c +0
-0@@ -2095,11 +2095,16 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio);Quoted 6 lines
mm/rmap.c static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, · line 2101
Quoted 4 lines
mm/rmap.c +0
-4-/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ That comment now actually belongs above the
pte_present()check below.
Quoted 4 lines
Oops I mindlessly just copied the comment.
I will then rather put it in the else block which gets the pfn from the softleaf, that is the most appropriate.
Works for me, thanks
Signature · 3 lines
Cheers, David
PATCH v2 3/6 mm/rmap: use huge_ptep_get() in try_to_migrate_one()
In reply to Dev Jaintry_to_migrate_one() is used by folio migration to replace a present mapping with a migration entry. For hugetlb folios, page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads the huge pte entry with ptep_get().
On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave.
It is not clear whether this has a trivially visible effect to userspace.
Use huge_ptep_get() to dereference a huge pte pointer.
Commit a98a2f0c8ce1 copied the bug from try_to_unmap_one into try_to_migrate_one.
--- mm/rmap.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
mm/rmap.c +10 -6
@@ -2505,11 +2505,16 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages. -*/ - pteval =ptep_get (pvmw.pte); + address = pvmw.address; +if (folio_test_hugetlb (folio)) { + pteval =huge_ptep_get (mm, address, pvmw.pte); + }else { +/* +* Handle PFN swap PTEs, such as device-exclusive ones, +* that actually map pages. +*/ + pteval =ptep_get (pvmw.pte); + }if (likely (pte_present (pteval))) { pfn =pte_pfn (pteval); }else {
@@ -2520,7 +2525,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, } subpage =folio_page (folio, pfn -folio_pfn (folio)); - address = pvmw.address; anon_exclusive =folio_test_anon (folio) &&PageAnonExclusive (subpage);
Signature · 1 line
2.43.0
David Hildenbrand (Arm)
maintainerRe: [PATCH v2 3/6] mm/rmap: use huge_ptep_get() in try_to_migrate_one()
Quoted 35 lines · Dev Jain
try_to_migrate_one()is used by folio migration to replace a present mapping with a migration entry. For hugetlb folios,page_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads the huge pte entry withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Use
huge_ptep_get()to dereference a huge pte pointer.Commit
a98a2f0c8ce1copied the bug fromtry_to_unmap_oneintotry_to_migrate_one.Fixes a98a2f0c8ce1 ("mm/rmap: split migration into its own function")Acked-by Muchun Song <[email protected]>Signed-off-by Dev Jain <[email protected]>--- mm/rmap.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)mm/rmap.c +0
-3@@ -2505,11 +2505,16 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,/* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO (!pvmw.pte, folio); -/* -* Handle PFN swap PTEs, such as device-exclusive ones, that -* actually map pages.
mm/rmap.c static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, · line 2515
Quoted 10 lines
mm/rmap.c +8
-2-*/ - pteval =ptep_get (pvmw.pte); + address = pvmw.address; +if (folio_test_hugetlb (folio)) { + pteval =huge_ptep_get (mm, address, pvmw.pte); + }else { +/* +* Handle PFN swap PTEs, such as device-exclusive ones, +* that actually map pages. +*/
Again, the comment is related to the pte_present() handling, so best to move it down there.
Apart from that LGTM.
Signature · 3 lines
Cheers, David
PATCH v2 4/6 mm/migrate: use huge_ptep_get() in remove_migration_pte()
In reply to Dev Jainremove_migration_pte() converts migration entries back to present PTEs after folio migration completes. For hugetlb folios, page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it with ptep_get().
On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave.
It is not clear whether this has a trivially visible effect to userspace.
Use huge_ptep_get() to dereference a huge pte pointer.
--- mm/migrate.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
mm/migrate.c +5 -1
@@ -371,7 +371,11 @@ static bool remove_migration_pte(struct folio *folio, continue; }#endif - old_pte =ptep_get (pvmw.pte); +if (folio_test_hugetlb (folio)) + old_pte =huge_ptep_get (vma->vm_mm, pvmw.address, + pvmw.pte); +else + old_pte =ptep_get (pvmw.pte);if (rmap_walk_arg->map_unused_to_zeropage &&try_to_map_unused_to_zeropage (&pvmw, folio, old_pte, idx))continue ;
Signature · 1 line
2.43.0
Re: [PATCH v2 4/6] mm/migrate: use huge_ptep_get() in remove_migration_pte()
Quoted 10 lines · Dev Jain
remove_migration_pte()converts migration entries back to present PTEs after folio migration completes. For hugetlb folios,page_vma_mapped_walk()returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it withptep_get().On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.It is not clear whether this has a trivially visible effect to userspace.
Right, for non-present entries it's even weirder. Fortunately your patch #1 also handles that.
Signature · 3 lines
Cheers, David
PATCH v2 5/6 mm/page_vma_mapped: use huge_ptep_get() for hugetlb
In reply to Dev Jaincheck_pte() is the final validation step in page_vma_mapped_walk(). It reads pvmw->pte with ptep_get() to decide whether the entry maps the PFN range being walked. For hugetlb VMAs, that pointer refers to a hugetlb entry.
On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave.
It is not clear whether this has a trivially visible effect to userspace.
Use huge_ptep_get() to dereference a huge pte pointer.
--- mm/page_vma_mapped.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
mm/page_vma_mapped.c +7 -1
@@ -107,7 +107,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp,static bool check_pte (struct page_vma_mapped_walk *pvmw,unsigned long pte_nr) {unsigned long pfn; -pte_t ptent =ptep_get (pvmw->pte); +pte_t ptent; + +if (is_vm_hugetlb_page (pvmw->vma)) + ptent =huge_ptep_get (pvmw->vma->vm_mm, pvmw->address, + pvmw->pte); +else + ptent =ptep_get (pvmw->pte);if (pvmw->flags &PVMW_MIGRATION ) {const softleaf_t entry =softleaf_from_pte (ptent);
Signature · 1 line
2.43.0
Re: [PATCH v2 5/6] mm/page_vma_mapped: use huge_ptep_get() for hugetlb
Quoted 13 lines · Dev Jain
check_pte()is the final validation step inpage_vma_mapped_walk(). It readspvmw->ptewithptep_get()to decide whether the entry maps the PFN range being walked. For hugetlb VMAs, that pointer refers to a hugetlb entry.On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.
Quoted 7 lines
It is not clear whether this has a trivially visible effect to userspace.
Use
huge_ptep_get()to dereference a huge pte pointer.Fixes ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")Signed-off-by Dev Jain <[email protected]>
Thanks.
Re: [PATCH v2 5/6] mm/page_vma_mapped: use huge_ptep_get() for hugetlb
In reply to Dev JainQuoted 9 lines · Dev Jain
check_pte()is the final validation step inpage_vma_mapped_walk(). It readspvmw->ptewithptep_get()to decide whether the entry maps the PFN range being walked. For hugetlb VMAs, that pointer refers to a hugetlb entry.On arches which provide their own
huge_ptep_get()to dereference a huge pte pointer, accessing viaptep_get()would causepte_pfn(),pte_present()etc to misbehave.
Quoted 8 lines
It is not clear whether this has a trivially visible effect to userspace.
Use
huge_ptep_get()to dereference a huge pte pointer.Fixes ace71a19cec5 ("mm: introduce page_vma_mapped_walk()")Signed-off-by Dev Jain <[email protected]>---
Signature · 3 lines
Cheers, David
PATCH v2 6/6 mm/mprotect: use huge_ptep_get() for hugetlb
In reply to Dev Jainprot_none_hugetlb_entry() is the hugetlb callback for the early mprotect(PROT_NONE) PFN permission walk on x86.
The callback passes the decoded PFN to pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries need huge_ptep_get(), reading that entry with ptep_get() can make the permission check use the wrong PFN.
Use huge_ptep_get() before decoding the hugetlb PFN.
Currently there is no path which can trigger a bug: huge_ptep_get() is a simple ptep_get() for x86, and the prot_none walk occurs only for x86.
So no need to backport - use the correct helper anyways.
--- mm/mprotect.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
mm/mprotect.c +7 -1
@@ -699,14 +699,20 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } +#ifdef CONFIG_HUGETLB_PAGE static int prot_none_hugetlb_entry (pte_t *pte,unsigned long hmask,unsigned long addr,unsigned long next,struct mm_walk *walk) { -return pfn_modify_allowed (pte_pfn (ptep_get (pte)), +pte_t entry =huge_ptep_get (walk->mm, addr, pte); + +return pfn_modify_allowed (pte_pfn (entry), *(pgprot_t *)(walk->private)) ?0 : -EACCES ; } +#else +#define prot_none_hugetlb_entry NULL +#endif static int prot_none_test (unsigned long addr,unsigned long next,struct mm_walk *walk)
Signature · 1 line
2.43.0
Re: [PATCH v2 6/6] mm/mprotect: use huge_ptep_get() for hugetlb
Quoted 10 lines · Dev Jain
prot_none_hugetlb_entry()is the hugetlb callback for the earlymprotect(PROT_NONE)PFN permission walk on x86.The callback passes the decoded PFN to
pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries needhuge_ptep_get(), reading that entry withptep_get()can make the permission check use the wrong PFN.Use
huge_ptep_get()before decoding the hugetlb PFN.
Quoted 8 lines
Currently there is no path which can trigger a bug:
huge_ptep_get()is a simpleptep_get()for x86, and theprot_nonewalk occurs only for x86.So no need to backport - use the correct helper anyways.
Fixes 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings")Signed-off-by Dev Jain <[email protected]>---
@Muchun, I did not do the change you suggested: if we do #ifdef around the hugetlb callback, then for CONFIG_HUGETLB_PAGE=n, prot_none_hugetlb_entry() is defined but not used, giving compile error.
Quoted 29 lines
mm/mprotect.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/mprotect.c +7
-1@@ -699,14 +699,20 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } +#ifdef CONFIG_HUGETLB_PAGE static int prot_none_hugetlb_entry (pte_t *pte,unsigned long hmask,unsigned long addr,unsigned long next,struct mm_walk *walk) { -return pfn_modify_allowed (pte_pfn (ptep_get (pte)), +pte_t entry =huge_ptep_get (walk->mm, addr, pte); + +return pfn_modify_allowed (pte_pfn (entry), *(pgprot_t *)(walk->private)) ?0 : -EACCES ; } +#else +#define prot_none_hugetlb_entry NULL +#endif static int prot_none_test (unsigned long addr,unsigned long next,struct mm_walk *walk)
Muchun Song
maintainerRe: [PATCH v2 6/6] mm/mprotect: use huge_ptep_get() for hugetlb
Quoted 22 lines · Dev Jain
Quoted 15 lines · Dev Jain
prot_none_hugetlb_entry()is the hugetlb callback for the earlymprotect(PROT_NONE)PFN permission walk on x86.The callback passes the decoded PFN to
pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries needhuge_ptep_get(), reading that entry withptep_get()can make the permission check use the wrong PFN.Use
huge_ptep_get()before decoding the hugetlb PFN.Currently there is no path which can trigger a bug:
huge_ptep_get()is a simpleptep_get()for x86, and theprot_nonewalk occurs only for x86.So no need to backport - use the correct helper anyways.
Quoted 8 lines
Quoted 3 lines
Fixes 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings")Signed-off-by Dev Jain <[email protected]>---
@Muchun, I did not do the change you suggested: if we do #ifdef around the hugetlb callback, then for
CONFIG_HUGETLB_PAGE=n,prot_none_hugetlb_entry()is defined but not used, giving compile error.
Got it. Thanks for your explanation.
Re: [PATCH v2 6/6] mm/mprotect: use huge_ptep_get() for hugetlb
In reply to Dev JainQuoted 14 lines · Dev Jain
prot_none_hugetlb_entry()is the hugetlb callback for the earlymprotect(PROT_NONE)PFN permission walk on x86.The callback passes the decoded PFN to
pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries needhuge_ptep_get(), reading that entry withptep_get()can make the permission check use the wrong PFN.Use
huge_ptep_get()before decoding the hugetlb PFN.
Quoted 7 lines
Currently there is no path which can trigger a bug:
huge_ptep_get()is a simpleptep_get()for x86, and theprot_nonewalk occurs only for x86.So no need to backport - use the correct helper anyways.
Fixes 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings")Signed-off-by Dev Jain <[email protected]>
Thanks.
David Hildenbrand (Arm)
maintainerRe: [PATCH v2 6/6] mm/mprotect: use huge_ptep_get() for hugetlb
In reply to Dev JainQuoted 26 lines · Dev Jain
prot_none_hugetlb_entry()is the hugetlb callback for the earlymprotect(PROT_NONE)PFN permission walk on x86.The callback passes the decoded PFN to
pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries needhuge_ptep_get(), reading that entry withptep_get()can make the permission check use the wrong PFN.Use
huge_ptep_get()before decoding the hugetlb PFN.Currently there is no path which can trigger a bug:
huge_ptep_get()is a simpleptep_get()for x86, and theprot_nonewalk occurs only for x86.So no need to backport - use the correct helper anyways.
Fixes 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings")Signed-off-by Dev Jain <[email protected]>--- mm/mprotect.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/mprotect.c +0
-0
mm/mprotect.c static int prot_none_pte_entry(pte_t *pte, unsigned long addr, · line 707
Quoted 10 lines
mm/mprotect.c +2
-1@@ -699,14 +699,20 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } +#ifdef CONFIG_HUGETLB_PAGE static int prot_none_hugetlb_entry (pte_t *pte,unsigned long hmask,unsigned long addr,unsigned long next,struct mm_walk *walk) { - return pfn_modify_allowed(pte_pfn(ptep_get(pte)), +pte_t entry =huge_ptep_get (walk->mm, addr, pte);
Nit: can be const :)
mm/mprotect.c static int prot_none_pte_entry(pte_t *pte, unsigned long addr, · line 711
Quoted 4 lines
mm/mprotect.c +2
-0+ +return pfn_modify_allowed (pte_pfn (entry), *(pgprot_t *)(walk->private)) ?0 : -EACCES ;
Indentation is odd.
Can we just make this readable?
if (pfn_modify_allowed ...)
return 0
return -EACCESS;
Signature · 3 lines
Cheers, David
Re: [PATCH v2 6/6] mm/mprotect: use huge_ptep_get() for hugetlb
Quoted 47 lines · David Hildenbrand (Arm)
Quoted 26 lines · Dev Jain
prot_none_hugetlb_entry()is the hugetlb callback for the earlymprotect(PROT_NONE)PFN permission walk on x86.The callback passes the decoded PFN to
pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries needhuge_ptep_get(), reading that entry withptep_get()can make the permission check use the wrong PFN.Use
huge_ptep_get()before decoding the hugetlb PFN.Currently there is no path which can trigger a bug:
huge_ptep_get()is a simpleptep_get()for x86, and theprot_nonewalk occurs only for x86.So no need to backport - use the correct helper anyways.
Fixes 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings")Signed-off-by Dev Jain <[email protected]>--- mm/mprotect.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)mm/mprotect.c +0
-0mm/mprotect.c static int prot_none_pte_entry(pte_t *pte, unsigned long addr, · line 707
Quoted 10 lines
mm/mprotect.c +2
-1@@ -699,14 +699,20 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } +#ifdef CONFIG_HUGETLB_PAGE static int prot_none_hugetlb_entry (pte_t *pte,unsigned long hmask,unsigned long addr,unsigned long next,struct mm_walk *walk) { - return pfn_modify_allowed(pte_pfn(ptep_get(pte)), +pte_t entry =huge_ptep_get (walk->mm, addr, pte);Nit: can be const :)
mm/mprotect.c static int prot_none_pte_entry(pte_t *pte, unsigned long addr, · line 711
Quoted 4 lines
mm/mprotect.c +2
-0+ +return pfn_modify_allowed (pte_pfn (entry), *(pgprot_t *)(walk->private)) ?0 : -EACCES ;Indentation is odd.
Quoted 6 lines
Can we just make this readable?
if (
pfn_modify_allowed...)
return 0
return-EACCESS;
Sure I'll do this.