riscv: mm: Implement pmdp_collapse_flush for THP
PATCH 1/1 riscv: mm: Implement pmdp_collapse_flush for THP
When THP is enabled, 4K pages are collapsed into a single huge page using the generic pmdp_collapse_flush() which will further use flush_tlb_range() to shoot-down stale TLB entries. Unfortunately, the generic pmdp_collapse_flush() only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.
Provide a RISC-V specific pmdp_collapse_flush() which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.
--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
arch/riscv/include/asm/pgtable.h +24 -0
@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd); return __pmd(atomic_long_xchg((atomic_long_t *)pmdp, pmd_val(pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, +unsigned long address,pmd_t *pmdp) +{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp); + +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf +* PMD entry (huge page), a valid non-leaf PTE is converted into a +* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12 +* specification allows implementations to cache valid non-leaf PTEs, +* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA +* with rs1=x0. If any PTE along the traversal path had its G bit set, +* rs2 must be x0; otherwise, rs2 should be set to the ASID for which +* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever +* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); +return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /*
Signature · 7 lines
2.34.1 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 23 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately, the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +0
-0
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 729
Quoted 9 lines
arch/riscv/include/asm/pgtable.h +6
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd);return __pmd (atomic_long_xchg ((atomic_long_t *)pmdp,pmd_val (pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, +unsigned long address,pmd_t *pmdp) +{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp);
The generic version of this function has a couple sanity checks for kernels compiled with CONFIG_DEBUG_VM. Shouldn't we duplicate those?
Thanks,
drew
Quoted 24 lines
arch/riscv/include/asm/pgtable.h +18
-0+ +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf +* PMD entry (huge page), a valid non-leaf PTE is converted into a +* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12 +* specification allows implementations to cache valid non-leaf PTEs, +* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA +* with rs1=x0. If any PTE along the traversal path had its G bit set, +* rs2 must be x0; otherwise, rs2 should be set to the ASID for which +* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever +* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); + return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* Signature · 1 line
2.34.1
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 35 lines · Andrew Jones
Quoted 34 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately,
the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +3
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd);return __pmd (atomic_long_xchg ((atomic_long_t *)pmdp,pmd_val (pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush(struct vm_area_struct
Quoted 8 lines
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 729
Quoted 5 lines
arch/riscv/include/asm/pgtable.h +1
-0*vma, + unsigned long address, pmd_t*pmdp)
arch/riscv/include/asm/pgtable.h +2
-0+{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp);The generic version of this function has a couple sanity checks for kernels compiled with
CONFIG_DEBUG_VM. Shouldn't we duplicate those?
Ok. I will add those checks too.
Quoted 32 lines
Thanks,
drewQuoted 28 lines
arch/riscv/include/asm/pgtable.h +3
-0+ +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf
arch/riscv/include/asm/pgtable.h +1
-0+* PMD entry (huge page), a valid non-leaf PTE is converted into a
arch/riscv/include/asm/pgtable.h +1
-0+* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12
arch/riscv/include/asm/pgtable.h +1
-0+* specification allows implementations to cache valid non-leaf PTEs,
arch/riscv/include/asm/pgtable.h +3
-0+* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA
arch/riscv/include/asm/pgtable.h +1
-0+* with rs1=x0. If any PTE along the traversal path had its G bit set,
arch/riscv/include/asm/pgtable.h +1
-0+* rs2 must be x0; otherwise, rs2 should be set to the ASID for which
arch/riscv/include/asm/pgtable.h +2
-0+* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever
arch/riscv/include/asm/pgtable.h +5
-0+* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); + return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Quoted 9 lines
Quoted 4 lines
/*
Signature · 1 line
2.34.1_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Alexandre Ghiti
reviewerHi Mayuresh,
Quoted 7 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately, the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.
That's interesting! I'm wondering if the same issue will happen if a user maps 4K, unmaps it and at the same address maps a 2MB hugepage: I'm not sure the mm code would correctly flush the non-leaf PTE when unmapping the 4KB page. In that case, your patch only fixes the THP usecase and maybe we should try to catch this non-leaf -> leaf upgrade at some lower level page table functions, what do you think?
Alex
Quoted 35 lines
Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +17
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd);return __pmd (atomic_long_xchg ((atomic_long_t *)pmdp,pmd_val (pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, +unsigned long address,pmd_t *pmdp) +{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp); + +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf +* PMD entry (huge page), a valid non-leaf PTE is converted into a +* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12 +* specification allows implementations to cache valid non-leaf PTEs, +* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA +* with rs1=x0. If any PTE along the traversal path had its G bit set, +* rs2 must be x0; otherwise, rs2 should be set to the ASID for which
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 750
Quoted 10 lines
arch/riscv/include/asm/pgtable.h +7
-0+* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever +* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); + return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /*
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 13 lines · Alexandre Ghiti
Hi Mayuresh,
Quoted 7 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately, the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.
Quoted 6 lines
That's interesting! I'm wondering if the same issue will happen if a user maps 4K, unmaps it and at the same address maps a 2MB hugepage: I'm not sure the mm code would correctly flush the non-leaf PTE when unmapping the 4KB page. In that case, your patch only fixes the THP usecase and maybe we should try to catch this non-leaf -> leaf upgrade at some lower level page table functions, what do you think?
This issue can happen whenever existing/valid non-leaf PTE is modified.
We hit this issue in the THP case but we can also hit this issue in other scenarios where the page table programming pattern is similar.
Regards,
Anup
Quoted 48 lines
Alex
Quoted 44 lines
Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +24
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd); return __pmd(atomic_long_xchg((atomic_long_t *)pmdp, pmd_val(pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, +unsigned long address,pmd_t *pmdp) +{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp); + +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf +* PMD entry (huge page), a valid non-leaf PTE is converted into a +* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12 +* specification allows implementations to cache valid non-leaf PTEs, +* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA +* with rs1=x0. If any PTE along the traversal path had its G bit set, +* rs2 must be x0; otherwise, rs2 should be set to the ASID for which +* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever +* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); +return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Quoted 1 line
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 750
Quoted 1 line
arch/riscv/include/asm/pgtable.h +0
-0/*
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Alexandre Ghiti
reviewerQuoted 18 lines · Anup Patel
Quoted 11 lines · Alexandre Ghiti
Hi Mayuresh,
Quoted 7 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately, the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.Quoted 6 lines
That's interesting! I'm wondering if the same issue will happen if a user maps 4K, unmaps it and at the same address maps a 2MB hugepage: I'm not sure the mm code would correctly flush the non-leaf PTE when unmapping the 4KB page. In that case, your patch only fixes the THP usecase and maybe we should try to catch this non-leaf -> leaf upgrade at some lower level page table functions, what do you think?
Quoted 4 lines
This issue can happen whenever existing/valid non-leaf PTE is modified.
We hit this issue in the THP case but we can also hit this issue in other scenarios where the page table programming pattern is similar.
Then what about trying to get all those cases at once? We can easily catch those spurious page faults as the pte would be valid: we would just have to flush_tlb_mm at the first page fault, if any.
Quoted 51 lines
Regards,
AnupQuoted 47 lines
Alex
Quoted 44 lines
Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +24
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd); return __pmd(atomic_long_xchg((atomic_long_t *)pmdp, pmd_val(pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, +unsigned long address,pmd_t *pmdp) +{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp); + +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf +* PMD entry (huge page), a valid non-leaf PTE is converted into a +* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12 +* specification allows implementations to cache valid non-leaf PTEs, +* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA +* with rs1=x0. If any PTE along the traversal path had its G bit set, +* rs2 must be x0; otherwise, rs2 should be set to the ASID for which +* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever +* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); +return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Quoted 5 lines
Quoted 1 line
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 750
Quoted 1 line
arch/riscv/include/asm/pgtable.h +0
-0/* _______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 30 lines · Alexandre Ghiti
Quoted 29 lines · Anup Patel
Quoted 14 lines · Alexandre Ghiti
Hi Mayuresh,
Quoted 10 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately,
the genericpmdp_collapse_flush()only invalidates cached leaf PTEs
using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache
non-leaf PTEs.Quoted 10 lines
That's interesting! I'm wondering if the same issue will happen if a
user maps 4K, unmaps it and at the same address maps a 2MB
hugepage: I'm
not sure the mm code would correctly flush the non-leaf PTE when unmapping the 4KB page. In that case, your patch only fixes the THP
usecase and maybe we should try to catch this non-leaf -> leaf upgrade
at some lower level page table functions, what do you think?This issue can happen whenever existing/valid non-leaf PTE is modified.
Quoted 7 lines
Quoted 3 lines
We hit this issue in the THP case but we can also hit this issue in other
scenarios where the page table programming pattern is similar.Then what about trying to get all those cases at once? We can easily catch those spurious page faults as the pte would be valid: we would just have to
flush_tlb_mmat the first page fault, if any.
IMO, we can consider fixing those cases in a separate patch.
Quoted 78 lines
Quoted 72 lines
Regards,
AnupQuoted 67 lines
Alex
Quoted 64 lines
Provide a RISC-V specific
pmdp_collapse_flush()which ensures both
cached leaf and non-leaf PTEs are invalidated by using non- address
specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +3
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd);return __pmd (atomic_long_xchg ((atomic_long_t *)pmdp,pmd_val (pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush(struct vm_area_struct*vma,
+ unsigned long address,pmd_t*pmdp)arch/riscv/include/asm/pgtable.h +2
-0+{ + pmd_t pmd = pmdp_huge_get_and_clear(vma->vm_mm, address,pmdp);
arch/riscv/include/asm/pgtable.h +3
-0+ +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf
+ * PMD entry (huge page), a valid non-leaf PTE is
converted into a
+ * valid leaf PTE at the level 1 page table. The RISC-V
privileged v1.12
+ * specification allows implementations to cache valid
non-leaf PTEs,
+ * but the section "4.2.1 Supervisor Memory-Management
Fencearch/riscv/include/asm/pgtable.h +2
-0+* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA
+ * with rs1=x0. If any PTE along the traversal path had
its G bit set,
+ * rs2 must be x0; otherwise, rs2 should be set to the
ASID for whicharch/riscv/include/asm/pgtable.h +2
-0+* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever
arch/riscv/include/asm/pgtable.h +5
-0+* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); + return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ Quoted 5 lines
Quoted 1 line
Quoted 1 line
/*_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 4 lines
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 12 lines · Alexandre Ghiti
Hi Mayuresh,
Quoted 8 lines · Mayuresh Chitale
When THP is enabled, 4K pages are collapsed into a single huge page using the generic
pmdp_collapse_flush()which will further useflush_tlb_range()to shoot-down stale TLB entries. Unfortunately,
the genericpmdp_collapse_flush()only invalidates cached leaf PTEs using address specific SFENCEs which results in repetitive (or unpredictable) page faults on RISC-V implementations which cache non-leaf PTEs.
Quoted 8 lines
That's interesting! I'm wondering if the same issue will happen if a user maps 4K, unmaps it and at the same address maps a 2MB hugepage:
I'm
not sure the mm code would correctly flush the non-leaf PTE when unmapping the 4KB page. In that case, your patch only fixes the THP usecase and maybe we should try to catch this non-leaf -> leaf upgrade
at some lower level page table functions, what do you think?
I will look into it but I dont know how to reproduce the issue without the THP use case. It would be great if you could share the test case or test code to reproduce it.
Quoted 62 lines
Alex
Quoted 58 lines
Provide a RISC-V specific
pmdp_collapse_flush()which ensures both cached leaf and non-leaf PTEs are invalidated by using non-address specific SFENCEs as recommended by the RISC-V privileged specification.Fixes e88b333142e4 ("riscv: mm: add THP support on 64-bit")Signed-off-by Mayuresh Chitale <[email protected]>--- arch/riscv/include/asm/pgtable.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)arch/riscv/include/asm/pgtable.h +4
-0@@ -721,6 +721,30 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,page_table_check_pmd_set (vma->vm_mm, address, pmdp, pmd);return __pmd (atomic_long_xchg ((atomic_long_t *)pmdp,pmd_val (pmd))); } + +#define pmdp_collapse_flush pmdp_collapse_flush +static inline pmd_t pmdp_collapse_flush (struct vm_area_struct *vma, + unsigned long address, pmd_t*pmdp)
arch/riscv/include/asm/pgtable.h +5
-0+{ +pmd_t pmd =pmdp_huge_get_and_clear (vma->vm_mm, address, pmdp); + +/* +* When leaf PTE enteries (regular pages) are collapsed into a leaf
arch/riscv/include/asm/pgtable.h +1
-0+* PMD entry (huge page), a valid non-leaf PTE is converted into a
arch/riscv/include/asm/pgtable.h +1
-0+* valid leaf PTE at the level 1 page table. The RISC-V privileged v1.12
arch/riscv/include/asm/pgtable.h +1
-0+* specification allows implementations to cache valid non-leaf PTEs,
arch/riscv/include/asm/pgtable.h +3
-0+* but the section "4.2.1 Supervisor Memory-Management Fence +* Instruction" recommends the following: +* "If software modifies a non-leaf PTE, it should execute SFENCE.VMA
arch/riscv/include/asm/pgtable.h +1
-0+* with rs1=x0. If any PTE along the traversal path had its G bit set,
arch/riscv/include/asm/pgtable.h +1
-0+* rs2 must be x0; otherwise, rs2 should be set to the ASID for which
arch/riscv/include/asm/pgtable.h +2
-0+* the translation is being modified." +* Based on the above recommendation, we should do full flush whenever
arch/riscv/include/asm/pgtable.h +5
-0+* leaf PTE entries are collapsed into a leaf PMD entry. +*/ +flush_tlb_mm (vma->vm_mm); + return pmd; +}#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Quoted 6 lines
Quoted 1 line
/*
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv