mm/x86/pat: Fix two possible issues
Subsystems (4)
cover mm/x86/pat: Fix two possible issues
I'm recently looking at the possibility of mapping PCIe large bars to use huge mappings (just like thp or hugetlb). Then I noticed these only when reading the code.
I don't think I'm familiar enough with the whole PAT system, however I figured I should post them out to collect some comments, hencing marking this small series as RFC.
Please feel free to have a look at each of them; I've put more words in the commit message than here, as the two issues are not related. Any comments are welcomed.
Thanks,
Peter Xu (2):
mm/x86/pat: Only untrack the pfn range if unmap region mm/x86/pat: Do proper PAT bit shift for large mappings
mm/huge_memory.c | 4 ++-- mm/memory.c | 5 ++--- 2 files changed, 4 insertions(+), 5 deletions(-)
Signature · 1 line
2.45.0
RFC PATCH 1/2 mm/x86/pat: Only untrack the pfn range if unmap region
X86 uses pfn tracking to do pfnmaps. It looks like the tracking is normally started at mmap() of device drivers, then untracked when munmap().
However in the current code the untrack is done in unmap_single_vma(). This might be problematic.
For example, unmap_single_vma() can be used nowadays even for zapping a single page rather than the whole vmas. It's very confusing to do whole vma untracking in this function even if a caller would like to zap one page.
That may not yet be a problem, may because of multiple reasons:
(1) Things like MADV_DONTNEED won't ever work for pfnmaps and it'll fail already before reaching here.
(2) If some pfn tracking is lost by accident, the default fallback is to use UC-, which might be safe enough even if it may not be wanted. One can see track_pfn_insert() -> lookup_memtype() which can fall back to the default _PAGE_CACHE_MODE_UC_MINUS for a MMIO range.
However there's ongoing work [1] on VFIO driver to allow tearing down MMIO pgtables before an munmap(), in which case we may not want to untrack the pfns if we're only tearing down the pgtables. IOW, we may want to keep the pfn tracking information as those pfn mappings can be restored later with the same vma object. In VFIO's use case it can be remapped later if the VFIO mapped MMIO region got re-enabled (e.g. PCI_COMMAND_MEMORY set). In this case we should do pfn track for the whole lifecycle of the vma.
IIUC, this was overlooked when zap_page_range_single() was introduced, while in the past it was only used in the munmap() path which wants to always unmap the region completely. E.g., commit f5cc4eef9987 ("VM: make zap_page_range() callers that act on a single VMA use separate helper") is the initial commit that introduced unmap_single_vma(), in which the chunk of untrack_pfn() was moved over from unmap_vmas().
Recover that behavior to untrack pfnmap only when unmap regions.
[1] https://lore.kernel.org/r/[email protected]
--- mm/memory.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
mm/memory.c +2 -3
@@ -1821,9 +1821,6 @@ static void unmap_single_vma(struct mmu_gather *tlb,if (vma->vm_file)uprobe_munmap (vma, start, end); -if (unlikely (vma->vm_flags &VM_PFNMAP )) -untrack_pfn (vma,0 ,0 , mm_wr_locked); -if (start != end) {if (unlikely (is_vm_hugetlb_page (vma))) {/*
@@ -1888,6 +1885,8 @@ void unmap_vmas(struct mmu_gather *tlb, struct ma_state *mas,unsigned long start = start_addr;unsigned long end = end_addr;hugetlb_zap_begin (vma, &start, &end); +if (unlikely (vma->vm_flags &VM_PFNMAP )) +untrack_pfn (vma,0 ,0 , mm_wr_locked);unmap_single_vma (tlb, vma, start, end, &details, mm_wr_locked);hugetlb_zap_end (vma, &details);
Signature · 1 line
2.45.0
RFC PATCH 2/2 mm/x86/pat: Do proper PAT bit shift for large mappings
In reply to Peter XuFor large mappings, the pgtable PAT is set on bit 12 (_PAGE_PAT_LARGE) rather than bit 9 (_PAGE_PAT), while bit 9 is used as PAE hint. Do proper shifting when inject large pfn pgtable mappings to make cache mode alright.
--- mm/huge_memory.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/huge_memory.c +2 -2
@@ -1135,7 +1135,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; } - entry =pmd_mkhuge (pfn_t_pmd (pfn, prot)); + entry =pmd_mkhuge (pfn_t_pmd (pfn,pgprot_4k_2_large (prot)));if (pfn_t_devmap (pfn)) entry =pmd_mkdevmap (entry);if (write) {
@@ -1233,7 +1233,7 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; } - entry =pud_mkhuge (pfn_t_pud (pfn, prot)); + entry =pud_mkhuge (pfn_t_pud (pfn,pgprot_4k_2_large (prot)));if (pfn_t_devmap (pfn)) entry =pud_mkdevmap (entry);if (write) {
Signature · 1 line
2.45.0
Re: [PATCH RFC 2/2] mm/x86/pat: Do proper PAT bit shift for large mappings
mm/huge_memory.c static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, · line 1141
Quoted 13 lines · Peter Xu
mm/huge_memory.c +1
-1@@ -1135,7 +1135,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; } - entry =pmd_mkhuge (pfn_t_pmd (pfn, prot)); + entry =pmd_mkhuge (pfn_t_pmd (pfn,pgprot_4k_2_large (prot)));if (pfn_t_devmap (pfn)) entry =pmd_mkdevmap (entry);if (write) {
Does this even compile on non-x86 architectures?
Re: [PATCH RFC 2/2] mm/x86/pat: Do proper PAT bit shift for large mappings
Quoted 9 lines · Dave Hansen
Quoted 8 lines · Peter Xu
mm/huge_memory.c +0
-0@@ -1135,7 +1135,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; }
Quoted 7 lines
mm/huge_memory.c static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, · line 1141
Quoted 5 lines
mm/huge_memory.c +1
-1- entry =pmd_mkhuge (pfn_t_pmd (pfn, prot)); + entry =pmd_mkhuge (pfn_t_pmd (pfn,pgprot_4k_2_large (prot)));if (pfn_t_devmap (pfn)) entry =pmd_mkdevmap (entry);if (write) {Does this even compile on non-x86 architectures?
Probably not.. I think I can define a pgprot_to_large() globally, pointing that to pgprot_4k_2_large() on x86 and make the fallback to be noop. And if there's a new version I'll guarantee to run over my cross compilers.
Any comments on the idea itself? Do we have a problem, or maybe I overlooked something?
Thanks,
Signature · 1 line
Peter Xu
Re: [PATCH RFC 2/2] mm/x86/pat: Do proper PAT bit shift for large mappings
Quoted 18 lines · Peter Xu
Quoted 9 lines · Dave Hansen
Quoted 8 lines · Peter Xu
mm/huge_memory.c +0
-0@@ -1135,7 +1135,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; }Quoted 7 lines
mm/huge_memory.c static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, · line 1141
Quoted 5 lines
mm/huge_memory.c +1
-1- entry =pmd_mkhuge (pfn_t_pmd (pfn, prot)); + entry =pmd_mkhuge (pfn_t_pmd (pfn,pgprot_4k_2_large (prot)));if (pfn_t_devmap (pfn)) entry =pmd_mkdevmap (entry);if (write) {Does this even compile on non-x86 architectures?
Quoted 6 lines
Probably not.. I think I can define a
pgprot_to_large()globally, pointing that topgprot_4k_2_large()on x86 and make the fallback to be noop. And if there's a new version I'll guarantee to run over my cross compilers.Any comments on the idea itself? Do we have a problem, or maybe I overlooked something?
I also attached one new version of patch 2 that should pass the cross builds. Please reviewers feel free to look at this one instead. From x86 perspective they should be the same thing.
Thanks,
===8<===
From 1cce12c872cb01aaa8686d8f5c7cd6b266ca4e38 Mon Sep 17 00:00:00 2001
From: Peter Xu <[email protected]>
Date: Thu, 23 May 2024 18:19:35 -0400
Subject: [PATCH rfcv1.1] mm/x86/pat: Do proper PAT bit shift for large mappings
For large mappings, the pgtable PAT is set on bit 12 (_PAGE_PAT_LARGE) rather than bit 9 (_PAGE_PAT), while bit 9 is used as PAE hint. Do proper shifting when inject large pfn pgtable mappings to make cache mode alright.
--- arch/x86/include/asm/pgtable_types.h | 1 + include/linux/pgtable.h | 4 ++++ mm/huge_memory.c | 4 ++-- 3 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/include/asm/pgtable_types.h +1 -0
@@ -512,6 +512,7 @@ static inline pgprot_t pgprot_large_2_4k(pgprot_t pgprot)return __pgprot (protval_large_2_4k (pgprot_val (pgprot))); } +#define pgprot_to_large (pgprot) pgprot_4k_2_large(pgprot)typedef struct page *pgtable_t ;
include/linux/pgtable.h +4 -0
@@ -1956,4 +1956,8 @@ pgprot_t vm_get_page_prot(unsigned long vm_flags) \ } \EXPORT_SYMBOL (vm_get_page_prot); +#ifndef pgprot_to_large +#define pgprot_to_large (pgprot) pgprot +#endif +#endif /* _LINUX_PGTABLE_H */
mm/huge_memory.c +2 -2
@@ -1135,7 +1135,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; } - entry =pmd_mkhuge (pfn_t_pmd (pfn, prot)); + entry =pmd_mkhuge (pfn_t_pmd (pfn,pgprot_to_large (prot)));if (pfn_t_devmap (pfn)) entry =pmd_mkdevmap (entry);if (write) {
@@ -1233,7 +1233,7 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, goto out_unlock; } - entry =pud_mkhuge (pfn_t_pud (pfn, prot)); + entry =pud_mkhuge (pfn_t_pud (pfn,pgprot_to_large (prot)));if (pfn_t_devmap (pfn)) entry =pud_mkdevmap (entry);if (write) {
Signature · 4 lines
2.45.0 -- Peter Xu
Re: [PATCH RFC 2/2] mm/x86/pat: Do proper PAT bit shift for large mappings
In reply to Peter XuQuoted 3 lines · Peter Xu
Probably not.. I think I can define a
pgprot_to_large()globally, pointing that topgprot_4k_2_large()on x86 and make the fallback to be noop. And if there's a new version I'll guarantee to run over my cross compilers.
I guess that would be functional, but it would be a bit mean to everybody else.
Quoted 2 lines
Any comments on the idea itself? Do we have a problem, or maybe I overlooked something?
I think it's probably unnecessary to inflict this particular x86-ism on generic code. The arch-generic 'prot' should have PAT at its 4k (_PAGE_BIT_PAT) position and then p*d_mkhuge() can shift it into the _PAGE_BIT_PAT_LARGE spot.
Re: [PATCH RFC 2/2] mm/x86/pat: Do proper PAT bit shift for large mappings
Quoted 8 lines · Dave Hansen
Quoted 3 lines · Peter Xu
Probably not.. I think I can define a
pgprot_to_large()globally, pointing that topgprot_4k_2_large()on x86 and make the fallback to be noop. And if there's a new version I'll guarantee to run over my cross compilers.I guess that would be functional, but it would be a bit mean to everybody else.
Quoted 7 lines
Quoted 2 lines
Any comments on the idea itself? Do we have a problem, or maybe I overlooked something?
I think it's probably unnecessary to inflict this particular x86-ism on generic code. The arch-generic 'prot' should have PAT at its 4k (
_PAGE_BIT_PAT) position and then p*d_mkhuge() can shift it into the_PAGE_BIT_PAT_LARGEspot.
Right that's another option indeed.
It's just that I found it might in many cases be better when we have the API separately properly and making the pairs matching each other.
For example, it could be clearer if pxx_mkhuge() does exactly what pxx_leaf() would check against.
PS: I hoped it's called pxx_huge() already to make the name paired with each other; afaict we called it pxx_leaf() only because pxx_huge() used to be "abused" by hugetlbfs before.. now it's gone.
The other thing is we mostly only need these knobs for special maps like pfnmaps, am I right? OTOH we use WB for RAMs, and maybe we don't want to bother any PAT stuff when the kernel is installing a THP anonymous?
IMHO having pgprot_to_large() is fine even if only x86 has it; it's really like pfn tracking itself which is noop for !x86. but I'll follow your advise if you still insist; I don't really have a strong opinion.
But if so I'd also like to mention a 3rd option, which is to have pxx_mkhuge_prot(), fallback to pxx_mkhuge() for !x86. That'll make pxx_huge() untainted for x86. I'm not sure whether that would ease the same concern, though.
In all cases, thanks for confirming this issue, I appreciate that. Let me know if you have any comment on patch 1 too; that one isn't a problem so far iiuc, but it can be soon.
Thanks,
Signature · 1 line
Peter Xu