Remove uses of struct page from x86 and arm64 MMU
v2 cover Remove uses of struct page from x86 and arm64 MMU
KVM supports mapping VM_IO and VM_PFNMAP memory into the guest by using follow_pte in gfn_to_pfn. However, the resolved pfns may not have assoicated struct pages, so they should not be passed to pfn_to_page. This series removes such calls from the x86 and arm64 secondary MMU. To do this, this series introduces gfn_to_pfn_page functions that parallel the gfn_to_pfn functions. These functions take an extra out parameter which contains the page if the hva was resovled by gup. This allows the caller to call put_page only when necessated by gup.
The gfn_to_pfn functions are depreciated. For now the functions remain with identical behavior to before, but callers should be migrated to the new gfn_to_pfn_page functions. I added new functions instead of simply adding another parameter to the existing functions to make it easier to track down users of the deprecated functions.
I have migrated the x86 and arm64 secondary MMUs to the new gfn_to_pfn_page functions.
This addresses CVE-2021-22543 on x86 and arm64.
v1 -> v2:
- Introduce new gfn_to_pfn_page functions instead of modifying the behavior of existing gfn_to_pfn functions, to make the change less invasive.
- Drop changes to mmu_audit.c
- Include Nicholas Piggin's patch to avoid corrupting refcount in the follow_pte case, and use it in depreciated gfn_to_pfn functions.
- Rebase on kvm/next
David Stevens (4):
KVM: mmu: introduce new gfn_to_pfn_page functions
KVM: x86/mmu: use gfn_to_pfn_page
KVM: arm64/mmu: use gfn_to_pfn_page
KVM: mmu: remove over-aggressive warnings
Nicholas Piggin (1):
KVM: do not allow mapping valid but non-refcounted pages
arch/arm64/kvm/mmu.c | 26 +++-- arch/x86/kvm/mmu/mmu.c | 50 ++++----- arch/x86/kvm/mmu/mmu_internal.h | 3 +- arch/x86/kvm/mmu/paging_tmpl.h | 23 +++-- arch/x86/kvm/mmu/tdp_mmu.c | 6 +- arch/x86/kvm/mmu/tdp_mmu.h | 4 +- arch/x86/kvm/x86.c | 6 +- include/linux/kvm_host.h | 17 +++ virt/kvm/kvm_main.c | 177 +++++++++++++++++++++++++------- 9 files changed, 222 insertions(+), 90 deletions(-)
Signature · 1 line
2.32.0.93.g670b81a890-goog
PATCH v2 1/5 KVM: do not allow mapping valid but non-refcounted pages
From: Nicholas Piggin <[email protected]>
It's possible to create a region which maps valid but non-refcounted pages (e.g., tail pages of non-compound higher order allocations). These host pages can then be returned by gfn_to_page, gfn_to_pfn, etc., family of APIs, which take a reference to the page, which takes it from 0 to 1. When the reference is dropped, this will free the page incorrectly.
Fix this by only taking a reference on the page if it was non-zero, which indicates it is participating in normal refcounting (and can be released with put_page).
--- virt/kvm/kvm_main.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +17 -2
@@ -2175,6 +2175,13 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault)return true; } +static int kvm_try_get_pfn (kvm_pfn_t pfn) +{ +if (kvm_is_reserved_pfn (pfn)) +return 1 ; +return get_page_unless_zero (pfn_to_page (pfn)); +} + static int hva_to_pfn_remapped(struct vm_area_struct *vma, unsigned long addr, bool *async, bool write_fault, bool *writable,
@@ -2224,13 +2231,21 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma,* Whoever called remap_pfn_range is also going to call e.g. * unmap_mapping_range before the underlying pages are freed, * causing a call to our MMU notifier. +* +* Certain IO or PFNMAP mappings can be backed with valid +* struct pages, but be allocated without refcounting e.g., +* tail pages of non-compound higher order allocations, which +* would then underflow the refcount when the caller does the +* required put_page. Don't allow those pages here. */ -kvm_get_pfn (pfn); +if (!kvm_try_get_pfn (pfn)) + r = -EFAULT ; out:pte_unmap_unlock (ptep, ptl); *p_pfn = pfn; -return 0 ; + +return r; }/*
Signature · 1 line
2.32.0.93.g670b81a890-goog
Re: [PATCH v2 1/5] KVM: do not allow mapping valid but non-refcounted pages
Quoted 8 lines · David Stevens
From: Nicholas Piggin <[email protected]>
It's possible to create a region which maps valid but non-refcounted pages (e.g., tail pages of non-compound higher order allocations). These host pages can then be returned by
gfn_to_page,gfn_to_pfn, etc., family of APIs, which take a reference to the page, which takes it from 0 to 1. When the reference is dropped, this will free the page incorrectly.
Quoted 5 lines
Fix this by only taking a reference on the page if it was non-zero, which indicates it is participating in normal refcounting (and can be released with
put_page).Signed-off-by Nicholas Piggin <[email protected]>
I guess this would be the small fix for stable? Do we want to add that cc?
Quoted 47 lines
--- virt/kvm/kvm_main.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +17
-2@@ -2175,6 +2175,13 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault)return true; } +static int kvm_try_get_pfn (kvm_pfn_t pfn) +{ +if (kvm_is_reserved_pfn (pfn)) +return 1 ; +return get_page_unless_zero (pfn_to_page (pfn)); +} + static int hva_to_pfn_remapped(struct vm_area_struct *vma, unsigned long addr, bool *async, bool write_fault, bool *writable,@@ -2224,13 +2231,21 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma,* Whoever called remap_pfn_range is also going to call e.g. * unmap_mapping_range before the underlying pages are freed, * causing a call to our MMU notifier. +* +* Certain IO or PFNMAP mappings can be backed with valid +* struct pages, but be allocated without refcounting e.g., +* tail pages of non-compound higher order allocations, which +* would then underflow the refcount when the caller does the +* required put_page. Don't allow those pages here. */ -kvm_get_pfn (pfn); +if (!kvm_try_get_pfn (pfn)) + r = -EFAULT ; out:pte_unmap_unlock (ptep, ptl); *p_pfn = pfn; -return 0 ; + +return r; }/*
Paolo Bonzini
maintainerRe: [PATCH v2 1/5] KVM: do not allow mapping valid but non-refcounted pages
Quoted 15 lines · Christian Borntraeger
Quoted 12 lines · David Stevens
From: Nicholas Piggin <[email protected]>
It's possible to create a region which maps valid but non-refcounted pages (e.g., tail pages of non-compound higher order allocations). These host pages can then be returned by
gfn_to_page,gfn_to_pfn, etc., family of APIs, which take a reference to the page, which takes it from 0 to 1. When the reference is dropped, this will free the page incorrectly.Fix this by only taking a reference on the page if it was non-zero, which indicates it is participating in normal refcounting (and can be released with
put_page).
Quoted 5 lines
Quoted 1 line
Signed-off-by Nicholas Piggin <[email protected]>I guess this would be the small fix for stable? Do we want to add that cc?
Reviewed-by Christian Borntraeger <[email protected]>
Yes, this one is going to Linus today. The rest is for 5.15.
Paolo
Quoted 50 lines
Quoted 49 lines
---
virt/kvm/kvm_main.c| 19 +++++++++++++++++--1 file changed, 17 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +7
-0@@ -2175,6 +2175,13 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault) return true; } +static int kvm_try_get_pfn (kvm_pfn_t pfn) +{ + if (kvm_is_reserved_pfn(pfn)) + return 1; + return get_page_unless_zero(pfn_to_page(pfn)); +} + static int hva_to_pfn_remapped(struct vm_area_struct *vma, unsigned long addr, bool *async,bool
write_fault, bool *writable,virt/kvm/kvm_main.c +8
-1@@ -2224,13 +2231,21 @@ static int hva_to_pfn_remapped(structvm_area_struct *vma, * Whoever called remap_pfn_range is also going to call e.g. * unmap_mapping_range before the underlying pages are freed, * causing a call to our MMU notifier. +* +* Certain IO or PFNMAP mappings can be backed with valid +* struct pages, but be allocated without refcounting e.g., +* tail pages of non-compound higher order allocations, which +* would then underflow the refcount when the caller does the +* required put_page. Don't allow those pages here. */ - kvm_get_pfn(pfn); + if (!kvm_try_get_pfn(pfn)) + r = -EFAULT;out:
pte_unmap_unlock(ptep, ptl);
*p_pfn= pfn;
- return 0;
+
+ return r;
}/*
PATCH v2 2/5 KVM: mmu: introduce new gfn_to_pfn_page functions
In reply to David StevensFrom: David Stevens <[email protected]>
Introduce new gfn_to_pfn_page functions that parallel existing gfn_to_pfn functions. The new functions are identical except they take an additional out parameter that is used to return the struct page if the hva was resolved by gup. This allows callers to differentiate the gup and follow_pte cases, which in turn allows callers to only touch the page refcount when necessitated by gup.
The old gfn_to_pfn functions are depreciated, and all callers should be migrated to the new gfn_to_pfn_page functions. In the interim, the gfn_to_pfn functions are reimplemented as wrappers of the corresponding gfn_to_pfn_page functions. The wrappers take a reference to the pfn's page that had previously been taken in hva_to_pfn_remapped.
--- include/linux/kvm_host.h | 17 ++++ virt/kvm/kvm_main.c | 186 ++++++++++++++++++++++++++++----------- 2 files changed, 152 insertions(+), 51 deletions(-)
include/linux/kvm_host.h +17 -0
@@ -820,6 +820,19 @@ kvm_pfn_t __gfn_to_pfn_memslot(struct kvm_memory_slot *slot, gfn_t gfn, bool atomic, bool *async, bool write_fault, bool *writable, hva_t *hva); +kvm_pfn_t gfn_to_pfn_page (struct kvm *kvm,gfn_t gfn,struct page **page); +kvm_pfn_t gfn_to_pfn_page_prot (struct kvm *kvm,gfn_t gfn, +bool write_fault,bool *writable, +struct page **page); +kvm_pfn_t gfn_to_pfn_page_memslot (struct kvm_memory_slot *slot, +gfn_t gfn,struct page **page); +kvm_pfn_t gfn_to_pfn_page_memslot_atomic (struct kvm_memory_slot *slot, +gfn_t gfn,struct page **page); +kvm_pfn_t __gfn_to_pfn_page_memslot (struct kvm_memory_slot *slot, +gfn_t gfn,bool atomic,bool *async, +bool write_fault,bool *writable, +hva_t *hva,struct page **page); +void kvm_release_pfn_clean (kvm_pfn_t pfn);void kvm_release_pfn_dirty (kvm_pfn_t pfn);void kvm_set_pfn_dirty (kvm_pfn_t pfn);
@@ -901,6 +914,10 @@ struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu);struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot (struct kvm_vcpu *vcpu,gfn_t gfn);kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic (struct kvm_vcpu *vcpu,gfn_t gfn);kvm_pfn_t kvm_vcpu_gfn_to_pfn (struct kvm_vcpu *vcpu,gfn_t gfn); +kvm_pfn_t kvm_vcpu_gfn_to_pfn_page_atomic (struct kvm_vcpu *vcpu,gfn_t gfn, +struct page **page); +kvm_pfn_t kvm_vcpu_gfn_to_pfn_page (struct kvm_vcpu *vcpu,gfn_t gfn, +struct page **page);int kvm_vcpu_map (struct kvm_vcpu *vcpu,gpa_t gpa,struct kvm_host_map *map);int kvm_map_gfn (struct kvm_vcpu *vcpu,gfn_t gfn,struct kvm_host_map *map,struct gfn_to_pfn_cache *cache,bool atomic);
virt/kvm/kvm_main.c +135 -51
296 lines not on this page · raw message
Signature · 1 line
2.32.0.93.g670b81a890-goog
PATCH v2 3/5 KVM: x86/mmu: use gfn_to_pfn_page
In reply to David StevensFrom: David Stevens <[email protected]>
Covert usages of the deprecated gfn_to_pfn functions to the new gfn_to_pfn_page functions.
--- arch/x86/kvm/mmu/mmu.c | 43 ++++++++++++++++++++------------- arch/x86/kvm/mmu/mmu_internal.h | 3 ++- arch/x86/kvm/mmu/paging_tmpl.h | 23 +++++++++++------- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++--- arch/x86/kvm/mmu/tdp_mmu.h | 4 +-- arch/x86/kvm/x86.c | 6 +++-- 6 files changed, 51 insertions(+), 34 deletions(-)
arch/x86/kvm/mmu/mmu.c +26 -17
@@ -2702,8 +2702,9 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, u64 *sptep,return ret; } -static kvm_pfn_t pte_prefetch_gfn_to_pfn (struct kvm_vcpu *vcpu,gfn_t gfn, -bool no_dirty_log) +static kvm_pfn_t pte_prefetch_gfn_to_pfn_page (struct kvm_vcpu *vcpu, +gfn_t gfn,bool no_dirty_log, +struct page **page) {struct kvm_memory_slot *slot;
@@ -2711,7 +2712,7 @@ static kvm_pfn_t pte_prefetch_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn,if (!slot)return KVM_PFN_ERR_FAULT ; -return gfn_to_pfn_memslot_atomic (slot, gfn); +return gfn_to_pfn_page_memslot_atomic (slot, gfn, page); } static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu,
@@ -2840,7 +2841,8 @@ int kvm_mmu_max_mapping_level(struct kvm *kvm,int kvm_mmu_hugepage_adjust (struct kvm_vcpu *vcpu,gfn_t gfn,int max_level,kvm_pfn_t *pfnp, -bool huge_page_disallowed,int *req_level) +struct page *page,bool huge_page_disallowed, +int *req_level) {struct kvm_memory_slot *slot;kvm_pfn_t pfn = *pfnp;
@@ -2852,6 +2854,9 @@ int kvm_mmu_hugepage_adjust(struct kvm_vcpu *vcpu, gfn_t gfn,if (unlikely (max_level =PG_LEVEL_4K ))return PG_LEVEL_4K ; +if (!page) +return PG_LEVEL_4K ; +if (is_error_noslot_pfn (pfn) ||kvm_is_reserved_pfn (pfn))return PG_LEVEL_4K ;
@@ -2906,7 +2911,8 @@ void disallowed_hugepage_adjust(u64 spte, gfn_t gfn, int cur_level, }static int __direct_map (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code, -int map_writable,int max_level,kvm_pfn_t pfn, +int map_writable,int max_level, +kvm_pfn_t pfn,struct page *page,bool prefault,bool is_tdp) {bool nx_huge_page_workaround_enabled =is_nx_huge_page_enabled ();
@@ -2919,7 +2925,7 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,gfn_t gfn = gpa >>PAGE_SHIFT ;gfn_t base_gfn = gfn; - level =kvm_mmu_hugepage_adjust (vcpu, gfn, max_level, &pfn, + level =kvm_mmu_hugepage_adjust (vcpu, gfn, max_level, &pfn, page, huge_page_disallowed, &req_level);trace_kvm_mmu_spte_requested (gpa, level, pfn);
@@ -3768,8 +3774,9 @@ static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, }static bool try_async_pf (struct kvm_vcpu *vcpu,bool prefault,gfn_t gfn, -gpa_t cr2_or_gpa,kvm_pfn_t *pfn,hva_t *hva, -bool write,bool *writable) +gpa_t cr2_or_gpa,kvm_pfn_t *pfn, +hva_t *hva,bool write,bool *writable, +struct page **page) {struct kvm_memory_slot *slot =kvm_vcpu_gfn_to_memslot (vcpu, gfn);bool async;
@@ -3790,8 +3797,8 @@ static bool try_async_pf(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn, } async = false; - *pfn =__gfn_to_pfn_memslot (slot, gfn, false, &async, - write, writable, hva); + *pfn =__gfn_to_pfn_page_memslot (slot, gfn, false, &async, + write, writable, hva, page);if (!async)return false;/* *pfn has correct page already */
@@ -3805,8 +3812,8 @@ static bool try_async_pf(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn,return true; } - *pfn =__gfn_to_pfn_memslot (slot, gfn, false,NULL , - write, writable, hva); + *pfn =__gfn_to_pfn_page_memslot (slot, gfn, false,NULL , + write, writable, hva, page);return false; }
@@ -3820,6 +3827,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,gfn_t gfn = gpa >>PAGE_SHIFT ;unsigned long mmu_seq;kvm_pfn_t pfn; +struct page *page;hva_t hva;int r;
@@ -3840,7 +3848,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,smp_rmb ();if (try_async_pf (vcpu, prefault, gfn, gpa, &pfn, &hva, - write, &map_writable)) + write, &map_writable, &page))return RET_PF_RETRY ;if (handle_abnormal_pfn (vcpu, is_tdp ?0 : gpa, gfn, pfn,ACC_ALL , &r))
@@ -3861,17 +3869,18 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,if (is_tdp_mmu_fault) r =kvm_tdp_mmu_map (vcpu, gpa, error_code, map_writable, max_level, - pfn, prefault); + pfn, page, prefault);else - r =__direct_map (vcpu, gpa, error_code, map_writable, max_level, pfn, - prefault, is_tdp); + r =__direct_map (vcpu, gpa, error_code, map_writable, max_level, + pfn, page, prefault, is_tdp); out_unlock:if (is_tdp_mmu_fault)read_unlock (&vcpu->kvm->mmu_lock);else write_unlock (&vcpu->kvm->mmu_lock); -kvm_release_pfn_clean (pfn); +if (page) +put_page (page);return r; }
arch/x86/kvm/mmu/mmu_internal.h +2 -1
@@ -159,7 +159,8 @@ int kvm_mmu_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, int max_level);int kvm_mmu_hugepage_adjust (struct kvm_vcpu *vcpu,gfn_t gfn,int max_level,kvm_pfn_t *pfnp, -bool huge_page_disallowed,int *req_level); +struct page *page,bool huge_page_disallowed, +int *req_level);void disallowed_hugepage_adjust (u64 spte,gfn_t gfn,int cur_level,kvm_pfn_t *pfnp,int *goal_levelp);
arch/x86/kvm/mmu/paging_tmpl.h +14 -9
@@ -564,6 +564,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,unsigned pte_access;gfn_t gfn;kvm_pfn_t pfn; +struct page *page;if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;
@@ -573,8 +574,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, gfn =gpte_to_gfn (gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); - pfn =pte_prefetch_gfn_to_pfn (vcpu, gfn, - no_dirty_log && (pte_access &ACC_WRITE_MASK )); + pfn =pte_prefetch_gfn_to_pfn_page (vcpu, gfn, + no_dirty_log && (pte_access &ACC_WRITE_MASK ), &page);if (is_error_pfn (pfn))return false;
@@ -585,7 +586,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,mmu_set_spte (vcpu, spte, pte_access, false,PG_LEVEL_4K , gfn, pfn, true, true); -kvm_release_pfn_clean (pfn); +if (page) +put_page (page);return true; }
@@ -665,8 +667,8 @@ static void FNAME(pte_prefetch)(struct kvm_vcpu *vcpu, struct guest_walker *gw,*/ static int FNAME (fetch )(struct kvm_vcpu *vcpu,gpa_t addr,struct guest_walker *gw,u32 error_code, -int max_level,kvm_pfn_t pfn,bool map_writable, -bool prefault) +int max_level,kvm_pfn_t pfn,struct page *page, +bool map_writable,bool prefault) {bool nx_huge_page_workaround_enabled =is_nx_huge_page_enabled ();bool write_fault = error_code &PFERR_WRITE_MASK ;
@@ -723,7 +725,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, } level =kvm_mmu_hugepage_adjust (vcpu, gw->gfn, max_level, &pfn, - huge_page_disallowed, &req_level); + page, huge_page_disallowed, + &req_level);trace_kvm_mmu_spte_requested (addr, gw->level, pfn);
@@ -830,6 +833,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,struct guest_walker walker;int r;kvm_pfn_t pfn; +struct page *page;hva_t hva;unsigned long mmu_seq;bool map_writable, is_self_change_mapping;
@@ -882,7 +886,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code,smp_rmb ();if (try_async_pf (vcpu, prefault, walker.gfn, addr, &pfn, &hva, - write_fault, &map_writable)) + write_fault, &map_writable, &page))return RET_PF_RETRY ;if (handle_abnormal_pfn (vcpu, addr, walker.gfn, pfn, walker.pte_access, &r))
@@ -916,13 +920,14 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, gpa_t addr, u32 error_code, r =make_mmu_pages_available (vcpu);if (r) goto out_unlock; - r =FNAME (fetch)(vcpu, addr, &walker, error_code, max_level, pfn, + r =FNAME (fetch)(vcpu, addr, &walker, error_code, max_level, pfn, page, map_writable, prefault);kvm_mmu_audit (vcpu,AUDIT_POST_PAGE_FAULT ); out_unlock:write_unlock (&vcpu->kvm->mmu_lock); -kvm_release_pfn_clean (pfn); +if (page) +put_page (page);return r; }
arch/x86/kvm/mmu/tdp_mmu.c +3 -3
@@ -966,8 +966,8 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, int write,* page tables and SPTEs to translate the faulting guest physical address. */ int kvm_tdp_mmu_map (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code, -int map_writable,int max_level,kvm_pfn_t pfn, -bool prefault) +int map_writable,int max_level, +kvm_pfn_t pfn,struct page *page,bool prefault) {bool nx_huge_page_workaround_enabled =is_nx_huge_page_enabled ();bool write = error_code &PFERR_WRITE_MASK ;
@@ -983,7 +983,7 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,int level;int req_level; - level =kvm_mmu_hugepage_adjust (vcpu, gfn, max_level, &pfn, + level =kvm_mmu_hugepage_adjust (vcpu, gfn, max_level, &pfn, page, huge_page_disallowed, &req_level);trace_kvm_mmu_spte_requested (gpa, level, pfn);
arch/x86/kvm/mmu/tdp_mmu.h +2 -2
@@ -52,8 +52,8 @@ void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm);void kvm_tdp_mmu_zap_invalidated_roots (struct kvm *kvm);int kvm_tdp_mmu_map (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code, -int map_writable,int max_level,kvm_pfn_t pfn, -bool prefault); +int map_writable,int max_level, +kvm_pfn_t pfn,struct page *page,bool prefault);bool kvm_tdp_mmu_unmap_gfn_range (struct kvm *kvm,struct kvm_gfn_range *range,bool flush);
arch/x86/kvm/x86.c +4 -2
@@ -7511,6 +7511,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, {gpa_t gpa = cr2_or_gpa;kvm_pfn_t pfn; +struct page *page;if (!(emulation_type &EMULTYPE_ALLOW_RETRY_PF ))return false;
@@ -7540,7 +7541,7 @@ static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,* retry instruction -> write #PF -> emulation fail -> retry * instruction -> ... */ - pfn =gfn_to_pfn (vcpu->kvm,gpa_to_gfn (gpa)); + pfn =gfn_to_pfn_page (vcpu->kvm,gpa_to_gfn (gpa), &page);/* * If the instruction failed on the error pfn, it can not be fixed,
@@ -7549,7 +7550,8 @@ static bool reexecute_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,if (is_error_noslot_pfn (pfn))return false; -kvm_release_pfn_clean (pfn); +if (page) +put_page (page);/* The instructions are well-emulated on direct mmu. */ if (vcpu->arch.mmu->direct_map) {
Signature · 1 line
2.32.0.93.g670b81a890-goog
PATCH v2 4/5 KVM: arm64/mmu: use gfn_to_pfn_page
In reply to David StevensFrom: David Stevens <[email protected]>
Covert usages of the deprecated gfn_to_pfn functions to the new gfn_to_pfn_page functions.
--- arch/arm64/kvm/mmu.c | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-)
arch/arm64/kvm/mmu.c +16 -10
@@ -780,7 +780,7 @@ static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot,static unsigned long transparent_hugepage_adjust (struct kvm_memory_slot *memslot,unsigned long hva,kvm_pfn_t *pfnp, -phys_addr_t *ipap) +struct page **page,phys_addr_t *ipap) {kvm_pfn_t pfn = *pfnp;
@@ -789,7 +789,7 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,* sure that the HVA and IPA are sufficiently aligned and that the * block map is contained within the memslot. */ -if (kvm_is_transparent_hugepage (pfn) && +if (*page &&kvm_is_transparent_hugepage (pfn) &&fault_supports_stage2_huge_mapping (memslot, hva,PMD_SIZE )) {/* * The address we faulted on is backed by a transparent huge
@@ -810,10 +810,11 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot,* page accordingly. */ *ipap &=PMD_MASK ; -kvm_release_pfn_clean (pfn); +put_page (*page); pfn &= ~(PTRS_PER_PMD -1 ); -kvm_get_pfn (pfn); *pfnp = pfn; + *page =pfn_to_page (pfn); +get_page (*page);return PMD_SIZE ; }
@@ -837,6 +838,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,short vma_shift;gfn_t gfn;kvm_pfn_t pfn; +struct page *page;bool logging_active =memslot_is_logging (memslot);unsigned long fault_level =kvm_vcpu_trap_get_fault_level (vcpu);unsigned long vma_pagesize, fault_granule;
@@ -933,8 +935,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,*/ smp_rmb (); - pfn =__gfn_to_pfn_memslot (memslot, gfn, false,NULL , - write_fault, &writable,NULL ); + pfn =__gfn_to_pfn_page_memslot (memslot, gfn, false,NULL , + write_fault, &writable,NULL , &page);if (pfn =KVM_PFN_ERR_HWPOISON ) {kvm_send_hwpoison_signal (hva, vma_shift);return 0 ;
@@ -967,7 +969,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,*/ if (vma_pagesize =PAGE_SIZE && !force_pte) vma_pagesize =transparent_hugepage_adjust (memslot, hva, - &pfn, &fault_ipa); + &pfn, &page, + &fault_ipa);if (writable) prot |=KVM_PGTABLE_PROT_W ;
@@ -999,14 +1002,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,/* Mark the page dirty only if the fault is handled successfully */ if (writable && !ret) { -kvm_set_pfn_dirty (pfn); +if (page) +kvm_set_pfn_dirty (pfn);mark_page_dirty_in_slot (kvm, memslot, gfn); } out_unlock:spin_unlock (&kvm->mmu_lock); -kvm_set_pfn_accessed (pfn); -kvm_release_pfn_clean (pfn); +if (page) { +kvm_set_pfn_accessed (pfn); +put_page (page); + }return ret != -EAGAIN ? ret :0 ; }
Signature · 1 line
2.32.0.93.g670b81a890-goog
PATCH v2 5/5 KVM: mmu: remove over-aggressive warnings
In reply to David StevensFrom: David Stevens <[email protected]>
Remove two warnings that require ref counts for pages to be non-zero, as mapped pfns from follow_pfn may not have an initialized ref count.
--- arch/x86/kvm/mmu/mmu.c | 7 ------- virt/kvm/kvm_main.c | 2 +- 2 files changed, 1 insertion(+), 8 deletions(-)
arch/x86/kvm/mmu/mmu.c +0 -7
@@ -607,13 +607,6 @@ static int mmu_spte_clear_track_bits(u64 *sptep) pfn =spte_to_pfn (old_spte); -/* -* KVM does not hold the refcount of the page used by -* kvm mmu, before reclaiming the page, we should -* unmap it from mmu first. -*/ -WARN_ON (!kvm_is_reserved_pfn (pfn) && !page_count (pfn_to_page (pfn))); -if (is_accessed_spte (old_spte))kvm_set_pfn_accessed (pfn);
virt/kvm/kvm_main.c +1 -1
@@ -168,7 +168,7 @@ bool kvm_is_zone_device_pfn(kvm_pfn_t pfn)* the device has been pinned, e.g. by get_user_pages(). WARN if the * page_count() is zero to help detect bad usage of this helper. */ -if (!pfn_valid (pfn) ||WARN_ON_ONCE (!page_count (pfn_to_page (pfn)))) +if (!pfn_valid (pfn) || !page_count (pfn_to_page (pfn)))return false;return is_zone_device_page (pfn_to_page (pfn));
Signature · 1 line
2.32.0.93.g670b81a890-goog
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BUILD: failure for Remove uses of struct page from x86 and arm64 MMU
In reply to David Stevens== Series Details ==
== Series Details ==
Series: Remove uses of struct page from x86 and arm64 MMU
URL : https://patchwork.freedesktop.org/series/91908/
State : failure
== Summary ==
Applying: KVM: do not allow mapping valid but non-refcounted pages
Applying: KVM: mmu: introduce new gfn_to_pfn_page functions
Applying: KVM: x86/mmu: use gfn_to_pfn_page
error: sha1 information is lacking or useless (arch/x86/kvm/mmu/mmu.c).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch Patch failed at 0003 KVM: x86/mmu: use gfn_to_pfn_page
When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx