KVM: MMU: Fix a refcount bug with ZONE_DEVICE pages
v2 cover KVM: MMU: Fix a refcount bug with ZONE_DEVICE pages
This mini-series fixes a suspected, but technically unconfirmed, bug in KVM related to ZONE_DEVICE pages. The suspected issue is that KVM treats ZONE_DEVICE pages as reserved PFNs, and so doesn't put references to such pages when dropping references via KVM's generic kvm_release_pfn_clean().
David Hildenbrand uncovered the bug during a discussion about removing PG_reserved from ZONE_DEVICE pages, after Dan Williams pointed out[1] that there was a bug report from Adam Borowski[2] that was likely related to KVM's interaction with PageReserved().
Patch 1/3 contains the actual fix, patches 2/3 and 3/3 are minor cleanup that is mostly unrelated, but dependent and prompted by the fix itself.
v2:
- Remove the kvm_is_zone_device_pfn(pfn) check from kvm_get_pfn(). It's not entirely clear whether or not the hva_to_pfn_remapped() case is actually broken, e.g. KVM's page fault handler is likely ok, whereas not calling get_page() willl definitely cause breakage as KVM would later call put_page() on the pfn/page. [Paolo]
- WARN if kvm_is_zone_device_pfn() is called without the underlying page being pinned. This won't necessarily catch all bugs, e.g. if the above hva_to_pfn_remapped case is indeed broken, but will prevent completely bogus usage. [Dan]
- Remove the is_error_pfn() check from transparent_hugepage_adjust() instead of carrying it forward into the new kvm_is_hugepage_allowed() helper. [Paolo]
[1] http://lkml.kernel.org/r/[email protected]
[2] https://lkml.kernel.org/r/[email protected]
Sean Christopherson (3):
KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
KVM: x86/mmu: Remove superfluous is_error_pfn() check from THP adjust
KVM: x86/mmu: Add helper to consolidate huge page promotion
arch/x86/kvm/mmu.c | 15 +++++++++------ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 26 +++++++++++++++++++++++--- 3 files changed, 33 insertions(+), 9 deletions(-)
Signature · 1 line
2.24.0
PATCH v2 1/3 KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
Explicitly exempt ZONE_DEVICE pages from kvm_is_reserved_pfn() and instead manually handle ZONE_DEVICE on a case-by-case basis. For things like page refcounts, KVM needs to treat ZONE_DEVICE pages like normal pages, e.g. put pages grabbed via gup(). But for flows such as setting A/D bits or shifting refcounts for transparent huge pages, KVM needs to to avoid processing ZONE_DEVICE pages as the flows in question lack the underlying machinery for proper handling of ZONE_DEVICE pages.
This fixes a hang reported by Adam Borowski[*] in dev_pagemap_cleanup() when running a KVM guest backed with /dev/dax memory, as KVM straight up doesn't put any references to ZONE_DEVICE pages acquired by gup().
Note, Dan Williams proposed an alternative solution of doing put_page() on ZONE_DEVICE pages immediately after gup() in order to simplify the auditing needed to ensure is_zone_device_page() is called if and only if the backing device is pinned (via gup()). But that approach would break kvm_vcpu_{un}map() as KVM requires the page to be pinned from map() 'til unmap() when accessing guest memory, unlike KVM's secondary MMU, which coordinates with mmu_notifier invalidations to avoid creating stale page references, i.e. doesn't rely on pages being pinned.
[*] http://lkml.kernel.org/r/[email protected]
--- arch/x86/kvm/mmu.c | 8 ++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 26 +++++++++++++++++++++++--- 3 files changed, 28 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu.c +4 -4
@@ -3306,7 +3306,7 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu,* here. */ if (!is_error_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && - level ==PT_PAGE_TABLE_LEVEL && + !kvm_is_zone_device_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL &&PageTransCompoundMap (pfn_to_page (pfn)) && !mmu_gfn_lpage_is_disallowed (vcpu, gfn,PT_DIRECTORY_LEVEL )) {unsigned long mask;
@@ -5914,9 +5914,9 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,* the guest, and the guest page table is using 4K page size * mapping if the indirect sp has level = 1. */ -if (sp->role.direct && - !kvm_is_reserved_pfn (pfn) && -PageTransCompoundMap (pfn_to_page (pfn))) { +if (sp->role.direct && !kvm_is_reserved_pfn (pfn) && + !kvm_is_zone_device_pfn (pfn) && +PageTransCompoundMap (pfn_to_page (pfn))) {pte_list_remove (rmap_head, sptep);if (kvm_available_flush_tlb_with_range ())
include/linux/kvm_host.h +1 -0
@@ -966,6 +966,7 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu);void kvm_vcpu_kick (struct kvm_vcpu *vcpu);bool kvm_is_reserved_pfn (kvm_pfn_t pfn); +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn);struct kvm_irq_ack_notifier {struct hlist_node link;
virt/kvm/kvm_main.c +23 -3
@@ -149,10 +149,30 @@ __weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,return 0 ; } +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn) +{ +/* +* The metadata used by is_zone_device_page() to determine whether or +* not a page is ZONE_DEVICE is guaranteed to be valid if and only if +* the device has been pinned, e.g. by get_user_pages(). WARN if the +* page_count() is zero to help detect bad usage of this helper. +*/ +if (!pfn_valid (pfn) ||WARN_ON_ONCE (!page_count (pfn_to_page (pfn)))) +return false; + +return is_zone_device_page (pfn_to_page (pfn)); +} +bool kvm_is_reserved_pfn (kvm_pfn_t pfn) { +/* +* ZONE_DEVICE pages currently set PG_reserved, but from a refcounting +* perspective they are "normal" pages, albeit with slightly different +* usage rules. +*/ if (pfn_valid (pfn)) -return PageReserved (pfn_to_page (pfn)); +return PageReserved (pfn_to_page (pfn)) && + !kvm_is_zone_device_pfn (pfn);return true; }
@@ -1865,7 +1885,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty);void kvm_set_pfn_dirty (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) { +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn)) {struct page *page =pfn_to_page (pfn);SetPageDirty (page);
@@ -1875,7 +1895,7 @@ EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty);void kvm_set_pfn_accessed (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn))mark_page_accessed (pfn_to_page (pfn)); }EXPORT_SYMBOL_GPL (kvm_set_pfn_accessed);
Signature · 1 line
2.24.0
Re: [PATCH v2 1/3] KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
Quoted 119 lines · Sean Christopherson
Explicitly exempt
ZONE_DEVICEpages fromkvm_is_reserved_pfn()and instead manually handleZONE_DEVICEon a case-by-case basis. For things like page refcounts, KVM needs to treatZONE_DEVICEpages like normal pages, e.g. put pages grabbed viagup(). But for flows such as setting A/D bits or shifting refcounts for transparent huge pages, KVM needs to to avoid processingZONE_DEVICEpages as the flows in question lack the underlying machinery for proper handling ofZONE_DEVICEpages.This fixes a hang reported by Adam Borowski[*] in
dev_pagemap_cleanup()when running a KVM guest backed with/dev/daxmemory, as KVM straight up doesn't put any references toZONE_DEVICEpages acquired bygup().Note, Dan Williams proposed an alternative solution of doing
put_page()onZONE_DEVICEpages immediately aftergup()in order to simplify the auditing needed to ensureis_zone_device_page()is called if and only if the backing device is pinned (viagup()). But that approach would break kvm_vcpu_{un}map() as KVM requires the page to be pinned frommap()'tilunmap()when accessing guest memory, unlike KVM's secondary MMU, which coordinates withmmu_notifierinvalidations to avoid creating stale page references, i.e. doesn't rely on pages being pinned.[*] http://lkml.kernel.org/r/[email protected]
Reported-by Adam Borowski <[email protected]>Debugged-by David Hildenbrand <[email protected]>Cc Dan Williams <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/mmu.c | 8 ++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 26 +++++++++++++++++++++++--- 3 files changed, 28 insertions(+), 7 deletions(-)arch/x86/kvm/mmu.c +4
-4@@ -3306,7 +3306,7 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu,* here. */ if (!is_error_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && - level ==PT_PAGE_TABLE_LEVEL && + !kvm_is_zone_device_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL &&PageTransCompoundMap (pfn_to_page (pfn)) && !mmu_gfn_lpage_is_disallowed (vcpu, gfn,PT_DIRECTORY_LEVEL )) {unsigned long mask;@@ -5914,9 +5914,9 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,* the guest, and the guest page table is using 4K page size * mapping if the indirect sp has level = 1. */ -if (sp->role.direct && - !kvm_is_reserved_pfn (pfn) && -PageTransCompoundMap (pfn_to_page (pfn))) { +if (sp->role.direct && !kvm_is_reserved_pfn (pfn) && + !kvm_is_zone_device_pfn (pfn) && +PageTransCompoundMap (pfn_to_page (pfn))) {pte_list_remove (rmap_head, sptep);if (kvm_available_flush_tlb_with_range ())include/linux/kvm_host.h +1
-0@@ -966,6 +966,7 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu);void kvm_vcpu_kick (struct kvm_vcpu *vcpu);bool kvm_is_reserved_pfn (kvm_pfn_t pfn); +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn);struct kvm_irq_ack_notifier {struct hlist_node link;virt/kvm/kvm_main.c +22
-2@@ -149,10 +149,30 @@ __weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,return 0 ; } +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn) +{ +/* +* The metadata used by is_zone_device_page() to determine whether or +* not a page is ZONE_DEVICE is guaranteed to be valid if and only if +* the device has been pinned, e.g. by get_user_pages(). WARN if the +* page_count() is zero to help detect bad usage of this helper. +*/ +if (!pfn_valid (pfn) ||WARN_ON_ONCE (!page_count (pfn_to_page (pfn)))) +return false; + +return is_zone_device_page (pfn_to_page (pfn)); +} +bool kvm_is_reserved_pfn (kvm_pfn_t pfn) { +/* +* ZONE_DEVICE pages currently set PG_reserved, but from a refcounting +* perspective they are "normal" pages, albeit with slightly different +* usage rules. +*/ if (pfn_valid (pfn)) -return PageReserved (pfn_to_page (pfn)); +return PageReserved (pfn_to_page (pfn)) && + !kvm_is_zone_device_pfn (pfn);return true; }@@ -1865,7 +1885,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty);void kvm_set_pfn_dirty (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) { +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn)) {struct page *page =pfn_to_page (pfn);SetPageDirty (page);
virt/kvm/kvm_main.c EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); · line 1901
Quoted 8 lines
virt/kvm/kvm_main.c +1
-1@@ -1875,7 +1895,7 @@ EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty);void kvm_set_pfn_accessed (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn))mark_page_accessed (pfn_to_page (pfn)); }EXPORT_SYMBOL_GPL (kvm_set_pfn_accessed);
Queued, thanks -- the other two will wait for after the merge window to avoid pointless conflicts.
Paolo
Re: [PATCH v2 1/3] KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
In reply to Sean ChristophersonQuoted 22 lines · Sean Christopherson
Explicitly exempt
ZONE_DEVICEpages fromkvm_is_reserved_pfn()and instead manually handleZONE_DEVICEon a case-by-case basis. For things like page refcounts, KVM needs to treatZONE_DEVICEpages like normal pages, e.g. put pages grabbed viagup(). But for flows such as setting A/D bits or shifting refcounts for transparent huge pages, KVM needs to to avoid processingZONE_DEVICEpages as the flows in question lack the underlying machinery for proper handling ofZONE_DEVICEpages.This fixes a hang reported by Adam Borowski[*] in
dev_pagemap_cleanup()when running a KVM guest backed with/dev/daxmemory, as KVM straight up doesn't put any references toZONE_DEVICEpages acquired bygup().Note, Dan Williams proposed an alternative solution of doing
put_page()onZONE_DEVICEpages immediately aftergup()in order to simplify the auditing needed to ensureis_zone_device_page()is called if and only if the backing device is pinned (viagup()). But that approach would break kvm_vcpu_{un}map() as KVM requires the page to be pinned frommap()'tilunmap()when accessing guest memory, unlike KVM's secondary MMU, which coordinates withmmu_notifierinvalidations to avoid creating stale page references, i.e. doesn't rely on pages being pinned.
Quoted 5 lines
[*] http://lkml.kernel.org/r/[email protected]
Reported-by Adam Borowski <[email protected]>Debugged-by David Hildenbrand <[email protected]>Cc Dan Williams <[email protected]>
Quoted 1 line
Perhaps add:
...since that was the first kernel that broke KVM's assumption about which pfn types needed to have the reference count managed.
Paolo Bonzini
maintainerRe: [PATCH v2 1/3] KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
Quoted 7 lines · Dan Williams
Quoted 5 lines
[*] http://lkml.kernel.org/r/[email protected]
Reported-by Adam Borowski <[email protected]>Debugged-by David Hildenbrand <[email protected]>Cc Dan Williams <[email protected]>Acked-by Dan Williams <[email protected]>
Quoted 8 lines
Quoted 1 line
Perhaps add:
Fixes 3565fce3a659 ("mm, x86: get_user_pages() for dax mappings")...since that was the first kernel that broke KVM's assumption about which pfn types needed to have the reference count managed.
Done, thanks!
Paolo
Re: [PATCH v2 1/3] KVM: MMU: Do not treat ZONE_DEVICE pages as being reserved
In reply to Sean ChristophersonQuoted 32 lines
Am 11.11.2019 um 23:12 schrieb Sean Christopherson <[email protected]>:
Explicitly exempt
ZONE_DEVICEpages fromkvm_is_reserved_pfn()and instead manually handleZONE_DEVICEon a case-by-case basis. For things like page refcounts, KVM needs to treatZONE_DEVICEpages like normal pages, e.g. put pages grabbed viagup(). But for flows such as setting A/D bits or shifting refcounts for transparent huge pages, KVM needs to to avoid processingZONE_DEVICEpages as the flows in question lack the underlying machinery for proper handling ofZONE_DEVICEpages.This fixes a hang reported by Adam Borowski[*] in
dev_pagemap_cleanup()when running a KVM guest backed with/dev/daxmemory, as KVM straight up doesn't put any references toZONE_DEVICEpages acquired bygup().Note, Dan Williams proposed an alternative solution of doing
put_page()onZONE_DEVICEpages immediately aftergup()in order to simplify the auditing needed to ensureis_zone_device_page()is called if and only if the backing device is pinned (viagup()). But that approach would break kvm_vcpu_{un}map() as KVM requires the page to be pinned frommap()'tilunmap()when accessing guest memory, unlike KVM's secondary MMU, which coordinates withmmu_notifierinvalidations to avoid creating stale page references, i.e. doesn't rely on pages being pinned.[*] http://lkml.kernel.org/r/[email protected]
Reported-by Adam Borowski <[email protected]>Debugged-by David Hildenbrand <[email protected]>Cc Dan Williams <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 6 lines
---
arch/x86/kvm/mmu.c | 8 ++++---- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 26 +++++++++++++++++++++++--- 3 files changed, 28 insertions(+), 7 deletions(-)
Thanks for taking care of this! Other KVM related code (PPC, vfio) also has a reserved check (see my series), I didn‘t have a look yet at the details, how reserved pages are treated. Will do so in the next weeks, after hollidays.
Quoted 95 lines
arch/x86/kvm/mmu.c +4
-4@@ -3306,7 +3306,7 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu,* here. */ if (!is_error_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && - level ==PT_PAGE_TABLE_LEVEL && + !kvm_is_zone_device_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL &&PageTransCompoundMap (pfn_to_page (pfn)) && !mmu_gfn_lpage_is_disallowed (vcpu, gfn,PT_DIRECTORY_LEVEL )) {unsigned long mask;@@ -5914,9 +5914,9 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,* the guest, and the guest page table is using 4K page size * mapping if the indirect sp has level = 1. */ -if (sp->role.direct && - !kvm_is_reserved_pfn (pfn) && -PageTransCompoundMap (pfn_to_page (pfn))) { +if (sp->role.direct && !kvm_is_reserved_pfn (pfn) && + !kvm_is_zone_device_pfn (pfn) && +PageTransCompoundMap (pfn_to_page (pfn))) {pte_list_remove (rmap_head, sptep);if (kvm_available_flush_tlb_with_range ())include/linux/kvm_host.h +1
-0@@ -966,6 +966,7 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu);void kvm_vcpu_kick (struct kvm_vcpu *vcpu);bool kvm_is_reserved_pfn (kvm_pfn_t pfn); +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn);struct kvm_irq_ack_notifier {struct hlist_node link;virt/kvm/kvm_main.c +23
-3@@ -149,10 +149,30 @@ __weak int kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,return 0 ; } +bool kvm_is_zone_device_pfn (kvm_pfn_t pfn) +{ +/* +* The metadata used by is_zone_device_page() to determine whether or +* not a page is ZONE_DEVICE is guaranteed to be valid if and only if +* the device has been pinned, e.g. by get_user_pages(). WARN if the +* page_count() is zero to help detect bad usage of this helper. +*/ +if (!pfn_valid (pfn) ||WARN_ON_ONCE (!page_count (pfn_to_page (pfn)))) +return false; + +return is_zone_device_page (pfn_to_page (pfn)); +} +bool kvm_is_reserved_pfn (kvm_pfn_t pfn) { +/* +* ZONE_DEVICE pages currently set PG_reserved, but from a refcounting +* perspective they are "normal" pages, albeit with slightly different +* usage rules. +*/ if (pfn_valid (pfn)) -return PageReserved (pfn_to_page (pfn)); +return PageReserved (pfn_to_page (pfn)) && + !kvm_is_zone_device_pfn (pfn);return true; }@@ -1865,7 +1885,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty);void kvm_set_pfn_dirty (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) { +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn)) {struct page *page =pfn_to_page (pfn);SetPageDirty (page);@@ -1875,7 +1895,7 @@ EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty);void kvm_set_pfn_accessed (kvm_pfn_t pfn) { -if (!kvm_is_reserved_pfn (pfn)) +if (!kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn))mark_page_accessed (pfn_to_page (pfn)); }EXPORT_SYMBOL_GPL (kvm_set_pfn_accessed);Signature · 1 line
2.24.0
PATCH v2 2/3 KVM: x86/mmu: Remove superfluous is_error_pfn() check from THP adjust
In reply to Sean ChristophersonReplace the is_error_noslot_pfn() check in transparent_hugepage_adjust() with an is_noslot_pfn() check. thp_adjust() cannot be reached with an error pfn as it is always called after handle_abnormal_pfn(), which aborts the page fault handler if an error pfn is encountered. Don't bother future proofing thp_adjust() with a WARN on is_error_pfn(), as calling thp_adjust() before handle_abnormal_pfn() is impossible for all intents and purposes, e.g. thp_adjust() relies on being called after mmu_notifier_retry() and while holding mmu_lock, thus moving it would essentially require a complete rewrite of KVM's page fault handlers.
No functional change intended.
--- arch/x86/kvm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu.c +1 -1
@@ -3305,7 +3305,7 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu,* PT_PAGE_TABLE_LEVEL and there would be no adjustment done * here. */ -if (!is_error_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && +if (!is_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL &&PageTransCompoundMap (pfn_to_page (pfn)) && !mmu_gfn_lpage_is_disallowed (vcpu, gfn,PT_DIRECTORY_LEVEL )) {
Signature · 1 line
2.24.0
PATCH v2 3/3 KVM: x86/mmu: Add helper to consolidate huge page promotion
In reply to Sean ChristophersonAdd a helper, kvm_is_hugepage_allowed(), to consolidate identical code across transparent_hugepage_adjust() and kvm_mmu_zap_collapsible_spte().
No functional change intended.
--- arch/x86/kvm/mmu.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
arch/x86/kvm/mmu.c +9 -6
@@ -3292,6 +3292,12 @@ static int kvm_handle_bad_page(struct kvm_vcpu *vcpu, gfn_t gfn, kvm_pfn_t pfn)return -EFAULT ; } +static bool kvm_is_hugepage_allowed (kvm_pfn_t pfn) +{ +return !kvm_is_reserved_pfn (pfn) && !kvm_is_zone_device_pfn (pfn) && +PageTransCompoundMap (pfn_to_page (pfn)); +} +static void transparent_hugepage_adjust (struct kvm_vcpu *vcpu,gfn_t gfn,kvm_pfn_t *pfnp,int *levelp)
@@ -3305,9 +3311,8 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu,* PT_PAGE_TABLE_LEVEL and there would be no adjustment done * here. */ -if (!is_noslot_pfn (pfn) && !kvm_is_reserved_pfn (pfn) && - !kvm_is_zone_device_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL && -PageTransCompoundMap (pfn_to_page (pfn)) && +if (!is_noslot_pfn (pfn) && level ==PT_PAGE_TABLE_LEVEL && +kvm_is_hugepage_allowed (pfn) && !mmu_gfn_lpage_is_disallowed (vcpu, gfn,PT_DIRECTORY_LEVEL )) {unsigned long mask;/*
@@ -5914,9 +5919,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,* the guest, and the guest page table is using 4K page size * mapping if the indirect sp has level = 1. */ -if (sp->role.direct && !kvm_is_reserved_pfn (pfn) && - !kvm_is_zone_device_pfn (pfn) && -PageTransCompoundMap (pfn_to_page (pfn))) { +if (sp->role.direct &&kvm_is_hugepage_allowed (pfn)) {pte_list_remove (rmap_head, sptep);if (kvm_available_flush_tlb_with_range ())
Signature · 1 line
2.24.0