KVM: x86/mmu: Use separate namespaces gPTEs and SPTEs
Sean Christopherson
maintainerv2 cover KVM: x86/mmu: Use separate namespaces gPTEs and SPTEs
This is based on kvm/queue + "Remove unused PT32_DIR_BASE_ADDR_MASK from mmu.c", so I think it should apply cleanly on top of what you already grabbed.
Clean up the x86 MMU code to use separate namespaces for guest PTEs and shadow PTEs. While there is most definitely overlap, especially in attribute bits, the rules for walking gPTEs and for generating/walking SPTEs differ in subtle ways, e.g. see commit fc9bf2e087ef ("KVM: x86/mmu:
Do not apply HPA (memory encryption) mask to GPAs"). The paging32 macros in particular should never be used outside of paging_tmpl.h.
Separating gPTEs from SPTEs actually provides for nice cleanups (see the diffstat) as KVM has ended up with a fair bit of copy+paste code that can be deduplicated once KVM isn't trying to use PT64_* defines for both gPTEs and SPTEs.
This is a spiritual successor to patches 4-7 of the series[*] that added the aformentenioned commit.
v2:
- Don't move is_cpuid_PSE36(). [Lai Jiangshan]
- Change author for patch 1 (yeah, I was lazy). [Lai Jiangshan]
- Fix a random typo in patch 7's changelog.
- Rebase to play nice with PT32_DIR_BASE_ADDR_MASK.
v1: https://lore.kernel.org/all/[email protected]
[*] https://lore.kernel.org/all/[email protected]
Lai Jiangshan (1):
KVM: x86/mmu: Drop unused CMPXCHG macro from paging_tmpl.h
Sean Christopherson (7):
KVM: VMX: Refactor 32-bit PSE PT creation to avoid using MMU macro
KVM: x86/mmu: Bury 32-bit PSE paging helpers in paging_tmpl.h
KVM: x86/mmu: Dedup macros for computing various page table masks
KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
KVM: x86/mmu: Use common macros to compute 32/64-bit paging masks
KVM: x86/mmu: Truncate paging32's PT_BASE_ADDR_MASK to 32 bits
KVM: x86/mmu: Use common logic for computing the 32/64-bit base PA
mask
arch/x86/kvm/mmu.h | 10 ------ arch/x86/kvm/mmu/mmu.c | 57 +++++++++------------------------ arch/x86/kvm/mmu/mmu_internal.h | 17 ++++++++++ arch/x86/kvm/mmu/paging.h | 14 -------- arch/x86/kvm/mmu/paging_tmpl.h | 55 ++++++++++++++++--------------- arch/x86/kvm/mmu/spte.c | 2 +- arch/x86/kvm/mmu/spte.h | 28 +++++++--------- arch/x86/kvm/mmu/tdp_iter.c | 6 ++-- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++-- arch/x86/kvm/vmx/vmx.c | 2 +- 10 files changed, 82 insertions(+), 115 deletions(-) delete mode 100644 arch/x86/kvm/mmu/paging.h
base-commit: ccf1c220033d8d6fe50d8d11daa3dec5640f8c4d
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 1/8 KVM: x86/mmu: Drop unused CMPXCHG macro from paging_tmpl.h
From: Lai Jiangshan <[email protected]>
Drop the CMPXCHG macro from paging_tmpl.h, it's no longer used now that KVM uses a common uaccess helper to do 8-byte CMPXCHG.
--- arch/x86/kvm/mmu/paging_tmpl.h | 6 ------ 1 file changed, 6 deletions(-)
arch/x86/kvm/mmu/paging_tmpl.h +0 -6
@@ -34,7 +34,6 @@#define PT_HAVE_ACCESSED_DIRTY (mmu) true#ifdef CONFIG_X86_64 #define PT_MAX_FULL_LEVELS PT64_ROOT_MAX_LEVEL -#define CMPXCHG "cmpxchgq"#else #define PT_MAX_FULL_LEVELS 2#endif
@@ -51,7 +50,6 @@#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT#define PT_HAVE_ACCESSED_DIRTY (mmu) true -#define CMPXCHG "cmpxchgl"#elif PTTYPE ==PTTYPE_EPT #define pt_element_t u64#define guest_walker guest_walkerEPT
@@ -64,9 +62,6 @@#define PT_GUEST_DIRTY_SHIFT 9#define PT_GUEST_ACCESSED_SHIFT 8#define PT_HAVE_ACCESSED_DIRTY (mmu) (!(mmu)->cpu_role.base.ad_disabled) -#ifdef CONFIG_X86_64 -#define CMPXCHG "cmpxchgq" -#endif #define PT_MAX_FULL_LEVELS PT64_ROOT_MAX_LEVEL#else #error Invalid PTTYPE value
@@ -1100,7 +1095,6 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)#undef PT_MAX_FULL_LEVELS#undef gpte_to_gfn#undef gpte_to_gfn_lvl -#undef CMPXCHG#undef PT_GUEST_ACCESSED_MASK#undef PT_GUEST_DIRTY_MASK#undef PT_GUEST_DIRTY_SHIFT
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 2/8 KVM: VMX: Refactor 32-bit PSE PT creation to avoid using MMU macro
In reply to Sean ChristophersonCompute the number of PTEs to be filled for the 32-bit PSE page tables using the page size and the size of each entry. While using the MMU's PT32_ENT_PER_PAGE macro is arguably better in isolation, removing VMX's usage will allow a future namespacing cleanup to move the guest page table macros into paging_tmpl.h, out of the reach of code that isn't directly related to shadow paging.
No functional change intended.
--- arch/x86/kvm/vmx/vmx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -3704,7 +3704,7 @@ static int init_rmode_identity_map(struct kvm *kvm) }/* Set up identity-mapping pagetable for EPT in real mode */ -for (i =0 ; i <PT32_ENT_PER_PAGE ; i++) { +for (i =0 ; i < (PAGE_SIZE /sizeof (tmp)); i++) { tmp = (i <<22 ) + (_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE);if (__copy_to_user (uaddr + i *sizeof (tmp), &tmp,sizeof (tmp))) {
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 3/8 KVM: x86/mmu: Bury 32-bit PSE paging helpers in paging_tmpl.h
In reply to Sean ChristophersonMove a handful of one-off macros and helpers for 32-bit PSE paging into paging_tmpl.h and hide them behind "PTTYPE == 32". Under no circumstance should anything but 32-bit shadow paging care about PSE paging.
No functional change intended.
--- arch/x86/kvm/mmu.h | 5 ----- arch/x86/kvm/mmu/mmu.c | 7 ------- arch/x86/kvm/mmu/paging_tmpl.h | 18 +++++++++++++++++- 3 files changed, 17 insertions(+), 13 deletions(-)
arch/x86/kvm/mmu.h +0 -5
@@ -34,11 +34,6 @@#define PT_DIR_PAT_SHIFT 12#define PT_DIR_PAT_MASK (1ULL << PT_DIR_PAT_SHIFT) -#define PT32_DIR_PSE36_SIZE 4 -#define PT32_DIR_PSE36_SHIFT 13 -#define PT32_DIR_PSE36_MASK \ - (((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT) -#define PT64_ROOT_5LEVEL 5#define PT64_ROOT_4LEVEL 4#define PT32_ROOT_LEVEL 2
arch/x86/kvm/mmu/mmu.c +0 -7
@@ -324,13 +324,6 @@ static int is_cpuid_PSE36(void)return 1 ; } -static gfn_t pse36_gfn_delta (u32 gpte) -{ -int shift =32 -PT32_DIR_PSE36_SHIFT -PAGE_SHIFT ; - -return (gpte &PT32_DIR_PSE36_MASK ) << shift; -} -#ifdef CONFIG_X86_64 static void __set_spte (u64 *sptep,u64 spte) {
arch/x86/kvm/mmu/paging_tmpl.h +17 -1
@@ -50,6 +50,11 @@#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT#define PT_HAVE_ACCESSED_DIRTY (mmu) true + +#define PT32_DIR_PSE36_SIZE 4 +#define PT32_DIR_PSE36_SHIFT 13 +#define PT32_DIR_PSE36_MASK \ + (((1ULL << PT32_DIR_PSE36_SIZE) - 1) << PT32_DIR_PSE36_SHIFT)#elif PTTYPE ==PTTYPE_EPT #define pt_element_t u64#define guest_walker guest_walkerEPT
@@ -92,6 +97,15 @@ struct guest_walker {struct x86_exception fault; }; +#if PTTYPE ==32 +static inline gfn_t pse36_gfn_delta (u32 gpte) +{ +int shift =32 -PT32_DIR_PSE36_SHIFT -PAGE_SHIFT ; + +return (gpte &PT32_DIR_PSE36_MASK ) << shift; +} +#endif +static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) {return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ;
@@ -416,8 +430,10 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, gfn =gpte_to_gfn_lvl (pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ; -if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ()) +#if PTTYPE ==32 +if (walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ()) gfn +=pse36_gfn_delta (pte); +#endif real_gpa =kvm_translate_gpa (vcpu, mmu,gfn_to_gpa (gfn), access, &walker->fault);if (real_gpa ==UNMAPPED_GVA )
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 4/8 KVM: x86/mmu: Dedup macros for computing various page table masks
In reply to Sean ChristophersonProvide common helper macros to generate various masks, shifts, etc... for 32-bit vs. 64-bit page tables. Only the inputs differ, the actual calculations are identical.
No functional change intended.
--- arch/x86/kvm/mmu.h | 4 ++-- arch/x86/kvm/mmu/mmu.c | 14 +++++--------- arch/x86/kvm/mmu/mmu_internal.h | 14 ++++++++++++++ arch/x86/kvm/mmu/paging.h | 9 +++++---- arch/x86/kvm/mmu/spte.h | 7 +++---- 5 files changed, 29 insertions(+), 19 deletions(-)
arch/x86/kvm/mmu.h +2 -2
@@ -7,9 +7,9 @@#include "cpuid.h" #define PT64_PT_BITS 9 -#define PT64_ENT_PER_PAGE (1 << PT64_PT_BITS) +#define PT64_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT64_PT_BITS)#define PT32_PT_BITS 10 -#define PT32_ENT_PER_PAGE (1 << PT32_PT_BITS) +#define PT32_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT32_PT_BITS)#define PT_WRITABLE_SHIFT 1#define PT_USER_SHIFT 2
arch/x86/kvm/mmu/mmu.c +5 -9
@@ -113,21 +113,17 @@ module_param(dbg, bool, 0644);#define PT32_LEVEL_BITS 10 -#define PT32_LEVEL_SHIFT (level) \ - (PAGE_SHIFT + (level - 1) * PT32_LEVEL_BITS) +#define PT32_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS)#define PT32_LVL_OFFSET_MASK (level) \ - (PT32_BASE_ADDR_MASK & ((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT32_LEVEL_BITS))) - 1)) - -#define PT32_INDEX (address, level)\ - (((address) >> PT32_LEVEL_SHIFT(level)) & ((1 << PT32_LEVEL_BITS) - 1)) + __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) +#define PT32_INDEX (address, level) __PT_INDEX(address, level, PT32_LEVEL_BITS)#define PT32_BASE_ADDR_MASK PAGE_MASK +#define PT32_LVL_ADDR_MASK (level) \ - (PAGE_MASK & ~((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT32_LEVEL_BITS))) - 1)) + __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS)#include <trace/events/kvm.h>
arch/x86/kvm/mmu/mmu_internal.h +14 -0
@@ -20,6 +20,20 @@ extern bool dbg;#define MMU_WARN_ON (x) do { } while (0)#endif +/* Page table builder macros common to shadow (host) PTEs and guest PTEs. */ +#define __PT_LEVEL_SHIFT (level, bits_per_level) \ + (PAGE_SHIFT + ((level) - 1) * (bits_per_level)) +#define __PT_INDEX (address, level, bits_per_level) \ + (((address) >> __PT_LEVEL_SHIFT(level, bits_per_level)) & ((1 << (bits_per_level)) - 1)) + +#define __PT_LVL_ADDR_MASK (base_addr_mask, level, bits_per_level) \ + ((base_addr_mask) & ~((1ULL << (PAGE_SHIFT + (((level) - 1) * (bits_per_level)))) - 1)) + +#define __PT_LVL_OFFSET_MASK (base_addr_mask, level, bits_per_level) \ + ((base_addr_mask) & ((1ULL << (PAGE_SHIFT + (((level) - 1) * (bits_per_level)))) - 1)) + +#define __PT_ENT_PER_PAGE (bits_per_level) (1 << (bits_per_level)) +/* * Unlike regular MMU roots, PAE "roots", a.k.a. PDPTEs/PDPTRs, have a PRESENT * bit, and thus are guaranteed to be non-zero when valid. And, when a guest
arch/x86/kvm/mmu/paging.h +5 -4
@@ -4,11 +4,12 @@#define __KVM_X86_PAGING_H#define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) +#define PT64_LVL_ADDR_MASK (level) \ - (GUEST_PT64_BASE_ADDR_MASK & ~((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT64_LEVEL_BITS))) - 1)) + __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) +#define PT64_LVL_OFFSET_MASK (level) \ - (GUEST_PT64_BASE_ADDR_MASK & ((1ULL << (PAGE_SHIFT + (((level) - 1) \ - * PT64_LEVEL_BITS))) - 1)) + __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) +#endif /* __KVM_X86_PAGING_H */
arch/x86/kvm/mmu/spte.h +3 -4
@@ -55,11 +55,10 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK == 0);#define PT64_LEVEL_BITS 9 -#define PT64_LEVEL_SHIFT (level) \ - (PAGE_SHIFT + (level - 1) * PT64_LEVEL_BITS) +#define PT64_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, PT64_LEVEL_BITS) + +#define PT64_INDEX (address, level) __PT_INDEX(address, level, PT64_LEVEL_BITS) -#define PT64_INDEX (address, level)\ - (((address) >> PT64_LEVEL_SHIFT(level)) & ((1 << PT64_LEVEL_BITS) - 1))#define SHADOW_PT_INDEX (addr, level) PT64_INDEX(addr, level)/*
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 5/8 KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
In reply to Sean ChristophersonSeparate the macros for KVM's shadow PTEs (SPTE) from guest 64-bit PTEs (PT64). SPTE and PT64 are _mostly_ the same, but the few differences are quite critical, e.g. *_BASE_ADDR_MASK must differentiate between host and guest physical address spaces, and SPTE_PERM_MASK (was PT64_PERM_MASK) is very much specific to SPTEs.
Opportunistically (and temporarily) move most guest macros into paging.h to clearly associate them with shadow paging, and to ensure that they're not used as of this commit. A future patch will eliminate them entirely.
Sadly, PT32_LEVEL_BITS is left behind in mmu_internal.h because it's needed for the quadrant calculation in kvm_mmu_get_page(). The quadrant calculation is hot enough (when using shadow paging with 32-bit guests) that adding a per-context helper is undesirable, and burying the computation in paging_tmpl.h with a forward declaration isn't exactly an improvement.
--- arch/x86/kvm/mmu.h | 5 ---- arch/x86/kvm/mmu/mmu.c | 44 +++++++++++---------------------- arch/x86/kvm/mmu/mmu_internal.h | 3 +++ arch/x86/kvm/mmu/paging.h | 17 +++++++++++++ arch/x86/kvm/mmu/paging_tmpl.h | 4 +-- arch/x86/kvm/mmu/spte.c | 2 +- arch/x86/kvm/mmu/spte.h | 27 +++++++++----------- arch/x86/kvm/mmu/tdp_iter.c | 6 ++--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++--- 9 files changed, 56 insertions(+), 58 deletions(-)
arch/x86/kvm/mmu.h +0 -5
@@ -6,11 +6,6 @@#include "kvm_cache_regs.h" #include "cpuid.h" -#define PT64_PT_BITS 9 -#define PT64_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT64_PT_BITS) -#define PT32_PT_BITS 10 -#define PT32_ENT_PER_PAGE __PT_ENT_PER_PAGE(PT32_PT_BITS) -#define PT_WRITABLE_SHIFT 1#define PT_USER_SHIFT 2
arch/x86/kvm/mmu/mmu.c +15 -29
131 lines not on this page · raw message
arch/x86/kvm/mmu/mmu_internal.h +3 -0
@@ -20,6 +20,9 @@ extern bool dbg;#define MMU_WARN_ON (x) do { } while (0)#endif +/* The number of bits for 32-bit PTEs is to needed compute the quandrant. */ +#define PT32_LEVEL_BITS 10 +/* Page table builder macros common to shadow (host) PTEs and guest PTEs. */ #define __PT_LEVEL_SHIFT (level, bits_per_level) \ (PAGE_SHIFT + ((level) - 1) * (bits_per_level))
arch/x86/kvm/mmu/paging.h +17 -0
@@ -5,11 +5,28 @@#define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) +#define PT64_LEVEL_BITS 9 + +#define PT64_INDEX (address, level) __PT_INDEX(address, level, PT64_LEVEL_BITS) +#define PT64_LVL_ADDR_MASK (level) \ __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS)#define PT64_LVL_OFFSET_MASK (level) \ __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) + +#define PT32_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) + +#define PT32_LVL_OFFSET_MASK (level) \ + __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) + +#define PT32_INDEX (address, level) __PT_INDEX(address, level, PT32_LEVEL_BITS) + +#define PT32_BASE_ADDR_MASK PAGE_MASK + +#define PT32_LVL_ADDR_MASK (level) \ + __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) +#endif /* __KVM_X86_PAGING_H */
arch/x86/kvm/mmu/paging_tmpl.h +2 -2
@@ -899,7 +899,7 @@ static gpa_t FNAME(get_level1_sp_gpa)(struct kvm_mmu_page *sp)WARN_ON (sp->role.level !=PG_LEVEL_4K );if (PTTYPE ==32 ) - offset = sp->role.quadrant <<PT64_LEVEL_BITS ; + offset = sp->role.quadrant <<SPTE_LEVEL_BITS ;return gfn_to_gpa (sp->gfn) + offset *sizeof (pt_element_t); }
@@ -1034,7 +1034,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) first_pte_gpa =FNAME (get_level1_sp_gpa)(sp); -for (i =0 ; i <PT64_ENT_PER_PAGE ; i++) { +for (i =0 ; i <SPTE_ENT_PER_PAGE ; i++) {u64 *sptep, spte;struct kvm_memory_slot *slot;unsigned pte_access;
arch/x86/kvm/mmu/spte.c +1 -1
@@ -301,7 +301,7 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn) {u64 new_spte; - new_spte = old_spte & ~PT64_BASE_ADDR_MASK ; + new_spte = old_spte & ~SPTE_BASE_ADDR_MASK ; new_spte |= (u64 )new_pfn <<PAGE_SHIFT ; new_spte &= ~PT_WRITABLE_MASK ;
arch/x86/kvm/mmu/spte.h +12 -15
@@ -36,12 +36,12 @@ extern bool __read_mostly enable_mmio_caching;static_assert (SPTE_TDP_AD_ENABLED_MASK ==0 );#ifdef CONFIG_DYNAMIC_PHYSICAL_MASK -#define PT64_BASE_ADDR_MASK (physical_mask & ~(u64)(PAGE_SIZE-1)) +#define SPTE_BASE_ADDR_MASK (physical_mask & ~(u64)(PAGE_SIZE-1))#else -#define PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) +#define SPTE_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1))#endif -#define PT64_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \ +#define SPTE_PERM_MASK (PT_PRESENT_MASK | PT_WRITABLE_MASK | shadow_user_mask \ | shadow_x_mask | shadow_nx_mask | shadow_me_mask)#define ACC_EXEC_MASK 1
@@ -50,16 +50,13 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK == 0);#define ACC_ALL (ACC_EXEC_MASK | ACC_WRITE_MASK | ACC_USER_MASK)/* The mask for the R/X bits in EPT PTEs */ -#define PT64_EPT_READABLE_MASK 0x1ull -#define PT64_EPT_EXECUTABLE_MASK 0x4ull +#define SPTE_EPT_READABLE_MASK 0x1ull +#define SPTE_EPT_EXECUTABLE_MASK 0x4ull -#define PT64_LEVEL_BITS 9 - -#define PT64_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, PT64_LEVEL_BITS) - -#define PT64_INDEX (address, level) __PT_INDEX(address, level, PT64_LEVEL_BITS) - -#define SHADOW_PT_INDEX (addr, level) PT64_INDEX(addr, level) +#define SPTE_LEVEL_BITS 9 +#define SPTE_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, SPTE_LEVEL_BITS) +#define SPTE_INDEX (address, level) __PT_INDEX(address, level, SPTE_LEVEL_BITS) +#define SPTE_ENT_PER_PAGE __PT_ENT_PER_PAGE(SPTE_LEVEL_BITS)/* * The mask/shift to use for saving the original R/X bits when marking the PTE
@@ -68,8 +65,8 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK == 0);* restored only when a write is attempted to the page. This mask obviously * must not overlap the A/D type mask. */ -#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (PT64_EPT_READABLE_MASK | \ - PT64_EPT_EXECUTABLE_MASK) +#define SHADOW_ACC_TRACK_SAVED_BITS_MASK (SPTE_EPT_READABLE_MASK | \ + SPTE_EPT_EXECUTABLE_MASK)#define SHADOW_ACC_TRACK_SAVED_BITS_SHIFT 54#define SHADOW_ACC_TRACK_SAVED_MASK (SHADOW_ACC_TRACK_SAVED_BITS_MASK << \ SHADOW_ACC_TRACK_SAVED_BITS_SHIFT)
@@ -281,7 +278,7 @@ static inline bool is_executable_pte(u64 spte)static inline kvm_pfn_t spte_to_pfn (u64 pte) { -return (pte &PT64_BASE_ADDR_MASK ) >>PAGE_SHIFT ; +return (pte &SPTE_BASE_ADDR_MASK ) >>PAGE_SHIFT ; }static inline bool is_accessed_spte (u64 spte)
arch/x86/kvm/mmu/tdp_iter.c +3 -3
@@ -11,7 +11,7 @@static void tdp_iter_refresh_sptep (struct tdp_iter *iter) { iter->sptep = iter->pt_path[iter->level -1 ] + -SHADOW_PT_INDEX (iter->gfn <<PAGE_SHIFT , iter->level); +SPTE_INDEX (iter->gfn <<PAGE_SHIFT , iter->level); iter->old_spte =kvm_tdp_mmu_read_spte (iter->sptep); }
@@ -116,8 +116,8 @@ static bool try_step_side(struct tdp_iter *iter)* Check if the iterator is already at the end of the current page * table. */ -if (SHADOW_PT_INDEX (iter->gfn <<PAGE_SHIFT , iter->level) == - (PT64_ENT_PER_PAGE -1 )) +if (SPTE_INDEX (iter->gfn <<PAGE_SHIFT , iter->level) == + (SPTE_ENT_PER_PAGE -1 ))return false; iter->gfn +=KVM_PAGES_PER_HPAGE (iter->level);
arch/x86/kvm/mmu/tdp_mmu.c +3 -3
@@ -425,7 +425,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared)tdp_mmu_unlink_sp (kvm, sp, shared); -for (i =0 ; i <PT64_ENT_PER_PAGE ; i++) { +for (i =0 ; i <SPTE_ENT_PER_PAGE ; i++) {tdp_ptep_t sptep = pt + i;gfn_t gfn = base_gfn + i *KVM_PAGES_PER_HPAGE (level);u64 old_spte;
@@ -1487,7 +1487,7 @@ static int tdp_mmu_split_huge_page(struct kvm *kvm, struct tdp_iter *iter,* No need for atomics when writing to sp->spt since the page table has * not been linked in yet and thus is not reachable from any other CPU. */ -for (i =0 ; i <PT64_ENT_PER_PAGE ; i++) +for (i =0 ; i <SPTE_ENT_PER_PAGE ; i++) sp->spt[i] =make_huge_page_split_spte (huge_spte, level, i);/*
@@ -1507,7 +1507,7 @@ static int tdp_mmu_split_huge_page(struct kvm *kvm, struct tdp_iter *iter,* are overwriting from the page stats. But we have to manually update * the page stats with the new present child pages. */ -kvm_update_page_stats (kvm, level -1 ,PT64_ENT_PER_PAGE ); +kvm_update_page_stats (kvm, level -1 ,SPTE_ENT_PER_PAGE ); out:trace_kvm_mmu_split_huge_page (iter->gfn, huge_spte, level, ret);
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Paolo Bonzini
maintainerRe: [PATCH v2 5/8] KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2018
Quoted 9 lines · Sean Christopherson
arch/x86/kvm/mmu/mmu.c +2
-2@@ -2027,8 +2013,8 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.direct = direct; role.access = access;if (role.has_4_byte_gpte) { - quadrant = gaddr >> (PAGE_SHIFT + (PT64_PT_BITS * level)); - quadrant &= (1 << ((PT32_PT_BITS -PT64_PT_BITS ) * level)) -1 ; + quadrant = gaddr >> (PAGE_SHIFT + (SPTE_LEVEL_BITS * level)); + quadrant &= (1 << ((PT32_LEVEL_BITS -SPTE_LEVEL_BITS ) * level)) -1 ; role.quadrant = quadrant;
That's just a fancy 1, though, and this is just
/*
* Isolate the bits of the address that have already been used by the
* 8-byte shadow page table structures, but not yet in the 4-byte guest
* page tables. For example, a 4-byte PDE consumes bits 31:22 and an
* 8-byte PDE consumes bits 29:21, so bits 31:30 go in the hash
* key. The hash table look up up ensures that each sPTE points to
* the page for the correct portion of the guest page table structure. */
quadrant = gaddr >> (PAGE_SHIFT + (SPTE_LEVEL_BITS * level)); quadrant &= (1 << level) - 1;
(Not the best comment, understood).
Paolo
Paolo Bonzini
maintainerRe: [PATCH v2 5/8] KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
In reply to Sean ChristophersonQuoted 10 lines · Sean Christopherson
Separate the macros for KVM's shadow PTEs (SPTE) from guest 64-bit PTEs (PT64). SPTE and PT64 are _mostly_ the same, but the few differences are quite critical, e.g.
*_BASE_ADDR_MASKmust differentiate between host and guest physical address spaces, andSPTE_PERM_MASK(wasPT64_PERM_MASK) is very much specific to SPTEs.Opportunistically (and temporarily) move most guest macros into
paging.hto clearly associate them with shadow paging, and to ensure that they're not used as of this commit. A future patch will eliminate them entirely.
Quoted 8 lines
Sadly,
PT32_LEVEL_BITSis left behind inmmu_internal.hbecause it's needed for the quadrant calculation inkvm_mmu_get_page(). The quadrant calculation is hot enough (when using shadow paging with 32-bit guests) that adding a per-context helper is undesirable, and burying the computation inpaging_tmpl.hwith a forward declaration isn't exactly an improvement.Signed-off-by Sean Christopherson <[email protected]>
A better try:
arch/x86/kvm/mmu/mmu.c +14 -1
@@ -2011,8 +2011,21 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.direct = direct; role.access = access;if (role.has_4_byte_gpte) { +/* +* The "quadrant" value corresponds to those bits of the address +* that have already been used by the 8-byte shadow page table +* lookup, but not yet in the 4-byte guest page tables. Having +* the quadrant as part of the role ensures that each upper sPTE +* points to the the correct portion of the guest page table +* structure. +* +* For example, a 4-byte PDE consumes bits 31:22 and an 8-byte PDE +* consumes bits 29:21. Each guest PD must be expanded into four +* shadow PDs, one for each value of bits 31:30, and the PDPEs +* will use the four quadrants in round-robin fashion. +*/ quadrant = gaddr >> (PAGE_SHIFT + (SPTE_LEVEL_BITS * level)); - quadrant &= (1 << ((PT32_LEVEL_BITS -SPTE_LEVEL_BITS ) * level)) -1 ; + quadrant &= (1 << level) -1 ; role.quadrant = quadrant; }if (level <= vcpu->arch.mmu->cpu_role.base.level)
arch/x86/kvm/mmu/mmu_internal.h +0 -3
@@ -20,9 +20,6 @@ extern bool dbg;#define MMU_WARN_ON (x) do { } while (0)#endif -/* The number of bits for 32-bit PTEs is to needed compute the quandrant. */ -#define PT32_LEVEL_BITS 10 -/* Page table builder macros common to shadow (host) PTEs and guest PTEs. */ #define __PT_LEVEL_SHIFT (level, bits_per_level) \ (PAGE_SHIFT + ((level) - 1) * (bits_per_level))
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -38,7 +38,7 @@#define pt_element_t u32#define guest_walker guest_walker32#define FNAME (name) paging##32_##name -#define PT_LEVEL_BITS PT32_LEVEL_BITS +#define PT_LEVEL_BITS 10#define PT_MAX_FULL_LEVELS 2#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT
Paolo
Sean Christopherson
maintainerRe: [PATCH v2 5/8] KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
Quoted 18 lines · Paolo Bonzini
Quoted 17 lines · Sean Christopherson
Separate the macros for KVM's shadow PTEs (SPTE) from guest 64-bit PTEs (PT64). SPTE and PT64 are _mostly_ the same, but the few differences are quite critical, e.g.
*_BASE_ADDR_MASKmust differentiate between host and guest physical address spaces, andSPTE_PERM_MASK(wasPT64_PERM_MASK) is very much specific to SPTEs.Opportunistically (and temporarily) move most guest macros into
paging.hto clearly associate them with shadow paging, and to ensure that they're not used as of this commit. A future patch will eliminate them entirely.Sadly,
PT32_LEVEL_BITSis left behind inmmu_internal.hbecause it's needed for the quadrant calculation inkvm_mmu_get_page(). The quadrant calculation is hot enough (when using shadow paging with 32-bit guests) that adding a per-context helper is undesirable, and burying the computation inpaging_tmpl.hwith a forward declaration isn't exactly an improvement.
Quoted 3 lines
Quoted 1 line
Signed-off-by Sean Christopherson <[email protected]>A better try:
Very nice! It's obvious once someone else writes the code. :-)
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +2
-0@@ -2011,8 +2011,21 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.direct = direct; role.access = access;if (role.has_4_byte_gpte) { +/* +* The "quadrant" value corresponds to those bits of the address
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2025
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10
-0+* that have already been used by the 8-byte shadow page table +* lookup, but not yet in the 4-byte guest page tables. Having +* the quadrant as part of the role ensures that each upper sPTE +* points to the the correct portion of the guest page table +* structure. +* +* For example, a 4-byte PDE consumes bits 31:22 and an 8-byte PDE +* consumes bits 29:21. Each guest PD must be expanded into four +* shadow PDs, one for each value of bits 31:30, and the PDPEs +* will use the four quadrants in round-robin fashion.
It's not round-robin, that would imply KVM rotates through each quadrant on its own. FWIW, I like David's comment from his patch that simplifies this mess in a similar way.
Quoted 12 lines
arch/x86/kvm/mmu/mmu.c +2
-1+*/ quadrant = gaddr >> (PAGE_SHIFT + (SPTE_LEVEL_BITS * level)); - quadrant &= (1 << ((PT32_LEVEL_BITS -SPTE_LEVEL_BITS ) * level)) -1 ; + quadrant &= (1 << level) -1 ; role.quadrant = quadrant; }if (level <= vcpu->arch.mmu->cpu_role.base.level)arch/x86/kvm/mmu/mmu_internal.h +0
-0
arch/x86/kvm/mmu/mmu_internal.h extern bool dbg; · line 22
Quoted 3 lines
arch/x86/kvm/mmu/mmu_internal.h +0
-1@@ -20,9 +20,6 @@ extern bool dbg;#define MMU_WARN_ON (x) do { } while (0)#endif -/* The number of bits for 32-bit PTEs is to needed compute the quandrant. */
Heh, and it gets rid of my typo.
Quoted 21 lines
arch/x86/kvm/mmu/mmu_internal.h +0
-2-#define PT32_LEVEL_BITS 10 -/* Page table builder macros common to shadow (host) PTEs and guest PTEs. */ #define __PT_LEVEL_SHIFT (level, bits_per_level) \ (PAGE_SHIFT + ((level) - 1) * (bits_per_level))arch/x86/kvm/mmu/paging_tmpl.h +1
-1@@ -38,7 +38,7 @@#define pt_element_t u32#define guest_walker guest_walker32#define FNAME (name) paging##32_##name -#define PT_LEVEL_BITS PT32_LEVEL_BITS +#define PT_LEVEL_BITS 10#define PT_MAX_FULL_LEVELS 2#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFTPaolo
Paolo Bonzini
maintainerRe: [PATCH v2 5/8] KVM: x86/mmu: Use separate namespaces for guest PTEs and shadow PTEs
Quoted 9 lines · Sean Christopherson
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2025
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +4
-0+* For example, a 4-byte PDE consumes bits 31:22 and an 8-byte PDE +* consumes bits 29:21. Each guest PD must be expanded into four +* shadow PDs, one for each value of bits 31:30, and the PDPEs +* will use the four quadrants in round-robin fashion. It's not round-robin, that would imply KVM rotates through each quadrant on its own. FWIW, I like David's comment from his patch that simplifies this mess in a similar way.
Yeah, by round-robin I meant that the 512 entries will look like q=0 q=1 q=2 q=3 q=0 q=1 etc.
I'll incorporate David's comment, minus the last paragraph.
Paolo
Sean Christopherson
maintainerPATCH v2 6/8 KVM: x86/mmu: Use common macros to compute 32/64-bit paging masks
In reply to Sean ChristophersonDedup the code for generating (most of) the per-type PT_* masks in paging_tmpl.h. The relevant macros only vary based on the number of bits per level, and that smidge of info is already provided in a common form as PT_LEVEL_BITS.
No functional change intended.
--- arch/x86/kvm/mmu/paging.h | 23 ----------------------- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++++-------------- 2 files changed, 11 insertions(+), 37 deletions(-)
arch/x86/kvm/mmu/paging.h +0 -23
@@ -5,28 +5,5 @@#define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) -#define PT64_LEVEL_BITS 9 - -#define PT64_INDEX (address, level) __PT_INDEX(address, level, PT64_LEVEL_BITS) - -#define PT64_LVL_ADDR_MASK (level) \ - __PT_LVL_ADDR_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) - -#define PT64_LVL_OFFSET_MASK (level) \ - __PT_LVL_OFFSET_MASK(GUEST_PT64_BASE_ADDR_MASK, level, PT64_LEVEL_BITS) - - -#define PT32_LEVEL_SHIFT (level) __PT_LEVEL_SHIFT(level, PT32_LEVEL_BITS) - -#define PT32_LVL_OFFSET_MASK (level) \ - __PT_LVL_OFFSET_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) - -#define PT32_INDEX (address, level) __PT_INDEX(address, level, PT32_LEVEL_BITS) - -#define PT32_BASE_ADDR_MASK PAGE_MASK - -#define PT32_LVL_ADDR_MASK (level) \ - __PT_LVL_ADDR_MASK(PT32_BASE_ADDR_MASK, level, PT32_LEVEL_BITS) -#endif /* __KVM_X86_PAGING_H */
arch/x86/kvm/mmu/paging_tmpl.h +11 -14
@@ -16,8 +16,9 @@*/ /* -* We need the mmu code to access both 32-bit and 64-bit guest ptes, -* so the code in this file is compiled twice, once per pte size. +* The MMU needs to be able to access/walk 32-bit and 64-bit guest page tables, +* as well as guest EPT tables, so the code in this file is compiled thrice, +* once per guest PTE type. The per-type defines are #undef'd at the end. */ #if PTTYPE ==64
@@ -25,10 +26,7 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) -#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl) -#define PT_INDEX (addr, level) PT64_INDEX(addr, level) -#define PT_LEVEL_BITS PT64_LEVEL_BITS +#define PT_LEVEL_BITS 9#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT#define PT_HAVE_ACCESSED_DIRTY (mmu) true
@@ -41,10 +39,7 @@#define pt_element_t u32#define guest_walker guest_walker32#define FNAME (name) paging##32_##name -#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) -#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl) -#define PT_INDEX (addr, level) PT32_INDEX(addr, level) +#define PT_BASE_ADDR_MASK PAGE_MASK#define PT_LEVEL_BITS PT32_LEVEL_BITS#define PT_MAX_FULL_LEVELS 2#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT
@@ -60,10 +55,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) -#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl) -#define PT_INDEX (addr, level) PT64_INDEX(addr, level) -#define PT_LEVEL_BITS PT64_LEVEL_BITS +#define PT_LEVEL_BITS 9#define PT_GUEST_DIRTY_SHIFT 9#define PT_GUEST_ACCESSED_SHIFT 8#define PT_HAVE_ACCESSED_DIRTY (mmu) (!(mmu)->cpu_role.base.ad_disabled)
@@ -72,6 +64,11 @@#error Invalid PTTYPE value#endif +/* Common logic, but per-type values. These also need to be undefined. */ +#define PT_LVL_ADDR_MASK (lvl) __PT_LVL_ADDR_MASK(PT_BASE_ADDR_MASK, lvl, PT_LEVEL_BITS) +#define PT_LVL_OFFSET_MASK (lvl) __PT_LVL_OFFSET_MASK(PT_BASE_ADDR_MASK, lvl, PT_LEVEL_BITS) +#define PT_INDEX (addr, lvl) __PT_INDEX(addr, lvl, PT_LEVEL_BITS) +#define PT_GUEST_DIRTY_MASK (1 << PT_GUEST_DIRTY_SHIFT)#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 7/8 KVM: x86/mmu: Truncate paging32's PT_BASE_ADDR_MASK to 32 bits
In reply to Sean ChristophersonTruncate paging32's PT_BASE_ADDR_MASK to a pt_element_t, i.e. to 32 bits. Ignoring PSE huge pages, the mask is only used in conjunction with gPTEs, which are 32 bits, and so the address is limited to bits 31:12.
PSE huge pages encoded PA bits 39:32 in PTE bits 20:13, i.e. need custom logic to handle their funky encoding regardless of PT_BASE_ADDR_MASK.
Note, PT_LVL_OFFSET_MASK is somewhat confusing in that it computes the offset of the _gfn_, not of the gpa, i.e. not having bits 63:32 set in PT_BASE_ADDR_MASK is again correct.
No functional change intended.
--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -39,7 +39,7 @@#define pt_element_t u32#define guest_walker guest_walker32#define FNAME (name) paging##32_##name -#define PT_BASE_ADDR_MASK PAGE_MASK +#define PT_BASE_ADDR_MASK ((pt_element_t)PAGE_MASK)#define PT_LEVEL_BITS PT32_LEVEL_BITS#define PT_MAX_FULL_LEVELS 2#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT
Signature · 1 line
2.36.1.476.g0c4daa206d-goog
Sean Christopherson
maintainerPATCH v2 8/8 KVM: x86/mmu: Use common logic for computing the 32/64-bit base PA mask
In reply to Sean ChristophersonUse common logic for computing PT_BASE_ADDR_MASK for 32-bit, 64-bit, and EPT paging. Both PAGE_MASK and the new-common logic are supsersets of what is actually needed for 32-bit paging. PAGE_MASK sets bits 63:12 and the former GUEST_PT64_BASE_ADDR_MASK sets bits 51:12, so regardless of which value is used, the result will always be bits 31:12.
No functional change intended.
--- arch/x86/kvm/mmu/mmu.c | 2 -- arch/x86/kvm/mmu/paging.h | 9 --------- arch/x86/kvm/mmu/paging_tmpl.h | 4 +--- 3 files changed, 1 insertion(+), 14 deletions(-) delete mode 100644 arch/x86/kvm/mmu/paging.h
arch/x86/kvm/mmu/mmu.c +0 -2
@@ -53,8 +53,6 @@#include <asm/kvm_page_track.h> #include "trace.h" -#include "paging.h" -extern bool itlb_multihit_kvm_mitigation;int __read_mostly nx_huge_pages =-1 ;
arch/x86/kvm/mmu/paging.h deleted +0 -9
@@ -1,9 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* Shadow paging constants/helpers that don't need to be #undef'd. */ -#ifndef __KVM_X86_PAGING_H -#define __KVM_X86_PAGING_H - -#define GUEST_PT64_BASE_ADDR_MASK (((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)) - -#endif /* __KVM_X86_PAGING_H */ -
arch/x86/kvm/mmu/paging_tmpl.h +1 -3
@@ -25,7 +25,6 @@#define pt_element_t u64#define guest_walker guest_walker64#define FNAME (name) paging##64_##name -#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK#define PT_LEVEL_BITS 9#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT#define PT_GUEST_ACCESSED_SHIFT PT_ACCESSED_SHIFT
@@ -39,7 +38,6 @@#define pt_element_t u32#define guest_walker guest_walker32#define FNAME (name) paging##32_##name -#define PT_BASE_ADDR_MASK ((pt_element_t)PAGE_MASK)#define PT_LEVEL_BITS PT32_LEVEL_BITS#define PT_MAX_FULL_LEVELS 2#define PT_GUEST_DIRTY_SHIFT PT_DIRTY_SHIFT
@@ -54,7 +52,6 @@#define pt_element_t u64#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name -#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK#define PT_LEVEL_BITS 9#define PT_GUEST_DIRTY_SHIFT 9#define PT_GUEST_ACCESSED_SHIFT 8
@@ -65,6 +62,7 @@#endif /* Common logic, but per-type values. These also need to be undefined. */ +#define PT_BASE_ADDR_MASK ((pt_element_t)(((1ULL << 52) - 1) & ~(u64)(PAGE_SIZE-1)))#define PT_LVL_ADDR_MASK (lvl) __PT_LVL_ADDR_MASK(PT_BASE_ADDR_MASK, lvl, PT_LEVEL_BITS)#define PT_LVL_OFFSET_MASK (lvl) __PT_LVL_OFFSET_MASK(PT_BASE_ADDR_MASK, lvl, PT_LEVEL_BITS)#define PT_INDEX (addr, lvl) __PT_INDEX(addr, lvl, PT_LEVEL_BITS)
Signature · 1 line
2.36.1.476.g0c4daa206d-goog