KVM: PPC: BOok3S HV: Support for nested HPT guests
Subsystems (3)
cover KVM: PPC: BOok3S HV: Support for nested HPT guests
This patch series adds support for running a nested kvm guest which uses the hpt (hash page table) mmu type.
Patch series based on v5.3-rc6.
The first 8 patches in this series enable a radix guest (L1) running under a radix hypervisor (L0) to act as a guest hypervisor for it's own nested (L2) guest where the nested guest is using hash page table translation. This mainly involved ensuring that the guest hypervisor uses the new run_single_vcpu() entry path and ensuring that the appropriate functions which are normally called in the real mode entry path in book3s_hv_rmhandlers.S are called on the new virtual mode entry path when a hpt guest is being run.
The remainder of the patches enable a (L0) hypervisor to perform hash page table translation for a nested (L2) hpt guest which is running under one of it's radix (L1) it's which is acting as a guest hypervisor. This primarily required changes to the nested guest entry patch to ensure that a shadow hpt would be allocated for the nested hpt guest, that the slb was context switched and that the real mode entry path in book3s_hv_rmhandlers.S could be used to enter/exit a nested hpt guest.
It was also necessary to be able to create translations by inserting ptes into the shadow page table which provided the combination of the translation from L2 virtual address to L1 guest physical address and the translation from L1 guest physical address to L0 host real address. Additionally invalidations of these translations need to be handled at both levels, by L1 via the H_TLB_INVALIDATE hcall to invalidate a L2 virtual address to L1 guest physical address translation, and by L0 when paging out a L1 guest page which had been subsequently mapped through to L2 thus invalidating the L1 guest physical address to L0 host real address translation.
Still lacking support for:
Passthrough of emulated mmio devices to nested hpt guests since the current method of reading nested guest memory relies on using quadrants which are only available when using radix translation.
Paul Mackerras (1):
KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page fault
handler
Suraj Jitindar Singh (22):
KVM: PPC: Book3S HV: Increment mmu_notifier_seq when modifying radix
pte rc bits
KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested
guests
KVM: PPC: Book3S HV: Handle making H_ENTER_NESTED hcall in a separate
function
KVM: PPC: Book3S HV: Enable calling kvmppc_hpte_hv_fault in virtual
mode
KVM: PPC: Book3S HV: Allow hpt manipulation hcalls to be called in
virtual mode
KVM: PPC: Book3S HV: Make kvmppc_invalidate_hpte() take lpid not a kvm
struct
KVM: PPC: Book3S HV: Nested: Allow pseries hypervisor to run hpt
nested guest
KVM: PPC: Book3S HV: Nested: Improve comments and naming of nest rmap
functions
KVM: PPC: Book3S HV: Nested: Increase gpa field in nest rmap to 46
bits
KVM: PPC: Book3S HV: Nested: Remove single nest rmap entries
KVM: PPC: Book3S HV: Nested: add kvmhv_remove_all_nested_rmap_lpid()
KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup
KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest
KVM: PPC: Book3S HV: Store lpcr and hdec_exp in the vcpu struct
KVM: PPC: Book3S HV: Nested: Make kvmppc_run_vcpu() entry path nested
capable
KVM: PPC: Book3S HV: Nested: Rename kvmhv_xlate_addr_nested_radix
KVM: PPC: Book3S HV: Separate out hashing from
kvmppc_hv_find_lock_hpte()
KVM: PPC: Book3S HV: Nested: Implement nested hpt mmu translation
KVM: PPC: Book3S HV: Nested: Handle tlbie hcall for nested hpt guest
KVM: PPC: Book3S HV: Nested: Implement nest rmap invalidations for hpt
guests
KVM: PPC: Book3S HV: Nested: Enable nested hpt guests
KVM: PPC: Book3S HV: Add nested hpt pte information to debugfs
arch/powerpc/include/asm/book3s/64/mmu-hash.h | 15 + arch/powerpc/include/asm/book3s/64/mmu.h | 9 + arch/powerpc/include/asm/hvcall.h | 36 - arch/powerpc/include/asm/kvm_asm.h | 5 + arch/powerpc/include/asm/kvm_book3s.h | 30 +- arch/powerpc/include/asm/kvm_book3s_64.h | 87 +- arch/powerpc/include/asm/kvm_host.h | 57 + arch/powerpc/include/asm/kvm_ppc.h | 5 +- arch/powerpc/kernel/asm-offsets.c | 5 + arch/powerpc/kvm/book3s.c | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 136 ++- arch/powerpc/kvm/book3s_64_mmu_radix.c | 167 +-- arch/powerpc/kvm/book3s_hv.c | 327 ++++-- arch/powerpc/kvm/book3s_hv_builtin.c | 33 +- arch/powerpc/kvm/book3s_hv_interrupts.S | 25 +- arch/powerpc/kvm/book3s_hv_nested.c | 1381 ++++++++++++++++++++++--- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 298 ++++-- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 126 ++- arch/powerpc/kvm/book3s_xive.h | 15 + arch/powerpc/kvm/powerpc.c | 3 +- 20 files changed, 2136 insertions(+), 625 deletions(-)
Signature · 1 line
2.13.6
PATCH 1/23 KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page fault handler
From: Paul Mackerras <[email protected]>
This makes the same changes in the page fault handler for HPT guests that commits 31c8b0d0694a ("KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot() in page fault handler", 2018-03-01), 71d29f43b633 ("KVM: PPC: Book3S HV:
Don't use compound_order to determine host mapping size", 2018-09-11) and 6579804c4317 ("KVM: PPC: Book3S HV: Avoid crash from THP collapse during radix page fault", 2018-10-04) made for the page fault handler for radix guests.
In summary, where we used to call get_user_pages_fast() and then do special handling for VM_PFNMAP vmas, we now call __get_user_pages_fast() and then __gfn_to_pfn_memslot() if that fails, followed by reading the Linux PTE to get the host PFN, host page size and mapping attributes.
This also brings in the change from SetPageDirty() to set_page_dirty_lock() which was done for the radix page fault handler in commit c3856aeb2940 ("KVM: PPC: Book3S HV: Fix handling of large pages in radix page fault handler", 2018-02-23).
--- arch/powerpc/kvm/book3s_64_mmu_hv.c | 118 +++++++++++++++++------------------- 1 file changed, 57 insertions(+), 61 deletions(-)
arch/powerpc/kvm/book3s_64_mmu_hv.c +57 -61
@@ -497,17 +497,18 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,__be64 *hptep;unsigned long mmu_seq, psize, pte_size;unsigned long gpa_base, gfn_base; -unsigned long gpa, gfn, hva, pfn; +unsigned long gpa, gfn, hva, pfn, hpa;struct kvm_memory_slot *memslot;unsigned long *rmap;struct revmap_entry *rev; -struct page *page, *pages[1 ]; -long index, ret, npages; +struct page *page; +long index, ret;bool is_ci; -unsigned int writing, write_ok; -struct vm_area_struct *vma; +bool writing, write_ok; +unsigned int shift;unsigned long rcbits;long mmio_update; +pte_t pte, *ptep;if (kvm_is_radix (kvm))return kvmppc_book3s_radix_page_fault (run, vcpu, ea, dsisr);
@@ -581,59 +582,62 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,smp_rmb (); ret = -EFAULT ; - is_ci = false; - pfn =0 ; page =NULL ; - pte_size =PAGE_SIZE ; writing = (dsisr &DSISR_ISSTORE ) !=0 ;/* If writing != 0, then the HPTE must allow writing, if we get here */ write_ok = writing; hva =gfn_to_hva_memslot (memslot, gfn); - npages =get_user_pages_fast (hva,1 , writing ?FOLL_WRITE :0 , pages); -if (npages <1 ) { -/* Check if it's an I/O mapping */ -down_read (¤t->mm->mmap_sem); - vma =find_vma (current->mm, hva); -if (vma && vma->vm_start <= hva && hva + psize <= vma->vm_end && - (vma->vm_flags &VM_PFNMAP )) { - pfn = vma->vm_pgoff + - ((hva - vma->vm_start) >>PAGE_SHIFT ); - pte_size = psize; - is_ci =pte_ci (__pte ((pgprot_val (vma->vm_page_prot)))); - write_ok = vma->vm_flags &VM_WRITE ; - } -up_read (¤t->mm->mmap_sem); -if (!pfn) - goto out_put; + +/* +* Do a fast check first, since __gfn_to_pfn_memslot doesn't +* do it with !atomic && !async, which is how we call it. +* We always ask for write permission since the common case +* is that the page is writable. +*/ +if (__get_user_pages_fast (hva,1 ,1 , &page) =1 ) { + write_ok = true; }else { - page = pages[0 ]; - pfn =page_to_pfn (page); -if (PageHuge (page)) { - page =compound_head (page); - pte_size <<=compound_order (page); - } -/* if the guest wants write access, see if that is OK */ -if (!writing &&hpte_is_writable (r)) { -pte_t *ptep, pte; -unsigned long flags; -/* -* We need to protect against page table destruction -* hugepage split and collapse. -*/ -local_irq_save (flags); - ptep =find_current_mm_pte (current->mm->pgd, - hva,NULL ,NULL ); -if (ptep) { - pte =kvmppc_read_update_linux_pte (ptep,1 ); -if (__pte_write (pte)) - write_ok =1 ; - } -local_irq_restore (flags); +/* Call KVM generic code to do the slow-path check */ + pfn =__gfn_to_pfn_memslot (memslot, gfn, false,NULL , + writing, &write_ok); +if (is_error_noslot_pfn (pfn)) +return -EFAULT ; + page =NULL ; +if (pfn_valid (pfn)) { + page =pfn_to_page (pfn); +if (PageReserved (page)) + page =NULL ; } } +/* +* Read the PTE from the process' radix tree and use that +* so we get the shift and attribute bits. +*/ +local_irq_disable (); + ptep =__find_linux_pte (vcpu->arch.pgdir, hva,NULL , &shift); +/* +* If the PTE disappeared temporarily due to a THP +* collapse, just return and let the guest try again. +*/ +if (!ptep) { +local_irq_enable (); +if (page) +put_page (page); +return RESUME_GUEST ; + } + pte = *ptep; +local_irq_enable (); + hpa =pte_pfn (pte) <<PAGE_SHIFT ; + pte_size =PAGE_SIZE ; +if (shift) + pte_size =1ul << shift; + is_ci =pte_ci (pte); +if (psize > pte_size) goto out_put; +if (pte_size > psize) + hpa |= hva & (pte_size - psize);/* Check WIMG vs. the actual page we're accessing */ if (!hpte_cache_flags_ok (r, is_ci)) {
@@ -647,14 +651,13 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, }/* -* Set the HPTE to point to pfn. -* Since the pfn is at PAGE_SIZE granularity, make sure we +* Set the HPTE to point to hpa. +* Since the hpa is at PAGE_SIZE granularity, make sure we * don't mask out lower-order bits if psize < PAGE_SIZE. */ if (psize <PAGE_SIZE ) psize =PAGE_SIZE ; - r = (r &HPTE_R_KEY_HI ) | (r & ~(HPTE_R_PP0 - psize)) | - ((pfn <<PAGE_SHIFT ) & ~(psize -1 )); + r = (r &HPTE_R_KEY_HI ) | (r & ~(HPTE_R_PP0 - psize)) | hpa;if (hpte_is_writable (r) && !write_ok) r =hpte_make_readonly (r); ret =RESUME_GUEST ;
@@ -719,20 +722,13 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, asmvolatile ("ptesync" : : :"memory" );preempt_enable ();if (page &&hpte_is_writable (r)) -SetPageDirty (page); +set_page_dirty_lock (page); out_put:trace_kvm_page_fault_exit (vcpu, hpte, ret); -if (page) { -/* -* We drop pages[0] here, not page because page might -* have been set to the head page of a compound, but -* we have to drop the reference on the correct tail -* page to match the get inside gup() -*/ -put_page (pages[0 ]); - } +if (page) +put_page (page);return ret; out_unlock:
Signature · 1 line
2.13.6
PATCH 2/23 KVM: PPC: Book3S HV: Increment mmu_notifier_seq when modifying radix pte rc bits
In reply to Suraj Jitindar SinghThe kvm mmu_notifier_seq is used to communicate that a page mapping has changed to code which is using that information in constructing a different but reliant page mapping. For example when constructing a mapping for a nested guest it is used to detect when the guest mapping has changed, which would render the nested guest mapping invalid.
When running nested guests it is important that the rc bits are kept in sync between the 2 ptes on the host in which they exist; the pte for the guest, and the pte for the nested guest. This is done when inserting the nested pte in __kvmhv_nested_page_fault_radix() by reducing the rc bits being set in the nested pte to those already set in the guest pte. And when setting the bits in the nested pte in response to an interrupt in kvmhv_handle_nested_set_rc_radix() the same bits are also set in the guest pte, with the bits not set in the nested pte if this fails.
When the host wants to remove rc bits from the guest pte in kvm_radix_test_clear_dirty(), if first removes then from the guest pte and then from any corresponding nested ptes which map the same guest page. This means that there is a window between which the rc bits could get out of sync between the two ptes as they might have been seen as set in the guest pte and thus updated in the nested pte assuming as such, while the host might be in the process of removing those rc bits leading to an inconsistency.
In kvm_radix_test_clear_dirty() the mmu_lock spin lock is held across removing the rc bits from the guest and nested pte, and the same is done across updating the rc bits in the guest and nested pte in kvmhv_handle_nested_set_rc_radix() and so there is no window for them to get out of sync in this case. However when constructing the pte in __kvmhv_nested_page_fault_radix() we drop the mmu_lock spin lock between reading the guest pte and inserting the nested pte, presenting a window for them to get out of sync. This is because the rc bits could have been observed as set in the guest pte and set in the nested pte accordingly, however in the mean time the rc bits in the guest pte could have been cleared, and since the nested pte wasn't yet inserted there is no way for the kvm_radix_test_clear_dirty() function to clear them and so an inconsistency can arise.
To avoid the possibility of the rc bits getting out of sync, increment the mmu_notifier_seq in kvm_radix_test_clear_dirty() under the mmu_lock when clearing rc bits. This means that when inserting the nested pte in __kvmhv_nested_page_fault_radix() we will bail out and retry when we see that the mmu_seq differs indicating that the guest pte has changed.
--- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 ++ 1 file changed, 2 insertions(+)
arch/powerpc/kvm/book3s_64_mmu_radix.c +2 -0
@@ -1044,6 +1044,8 @@ static int kvm_radix_test_clear_dirty(struct kvm *kvm,kvmhv_update_nest_rmap_rc_list (kvm, rmapp, _PAGE_DIRTY,0 , old &PTE_RPN_MASK ,1UL << shift); +/* Notify anyone trying to map the page that it has changed */ + kvm->mmu_notifier_seq++;spin_unlock (&kvm->mmu_lock); }return ret;
Signature · 1 line
2.13.6
PATCH 3/23 KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested guests
In reply to Suraj Jitindar SinghDon't allow hpt (hash page table) guests to act as guest hypervisors and thus be able to run nested guests. There is currently no support for this, if a nested guest is to be run it must be run at the lowest level. Explicitly disallow hash guests from enabling the nested kvm-hv capability at the hypervisor level.
--- arch/powerpc/kvm/book3s_hv.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +5 -1
@@ -5336,8 +5336,12 @@ static int kvmhv_enable_nested(struct kvm *kvm)return -ENODEV ;/* kvm = NULL means the caller is testing if the capability exists */ -if (kvm) +if (kvm) { +/* Only radix guests can act as nested hv and thus run guests */ +if (!kvm_is_radix (kvm)) +return -1 ; kvm->arch.nested_enable = true; + }return 0 ; }
Signature · 1 line
2.13.6
Re: [PATCH 03/23] KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested guests
Quoted 16 lines · Suraj Jitindar Singh
Don't allow hpt (hash page table) guests to act as guest hypervisors and thus be able to run nested guests. There is currently no support for this, if a nested guest is to be run it must be run at the lowest level. Explicitly disallow hash guests from enabling the nested kvm-hv capability at the hypervisor level.
Signed-off-by Suraj Jitindar Singh <[email protected]>--- arch/powerpc/kvm/book3s_hv.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)arch/powerpc/kvm/book3s_hv.c +0
-0
arch/powerpc/kvm/book3s_hv.c static int kvmhv_enable_nested(struct kvm *kvm) · line 5344
Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +5
-1@@ -5336,8 +5336,12 @@ static int kvmhv_enable_nested(struct kvm *kvm)return -ENODEV ;/* kvm = NULL means the caller is testing if the capability exists */ -if (kvm) +if (kvm) { +/* Only radix guests can act as nested hv and thus run guests */ +if (!kvm_is_radix (kvm)) +return -1 ; kvm->arch.nested_enable = true; + }
I don't think this is necessary, and is possibly undesirable, since a guest can switch between HPT and radix mode. In fact if a guest in HPT mode tries to do any of the hcalls for managing nested guests, it will get errors, because we have this:
static inline bool nesting_enabled(struct kvm *kvm)
{
return kvm->arch.nested_enable && kvm_is_radix(kvm);
}and H_SET_PARTITION_TABLE, H_ENTER_NESTED, etc. all return H_FUNCTION if nested_enabled() is false. (This is as the code is today without your patch). Furthermore, kvmppc_switch_mmu_to_hpt() does this:
if (nesting_enabled(kvm)) kvmhv_release_all_nested(kvm);
So I think it is all covered already without your patch.
Paul.
PATCH 4/23 KVM: PPC: Book3S HV: Handle making H_ENTER_NESTED hcall in a separate function
In reply to Suraj Jitindar SinghA pseries guest (that is a guest acting as a guest hypervisor) which would like to run it's own guest (a nested guest) must perform this by invoking the top level hypervisor via the H_ENTER_NESTED hcall.
Separate out the code which handles calling this into a separate function for code clarity and readability.
No functional change.
--- arch/powerpc/kvm/book3s_hv.c | 88 ++++++++++++++++++++++++-------------------- 1 file changed, 49 insertions(+), 39 deletions(-)
arch/powerpc/kvm/book3s_hv.c +49 -39
@@ -3367,7 +3367,55 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc) }/* +* Handle making the H_ENTER_NESTED hcall if we're pseries. +*/ +static int kvmhv_pseries_enter_guest (struct kvm_vcpu *vcpu,u64 time_limit, +unsigned long lpcr) +{ +/* call our hypervisor to load up HV regs and go */ +struct hv_guest_state hvregs; +/* we need to save/restore host & guest psscr since L0 doesn't for us */ +unsigned long host_psscr; +int trap; + + host_psscr =mfspr (SPRN_PSSCR_PR ); +mtspr (SPRN_PSSCR_PR , vcpu->arch.psscr); +kvmhv_save_hv_regs (vcpu, &hvregs); + hvregs.lpcr = lpcr; + vcpu->arch.regs.msr = vcpu->arch.shregs.msr; + hvregs.version =HV_GUEST_STATE_VERSION ; +if (vcpu->arch.nested) { + hvregs.lpid = vcpu->arch.nested->shadow_lpid; + hvregs.vcpu_token = vcpu->arch.nested_vcpu_id; + }else { + hvregs.lpid = vcpu->kvm->arch.lpid; + hvregs.vcpu_token = vcpu->vcpu_id; + } + hvregs.hdec_expiry = time_limit; + trap =plpar_hcall_norets (H_ENTER_NESTED ,__pa (&hvregs), +__pa (&vcpu->arch.regs)); +kvmhv_restore_hv_return_state (vcpu, &hvregs); + vcpu->arch.shregs.msr = vcpu->arch.regs.msr; + vcpu->arch.shregs.dar =mfspr (SPRN_DAR ); + vcpu->arch.shregs.dsisr =mfspr (SPRN_DSISR ); + vcpu->arch.psscr =mfspr (SPRN_PSSCR_PR ); +mtspr (SPRN_PSSCR_PR , host_psscr); + +/* H_CEDE has to be handled now, not later */ +if (trap =BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested && +kvmppc_get_gpr (vcpu,3 ) =H_CEDE ) { +kvmppc_nested_cede (vcpu); + trap =0 ; + } + +return trap; +} + +/* * Load up hypervisor-mode registers on P9. +* This is only called on baremetal (powernv) systems i.e. where +* CPU_FTR_HVMODE is set. This is only used for radix guests, however that +* radix guest may be a direct guest of this hypervisor or a nested guest. */ static int kvmhv_load_hv_regs_and_go (struct kvm_vcpu *vcpu,u64 time_limit,unsigned long lpcr)
@@ -3569,45 +3617,7 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,mtspr (SPRN_DEC , vcpu->arch.dec_expires -mftb ());if (kvmhv_on_pseries ()) { -/* -* We need to save and restore the guest visible part of the -* psscr (i.e. using SPRN_PSSCR_PR) since the hypervisor -* doesn't do this for us. Note only required if pseries since -* this is done in kvmhv_load_hv_regs_and_go() below otherwise. -*/ -unsigned long host_psscr; -/* call our hypervisor to load up HV regs and go */ -struct hv_guest_state hvregs; - - host_psscr =mfspr (SPRN_PSSCR_PR ); -mtspr (SPRN_PSSCR_PR , vcpu->arch.psscr); -kvmhv_save_hv_regs (vcpu, &hvregs); - hvregs.lpcr = lpcr; - vcpu->arch.regs.msr = vcpu->arch.shregs.msr; - hvregs.version =HV_GUEST_STATE_VERSION ; -if (vcpu->arch.nested) { - hvregs.lpid = vcpu->arch.nested->shadow_lpid; - hvregs.vcpu_token = vcpu->arch.nested_vcpu_id; - }else { - hvregs.lpid = vcpu->kvm->arch.lpid; - hvregs.vcpu_token = vcpu->vcpu_id; - } - hvregs.hdec_expiry = time_limit; - trap =plpar_hcall_norets (H_ENTER_NESTED ,__pa (&hvregs), -__pa (&vcpu->arch.regs)); -kvmhv_restore_hv_return_state (vcpu, &hvregs); - vcpu->arch.shregs.msr = vcpu->arch.regs.msr; - vcpu->arch.shregs.dar =mfspr (SPRN_DAR ); - vcpu->arch.shregs.dsisr =mfspr (SPRN_DSISR ); - vcpu->arch.psscr =mfspr (SPRN_PSSCR_PR ); -mtspr (SPRN_PSSCR_PR , host_psscr); - -/* H_CEDE has to be handled now, not later */ -if (trap =BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested && -kvmppc_get_gpr (vcpu,3 ) =H_CEDE ) { -kvmppc_nested_cede (vcpu); - trap =0 ; - } + trap =kvmhv_pseries_enter_guest (vcpu, time_limit, lpcr); }else { trap =kvmhv_load_hv_regs_and_go (vcpu, time_limit, lpcr); }
Signature · 1 line
2.13.6
PATCH 5/23 KVM: PPC: Book3S HV: Enable calling kvmppc_hpte_hv_fault in virtual mode
In reply to Suraj Jitindar SinghThe function kvmppc_hpte_hv_fault() is used to search a hpt (hash page table) for an existing entry given a slb entry, a fault code and a fault address.
Currently this function is only called in real mode. Modify this function so that it can be called in virtual mode and add a function parameter used to specify if the function is being called from real mode or not.
--- arch/powerpc/include/asm/kvm_ppc.h | 3 ++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 9 +++++++-- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++ 3 files changed, 11 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +2 -1
@@ -688,7 +688,8 @@ long kvmppc_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,long kvmppc_rm_h_page_init (struct kvm_vcpu *vcpu,unsigned long flags,unsigned long dest,unsigned long src);long kvmppc_hpte_hv_fault (struct kvm_vcpu *vcpu,unsigned long addr, -unsigned long slb_v,unsigned int status,bool data); +unsigned long slb_v,unsigned int status, +bool data,bool is_realmode);unsigned long kvmppc_rm_h_xirr (struct kvm_vcpu *vcpu);unsigned long kvmppc_rm_h_xirr_x (struct kvm_vcpu *vcpu);unsigned long kvmppc_rm_h_ipoll (struct kvm_vcpu *vcpu,unsigned long server);
arch/powerpc/kvm/book3s_hv_rm_mmu.c +7 -2
@@ -1182,9 +1182,12 @@ EXPORT_SYMBOL(kvmppc_hv_find_lock_hpte);* -1 to pass the fault up to host kernel mode code, -2 to do that * and also load the instruction word (for MMIO emulation), * or 0 if we should make the guest retry the access. +* For a nested hypervisor, this will be called in virtual mode +* (is_realmode = false) and should be called with preemption disabled. */ long kvmppc_hpte_hv_fault (struct kvm_vcpu *vcpu,unsigned long addr, -unsigned long slb_v,unsigned int status,bool data) +unsigned long slb_v,unsigned int status, +bool data,bool is_realmode) {struct kvm *kvm = vcpu->kvm;long int index;
@@ -1222,7 +1225,9 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr, v =hpte_new_to_old_v (v, r); r =hpte_new_to_old_r (r); } - rev =real_vmalloc_addr (&kvm->arch.hpt.rev[index]); + rev = &kvm->arch.hpt.rev[index]; +if (is_realmode) + rev =real_vmalloc_addr (rev); gr = rev->guest_rpte;unlock_hpte (hpte, orig_v);
arch/powerpc/kvm/book3s_hv_rmhandlers.S +2 -0
@@ -2066,6 +2066,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) /* Search the hash table. */ mr r3, r9 /* vcpu pointer */ li r7, 1 /* data fault */ + li r8, 1 /* is real mode */ bl kvmppc_hpte_hv_fault ld r9, HSTATE_KVM_VCPU(r13) ld r10, VCPU_PC(r9)
@@ -2158,6 +2159,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) mr r4, r10 mr r6, r11 li r7, 0 /* instruction fault */ + li r8, 1 /* is real mode */ bl kvmppc_hpte_hv_fault ld r9, HSTATE_KVM_VCPU(r13) ld r10, VCPU_PC(r9)
Signature · 1 line
2.13.6
PATCH 6/23 KVM: PPC: Book3S HV: Allow hpt manipulation hcalls to be called in virtual mode
In reply to Suraj Jitindar SinghThe hcalls H_ENTER, H_REMOVE, H_READ, H_CLEAR_MOD, H_CLEAR_READ, H_PROTECT and H_BULK_REMOVE are used by a guest to call into the hypervisor in order to control manipulation of it's hpt (hash page table) by the hypervisor on it'the guest's behalf.
Currently the functions which handle these hcalls are only called in real mode from the kvm exit path in book3s_hv_rmhandlers.c.
Modify the functions which handle these hcalls so that they can be called in real mode and call them from the virtual mode hcall handling function kvmppc_pseries_do_hcall() if we're a pseries machine. There is no need to call these functions again on powernv as they were already called from real mode.
--- arch/powerpc/include/asm/kvm_book3s.h | 16 ++++- arch/powerpc/kvm/book3s_64_mmu_hv.c | 10 +--- arch/powerpc/kvm/book3s_hv.c | 60 +++++++++++++++++++ arch/powerpc/kvm/book3s_hv_rm_mmu.c | 107 +++++++++++++++++++++++++--------- 4 files changed, 159 insertions(+), 34 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +15 -1
@@ -252,9 +252,23 @@ extern void kvmppc_unpin_guest_page(struct kvm *kvm, void *addr,extern long kvmppc_do_h_enter (struct kvm *kvm,unsigned long flags,long pte_index,unsigned long pteh,unsigned long ptel,pgd_t *pgdir,bool realmode,unsigned long *idx_ret); +extern long kvmppc_virtmode_do_h_enter (struct kvm *kvm,unsigned long flags, +long pte_index,unsigned long pteh, +unsigned long ptel, +unsigned long *pte_idx_ret);extern long kvmppc_do_h_remove (struct kvm *kvm,unsigned long flags,unsigned long pte_index,unsigned long avpn, -unsigned long *hpret); +bool realmode,unsigned long *hpret); +extern long kvmppc_do_h_read (struct kvm_vcpu *vcpu,unsigned long flags, +unsigned long pte_index,bool realmode); +extern long kvmppc_do_h_clear_mod (struct kvm_vcpu *vcpu,unsigned long flags, +unsigned long pte_index,bool realmode); +extern long kvmppc_do_h_clear_ref (struct kvm_vcpu *vcpu,unsigned long flags, +unsigned long pte_index,bool realmode); +extern long kvmppc_do_h_protect (struct kvm_vcpu *vcpu,unsigned long flags, +unsigned long pte_index,unsigned long avpn, +unsigned long va,bool realmode); +extern long kvmppc_do_h_bulk_remove (struct kvm_vcpu *vcpu,bool realmode);extern long kvmppc_hv_get_dirty_log_hpt (struct kvm *kvm,struct kvm_memory_slot *memslot,unsigned long *map); extern void kvmppc_harvest_vpa_dirty(struct kvmppc_vpa *vpa,
arch/powerpc/kvm/book3s_64_mmu_hv.c +3 -7
@@ -42,10 +42,6 @@ do { } while (0)#endif -static long kvmppc_virtmode_do_h_enter (struct kvm *kvm,unsigned long flags, -long pte_index,unsigned long pteh, -unsigned long ptel,unsigned long *pte_idx_ret); -struct kvm_resize_hpt {/* These fields read-only after init */ struct kvm *kvm;
@@ -287,7 +283,7 @@ static void kvmppc_mmu_book3s_64_hv_reset_msr(struct kvm_vcpu *vcpu)kvmppc_set_msr (vcpu, msr); } -static long kvmppc_virtmode_do_h_enter (struct kvm *kvm,unsigned long flags, +long kvmppc_virtmode_do_h_enter (struct kvm *kvm,unsigned long flags,long pte_index,unsigned long pteh,unsigned long ptel,unsigned long *pte_idx_ret) {
@@ -1907,7 +1903,7 @@ static ssize_t kvm_htab_write(struct file *file, const char __user *buf, nb +=HPTE_SIZE ;if (be64_to_cpu (hptp[0 ]) & (HPTE_V_VALID |HPTE_V_ABSENT )) -kvmppc_do_h_remove (kvm,0 , i,0 , tmp); +kvmppc_do_h_remove (kvm,0 , i,0 , false, tmp); err = -EIO ; ret =kvmppc_virtmode_do_h_enter (kvm,H_EXACT , i, v, r, tmp);
@@ -1937,7 +1933,7 @@ static ssize_t kvm_htab_write(struct file *file, const char __user *buf,for (j =0 ; j < hdr.n_invalid; ++j) {if (be64_to_cpu (hptp[0 ]) & (HPTE_V_VALID |HPTE_V_ABSENT )) -kvmppc_do_h_remove (kvm,0 , i,0 , tmp); +kvmppc_do_h_remove (kvm,0 , i,0 , false, tmp); ++i; hptp +=2 ; }
arch/powerpc/kvm/book3s_hv.c +60 -0
66 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +81 -26
241 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 7/23 KVM: PPC: Book3S HV: Make kvmppc_invalidate_hpte() take lpid not a kvm struct
In reply to Suraj Jitindar SinghThe function kvmppc_invalidate_hpte() is used to invalidate and perform the tlbies for a given pte in a hpt (hash page table). Currently the function takes a kvm struct as an argument, however the only member of this struct that it accesses in the lpid field. Modify this function to take an lpid argument in place of the kvm struct.
No functional change.
--- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 6 +++--- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 22 ++++++++++++---------- 3 files changed, 16 insertions(+), 14 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -1
@@ -241,7 +241,7 @@ extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev, unsigned long *rmap, long pte_index, int realmode);extern void kvmppc_update_dirty_map (const struct kvm_memory_slot *memslot,unsigned long gfn,unsigned long psize); -extern void kvmppc_invalidate_hpte (struct kvm *kvm,__be64 *hptep, +extern void kvmppc_invalidate_hpte (unsigned int lpid,__be64 *hptep,unsigned long pte_index);void kvmppc_clear_ref_hpte (struct kvm *kvm,__be64 *hptep,unsigned long pte_index);
arch/powerpc/kvm/book3s_64_mmu_hv.c +3 -3
@@ -701,7 +701,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,/* HPTE was previously valid, so we need to invalidate it */ unlock_rmap (rmap); hptep[0 ] |=cpu_to_be64 (HPTE_V_ABSENT ); -kvmppc_invalidate_hpte (kvm, hptep, index); +kvmppc_invalidate_hpte (kvm->arch.lpid, hptep, index);/* don't lose previous R and C bits */ r |=be64_to_cpu (hptep[1 ]) & (HPTE_R_R |HPTE_R_C ); }else {
@@ -829,7 +829,7 @@ static void kvmppc_unmap_hpte(struct kvm *kvm, unsigned long i,if ((be64_to_cpu (hptep[0 ]) &HPTE_V_VALID ) &&hpte_rpn (ptel, psize) = gfn) { hptep[0 ] |=cpu_to_be64 (HPTE_V_ABSENT ); -kvmppc_invalidate_hpte (kvm, hptep, i); +kvmppc_invalidate_hpte (kvm->arch.lpid, hptep, i); hptep[1 ] &= ~cpu_to_be64 (HPTE_R_KEY_HI |HPTE_R_KEY_LO );/* Harvest R and C */ rcbits =be64_to_cpu (hptep[1 ]) & (HPTE_R_R |HPTE_R_C );
@@ -1094,7 +1094,7 @@ static int kvm_test_clear_dirty_npages(struct kvm *kvm, unsigned long *rmapp)/* need to make it temporarily absent so C is stable */ hptep[0 ] |=cpu_to_be64 (HPTE_V_ABSENT ); -kvmppc_invalidate_hpte (kvm, hptep, i); +kvmppc_invalidate_hpte (kvm->arch.lpid, hptep, i); v =be64_to_cpu (hptep[0 ]); r =be64_to_cpu (hptep[1 ]);if (r &HPTE_R_C ) {
arch/powerpc/kvm/book3s_hv_rm_mmu.c +12 -10
@@ -440,7 +440,7 @@ static inline int is_mmio_hpte(unsigned long v, unsigned long r) (HPTE_R_KEY_HI | HPTE_R_KEY_LO)); } -static void do_tlbies (struct kvm *kvm,unsigned long *rbvalues, +static void do_tlbies (unsigned int lpid,unsigned long *rbvalues,long npages,int global,bool need_sync) {long i;
@@ -455,7 +455,7 @@ static void do_tlbies(struct kvm *kvm, unsigned long *rbvalues, asmvolatile ("ptesync" : : :"memory" );for (i =0 ; i < npages; ++i) { asm volatile(PPC_TLBIE_5(%0,%1,0,0,0) : : - "r" (rbvalues[i]), "r" (kvm->arch.lpid)); + "r" (rbvalues[i]), "r" (lpid)); }if (cpu_has_feature (CPU_FTR_P9_TLBIE_BUG )) {
@@ -465,7 +465,7 @@ static void do_tlbies(struct kvm *kvm, unsigned long *rbvalues,*/ asmvolatile ("ptesync" : : :"memory" ); asm volatile(PPC_TLBIE_5(%0,%1,0,0,0) : : - "r" (rbvalues[0]), "r" (kvm->arch.lpid)); + "r" (rbvalues[0]), "r" (lpid)); } asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" );
@@ -516,7 +516,8 @@ long kvmppc_do_h_remove(struct kvm *kvm, unsigned long flags,if (v &HPTE_V_VALID ) { hpte[0 ] &= ~cpu_to_be64 (HPTE_V_VALID ); rb =compute_tlbie_rb (v, pte_r, pte_index); -do_tlbies (kvm, &rb,1 ,global_invalidates (kvm), true); +do_tlbies (kvm->arch.lpid, &rb,1 ,global_invalidates (kvm), + true);/* * The reference (R) and change (C) bits in a HPT * entry can be set by hardware at any time up until
@@ -652,7 +653,7 @@ long kvmppc_do_h_bulk_remove(struct kvm_vcpu *vcpu, bool realmode)break ;/* Now that we've collected a batch, do the tlbies */ -do_tlbies (kvm, tlbrb, n, global, true); +do_tlbies (kvm->arch.lpid, tlbrb, n, global, true);/* Read PTE low words after tlbie to get final R/C values */ for (k =0 ; k < n; ++k) {
@@ -736,7 +737,8 @@ long kvmppc_do_h_protect(struct kvm_vcpu *vcpu, unsigned long flags, rb =compute_tlbie_rb (v, r, pte_index); hpte[0 ] =cpu_to_be64 ((pte_v & ~HPTE_V_VALID ) |HPTE_V_ABSENT ); -do_tlbies (kvm, &rb,1 ,global_invalidates (kvm), true); +do_tlbies (kvm->arch.lpid, &rb,1 , +global_invalidates (kvm), true);/* Don't lose R/C bit updates done by hardware */ r |=be64_to_cpu (hpte[1 ]) & (HPTE_R_R |HPTE_R_C ); hpte[1 ] =cpu_to_be64 (r);
@@ -898,7 +900,7 @@ long kvmppc_do_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,if (v &HPTE_V_VALID ) {/* need to make it temporarily absent so C is stable */ hpte[0 ] |=cpu_to_be64 (HPTE_V_ABSENT ); -kvmppc_invalidate_hpte (kvm, hpte, pte_index); +kvmppc_invalidate_hpte (kvm->arch.lpid, hpte, pte_index); r =be64_to_cpu (hpte[1 ]); gr |= r & (HPTE_R_R |HPTE_R_C );if (r &HPTE_R_C ) {
@@ -1064,7 +1066,7 @@ long kvmppc_rm_h_page_init(struct kvm_vcpu *vcpu, unsigned long flags,return ret; } -void kvmppc_invalidate_hpte (struct kvm *kvm,__be64 *hptep, +void kvmppc_invalidate_hpte (unsigned int lpid,__be64 *hptep,unsigned long pte_index) {unsigned long rb;
@@ -1078,7 +1080,7 @@ void kvmppc_invalidate_hpte(struct kvm *kvm, __be64 *hptep, hp1 =hpte_new_to_old_r (hp1); } rb =compute_tlbie_rb (hp0, hp1, pte_index); -do_tlbies (kvm, &rb,1 ,1 , true); +do_tlbies (lpid, &rb,1 ,1 , true); }EXPORT_SYMBOL_GPL (kvmppc_invalidate_hpte);
@@ -1099,7 +1101,7 @@ void kvmppc_clear_ref_hpte(struct kvm *kvm, __be64 *hptep, rbyte = (be64_to_cpu (hptep[1 ]) & ~HPTE_R_R ) >>8 ;/* modify only the second-last byte, which contains the ref bit */ *((char *)hptep +14 ) = rbyte; -do_tlbies (kvm, &rb,1 ,1 , false); +do_tlbies (kvm->arch.lpid, &rb,1 ,1 , false); }EXPORT_SYMBOL_GPL (kvmppc_clear_ref_hpte);
Signature · 1 line
2.13.6
PATCH 8/23 KVM: PPC: Book3S HV: Nested: Allow pseries hypervisor to run hpt nested guest
In reply to Suraj Jitindar SinghAllow a pseries guest hypervisor (must be a radix guest) to run a hpt (hash page table) nested guest.
Modify the entry path such that a pseries hypervisor will always use the kvmhv_run_single_vcpu() function to enter a guest which will result in it calling H_ENTER_NESTED. Also modify the API to H_ENTER_NESTED to add a version 2 which adds a slb pointer to the argument list which provides the slb state to be used to run the nested guest.
Also modify the exit path such that a pseries hypervisor will call kvmppc_hpte_hv_fault() when handling a page fault which would normally be called from real mode in book3s_hv_rmhandlers.c on a powernv hypervisor. This is required for subsequent functions which are invoked to handle the page fault. Also save the slb state on guest exit and don't zero slb_max.
Modify kvm_vm_ioctl_get_smmu_info_hv() such that only 4k and 64k page size support is reported to the guest. This is to ensure that we can maintain a 1-to-1 mapping between the guest hpt and the shadow hpt for simplicity (this could be relaxed later with appropriate support) since a 16M page would likely have to be broken into multiple smaller pages since the radix guest is likely to at most be backed by 2M pages.
Modify do_tlbies() such that a pseries hypervisor will make the H_TLB_INVALIDATE hcall to notify it's hypervisor of the invalidation of partition scoped translation information which is required to keep the shadow hpt in sync.
Finally allow a pseries hypervisor to run a nested hpt guest by reporting the KVM_CAP_PPC_MMU_HASH_V3 capability and allowing handling of kvmhv_configure_mmu().
--- arch/powerpc/include/asm/hvcall.h | 36 -------------- arch/powerpc/include/asm/kvm_book3s.h | 2 + arch/powerpc/include/asm/kvm_host.h | 55 +++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 90 ++++++++++++++++++++++++++++------- arch/powerpc/kvm/book3s_hv_nested.c | 22 ++++++++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 20 ++++++++ arch/powerpc/kvm/powerpc.c | 3 +- 7 files changed, 173 insertions(+), 55 deletions(-)
arch/powerpc/include/asm/hvcall.h +0 -36
@@ -481,42 +481,6 @@ struct h_cpu_char_result {u64 behaviour; }; -/* Register state for entering a nested guest with H_ENTER_NESTED */ -struct hv_guest_state { -u64 version;/* version of this structure layout */ -u32 lpid; -u32 vcpu_token; -/* These registers are hypervisor privileged (at least for writing) */ -u64 lpcr; -u64 pcr; -u64 amor; -u64 dpdes; -u64 hfscr; -s64 tb_offset; -u64 dawr0; -u64 dawrx0; -u64 ciabr; -u64 hdec_expiry; -u64 purr; -u64 spurr; -u64 ic; -u64 vtb; -u64 hdar; -u64 hdsisr; -u64 heir; -u64 asdr; -/* These are OS privileged but need to be set late in guest entry */ -u64 srr0; -u64 srr1; -u64 sprg[4 ]; -u64 pidr; -u64 cfar; -u64 ppr; -}; - -/* Latest version of hv_guest_state structure */ -#define HV_GUEST_STATE_VERSION 1 -#endif /* __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* _ASM_POWERPC_HVCALL_H */
arch/powerpc/include/asm/kvm_book3s.h +2 -0
@@ -317,6 +317,8 @@ long kvmhv_do_nested_tlbie(struct kvm_vcpu *vcpu);int kvmhv_run_single_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu,u64 time_limit,unsigned long lpcr);void kvmhv_save_hv_regs (struct kvm_vcpu *vcpu,struct hv_guest_state *hr); +void kvmhv_save_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp); +void kvmhv_restore_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp);void kvmhv_restore_hv_return_state (struct kvm_vcpu *vcpu,struct hv_guest_state *hr);long int kvmhv_nested_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu);
arch/powerpc/include/asm/kvm_host.h +55 -0
61 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +74 -16
150 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_nested.c +21 -1
@@ -51,6 +51,16 @@ void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr) hr->ppr = vcpu->arch.ppr; } +void kvmhv_save_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp) +{ +int i; + +for (i =0 ; i <64 ; i++) + slbp->slb[i] = vcpu->arch.slb[i]; + slbp->slb_max = vcpu->arch.slb_max; + slbp->slb_nr = vcpu->arch.slb_nr; +} +static void byteswap_pt_regs (struct pt_regs *regs) {unsigned long *addr = (unsigned long *) regs;
@@ -169,6 +179,16 @@ static void restore_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr) vcpu->arch.ppr = hr->ppr; } +void kvmhv_restore_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp) +{ +int i; + +for (i =0 ; i <64 ; i++) + vcpu->arch.slb[i] = slbp->slb[i]; + vcpu->arch.slb_max = slbp->slb_max; + vcpu->arch.slb_nr = slbp->slb_nr; +} +void kvmhv_restore_hv_return_state (struct kvm_vcpu *vcpu,struct hv_guest_state *hr) {
@@ -239,7 +259,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)return H_PARAMETER ;if (kvmppc_need_byteswap (vcpu))byteswap_hv_regs (&l2_hv); -if (l2_hv.version !=HV_GUEST_STATE_VERSION ) +if (l2_hv.version !=1 )return H_P2 ; regs_ptr =kvmppc_get_gpr (vcpu,5 );
arch/powerpc/kvm/book3s_hv_rm_mmu.c +20 -0
@@ -446,6 +446,25 @@ static void do_tlbies(unsigned int lpid, unsigned long *rbvalues,long i;/* +* Handle the case where we're running as a nested hypervisor and so +* have to make an hcall to handle invalidations for us. +*/ +if (kvmhv_on_pseries ()) { +unsigned long rc, ric =0 , prs =0 , r =0 ; + +for (i =0 ; i < npages; i++) { + rc =plpar_hcall_norets (H_TLB_INVALIDATE , +H_TLBIE_P1_ENC (ric, prs, r), + lpid, rbvalues[i]); +if (rc) +pr_err ("KVM: HPT TLB page invalidation hcall failed" +", rc=%ld\n" , rc); + } + +return ; + } + +/* * We use the POWER9 5-operand versions of tlbie and tlbiel here. * Since we are using RIC=0 PRS=0 R=0, and P7/P8 tlbiel ignores * the RS field, this is backwards-compatible with P7 and P8.
@@ -1355,3 +1374,4 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,return -1 ;/* send fault up to host kernel mode */ } +EXPORT_SYMBOL_GPL (kvmppc_hpte_hv_fault);
arch/powerpc/kvm/powerpc.c +1 -2
@@ -604,8 +604,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = !!(hv_enabled &&radix_enabled ());break ;case KVM_CAP_PPC_MMU_HASH_V3 : - r = !!(hv_enabled &&cpu_has_feature (CPU_FTR_ARCH_300 ) && -cpu_has_feature (CPU_FTR_HVMODE )); + r = !!(hv_enabled &&cpu_has_feature (CPU_FTR_ARCH_300 ));break ;case KVM_CAP_PPC_NESTED_HV : r = !!(hv_enabled && kvmppc_hv_ops->enable_nested &&
Signature · 1 line
2.13.6
PATCH 9/23 KVM: PPC: Book3S HV: Nested: Improve comments and naming of nest rmap functions
In reply to Suraj Jitindar SinghThe nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot.
Improve the naming of some of these functions such that it's clearer what they do, the functions with remove in the name remove the rmap _and_ perform an invalidation so rename them invalidate to reflect this.
kvmhv_insert_nest_rmap() takes a kvm struct as an argument which is unused, so remove this argument.
Additionally improve the function comments and add information about which locks must be held for clarity.
No functional change.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 4 +-- arch/powerpc/kvm/book3s_64_mmu_radix.c | 8 +++--- arch/powerpc/kvm/book3s_hv_nested.c | 49 +++++++++++++++++++++++--------- 3 files changed, 42 insertions(+), 19 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +2 -2
@@ -624,12 +624,12 @@ extern int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, unsigned long gpa, unsigned int level, unsigned long mmu_seq, unsigned int lpid, unsigned long *rmapp, struct rmap_nested **n_rmap); -extern void kvmhv_insert_nest_rmap (struct kvm *kvm,unsigned long *rmapp, +extern void kvmhv_insert_nest_rmap (unsigned long *rmapp,struct rmap_nested **n_rmap);extern void kvmhv_update_nest_rmap_rc_list (struct kvm *kvm,unsigned long *rmapp,unsigned long clr,unsigned long set,unsigned long hpa,unsigned long nbytes); -extern void kvmhv_remove_nest_rmap_range (struct kvm *kvm, +extern void kvmhv_invalidate_nest_rmap_range (struct kvm *kvm,const struct kvm_memory_slot *memslot,unsigned long gpa,unsigned long hpa,unsigned long nbytes);
arch/powerpc/kvm/book3s_64_mmu_radix.c +4 -4
@@ -405,7 +405,7 @@ void kvmppc_unmap_pte(struct kvm *kvm, pte_t *pte, unsigned long gpa, gpa &= ~(page_size -1 ); hpa = old &PTE_RPN_MASK ; -kvmhv_remove_nest_rmap_range (kvm, memslot, gpa, hpa, page_size); +kvmhv_invalidate_nest_rmap_range (kvm, memslot, gpa, hpa, page_size);if ((old & _PAGE_DIRTY) && memslot->dirty_bitmap)kvmppc_update_dirty_map (memslot, gfn, page_size);
@@ -643,7 +643,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, }kvmppc_radix_set_pte_at (kvm, gpa, (pte_t *)pud, pte);if (rmapp && n_rmap) -kvmhv_insert_nest_rmap (kvm, rmapp, n_rmap); +kvmhv_insert_nest_rmap (rmapp, n_rmap); ret =0 ; goto out_unlock; }
@@ -695,7 +695,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, }kvmppc_radix_set_pte_at (kvm, gpa,pmdp_ptep (pmd), pte);if (rmapp && n_rmap) -kvmhv_insert_nest_rmap (kvm, rmapp, n_rmap); +kvmhv_insert_nest_rmap (rmapp, n_rmap); ret =0 ; goto out_unlock; }
@@ -721,7 +721,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, }kvmppc_radix_set_pte_at (kvm, gpa, ptep, pte);if (rmapp && n_rmap) -kvmhv_insert_nest_rmap (kvm, rmapp, n_rmap); +kvmhv_insert_nest_rmap (rmapp, n_rmap); ret =0 ; out_unlock:
arch/powerpc/kvm/book3s_hv_nested.c +36 -13
100 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 10/23 KVM: PPC: Book3S HV: Nested: Increase gpa field in nest rmap to 46 bits
In reply to Suraj Jitindar SinghThe nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot.
Increase the size of the gpa (guest physical address) field in the nested rmap entry to 46 bits such that it can be reused to store a nested hpt (hash page table) guest entry where this field will hold the hpt index (which can be up to 46 bits).
Additionally introduce helper functions to access these bit fields for simplicity.
No functional change.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 5 ++-- arch/powerpc/kvm/book3s_hv_nested.c | 41 ++++++++++++++++++++++++-------- 2 files changed, 34 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +3 -2
@@ -50,12 +50,13 @@ struct kvm_nested_guest {/* * We define a nested rmap entry as a single 64-bit quantity * 0xFFF0000000000000 12-bit lpid field -* 0x000FFFFFFFFFF000 40-bit guest 4k page frame number +* 0x000FFFFFFFFFFFC0 46-bit guest page frame number (radix) or hpt index * 0x0000000000000001 1-bit single entry flag */ #define RMAP_NESTED_LPID_MASK 0xFFF0000000000000UL#define RMAP_NESTED_LPID_SHIFT (52) -#define RMAP_NESTED_GPA_MASK 0x000FFFFFFFFFF000UL +#define RMAP_NESTED_GPA_MASK 0x000FFFFFFFFFFFC0UL +#define RMAP_NESTED_GPA_SHIFT (6)#define RMAP_NESTED_IS_SINGLE_ENTRY 0x0000000000000001UL/* Structure for a nested guest rmap entry */
arch/powerpc/kvm/book3s_hv_nested.c +31 -10
72 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 11/23 KVM: PPC: Book3S HV: Nested: Remove single nest rmap entries
In reply to Suraj Jitindar SinghThe nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot. These entries are stored in the guest memslot as a singly linked list (llist) with the next pointer of the last entry in the list used to store a "single entry" to save having to add another list entry.
This approach while saving a small amount of memory significantly complicates the list handling. For simplicity and code clarity remove the existence of these "single entries" and always insert another list entry to hold the final value.
No functional change.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 65 ++++++++++---------------------- arch/powerpc/kvm/book3s_hv_nested.c | 51 ++++++++++--------------- 2 files changed, 39 insertions(+), 77 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +19 -46
@@ -48,66 +48,39 @@ struct kvm_nested_guest { };/* +* We use nested rmap entries to store information so that we can find pte +* entries in the shadow page table of a nested guest when the host is modifying +* a pte for a l1 guest page. For a radix nested guest this is the nested +* gpa which can then be used to walk the radix shadow page table to find a +* pte. For a hash nested guest this is an index into the shadow hpt which can +* be used to find a pte. Irrespective the lpid of the nested guest is stored. +* +* These entries are stored in a rmap_nested struct. There may be multiple +* entries for a single l1 guest page since that guest may have multiple nested +* guests and map the same page into more than 1, or a single nested guest may +* map the same l1 guest page with multiple hpt entries. To accommodate this +* the rmap_nested entries are linked together in a singly linked list with the +* corresponding rmap entry in the rmap array of the memslot containing the +* head pointer of the linked list (or NULL if list is empty). +*/ + +/* * We define a nested rmap entry as a single 64-bit quantity * 0xFFF0000000000000 12-bit lpid field * 0x000FFFFFFFFFFFC0 46-bit guest page frame number (radix) or hpt index -* 0x0000000000000001 1-bit single entry flag +* 0x000000000000003F 6-bit unused field */ #define RMAP_NESTED_LPID_MASK 0xFFF0000000000000UL#define RMAP_NESTED_LPID_SHIFT (52)#define RMAP_NESTED_GPA_MASK 0x000FFFFFFFFFFFC0UL#define RMAP_NESTED_GPA_SHIFT (6) -#define RMAP_NESTED_IS_SINGLE_ENTRY 0x0000000000000001UL/* Structure for a nested guest rmap entry */ struct rmap_nested {struct llist_node list; -u64 rmap; +u64 rmap;/* layout defined above */ }; -/* -* for_each_nest_rmap_safe - iterate over the list of nested rmap entries -* safe against removal of the list entry or NULL list -* @pos: a (struct rmap_nested *) to use as a loop cursor -* @node: pointer to the first entry -* NOTE: this can be NULL -* @rmapp: an (unsigned long *) in which to return the rmap entries on each -* iteration -* NOTE: this must point to already allocated memory -* -* The nested_rmap is a llist of (struct rmap_nested) entries pointed to by the -* rmap entry in the memslot. The list is always terminated by a "single entry" -* stored in the list element of the final entry of the llist. If there is ONLY -* a single entry then this is itself in the rmap entry of the memslot, not a -* llist head pointer. -* -* Note that the iterator below assumes that a nested rmap entry is always -* non-zero. This is true for our usage because the LPID field is always -* non-zero (zero is reserved for the host). -* -* This should be used to iterate over the list of rmap_nested entries with -* processing done on the u64 rmap value given by each iteration. This is safe -* against removal of list entries and it is always safe to call free on (pos). -* -* e.g. -* struct rmap_nested *cursor; -* struct llist_node *first; -* unsigned long rmap; -* for_each_nest_rmap_safe(cursor, first, &rmap) { -* do_something(rmap); -* free(cursor); -* } -*/ -#define for_each_nest_rmap_safe (pos, node, rmapp) \ - for ((pos) = llist_entry((node), typeof(*(pos)), list); \ - (node) && \ - (*(rmapp) = ((RMAP_NESTED_IS_SINGLE_ENTRY & ((u64) (node))) ? \ - ((u64) (node)) : ((pos)->rmap))) && \ - (((node) = ((RMAP_NESTED_IS_SINGLE_ENTRY & ((u64) (node))) ? \ - ((struct llist_node *) ((pos) = NULL)) : \ - (pos)->list.next)), true); \ - (pos) = llist_entry((node), typeof(*(pos)), list)) -struct kvm_nested_guest *kvmhv_get_nested (struct kvm *kvm,int l1_lpid,bool create);void kvmhv_put_nested (struct kvm_nested_guest *gp);
arch/powerpc/kvm/book3s_hv_nested.c +20 -31
91 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 12/23 KVM: PPC: Book3S HV: Nested: add kvmhv_remove_all_nested_rmap_lpid()
In reply to Suraj Jitindar SinghNested rmap entries are used to store a reverse mapping from a L1 guest page in a kvm memslot to a nested guest pte.
Implement a function to remove all nest rmap entries of a given lpid from all of the memslots for a given guest.
--- arch/powerpc/kvm/book3s_hv_nested.c | 47 +++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
arch/powerpc/kvm/book3s_hv_nested.c +47 -0
@@ -819,6 +819,53 @@ void kvmhv_insert_nest_rmap(unsigned long *rmapp, struct rmap_nested **n_rmap) *n_rmap =NULL ; } +/* called with kvm->mmu_lock held */ +static void kvmhv_remove_nested_rmap_lpid (unsigned long *rmapp,int l1_lpid) +{ +struct llist_node **next = &(((struct llist_head *) rmapp)->first); +u64 match =lpid_to_n_rmap (l1_lpid); + +while (*next) { +struct llist_node *entry = (*next); +struct rmap_nested *n_rmap =llist_entry (entry,typeof (*n_rmap), + list); + +if (kvmhv_n_rmap_is_equal (match, n_rmap->rmap, +RMAP_NESTED_LPID_MASK )) { + *next = entry->next; +kfree (n_rmap); + }else { + next = &(entry->next); + } + } +} + +/* +* caller must hold gp->tlb_lock +* For a given nested lpid, remove all of the rmap entries which match that +* nest lpid. Note that no invalidation/tlbie is done for the entries, it is +* assumed that the caller will perform an lpid wide invalidation after calling +* this function. +*/ +static void kvmhv_remove_all_nested_rmap_lpid (struct kvm *kvm,int l1_lpid) +{ +struct kvm_memory_slot *memslot; + +kvm_for_each_memslot (memslot,kvm_memslots (kvm)) { +unsigned long page; + +for (page =0 ; page < memslot->npages; page++) { +unsigned long *rmapp; + +spin_lock (&kvm->mmu_lock); + rmapp = &memslot->arch.rmap[page]; +if (*rmapp)/* Are there any rmap entries? */ +kvmhv_remove_nested_rmap_lpid (rmapp, l1_lpid); +spin_unlock (&kvm->mmu_lock); + } + } +} +/* * called with kvm->mmu_lock held * Given a single rmap entry, update the rc bits in the corresponding shadow
Signature · 1 line
2.13.6
PATCH 13/23 KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup
In reply to Suraj Jitindar SinghAdd the infrastructure to book3s_hv_nested.c to allow a nested hpt (hash page table) guest to be setup. As this patch doesn't add the capability of creating or removing mmu translations return H_PARAMETER when an attempt to actually run a nested hpt guest is made.
Add fields to the nested guest struct to store the hpt and the vrma slb entry.
Update kvmhv_update_ptbl_cache() to determine when a nested guest is switching from radix to hpt or hpt to radix and perform the required setup. A page table (radix) or hpt (hash) must be allocated with any existing table being freed and the radix field in the nested guest struct being updated under the mmu_lock (this means that when holding the mmu_lock the radix field can be tested and the existance of the correct type of page table guaranteed). Also remove all of the nest rmap entries which belong to this nested guest since a nested rmap entry is specific to whether the nested guest is hash or radix.
When a nested guest is initially created or when the partition table entry is empty we assume a radix guest since it is much less expensive to allocate a radix page table compared to a hpt.
The hpt which is allocated in the hypervisor for the nested guest (called the shadow hpt) is identical in size to the one allocated in the guest hypervisor to ensure a 1-to-1 mapping between page table entries. This simplifies handling of the entries however this requirement could be relaxed in future if support was added.
Introduce a hash nested_page_fault function to be envoked when the nested guest which experiences a page fault is hash, returns -EINVAL for now. Also return -EINVAL when handling the H_TLB_INVALIDATE hcall. Also lacking support for the hypervisor paging out a guest page which has been mapped through to a nested guest. These 3 portions of functionality added in proceeding patches.
--- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 2 + arch/powerpc/include/asm/book3s/64/mmu.h | 9 + arch/powerpc/include/asm/kvm_book3s_64.h | 4 +- arch/powerpc/kvm/book3s_hv_nested.c | 255 ++++++++++++++++++++++---- 4 files changed, 235 insertions(+), 35 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +2 -0
@@ -45,12 +45,14 @@#define SLB_VSID_KP ASM_CONST(0x0000000000000400)#define SLB_VSID_N ASM_CONST(0x0000000000000200)/* no-execute */ #define SLB_VSID_L ASM_CONST(0x0000000000000100) +#define SLB_VSID_L_SHIFT 8#define SLB_VSID_C ASM_CONST(0x0000000000000080)/* class */ #define SLB_VSID_LP ASM_CONST(0x0000000000000030)#define SLB_VSID_LP_00 ASM_CONST(0x0000000000000000)#define SLB_VSID_LP_01 ASM_CONST(0x0000000000000010)#define SLB_VSID_LP_10 ASM_CONST(0x0000000000000020)#define SLB_VSID_LP_11 ASM_CONST(0x0000000000000030) +#define SLB_VSID_LP_SHIFT 4#define SLB_VSID_LLP (SLB_VSID_L|SLB_VSID_LP)#define SLB_VSID_KERNEL (SLB_VSID_KP)
arch/powerpc/include/asm/book3s/64/mmu.h +9 -0
@@ -62,6 +62,7 @@ struct patb_entry {extern struct patb_entry *partition_tb;/* Bits in patb0 field */ +/* Radix */ #define PATB_HR (1UL << 63)#define RPDB_MASK 0x0fffffffffffff00UL#define RPDB_SHIFT (1UL << 8)
@@ -70,6 +71,14 @@ extern struct patb_entry *partition_tb;#define RTS2_SHIFT 5/* bottom 3 bits of radix tree size */ #define RTS2_MASK (7UL << RTS2_SHIFT)#define RPDS_MASK 0x1f/* root page dir. size field */ +/* Hash */ +#define PATB_HTABORG 0x0ffffffffffc0000UL/* hpt base */ +#define PATB_PS 0xe0/* page size */ +#define PATB_PS_L 0x80 +#define PATB_PS_L_SHIFT 7 +#define PATB_PS_LP 0x60 +#define PATB_PS_LP_SHIFT 5 +#define PATB_HTABSIZE 0x1f/* hpt size */ /* Bits in patb1 field */ #define PATB_GR (1UL << 63)/* guest uses radix; must match HR */
arch/powerpc/include/asm/kvm_book3s_64.h +3 -1
@@ -35,7 +35,9 @@ struct kvm_nested_guest {struct kvm *l1_host;/* L1 VM that owns this nested guest */ int l1_lpid;/* lpid L1 guest thinks this guest is */ int shadow_lpid;/* real lpid of this nested guest */ -pgd_t *shadow_pgtable;/* our page table for this guest */ +pgd_t *shadow_pgtable;/* page table for this guest if radix */ +struct kvm_hpt_info shadow_hpt;/* hpt for this guest if hash */ +u64 vrma_slb_v;/* vrma slb for this guest if hash */ u64 l1_gr_to_hr;/* L1's addr of part'n-scoped table */ u64 process_table;/* process table entry for this guest */ long refcnt;/* number of pointers to this struct */
arch/powerpc/kvm/book3s_hv_nested.c +221 -34
430 lines not on this page · raw message
Signature · 1 line
2.13.6
Re: [PATCH 13/23] KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup
Quoted 28 lines · Suraj Jitindar Singh
Add the infrastructure to
book3s_hv_nested.cto allow a nested hpt (hash page table) guest to be setup. As this patch doesn't add the capability of creating or removing mmu translations returnH_PARAMETERwhen an attempt to actually run a nested hpt guest is made.Add fields to the nested guest struct to store the hpt and the vrma slb entry.
Update
kvmhv_update_ptbl_cache()to determine when a nested guest is switching from radix to hpt or hpt to radix and perform the required setup. A page table (radix) or hpt (hash) must be allocated with any existing table being freed and the radix field in the nested guest struct being updated under themmu_lock(this means that when holding themmu_lockthe radix field can be tested and the existance of the correct type of page table guaranteed). Also remove all of the nest rmap entries which belong to this nested guest since a nested rmap entry is specific to whether the nested guest is hash or radix.When a nested guest is initially created or when the partition table entry is empty we assume a radix guest since it is much less expensive to allocate a radix page table compared to a hpt.
The hpt which is allocated in the hypervisor for the nested guest (called the shadow hpt) is identical in size to the one allocated in the guest hypervisor to ensure a 1-to-1 mapping between page table entries. This simplifies handling of the entries however this requirement could be relaxed in future if support was added.
Quoted 8 lines
Introduce a hash
nested_page_faultfunction to be envoked when the nested guest which experiences a page fault is hash, returns-EINVALfor now. Also return-EINVALwhen handling theH_TLB_INVALIDATEhcall. Also lacking support for the hypervisor paging out a guest page which has been mapped through to a nested guest. These 3 portions of functionality added in proceeding patches.Signed-off-by Suraj Jitindar Singh <[email protected]>
Small nit below...
Quoted 13 lines
arch/powerpc/kvm/book3s_hv_nested.c +13
-0+/* Caller must hold gp->tlb_lock */ +static int kvmhv_switch_to_radix_nested (struct kvm_nested_guest *gp) +{ +struct kvm *kvm = gp->l1_host; +pgd_t *pgtable; + +/* try to allocate a radix tree */ + pgtable =pgd_alloc (kvm->mm); +if (!pgtable) { +pr_err_ratelimited ("KVM: Couldn't alloc nested radix tree\n" ); +return -ENOMEM ; + } +
arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 642
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10
-0+/* mmu_lock protects shadow_hpt & radix in nested guest struct */ +spin_lock (&kvm->mmu_lock); +kvmppc_free_hpt (&gp->shadow_hpt); + gp->radix =1 ; + gp->shadow_pgtable = pgtable; +spin_unlock (&kvm->mmu_lock); + +/* remove all nested rmap entries and perform global invalidation */ +kvmhv_remove_all_nested_rmap_lpid (kvm, gp->l1_lpid); +kvmhv_flush_lpid (gp->shadow_lpid, gp->radix);
Shouldn't this flush be done using the old value of gp->radix, i.e. 0? Both because we want to flush the old translations for the guest, and because we haven't changed the partition table entry for the guest at this point, so it still says HPT.
Quoted 20 lines
arch/powerpc/kvm/book3s_hv_nested.c +20
-0+ +return 0 ; +} + +/* Caller must hold gp->tlb_lock */ +static int kvmhv_switch_to_hpt_nested (struct kvm_nested_guest *gp,int order) +{ +struct kvm *kvm = gp->l1_host; +struct kvm_hpt_info info; +int rc; + +/* try to allocate an hpt */ + rc =kvmppc_allocate_hpt (&info, order); +if (rc) { +pr_err_ratelimited ("KVM: Couldn't alloc nested hpt\n" ); +return rc; + } + +/* mmu_lock protects shadow_pgtable & radix in nested guest struct */ +spin_lock (&kvm->mmu_lock);
arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 672
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10
-0+kvmppc_free_pgtable_radix (kvm, gp->shadow_pgtable, gp->shadow_lpid); +pgd_free (kvm->mm, gp->shadow_pgtable); + gp->shadow_pgtable =NULL ; + gp->radix =0 ; + gp->shadow_hpt = info; +spin_unlock (&kvm->mmu_lock); + +/* remove all nested rmap entries and perform global invalidation */ +kvmhv_remove_all_nested_rmap_lpid (kvm, gp->l1_lpid); +kvmhv_flush_lpid (gp->shadow_lpid, gp->radix);
Similarly, shouldn't this be a radix flush?
arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 645
Quoted 3 lines
arch/powerpc/kvm/book3s_hv_nested.c +3
-0+ +return 0 ; +}
Paul.
PATCH 14/23 KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest
In reply to Suraj Jitindar SinghA version 2 of the H_ENTER_NESTED hcall was added with an argument to specify the slb entries which should be used to run the nested guest.
Add support for this version of the hcall structures to kvmhv_enter_nested_guest() and context switch the slb when the nested guest being run is a hpt (hash page table) guest.
--- arch/powerpc/kvm/book3s_hv_nested.c | 84 ++++++++++++++++++++++++++++++++++--- 1 file changed, 79 insertions(+), 5 deletions(-)
arch/powerpc/kvm/book3s_hv_nested.c +79 -5
@@ -104,6 +104,28 @@ static void byteswap_hv_regs(struct hv_guest_state *hr) hr->ppr =swab64 (hr->ppr); } +static void byteswap_guest_slb (struct guest_slb *slbp) +{ +int i; + +for (i =0 ; i <64 ; i++) { + slbp->slb[i].esid =swab64 (slbp->slb[i].esid); + slbp->slb[i].vsid =swab64 (slbp->slb[i].vsid); + slbp->slb[i].orige =swab64 (slbp->slb[i].orige); + slbp->slb[i].origv =swab64 (slbp->slb[i].origv); + slbp->slb[i].valid =swab32 (slbp->slb[i].valid); + slbp->slb[i].Ks =swab32 (slbp->slb[i].Ks); + slbp->slb[i].Kp =swab32 (slbp->slb[i].Kp); + slbp->slb[i].nx =swab32 (slbp->slb[i].nx); + slbp->slb[i].large =swab32 (slbp->slb[i].large); + slbp->slb[i].tb =swab32 (slbp->slb[i].tb); + slbp->slb[i].class =swab32 (slbp->slb[i].class); +/* base_page_size is u8 thus no need to byteswap */ + } + slbp->slb_max =swab64 (slbp->slb_max); + slbp->slb_nr =swab64 (slbp->slb_nr); +} +static void save_hv_return_state (struct kvm_vcpu *vcpu,int trap,struct hv_guest_state *hr) {
@@ -238,12 +260,13 @@ static void kvmhv_nested_mmio_needed(struct kvm_vcpu *vcpu, u64 regs_ptr)long kvmhv_enter_nested_guest (struct kvm_vcpu *vcpu) { -long int err, r; +long int err, r, ret =H_SUCCESS ;struct kvm_nested_guest *l2;struct pt_regs l2_regs, saved_l1_regs;struct hv_guest_state l2_hv, saved_l1_hv; +struct guest_slb *l2_slb =NULL , *saved_l1_slb =NULL ;struct kvmppc_vcore *vc = vcpu->arch.vcore; -u64 hv_ptr, regs_ptr; +u64 hv_ptr, regs_ptr, slb_ptr =0UL ;u64 hdec_exp;s64 delta_purr, delta_spurr, delta_ic, delta_vtb;u64 mask;
@@ -261,7 +284,9 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)return H_PARAMETER ;if (kvmppc_need_byteswap (vcpu))byteswap_hv_regs (&l2_hv); -if (l2_hv.version !=1 ) +/* Do we support the guest version of the argument structures */ +if ((l2_hv.version >HV_GUEST_STATE_MAX_VERSION ) || + (l2_hv.version <HV_GUEST_STATE_MIN_VERSION ))return H_P2 ; regs_ptr =kvmppc_get_gpr (vcpu,5 );
@@ -296,6 +321,9 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)return H_PARAMETER ; }else {return H_PARAMETER ; +/* must be at least V2 to support hpt guest */ +if (l2_hv.version <2 ) +return H_PARAMETER ;/* hpt doesn't support gtse or uprt and required vpm */ if ((l2_hv.lpcr & LPCR_HR) || (l2_hv.lpcr & LPCR_GTSE) || (l2_hv.lpcr & LPCR_UPRT) ||
@@ -307,6 +335,26 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.regs.msr = vcpu->arch.shregs.msr; saved_l1_regs = vcpu->arch.regs;kvmhv_save_hv_regs (vcpu, &saved_l1_hv); +/* if running hpt then context switch the slb in the vcpu struct */ +if (!radix) { + slb_ptr =kvmppc_get_gpr (vcpu,6 ); + l2_slb =kzalloc (sizeof (*l2_slb),GFP_KERNEL ); + saved_l1_slb =kzalloc (sizeof (*saved_l1_slb),GFP_KERNEL ); + +if ((!l2_slb) || (!saved_l1_slb)) { + ret =H_HARDWARE ; + goto out_free; + } + err =kvm_vcpu_read_guest (vcpu, slb_ptr, l2_slb, +sizeof (struct guest_slb )); +if (err) { + ret =H_PARAMETER ; + goto out_free; + } +if (kvmppc_need_byteswap (vcpu)) +byteswap_guest_slb (l2_slb); +kvmhv_save_guest_slb (vcpu, saved_l1_slb); + }/* convert TB values/offsets to host (L0) values */ hdec_exp = l2_hv.hdec_expiry - vc->tb_offset;
@@ -323,6 +371,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) lpcr = (vc->lpcr & ~mask) | (l2_hv.lpcr & mask);sanitise_hv_regs (vcpu, &l2_hv);restore_hv_regs (vcpu, &l2_hv); +if (!radix) +kvmhv_restore_guest_slb (vcpu, l2_slb); vcpu->arch.ret =RESUME_GUEST ; vcpu->arch.trap =0 ;
@@ -332,8 +382,11 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) r =RESUME_HOST ;break ; } - r =kvmhv_run_single_vcpu (vcpu->arch.kvm_run, vcpu, hdec_exp, - lpcr); +if (radix) + r =kvmhv_run_single_vcpu (vcpu->arch.kvm_run, vcpu, + hdec_exp, lpcr); +else + r =RESUME_HOST ;/* XXX TODO hpt entry path */ }while (is_kvmppc_resume_guest (r));/* save L2 state for return */
@@ -344,6 +397,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) delta_ic = vcpu->arch.ic - l2_hv.ic; delta_vtb = vc->vtb - l2_hv.vtb;save_hv_return_state (vcpu, vcpu->arch.trap, &l2_hv); +if (!radix) +kvmhv_save_guest_slb (vcpu, l2_slb);/* restore L1 state */ vcpu->arch.nested =NULL ;
@@ -354,6 +409,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.shregs.msr |=MSR_TS_S ; vc->tb_offset = saved_l1_hv.tb_offset;restore_hv_regs (vcpu, &saved_l1_hv); +if (!radix) +kvmhv_restore_guest_slb (vcpu, saved_l1_slb); vcpu->arch.purr += delta_purr; vcpu->arch.spurr += delta_spurr; vcpu->arch.ic += delta_ic;
@@ -363,9 +420,21 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)/* copy l2_hv_state and regs back to guest */ if (kvmppc_need_byteswap (vcpu)) { +if (!radix) +byteswap_guest_slb (l2_slb);byteswap_hv_regs (&l2_hv);byteswap_pt_regs (&l2_regs); } +if (!radix) { + err =kvm_vcpu_write_guest (vcpu, slb_ptr, l2_slb, +sizeof (struct guest_slb )); +if (err) { + ret =H_AUTHORITY ; + goto out_free; + } +kfree (l2_slb); +kfree (saved_l1_slb); + } err =kvm_vcpu_write_guest (vcpu, hv_ptr, &l2_hv,sizeof (struct hv_guest_state ));if (err)
@@ -384,6 +453,11 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) }return vcpu->arch.trap; + +out_free: +kfree (l2_slb); +kfree (saved_l1_slb); +return ret; }long kvmhv_nested_init (void )
Signature · 1 line
2.13.6
Re: [PATCH 14/23] KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest
Quoted 8 lines · Suraj Jitindar Singh
A version 2 of the
H_ENTER_NESTEDhcall was added with an argument to specify the slb entries which should be used to run the nested guest.Add support for this version of the hcall structures to
kvmhv_enter_nested_guest()and context switch the slb when the nested guest being run is a hpt (hash page table) guest.Signed-off-by Suraj Jitindar Singh <[email protected]>
Question below...
Quoted 13 lines
arch/powerpc/kvm/book3s_hv_nested.c +9
-0@@ -307,6 +335,26 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.regs.msr = vcpu->arch.shregs.msr; saved_l1_regs = vcpu->arch.regs;kvmhv_save_hv_regs (vcpu, &saved_l1_hv); +/* if running hpt then context switch the slb in the vcpu struct */ +if (!radix) { + slb_ptr =kvmppc_get_gpr (vcpu,6 ); + l2_slb =kzalloc (sizeof (*l2_slb),GFP_KERNEL ); + saved_l1_slb =kzalloc (sizeof (*saved_l1_slb),GFP_KERNEL ); + +if ((!l2_slb) || (!saved_l1_slb)) { + ret =H_HARDWARE ; + goto out_free;
arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) · line 356
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10
-0+ } + err =kvm_vcpu_read_guest (vcpu, slb_ptr, l2_slb, +sizeof (struct guest_slb )); +if (err) { + ret =H_PARAMETER ; + goto out_free; + } +if (kvmppc_need_byteswap (vcpu)) +byteswap_guest_slb (l2_slb); +kvmhv_save_guest_slb (vcpu, saved_l1_slb);
Why are we bothering to save the SLB state of the L1 guest, which has to be a radix guest? Won't the L1 SLB state always just have 0 entries?
arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) · line 413
Quoted 6 lines
arch/powerpc/kvm/book3s_hv_nested.c +2
-0@@ -354,6 +409,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.shregs.msr |=MSR_TS_S ; vc->tb_offset = saved_l1_hv.tb_offset;restore_hv_regs (vcpu, &saved_l1_hv); +if (!radix) +kvmhv_restore_guest_slb (vcpu, saved_l1_slb);
Likewise here can't we just set vcpu->arch.slb_max and vcpu->arch.slb_nr to zero?
Paul.
PATCH 15/23 KVM: PPC: Book3S HV: Store lpcr and hdec_exp in the vcpu struct
In reply to Suraj Jitindar SinghWhen running a single vcpu with kvmhv_run_single_vcpu() the lpcr and hypervisor decrementer expiry are passed as function arguments. When running a vcore with kvmppc_run_vcpu() the lpcr is taken from the vcore and there is no need to consider the hypervisor decrementer expiry as it only applies when running a nested guest.
These fields will need to be accessed in the guest entry path in book3s_hv_rmhandlers.S when running a nested hpt (hash page table) guest. To allow for this store the lpcr and hdec_exp in the vcpu struct.
No functional change.
--- arch/powerpc/include/asm/kvm_book3s.h | 3 +-- arch/powerpc/include/asm/kvm_host.h | 2 ++ arch/powerpc/kvm/book3s_hv.c | 40 +++++++++++++++++------------------ arch/powerpc/kvm/book3s_hv_nested.c | 10 ++++----- 4 files changed, 27 insertions(+), 28 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -2
@@ -314,8 +314,7 @@ void kvmhv_set_ptbl_entry(unsigned int lpid, u64 dw0, u64 dw1);void kvmhv_release_all_nested (struct kvm *kvm);long kvmhv_enter_nested_guest (struct kvm_vcpu *vcpu);long kvmhv_do_nested_tlbie (struct kvm_vcpu *vcpu); -int kvmhv_run_single_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu, -u64 time_limit,unsigned long lpcr); +int kvmhv_run_single_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu);void kvmhv_save_hv_regs (struct kvm_vcpu *vcpu,struct hv_guest_state *hr);void kvmhv_save_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp);void kvmhv_restore_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp);
arch/powerpc/include/asm/kvm_host.h +2 -0
@@ -793,10 +793,12 @@ struct kvm_vcpu_arch {u32 online; +unsigned long lpcr;/* For support of nested guests */ struct kvm_nested_guest *nested;u32 nested_vcpu_id;gpa_t nested_io_gpr; +u64 hdec_exp;#endif #ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING
arch/powerpc/kvm/book3s_hv.c +19 -21
@@ -3429,8 +3429,7 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc)/* * Handle making the H_ENTER_NESTED hcall if we're pseries. */ -static int kvmhv_pseries_enter_guest (struct kvm_vcpu *vcpu,u64 time_limit, -unsigned long lpcr) +static int kvmhv_pseries_enter_guest (struct kvm_vcpu *vcpu,u64 time_limit) {/* call our hypervisor to load up HV regs and go */ struct hv_guest_state hvregs;
@@ -3454,7 +3453,7 @@ static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit, host_psscr =mfspr (SPRN_PSSCR_PR );mtspr (SPRN_PSSCR_PR , vcpu->arch.psscr);kvmhv_save_hv_regs (vcpu, &hvregs); - hvregs.lpcr = lpcr; + hvregs.lpcr = vcpu->arch.lpcr; vcpu->arch.regs.msr = vcpu->arch.shregs.msr;if (vcpu->arch.nested) { hvregs.lpid = vcpu->arch.nested->shadow_lpid;
@@ -3536,8 +3535,7 @@ static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit,* CPU_FTR_HVMODE is set. This is only used for radix guests, however that * radix guest may be a direct guest of this hypervisor or a nested guest. */ -static int kvmhv_load_hv_regs_and_go (struct kvm_vcpu *vcpu,u64 time_limit, -unsigned long lpcr) +static int kvmhv_load_hv_regs_and_go (struct kvm_vcpu *vcpu,u64 time_limit) {struct kvmppc_vcore *vc = vcpu->arch.vcore;s64 hdec;
@@ -3594,7 +3592,7 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,mtspr (SPRN_AMOR , ~0UL ); -mtspr (SPRN_LPCR , lpcr); +mtspr (SPRN_LPCR , vcpu->arch.lpcr);isync ();kvmppc_xive_push_vcpu (vcpu);
@@ -3666,8 +3664,7 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,* Virtual-mode guest entry for POWER9 and later when the host and * guest are both using the radix MMU. The LPIDR has already been set. */ -int kvmhv_p9_guest_entry (struct kvm_vcpu *vcpu,u64 time_limit, -unsigned long lpcr) +int kvmhv_p9_guest_entry (struct kvm_vcpu *vcpu) {struct kvmppc_vcore *vc = vcpu->arch.vcore;unsigned long host_dscr =mfspr (SPRN_DSCR );
@@ -3675,7 +3672,7 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,unsigned long host_iamr =mfspr (SPRN_IAMR );unsigned long host_amr =mfspr (SPRN_AMR );s64 dec; -u64 tb; +u64 tb, time_limit;int trap, save_pmu; dec =mfspr (SPRN_DEC );
@@ -3683,8 +3680,8 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,if (dec <512 )return BOOK3S_INTERRUPT_HV_DECREMENTER ; local_paca->kvm_hstate.dec_expires = dec + tb; -if (local_paca->kvm_hstate.dec_expires < time_limit) - time_limit = local_paca->kvm_hstate.dec_expires; + time_limit =min_t (u64, local_paca->kvm_hstate.dec_expires, + vcpu->arch.hdec_exp); vcpu->arch.ceded =0 ;
@@ -3736,15 +3733,16 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,mtspr (SPRN_DEC , vcpu->arch.dec_expires -mftb ());if (kvmhv_on_pseries ()) { - trap =kvmhv_pseries_enter_guest (vcpu, time_limit, lpcr); + trap =kvmhv_pseries_enter_guest (vcpu, time_limit); }else { - trap =kvmhv_load_hv_regs_and_go (vcpu, time_limit, lpcr); + trap =kvmhv_load_hv_regs_and_go (vcpu, time_limit); }if (kvm_is_radix (vcpu->kvm)) vcpu->arch.slb_max =0 ; dec =mfspr (SPRN_DEC ); -if (!(lpcr &LPCR_LD ))/* Sign extend if not using large decrementer */ +/* Sign extend if not using large decrementer */ +if (!(vcpu->arch.lpcr &LPCR_LD )) dec = (s32 ) dec; tb =mftb (); vcpu->arch.dec_expires = dec + tb;
@@ -4145,9 +4143,7 @@ static int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)return vcpu->arch.ret; } -int kvmhv_run_single_vcpu (struct kvm_run *kvm_run, -struct kvm_vcpu *vcpu,u64 time_limit, -unsigned long lpcr) +int kvmhv_run_single_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu) {int trap, r, pcpu;int srcu_idx, lpid;
@@ -4206,7 +4202,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, }if (test_bit (BOOK3S_IRQPRIO_EXTERNAL , &vcpu->arch.pending_exceptions)) - lpcr |=LPCR_MER ; + vcpu->arch.lpcr |=LPCR_MER ; }else if (vcpu->arch.pending_exceptions || vcpu->arch.doorbell_request ||xive_interrupt_pending (vcpu)) {
@@ -4242,7 +4238,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run,/* Tell lockdep that we're about to enable interrupts */ trace_hardirqs_on (); - trap =kvmhv_p9_guest_entry (vcpu, time_limit, lpcr); + trap =kvmhv_p9_guest_entry (vcpu); vcpu->arch.trap = trap;trace_hardirqs_off ();
@@ -4399,6 +4395,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu) vcpu->arch.state =KVMPPC_VCPU_BUSY_IN_HOST ; do { + /* update vcpu->arch.lpcr in case a previous loop modified it */ + vcpu->arch.lpcr = vcpu->arch.vcore->lpcr; + vcpu->arch.hdec_exp = ~(u64)0; /* * The early POWER9 chips that can't mix radix and HPT threads * on the same core also need the workaround for the problem
@@ -4412,8 +4411,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)if (kvmhv_on_pseries () || (kvm->arch.threads_indep &&kvm_is_radix (kvm) && !no_mixing_hpt_and_radix)) - r =kvmhv_run_single_vcpu (run, vcpu, ~(u64 )0 , - vcpu->arch.vcore->lpcr); + r =kvmhv_run_single_vcpu (run, vcpu);else r =kvmppc_run_vcpu (run, vcpu);
arch/powerpc/kvm/book3s_hv_nested.c +5 -5
@@ -267,7 +267,6 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)struct guest_slb *l2_slb =NULL , *saved_l1_slb =NULL ;struct kvmppc_vcore *vc = vcpu->arch.vcore;u64 hv_ptr, regs_ptr, slb_ptr =0UL ; -u64 hdec_exp;s64 delta_purr, delta_spurr, delta_ic, delta_vtb;u64 mask;unsigned long lpcr;
@@ -357,7 +356,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) }/* convert TB values/offsets to host (L0) values */ - hdec_exp = l2_hv.hdec_expiry - vc->tb_offset; + vcpu->arch.hdec_exp = l2_hv.hdec_expiry - vc->tb_offset; vc->tb_offset += l2_hv.tb_offset;/* set L1 state to L2 state */
@@ -377,14 +376,15 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) vcpu->arch.ret =RESUME_GUEST ; vcpu->arch.trap =0 ;do { -if (mftb () >= hdec_exp) { +if (mftb () >= vcpu->arch.hdec_exp) { vcpu->arch.trap =BOOK3S_INTERRUPT_HV_DECREMENTER ; r =RESUME_HOST ;break ; } +/* update vcpu->arch.lpcr in case a previous loop modified it */ + vcpu->arch.lpcr = lpcr;if (radix) - r =kvmhv_run_single_vcpu (vcpu->arch.kvm_run, vcpu, - hdec_exp, lpcr); + r =kvmhv_run_single_vcpu (vcpu->arch.kvm_run, vcpu);else r =RESUME_HOST ;/* XXX TODO hpt entry path */ }while (is_kvmppc_resume_guest (r));
Signature · 1 line
2.13.6
PATCH 16/23 KVM: PPC: Book3S HV: Nested: Make kvmppc_run_vcpu() entry path nested capable
In reply to Suraj Jitindar SinghIn order to run a hpt (hash page table) guest the kvm entry path used must enter real mode before loading up the guest mmu state. Currently the only path which does this is calling kvmppc_run_vcpu() which then uses the entry path in book3s_hv_rmhandlers.S and until now this path didn't accomodate running a nested guest.
Have the nested hpt guest entry path call kvmppc_run_vcpu() and modify the entry path in book3s_hv_rmhandlers.S to be able to run a nested guest.
For the entry path this means loading the smaller of the guest hypervisor decrementer and the host decrementer into the host decrementer since we want control back when either expires. Additionally the correct LPID and LPCR must be loaded, and the guest slb entries must be restored. When checking if an interrupt can be injected to the guest in kvmppc_guest_entry_inject_int() we return -1 if entering a nested guest while there is something pending for the L1 guest to indicate that the nested guest shouldn't be entered and control should be passed back to the L1 guest.
On the exit path we must save the guest slb entries to be returned to the L1 guest hypervisor. Additionally the correct vrma_slb_v entry must be loaded for kvmppc_hpte_hv_fault() if the guest was in real mode. The correct hpt must be used in kvmppc_hpte_hv_fault(). And the correct handle_exit function must be called depending on whether a nested guest was being run or not.
--- arch/powerpc/include/asm/kvm_asm.h | 5 ++ arch/powerpc/include/asm/kvm_book3s.h | 3 +- arch/powerpc/include/asm/kvm_ppc.h | 2 +- arch/powerpc/kernel/asm-offsets.c | 5 ++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_hv.c | 55 +++++++------- arch/powerpc/kvm/book3s_hv_builtin.c | 33 ++++++--- arch/powerpc/kvm/book3s_hv_interrupts.S | 25 ++++++- arch/powerpc/kvm/book3s_hv_nested.c | 2 +- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 80 +++++++++++++++------ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 124 ++++++++++++++++++++++---------- arch/powerpc/kvm/book3s_xive.h | 15 ++++ 12 files changed, 252 insertions(+), 99 deletions(-)
arch/powerpc/include/asm/kvm_asm.h +5 -0
@@ -104,6 +104,11 @@* completely in the guest. */ #define BOOK3S_INTERRUPT_HV_RM_HARD 0x5555 +/* +* Special trap used when running a nested guest to communicate that control +* should be passed back to the L1 guest. e.g. Because interrupt pending +*/ +#define BOOK3S_INTERRUPT_HV_NEST_EXIT 0x5556#define BOOK3S_IRQPRIO_SYSTEM_RESET 0#define BOOK3S_IRQPRIO_DATA_SEGMENT 1
arch/powerpc/include/asm/kvm_book3s.h +2 -1
@@ -158,7 +158,7 @@ extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu);extern int kvmppc_book3s_hv_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned long addr,unsigned long status); -extern long kvmppc_hv_find_lock_hpte (struct kvm *kvm,gva_t eaddr, +extern long kvmppc_hv_find_lock_hpte (struct kvm_hpt_info *hpt,gva_t eaddr,unsigned long slb_v,unsigned long valid);extern int kvmppc_hv_emulate_mmio (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned long gpa,gva_t ea,int is_store);
@@ -315,6 +315,7 @@ void kvmhv_release_all_nested(struct kvm *kvm);long kvmhv_enter_nested_guest (struct kvm_vcpu *vcpu);long kvmhv_do_nested_tlbie (struct kvm_vcpu *vcpu);int kvmhv_run_single_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu); +int kvmppc_run_vcpu (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu);void kvmhv_save_hv_regs (struct kvm_vcpu *vcpu,struct hv_guest_state *hr);void kvmhv_save_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp);void kvmhv_restore_guest_slb (struct kvm_vcpu *vcpu,struct guest_slb *slbp);
arch/powerpc/include/asm/kvm_ppc.h +1 -1
@@ -697,7 +697,7 @@ int kvmppc_rm_h_ipi(struct kvm_vcpu *vcpu, unsigned long server, unsigned long mfrr);int kvmppc_rm_h_cppr (struct kvm_vcpu *vcpu,unsigned long cppr);int kvmppc_rm_h_eoi (struct kvm_vcpu *vcpu,unsigned long xirr); -void kvmppc_guest_entry_inject_int (struct kvm_vcpu *vcpu); +int kvmppc_guest_entry_inject_int (struct kvm_vcpu *vcpu);/* * Host-side operations we want to set up while running in real
arch/powerpc/kernel/asm-offsets.c +5 -0
@@ -511,9 +511,14 @@ int main(void)OFFSET (VCPU_VPA , kvm_vcpu, arch.vpa.pinned_addr);OFFSET (VCPU_VPA_DIRTY , kvm_vcpu, arch.vpa.dirty);OFFSET (VCPU_HEIR , kvm_vcpu, arch.emul_inst); +OFFSET (VCPU_LPCR , kvm_vcpu, arch.lpcr);OFFSET (VCPU_NESTED , kvm_vcpu, arch.nested); +OFFSET (VCPU_NESTED_LPID , kvm_nested_guest, shadow_lpid); +OFFSET (VCPU_NESTED_RADIX , kvm_nested_guest, radix); +OFFSET (VCPU_NESTED_VRMA_SLB_V , kvm_nested_guest, vrma_slb_v);OFFSET (VCPU_CPU , kvm_vcpu, cpu);OFFSET (VCPU_THREAD_CPU , kvm_vcpu, arch.thread_cpu); +OFFSET (VCPU_HDEC_EXP , kvm_vcpu, arch.hdec_exp);#endif #ifdef CONFIG_PPC_BOOK3S OFFSET (VCPU_PURR , kvm_vcpu, arch.purr);
arch/powerpc/kvm/book3s_64_mmu_hv.c +1 -1
@@ -361,7 +361,7 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,preempt_disable ();/* Find the HPTE in the hash table */ - index =kvmppc_hv_find_lock_hpte (kvm, eaddr, slb_v, + index =kvmppc_hv_find_lock_hpte (&kvm->arch.hpt, eaddr, slb_v,HPTE_V_VALID |HPTE_V_ABSENT );if (index <0 ) {preempt_enable ();
arch/powerpc/kvm/book3s_hv.c +29 -26
109 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_builtin.c +24 -9
@@ -759,26 +759,40 @@ void kvmhv_p9_restore_lpcr(struct kvm_split_mode *sip)* Is there a PRIV_DOORBELL pending for the guest (on POWER9)? * Can we inject a Decrementer or a External interrupt? */ -void kvmppc_guest_entry_inject_int (struct kvm_vcpu *vcpu) +int kvmppc_guest_entry_inject_int (struct kvm_vcpu *vcpu) {int ext;unsigned long vec =0 ; -unsigned long lpcr; +unsigned long old_lpcr, lpcr; + +#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE +/* +* Don't enter a nested guest if there is something pending for this +* vcpu for the l1 guest. Return -1 to indicate this. +*/ +if (vcpu->arch.nested && (vcpu->arch.pending_exceptions || + vcpu->arch.prodded || + vcpu->arch.doorbell_request || +xive_interrupt_pending (vcpu))) +return -1 ; +#endif /* Insert EXTERNAL bit into LPCR at the MER bit position */ ext = (vcpu->arch.pending_exceptions >>BOOK3S_IRQPRIO_EXTERNAL ) &1 ; - lpcr =mfspr (SPRN_LPCR ); - lpcr |= ext <<LPCR_MER_SH ; -mtspr (SPRN_LPCR , lpcr); -isync (); + old_lpcr =mfspr (SPRN_LPCR ); + lpcr = old_lpcr | (ext <<LPCR_MER_SH ); +if (lpcr != old_lpcr) { +mtspr (SPRN_LPCR , lpcr); +isync (); + }if (vcpu->arch.shregs.msr &MSR_EE ) {if (ext) { vec =BOOK3S_INTERRUPT_EXTERNAL ; }else { -long int dec =mfspr (SPRN_DEC ); +s64 dec =mfspr (SPRN_DEC );if (!(lpcr &LPCR_LD )) - dec = (int ) dec; + dec = (s32 ) dec;if (dec <0 ) vec =BOOK3S_INTERRUPT_DECREMENTER ; }
@@ -795,12 +809,13 @@ void kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu) vcpu->arch.shregs.msr = msr; } -if (vcpu->arch.doorbell_request) { +if (cpu_has_feature (CPU_FTR_ARCH_300 ) && vcpu->arch.doorbell_request) {mtspr (SPRN_DPDES ,1 ); vcpu->arch.vcore->dpdes =1 ;smp_wmb (); vcpu->arch.doorbell_request =0 ; } +return 0 ; }static void flush_guest_tlb (struct kvm *kvm)
arch/powerpc/kvm/book3s_hv_interrupts.S +22 -3
@@ -58,10 +58,20 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) /* * Put whatever is in the decrementer into the * hypervisor decrementer. + * If running a nested guest then put the lower of the host decrementer + * and the guest hypervisor decrementer into the hypervisor decrementer + * since we want control back from the nested guest when either expires. */ BEGIN_FTR_SECTION ld r5, HSTATE_KVM_VCORE(r13) - ld r6, VCORE_KVM(r5) + ld r6, HSTATE_KVM_VCPU(r13) + cmpdi cr1, r6, 0 /* Do we actually have a vcpu? */ + beq cr1, 33f + ld r7, VCPU_NESTED(r6) + cmpdi cr1, r7, 0 /* Do we have a nested guest? */ + beq cr1, 33f + ld r10, VCPU_HDEC_EXP(r6) /* If so load the hdec expiry */ +33: ld r6, VCORE_KVM(r5) ld r9, KVM_HOST_LPCR(r6) andis. r9, r9, LPCR_LD@h END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
@@ -72,8 +82,17 @@ BEGIN_FTR_SECTION bne 32f END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) extsw r8,r8 -32: mtspr SPRN_HDEC,r8 - add r8,r8,r7 +BEGIN_FTR_SECTION +32: beq cr1, 34f /* did we load hdec expiry above? */ + subf r10, r7, r10 /* r10 = guest_hdec = hdec_exp - tb */ + cmpd r8, r10 /* host decrementer < hdec? */ + ble 34f + mtspr SPRN_HDEC, r10 /* put guest_hdec into the hv decr */ + b 35f +34: +END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) + mtspr SPRN_HDEC,r8 /* put host decr into hv decr */ +35: add r8,r8,r7 std r8,HSTATE_DECEXP(r13) /* Jump to partition switch code */
arch/powerpc/kvm/book3s_hv_nested.c +1 -1
@@ -386,7 +386,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)if (radix) r =kvmhv_run_single_vcpu (vcpu->arch.kvm_run, vcpu);else - r =RESUME_HOST ;/* XXX TODO hpt entry path */ + r =kvmppc_run_vcpu (vcpu->arch.kvm_run, vcpu); }while (is_kvmppc_resume_guest (r));/* save L2 state for return */
arch/powerpc/kvm/book3s_hv_rm_mmu.c +59 -21
162 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rmhandlers.S +88 -36
227 lines not on this page · raw message
arch/powerpc/kvm/book3s_xive.h +15 -0
@@ -283,5 +283,20 @@ int kvmppc_xive_attach_escalation(struct kvm_vcpu *vcpu, u8 prio, bool single_escalation);struct kvmppc_xive *kvmppc_xive_get_device (struct kvm *kvm,u32 type); +static inline bool xive_interrupt_pending (struct kvm_vcpu *vcpu) +{ +if (!xics_on_xive ()) +return false; +return vcpu->arch.irq_pending || vcpu->arch.xive_saved_state.pipr < + vcpu->arch.xive_saved_state.cppr; +} + +#else /* !CONFIG_KVM_XICS */ + +static inline bool xive_interrupt_pending (struct kvm_vcpu *vcpu) +{ +return false; +} +#endif /* CONFIG_KVM_XICS */ #endif /* _KVM_PPC_BOOK3S_XICS_H */
Signature · 1 line
2.13.6
PATCH 17/23 KVM: PPC: Book3S HV: Nested: Rename kvmhv_xlate_addr_nested_radix
In reply to Suraj Jitindar SinghRename kvmhv_xlate_addr_nested() to kvmhv_xlate_addr_nested_radix() for clarity since there will need to be a hash function added.
Additionally if we can't permit an access since a nested guest is writing to a page which has the KVM_MEM_READONLY bit set in the memslot then give the nested guest an interrupt, there's not much else which can be done in this case. For now this is the only place where an interrupt is injected directly to the nested guest by the L0 hypervisor without involving the L1 guest hypervisor.
--- arch/powerpc/kvm/book3s.c | 1 + arch/powerpc/kvm/book3s_hv_nested.c | 81 +++++++++++++++++++++++++++---------- 2 files changed, 61 insertions(+), 21 deletions(-)
arch/powerpc/kvm/book3s.c +1 -0
@@ -140,6 +140,7 @@ void kvmppc_inject_interrupt(struct kvm_vcpu *vcpu, int vec, u64 flags)kvmppc_set_pc (vcpu,kvmppc_interrupt_offset (vcpu) + vec); vcpu->arch.mmu.reset_msr (vcpu); } +EXPORT_SYMBOL_GPL (kvmppc_inject_interrupt);static int kvmppc_book3s_vec2irqprio (unsigned int vec) {
arch/powerpc/kvm/book3s_hv_nested.c +60 -21
158 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 18/23 KVM: PPC: Book3S HV: Separate out hashing from kvmppc_hv_find_lock_hpte()
In reply to Suraj Jitindar SinghSeparate out the hashing function from kvmppc_hv_find_lock_hpte() as kvmppc_hv_get_hash_value() to allow this function to be reused and prevent code duplication.
No functional change.
--- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 60 ++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 24 deletions(-)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +36 -24
@@ -1162,6 +1162,40 @@ static struct mmio_hpte_cache_entry *return &vcpu->arch.mmio_cache.entry[index]; } +/* +* Given an effective address and a slb entry, work out the hash and the +* virtual page number +*/ +unsigned long kvmppc_hv_get_hash_value (struct kvm_hpt_info *hpt,gva_t eaddr, +unsigned long slb_v,unsigned long *avpn, +unsigned int *pshift_p) +{ +unsigned long hash, somask, vsid; +unsigned int pshift =12 ; + +if (slb_v &SLB_VSID_L ) + pshift = slb_base_page_shift[(slb_v &SLB_VSID_LP ) >>4 ]; +if (slb_v &SLB_VSID_B_1T ) { + somask = (1UL <<40 ) -1 ; + vsid = (slb_v & ~SLB_VSID_B ) >>SLB_VSID_SHIFT_1T ; + vsid ^= vsid <<25 ; + }else { + somask = (1UL <<28 ) -1 ; + vsid = (slb_v & ~SLB_VSID_B ) >>SLB_VSID_SHIFT ; + } + hash = (vsid ^ ((eaddr & somask) >> pshift)) &kvmppc_hpt_mask (hpt); + *avpn = slb_v & ~(somask >>16 );/* also includes B */ + *avpn |= (eaddr & somask) >>16 ; + +if (pshift >=24 ) + *avpn &= ~((1UL << (pshift -16 )) -1 ); +else + *avpn &= ~0x7fUL ; + *pshift_p = pshift; + +return hash; +} +/* When called from virtmode, this func should be protected by * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK * can trigger deadlock issue.
@@ -1171,39 +1205,17 @@ long kvmppc_hv_find_lock_hpte(struct kvm_hpt_info *hpt, gva_t eaddr, {unsigned int i;unsigned int pshift; -unsigned long somask; -unsigned long vsid, hash; -unsigned long avpn;__be64 *hpte; -unsigned long mask, val; +unsigned long hash, mask, val;unsigned long v, r, orig_v;/* Get page shift, work out hash and AVPN etc. */ mask =SLB_VSID_B |HPTE_V_AVPN |HPTE_V_SECONDARY ; - val =0 ; - pshift =12 ; + hash =kvmppc_hv_get_hash_value (hpt, eaddr, slb_v, &val, &pshift);if (slb_v &SLB_VSID_L ) { mask |=HPTE_V_LARGE ; val |=HPTE_V_LARGE ; - pshift = slb_base_page_shift[(slb_v &SLB_VSID_LP ) >>4 ]; - } -if (slb_v &SLB_VSID_B_1T ) { - somask = (1UL <<40 ) -1 ; - vsid = (slb_v & ~SLB_VSID_B ) >>SLB_VSID_SHIFT_1T ; - vsid ^= vsid <<25 ; - }else { - somask = (1UL <<28 ) -1 ; - vsid = (slb_v & ~SLB_VSID_B ) >>SLB_VSID_SHIFT ; } - hash = (vsid ^ ((eaddr & somask) >> pshift)) &kvmppc_hpt_mask (hpt); - avpn = slb_v & ~(somask >>16 );/* also includes B */ - avpn |= (eaddr & somask) >>16 ; - -if (pshift >=24 ) - avpn &= ~((1UL << (pshift -16 )) -1 ); -else - avpn &= ~0x7fUL ; - val |= avpn;for (;;) { hpte = (__be64 *)(hpt->virt + (hash <<7 ));
Signature · 1 line
2.13.6
PATCH 19/23 KVM: PPC: Book3S HV: Nested: Implement nested hpt mmu translation
In reply to Suraj Jitindar SinghImplement the insertion of nested hpt (hash page table) translations in to the shadow hpt. The shadow hpt is used to store ptes (page table entries) which provide the translation from nested guest virtual address to host real address. The translation from nested guest effective address to virtual address is provided by the slb which the nested guest can manage itself directly.
In order to construct this translation the hash page table in L1 memory must first be searched to provide a translation from nested guest virtual address to L1 guest real address. If no such entry is found then an interrupt is provided to the L1 guest hypervisor so that it can insert an entry.
The L1 guest real address is then translated to a host real address through the radix page tables for the L1 guest, with an entry created if one doesn't already exist.
The rc bits are then set for the pte in the L1 guest memory and the host radix pte for the L1 guest since these will be set by the hardware in the shadow pte automatically and so no interrupt will be provided to ensure these can be kept in sync. In fact the rc bits in the shadow hpt are set by software here. The c (change) bit is only set if the nested guest is writing, otherwise the page is mapped read only so that we can fault on a write to set the change bit and upgrade the write permissions.
The combination of the pte permissions is applied to the entry inserted and a nest rmap entry inserted.
Since we may come in with an existing entry when we just need to upgrade the write permissions on a page (in which case the index was found and stored in kvmppc_hpte_hv_fault()) we check for this case and just load the guest rpte entry from the rev map rather than having to look it up in L1 guest memory again.
This doesn't support the invalidation of translations by either the L0 or L1 hypervisors, this functionality is added in proceeding patches.
--- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 2 + arch/powerpc/include/asm/kvm_book3s.h | 4 + arch/powerpc/include/asm/kvm_book3s_64.h | 9 + arch/powerpc/kvm/book3s_hv_nested.c | 385 +++++++++++++++++++++++++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 6 +- 5 files changed, 404 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +2 -0
@@ -90,6 +90,8 @@#define HPTE_R_KEY_HI ASM_CONST(0x3000000000000000)#define HPTE_R_KEY_BIT0 ASM_CONST(0x2000000000000000)#define HPTE_R_KEY_BIT1 ASM_CONST(0x1000000000000000) +#define HPTE_R_B ASM_CONST(0x0c00000000000000) +#define HPTE_R_B_1T ASM_CONST(0x0400000000000000)#define HPTE_R_RPN_SHIFT 12#define HPTE_R_RPN ASM_CONST(0x0ffffffffffff000)#define HPTE_R_RPN_3_0 ASM_CONST(0x01fffffffffff000)
arch/powerpc/include/asm/kvm_book3s.h +4 -0
@@ -158,6 +158,10 @@ extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu);extern int kvmppc_book3s_hv_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned long addr,unsigned long status); +extern unsigned long kvmppc_hv_get_hash_value (struct kvm_hpt_info *hpt, +gva_t eaddr,unsigned long slb_v, +unsigned long *avpn, +unsigned int *pshift_p);extern long kvmppc_hv_find_lock_hpte (struct kvm_hpt_info *hpt,gva_t eaddr,unsigned long slb_v,unsigned long valid); extern int kvmppc_hv_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu,
arch/powerpc/include/asm/kvm_book3s_64.h +9 -0
@@ -369,6 +369,15 @@ static inline unsigned long hpte_make_readonly(unsigned long ptel)return ptel; } +static inline unsigned long hpte_make_writable (unsigned long ptel) +{ +if ((ptel &HPTE_R_PP0 ) || (ptel &HPTE_R_PP ) =PP_RWXX ) + ptel = (ptel & ~HPTE_R_PPP ) |PP_RWXX ; +else + ptel = (ptel & ~HPTE_R_PP ) |PP_RWRW ; +return ptel; +} +static inline bool hpte_cache_flags_ok (unsigned long hptel,bool is_ci) {unsigned int wimg = hptel &HPTE_R_WIMG ;
arch/powerpc/kvm/book3s_hv_nested.c +384 -1
402 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +5 -1
@@ -1195,6 +1195,7 @@ unsigned long kvmppc_hv_get_hash_value(struct kvm_hpt_info *hpt, gva_t eaddr,return hash; } +EXPORT_SYMBOL_GPL (kvmppc_hv_get_hash_value);/* When called from virtmode, this func should be protected by * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK
@@ -1291,8 +1292,11 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr, hpt = &kvm->arch.hpt; nested = vcpu->arch.nested; -if (nested) +if (nested) { hpt = &nested->shadow_hpt; +/* reuse fault_gpa field to save slb for nested pgfault funcn */ + vcpu->arch.fault_gpa = slb_v; + }/* For protection fault, expect to find a valid HPTE */ valid =HPTE_V_VALID ;
Signature · 1 line
2.13.6
PATCH 20/23 KVM: PPC: Book3S HV: Nested: Handle tlbie hcall for nested hpt guest
In reply to Suraj Jitindar SinghThe tlbie instruction is used to invalidate caching of translation information derived from the partition and process scoped page tables. This instruction is hypervisor privileged when specifying partition scoped translations to be invalidated. Thus this interface is paravirtualised with the H_TLB_INVALIDATE hcall which is used by a pseries hypervisor to perform partition scoped invalidations. This is then handled in the hypervisor in kvmhv_emulate_priv_tlbie().
When handling this hcall in the hypervisor it is necessary to invalidate caching of partition scoped translations which are in the shadow page table (radix) or shadow hpt (hash page table). This functionality already exists for radix, so implement it for a hash guest.
LPID wide invalidations are already handled and don't differ from the radix case. However the case where a single tlb entry corresponding to a virtual address is being invalidated needs to be implemented here. The information to find the entry is provided as an hcall argument and is expected to match the layout of rb specified for the tlbie instruction.
The rb value provides an abbreviated virtual address, base and actual page size and segment size to be used to search for the corresponding tlb entry. A hash is computed and this is used to find the pteg which needs to be searched. However since only 64 bits of the virtual address are supplied and depending on the segment and hpt size up to all 78 bits of the virtual address may be needed to compute the hash we have to mask out the bits which can't be determined and iterate through all possible combinations looking for a matching entry in all of the ptegs which are addressed by all of the possible hash values. Although there is in theory a 1-to-1 relationship between ptes in the shadow hpt and the hpt maintained by the guest hypervisor we need to invalidate all matches since they can't be differentiated.
When a matching entry is found it is invalidated if it was valid and the corresponding tlbie is issued by the hypervisor, irrespective the pte is then zeroed. An optimisation here would be to just make the pte absent and extend the rev map to store the host real doubleword since it is still valid.
Note: ric = 3 (cluster bombs) are not supported even though the ISA technically allows for them, their encoding is implemenatation dependant and linux doesn't use them.
--- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 11 + arch/powerpc/kvm/book3s_hv_nested.c | 293 ++++++++++++++++++++++---- 2 files changed, 258 insertions(+), 46 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +11 -0
@@ -129,6 +129,17 @@#define TLBIEL_INVAL_SET_MASK 0xfff000/* set number to inval. */ #define TLBIEL_INVAL_SET_SHIFT 12 +/* Fields in the rb registers for the tlbie instruction */ +#define TLBIE_RB_AVA_4K ASM_CONST(0xfffffffffffff000) +#define TLBIE_RB_AVA_L ASM_CONST(0xfffffffffff00000) +#define TLBIE_RB_LP ASM_CONST(0x00000000000ff000) +#define TLBIE_RB_B ASM_CONST(0x0000000000000300) +#define TLBIE_RB_B_1T ASM_CONST(0x0000000000000100) +#define TLBIE_RB_B_SHIFT 50/* Shift to match the pte location */ +#define TLBIE_RB_AVAL ASM_CONST(0x00000000000000fe) +#define TLBIE_RB_AVAL_SHIFT 12 +#define TLBIE_RB_L ASM_CONST(0x0000000000000001) +#define POWER7_TLB_SETS 128/* # sets in POWER7 TLB */ #define POWER8_TLB_SETS 512/* # sets in POWER8 TLB */ #define POWER9_TLB_SETS_HASH 256/* # sets in POWER9 TLB Hash mode */
arch/powerpc/kvm/book3s_hv_nested.c +247 -46
372 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 21/23 KVM: PPC: Book3S HV: Nested: Implement nest rmap invalidations for hpt guests
In reply to Suraj Jitindar SinghThe nest rmap is used to store a reverse mapping from the (L1) guest real address back to a pte in the shadow page table which maps it. This is used when the host is modifying a L1 guest pte (either invalidating it or modifying the rc bits) to make the necessary changes to the ptes in the shadow tables which map that L1 guest page. This is already implemented for a nested radix guest where the rmap entry stores the gpa (guest physical address) of the nested pte which can be used to traverse the shadow page table and find any matching ptes. Implement this nested rmap invalidation for nested hpt (hash page table) guests.
We reuse the nest rmap structure that already exists for radix nested guests for nested hpt guests. Instead of storing the gpa the hpt index of the pte is stored. This means that a pte in the shadow hpt can be uniquely identified by the nest rmap. As with the radix case we check that the same host page is being addressed to detect if this is a stale rmap entry, in which case we skip the invalidation.
When the host is invalidating a mapping for a L1 guest page use the nest rmap to find any shadow ptes in the shadow hpt which map that page and invalidate then, also invalidate any caching of the entry. A future optimisation would be to make the pte absend so that we can avoid having to lookup the guest rpte the next time an entry is faulted in.
When the host is clearing rc bits for a mapping for a L1 guest page use the nest rmap to find any shadow ptes in the shadow hpt which map that page and invalidate them as in the above case for invalidating a L1 guest page. It is not sufficient to clear the rc bits in the shadow pte since hardware can set them again without software intervention, so the mapping must be made invalid so that we will take a page fault and can ensure that the rc bits stay in sync in the page fault handler.
--- arch/powerpc/kvm/book3s_hv_nested.c | 114 +++++++++++++++++++++++++++--------- 1 file changed, 85 insertions(+), 29 deletions(-)
arch/powerpc/kvm/book3s_hv_nested.c +85 -29
@@ -25,6 +25,9 @@ static struct patb_entry *pseries_partition_tb;static void kvmhv_update_ptbl_cache (struct kvm_nested_guest *gp);static void kvmhv_remove_all_nested_rmap_lpid (struct kvm *kvm,int lpid);static void kvmhv_free_memslot_nest_rmap (struct kvm_memory_slot *free); +static void kvmhv_invalidate_shadow_pte_hash (struct kvm_hpt_info *hpt, +unsigned int lpid,__be64 *hptep, +unsigned long index);void kvmhv_save_hv_regs (struct kvm_vcpu *vcpu,struct hv_guest_state *hr) {
@@ -1135,30 +1138,57 @@ static void kvmhv_update_nest_rmap_rc(struct kvm *kvm, u64 n_rmap, unsigned long hpa, unsigned long mask) {struct kvm_nested_guest *gp; -unsigned long gpa; -unsigned int shift, lpid; -pte_t *ptep; +unsigned int lpid; - gpa =n_rmap_to_gpa (n_rmap); lpid =n_rmap_to_lpid (n_rmap);; gp =kvmhv_find_nested (kvm, lpid);if (!gp)return ; -/* Find the pte */ -if (gp->radix) - ptep =__find_linux_pte (gp->shadow_pgtable, gpa,NULL , &shift); -else - ptep =NULL ;/* XXX TODO */ /* -* If the pte is present and the pfn is still the same, update the pte. -* If the pfn has changed then this is a stale rmap entry, the nested -* gpa actually points somewhere else now, and there is nothing to do. -* XXX A future optimisation would be to remove the rmap entry here. +* Find the pte, and ensure it's valid and still points to the same +* host page. If the pfn has changed then this is a stale rmap entry, +* the shadow pte actually points somewhere else now, and there is +* nothing to do. Otherwise clear the requested rc bits from the shadow +* pte and perform the appropriate cache invalidation. +* XXX A future optimisation would be to remove the rmap entry */ -if (ptep &&pte_present (*ptep) && ((pte_val (*ptep) & mask) = hpa)) { -__radix_pte_update (ptep, clr, set); -kvmppc_radix_tlbie_page (kvm, gpa, shift, lpid); +if (gp->radix) { +unsigned long gpa =n_rmap_to_gpa (n_rmap); +unsigned int shift; +pte_t *ptep; + + ptep =__find_linux_pte (gp->shadow_pgtable, gpa,NULL , &shift); +/* pte present and still points to the same host page? */ +if (ptep &&pte_present (*ptep) && ((pte_val (*ptep) & mask) = + hpa)) { +__radix_pte_update (ptep, clr, set); +kvmppc_radix_tlbie_page (kvm, gpa, shift, lpid); + } + }else { +unsigned long v, r, index =n_rmap_to_index (n_rmap); +__be64 *hptep = (__be64 *)(gp->shadow_hpt.virt + (index <<4 )); + +preempt_disable (); +while (!try_lock_hpte (hptep,HPTE_V_HVLOCK )) +cpu_relax (); + v =be64_to_cpu (hptep[0 ]) & ~HPTE_V_HVLOCK ; + r =be64_to_cpu (hptep[1 ]); + +/* +* It's not enough to just clear the rc bits here since the +* hardware can just set them again transparently, we need to +* make the pte invalid so that an attempt to access the page +* will invoke the page fault handler and we can ensure +* consistency across the rc bits in the various ptes. +*/ +if ((v &HPTE_V_VALID ) && ((r & mask) = hpa)) +kvmhv_invalidate_shadow_pte_hash (&gp->shadow_hpt, + gp->shadow_lpid, hptep, + index); +else /* Leave pte unchanged */ +__unlock_hpte (hptep, v); +preempt_enable (); } }
@@ -1179,7 +1209,7 @@ void kvmhv_update_nest_rmap_rc_list(struct kvm *kvm, unsigned long *rmapp,if ((clr | set) & ~(_PAGE_DIRTY | _PAGE_ACCESSED))return ; - mask =PTE_RPN_MASK & ~(nbytes -1 ); + mask =HPTE_R_RPN_3_0 & ~(nbytes -1 ); hpa &= mask;llist_for_each_entry (cursor, head->first, list)
@@ -1195,24 +1225,50 @@ static void kvmhv_invalidate_nest_rmap(struct kvm *kvm, u64 n_rmap, unsigned long hpa, unsigned long mask) {struct kvm_nested_guest *gp; -unsigned long gpa; -unsigned int shift, lpid; -pte_t *ptep; +unsigned int lpid; - gpa =n_rmap_to_gpa (n_rmap); lpid =n_rmap_to_lpid (n_rmap);; gp =kvmhv_find_nested (kvm, lpid);if (!gp)return ; -/* Find and invalidate the pte */ -if (gp->radix) +/* +* Find the pte, and ensure it's valid and still points to the same +* host page. If the pfn has changed then this is a stale rmap entry, +* the shadow pte actually points somewhere else now, and there is +* nothing to do. Otherwise invalidate the shadow pte and perform the +* appropriate cache invalidation. +*/ +if (gp->radix) { +unsigned long gpa =n_rmap_to_gpa (n_rmap); +unsigned int shift; +pte_t *ptep; + ptep =__find_linux_pte (gp->shadow_pgtable, gpa,NULL , &shift); -else - ptep =NULL ;/* XXX TODO */ -/* Don't spuriously invalidate ptes if the pfn has changed */ -if (ptep &&pte_present (*ptep) && ((pte_val (*ptep) & mask) = hpa)) -kvmppc_unmap_pte (kvm, ptep, gpa, shift,NULL , gp->shadow_lpid); +/* pte present and still points to the same host page? */ +if (ptep &&pte_present (*ptep) && ((pte_val (*ptep) & mask) = + hpa)) +kvmppc_unmap_pte (kvm, ptep, gpa, shift,NULL , + gp->shadow_lpid); + }else { +unsigned long v, r, index =n_rmap_to_index (n_rmap); +__be64 *hptep = (__be64 *)(gp->shadow_hpt.virt + (index <<4 )); + +preempt_disable (); +while (!try_lock_hpte (hptep,HPTE_V_HVLOCK )) +cpu_relax (); + v =be64_to_cpu (hptep[0 ]) & ~HPTE_V_HVLOCK ; + r =be64_to_cpu (hptep[1 ]); + +/* Invalidate existing pte if valid and host addr matches */ +if ((v &HPTE_V_VALID ) && ((r & mask) = hpa)) +kvmhv_invalidate_shadow_pte_hash (&gp->shadow_hpt, + gp->shadow_lpid, hptep, + index); +else /* Leave pte unchanged */ +__unlock_hpte (hptep, v); +preempt_enable (); + } }/*
@@ -1252,7 +1308,7 @@ void kvmhv_invalidate_nest_rmap_range(struct kvm *kvm, gfn = (gpa >>PAGE_SHIFT ) - memslot->base_gfn; end_gfn = gfn + (nbytes >>PAGE_SHIFT ); - addr_mask =PTE_RPN_MASK & ~(nbytes -1 ); + addr_mask =HPTE_R_RPN_3_0 & ~(nbytes -1 ); hpa &= addr_mask;for (; gfn < end_gfn; gfn++) {
Signature · 1 line
2.13.6
PATCH 22/23 KVM: PPC: Book3S HV: Nested: Enable nested hpt guests
In reply to Suraj Jitindar SinghAllow nested hpt (hash page table) guests to be run.
--- arch/powerpc/kvm/book3s_hv_nested.c | 1 - 1 file changed, 1 deletion(-)
arch/powerpc/kvm/book3s_hv_nested.c +0 -1
@@ -330,7 +330,6 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) (l2_hv.lpcr & LPCR_VPM1))return H_PARAMETER ; }else { -return H_PARAMETER ;/* must be at least V2 to support hpt guest */ if (l2_hv.version <2 )return H_PARAMETER ;
Signature · 1 line
2.13.6
PATCH 23/23 KVM: PPC: Book3S HV: Add nested hpt pte information to debugfs
In reply to Suraj Jitindar SinghThe radix entry in the vm debugfs folder is used to export debug information about the ptes contained in the page table of a radix guest. There is a corrsponding htab entry which provides the same pte information for a hpt (hash page table) guest.
When a radix guest is running a nested guest this entry also provides information about the shadow ptes in the shadow page table.
Add the same information for a nested hpt guest running under a radix guest to this entry. The format followed is the same as that for the htab entry for a hpt guest.
--- arch/powerpc/kvm/book3s_64_mmu_radix.c | 157 +++++++++++++++++++++------------ 1 file changed, 101 insertions(+), 56 deletions(-)
arch/powerpc/kvm/book3s_64_mmu_radix.c +101 -56
@@ -1188,6 +1188,8 @@ static int debugfs_radix_release(struct inode *inode, struct file *file)return 0 ; } +#define HPTE_SIZE (2 * sizeof(unsigned long)) +static ssize_t debugfs_radix_read (struct file *file,char __user *buf,size_t len,loff_t *ppos) {
@@ -1197,6 +1199,7 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf,struct kvm *kvm;unsigned long gpa;pgd_t *pgt; +struct kvm_hpt_info *hpt;struct kvm_nested_guest *nested;pgd_t pgd, *pgdp;pud_t pud, *pudp;
@@ -1234,84 +1237,126 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf, gpa = p->gpa; nested =NULL ; pgt =NULL ; + hpt =NULL ;while (len !=0 && p->lpid >=0 ) { -if (gpa >=RADIX_PGTABLE_RANGE ) { - gpa =0 ; - pgt =NULL ; -if (nested) { -kvmhv_put_nested (nested); - nested =NULL ; - } - p->lpid =kvmhv_nested_next_lpid (kvm, p->lpid); - p->hdr =0 ; -if (p->lpid <0 ) -break ; - } -if (!pgt) { +if (!pgt && !hpt) {if (p->lpid =0 ) { pgt = kvm->arch.pgtable; }else { nested =kvmhv_get_nested (kvm, p->lpid, false);if (!nested) { - gpa =RADIX_PGTABLE_RANGE ; + p->lpid =kvmhv_nested_next_lpid (kvm, + p->lpid);continue ; } - pgt = nested->shadow_pgtable; +if (nested->radix) + pgt = nested->shadow_pgtable; +else + hpt = &nested->shadow_hpt; } } +if ((pgt && (gpa >=RADIX_PGTABLE_RANGE )) || (hpt && + (gpa >=kvmppc_hpt_npte (hpt)))) { + gpa =0 ; + pgt =NULL ; + hpt =NULL ; +if (nested) { +kvmhv_put_nested (nested); + nested =NULL ; + } + p->lpid =kvmhv_nested_next_lpid (kvm, p->lpid); + p->hdr =0 ; +continue ; + } n =0 ;if (!p->hdr) {if (p->lpid >0 ) n =scnprintf (p->buf,sizeof (p->buf), -"\nNested LPID %d: " , p->lpid); - n +=scnprintf (p->buf + n,sizeof (p->buf) - n, -"pgdir: %lx\n" , (unsigned long )pgt); +"\nNested LPID %d: \n" , p->lpid); +if (pgt) + n +=scnprintf (p->buf + n,sizeof (p->buf) - n, +"RADIX:\npgdir: %lx\n" , + (unsigned long )pgt); +else + n +=scnprintf (p->buf + n,sizeof (p->buf) - n, +"HASH:\nnpte: %lx\n" , +kvmppc_hpt_npte (hpt)); p->hdr =1 ; goto copy; } - pgdp = pgt +pgd_index (gpa); - pgd =READ_ONCE (*pgdp); -if (!(pgd_val (pgd) & _PAGE_PRESENT)) { - gpa = (gpa &PGDIR_MASK ) +PGDIR_SIZE ; -continue ; - } +if (pgt) { + pgdp = pgt +pgd_index (gpa); + pgd =READ_ONCE (*pgdp); +if (!(pgd_val (pgd) & _PAGE_PRESENT)) { + gpa = (gpa &PGDIR_MASK ) +PGDIR_SIZE ; +continue ; + } - pudp =pud_offset (&pgd, gpa); - pud =READ_ONCE (*pudp); -if (!(pud_val (pud) & _PAGE_PRESENT)) { - gpa = (gpa &PUD_MASK ) +PUD_SIZE ; -continue ; - } -if (pud_val (pud) & _PAGE_PTE) { - pte =pud_val (pud); - shift =PUD_SHIFT ; - goto leaf; - } + pudp =pud_offset (&pgd, gpa); + pud =READ_ONCE (*pudp); +if (!(pud_val (pud) & _PAGE_PRESENT)) { + gpa = (gpa &PUD_MASK ) +PUD_SIZE ; +continue ; + } +if (pud_val (pud) & _PAGE_PTE) { + pte =pud_val (pud); + shift =PUD_SHIFT ; + goto leaf; + } - pmdp =pmd_offset (&pud, gpa); - pmd =READ_ONCE (*pmdp); -if (!(pmd_val (pmd) & _PAGE_PRESENT)) { - gpa = (gpa &PMD_MASK ) +PMD_SIZE ; -continue ; - } -if (pmd_val (pmd) & _PAGE_PTE) { - pte =pmd_val (pmd); - shift =PMD_SHIFT ; - goto leaf; - } + pmdp =pmd_offset (&pud, gpa); + pmd =READ_ONCE (*pmdp); +if (!(pmd_val (pmd) & _PAGE_PRESENT)) { + gpa = (gpa &PMD_MASK ) +PMD_SIZE ; +continue ; + } +if (pmd_val (pmd) & _PAGE_PTE) { + pte =pmd_val (pmd); + shift =PMD_SHIFT ; + goto leaf; + } - ptep =pte_offset_kernel (&pmd, gpa); - pte =pte_val (READ_ONCE (*ptep)); -if (!(pte & _PAGE_PRESENT)) { - gpa +=PAGE_SIZE ; -continue ; - } - shift =PAGE_SHIFT ; - leaf: - n =scnprintf (p->buf,sizeof (p->buf), + ptep =pte_offset_kernel (&pmd, gpa); + pte =pte_val (READ_ONCE (*ptep)); +if (!(pte & _PAGE_PRESENT)) { + gpa +=PAGE_SIZE ; +continue ; + } + shift =PAGE_SHIFT ; + leaf: + n =scnprintf (p->buf,sizeof (p->buf)," %lx: %lx %d\n" , gpa, pte, shift); - gpa +=1ul << shift; + gpa +=1ul << shift; + }else {/* hpt */ +__be64 *hptp = (__be64 *)(hpt->virt + (gpa *HPTE_SIZE )); +unsigned long v, hr, gr; + +if (!(be64_to_cpu (hptp[0 ]) & (HPTE_V_VALID | +HPTE_V_ABSENT ))) { + gpa++; +continue ; + } +/* lock the HPTE so it's stable and read it */ +preempt_disable (); +while (!try_lock_hpte (hptp,HPTE_V_HVLOCK )) +cpu_relax (); + v =be64_to_cpu (hptp[0 ]) & ~HPTE_V_HVLOCK ; + hr =be64_to_cpu (hptp[1 ]); + gr = hpt->rev[gpa].guest_rpte; +unlock_hpte (hptp, v); +preempt_enable (); + +if (!(v & (HPTE_V_VALID |HPTE_V_ABSENT ))) { + gpa++; +continue ; + } + + n =scnprintf (p->buf,sizeof (p->buf), +"%6lx %.16lx %.16lx %.16lx\n" , + gpa, v, hr, gr); + gpa++; + } copy: p->chars_left = n;if (n > len)
Signature · 1 line
2.13.6