Skip to contentSkip to conversation

KVM: PPC: BOok3S HV: Support for nested HPT guests

PATCH In review Suraj Jitindar Singh 2 participants 27 messages Aug 26, 2019 – Oct 24, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover KVM: PPC: BOok3S HV: Support for nested HPT guests

This patch series adds support for running a nested kvm guest which uses the hpt (hash page table) mmu type.

Patch series based on v5.3-rc6.

The first 8 patches in this series enable a radix guest (L1) running under a radix hypervisor (L0) to act as a guest hypervisor for it's own nested (L2) guest where the nested guest is using hash page table translation. This mainly involved ensuring that the guest hypervisor uses the new run_single_vcpu() entry path and ensuring that the appropriate functions which are normally called in the real mode entry path in book3s_hv_rmhandlers.S are called on the new virtual mode entry path when a hpt guest is being run.

The remainder of the patches enable a (L0) hypervisor to perform hash page table translation for a nested (L2) hpt guest which is running under one of it's radix (L1) it's which is acting as a guest hypervisor. This primarily required changes to the nested guest entry patch to ensure that a shadow hpt would be allocated for the nested hpt guest, that the slb was context switched and that the real mode entry path in book3s_hv_rmhandlers.S could be used to enter/exit a nested hpt guest.

It was also necessary to be able to create translations by inserting ptes into the shadow page table which provided the combination of the translation from L2 virtual address to L1 guest physical address and the translation from L1 guest physical address to L0 host real address. Additionally invalidations of these translations need to be handled at both levels, by L1 via the H_TLB_INVALIDATE hcall to invalidate a L2 virtual address to L1 guest physical address translation, and by L0 when paging out a L1 guest page which had been subsequently mapped through to L2 thus invalidating the L1 guest physical address to L0 host real address translation.

Still lacking support for:
Passthrough of emulated mmio devices to nested hpt guests since the current method of reading nested guest memory relies on using quadrants which are only available when using radix translation.

Paul Mackerras (1):
KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page fault
handler

Suraj Jitindar Singh (22):
KVM: PPC: Book3S HV: Increment mmu_notifier_seq when modifying radix
pte rc bits
KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested
guests
KVM: PPC: Book3S HV: Handle making H_ENTER_NESTED hcall in a separate
function
KVM: PPC: Book3S HV: Enable calling kvmppc_hpte_hv_fault in virtual
mode
KVM: PPC: Book3S HV: Allow hpt manipulation hcalls to be called in
virtual mode
KVM: PPC: Book3S HV: Make kvmppc_invalidate_hpte() take lpid not a kvm
struct
KVM: PPC: Book3S HV: Nested: Allow pseries hypervisor to run hpt
nested guest
KVM: PPC: Book3S HV: Nested: Improve comments and naming of nest rmap
functions
KVM: PPC: Book3S HV: Nested: Increase gpa field in nest rmap to 46
bits
KVM: PPC: Book3S HV: Nested: Remove single nest rmap entries
KVM: PPC: Book3S HV: Nested: add kvmhv_remove_all_nested_rmap_lpid()
KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup
KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest
KVM: PPC: Book3S HV: Store lpcr and hdec_exp in the vcpu struct
KVM: PPC: Book3S HV: Nested: Make kvmppc_run_vcpu() entry path nested
capable
KVM: PPC: Book3S HV: Nested: Rename kvmhv_xlate_addr_nested_radix
KVM: PPC: Book3S HV: Separate out hashing from
kvmppc_hv_find_lock_hpte()
KVM: PPC: Book3S HV: Nested: Implement nested hpt mmu translation
KVM: PPC: Book3S HV: Nested: Handle tlbie hcall for nested hpt guest
KVM: PPC: Book3S HV: Nested: Implement nest rmap invalidations for hpt
guests
KVM: PPC: Book3S HV: Nested: Enable nested hpt guests
KVM: PPC: Book3S HV: Add nested hpt pte information to debugfs

 arch/powerpc/include/asm/book3s/64/mmu-hash.h |   15 +
 arch/powerpc/include/asm/book3s/64/mmu.h      |    9 +
 arch/powerpc/include/asm/hvcall.h             |   36 -
 arch/powerpc/include/asm/kvm_asm.h            |    5 +
 arch/powerpc/include/asm/kvm_book3s.h         |   30 +-
 arch/powerpc/include/asm/kvm_book3s_64.h      |   87 +-
 arch/powerpc/include/asm/kvm_host.h           |   57 +
 arch/powerpc/include/asm/kvm_ppc.h            |    5 +-
 arch/powerpc/kernel/asm-offsets.c             |    5 +
 arch/powerpc/kvm/book3s.c                     |    1 +
 arch/powerpc/kvm/book3s_64_mmu_hv.c           |  136 ++-
 arch/powerpc/kvm/book3s_64_mmu_radix.c        |  167 +--
 arch/powerpc/kvm/book3s_hv.c                  |  327 ++++--
 arch/powerpc/kvm/book3s_hv_builtin.c          |   33 +-
 arch/powerpc/kvm/book3s_hv_interrupts.S       |   25 +-
 arch/powerpc/kvm/book3s_hv_nested.c           | 1381 ++++++++++++++++++++++---
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |  298 ++++--
 arch/powerpc/kvm/book3s_hv_rmhandlers.S       |  126 ++-
 arch/powerpc/kvm/book3s_xive.h                |   15 +
 arch/powerpc/kvm/powerpc.c                    |    3 +-
 20 files changed, 2136 insertions(+), 625 deletions(-)
Signature · 1 line
2.13.6
#1 lore raw

PATCH 1/23 KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot in HPT page fault handler

From: Paul Mackerras <[email protected]>

This makes the same changes in the page fault handler for HPT guests that commits 31c8b0d0694a ("KVM: PPC: Book3S HV: Use __gfn_to_pfn_memslot() in page fault handler", 2018-03-01), 71d29f43b633 ("KVM: PPC: Book3S HV:
Don't use compound_order to determine host mapping size", 2018-09-11) and 6579804c4317 ("KVM: PPC: Book3S HV: Avoid crash from THP collapse during radix page fault", 2018-10-04) made for the page fault handler for radix guests.

In summary, where we used to call get_user_pages_fast() and then do special handling for VM_PFNMAP vmas, we now call __get_user_pages_fast() and then __gfn_to_pfn_memslot() if that fails, followed by reading the Linux PTE to get the host PFN, host page size and mapping attributes.

This also brings in the change from SetPageDirty() to set_page_dirty_lock() which was done for the radix page fault handler in commit c3856aeb2940 ("KVM: PPC: Book3S HV: Fix handling of large pages in radix page fault handler", 2018-02-23).

---
 arch/powerpc/kvm/book3s_64_mmu_hv.c | 118 +++++++++++++++++-------------------
 1 file changed, 57 insertions(+), 61 deletions(-)
arch/powerpc/kvm/book3s_64_mmu_hv.c +57 -61
index 9a75f0e1933b..a485bb018193 100644
@@ -497,17 +497,18 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	__be64 *hptep;
 	unsigned long mmu_seq, psize, pte_size;
 	unsigned long gpa_base, gfn_base;
-	unsigned long gpa, gfn, hva, pfn;
+	unsigned long gpa, gfn, hva, pfn, hpa;
 	struct kvm_memory_slot *memslot;
 	unsigned long *rmap;
 	struct revmap_entry *rev;
-	struct page *page, *pages[1];
-	long index, ret, npages;
+	struct page *page;
+	long index, ret;
 	bool is_ci;
-	unsigned int writing, write_ok;
-	struct vm_area_struct *vma;
+	bool writing, write_ok;
+	unsigned int shift;
 	unsigned long rcbits;
 	long mmio_update;
+	pte_t pte, *ptep;
 
 	if (kvm_is_radix(kvm))
 		return kvmppc_book3s_radix_page_fault(run, vcpu, ea, dsisr);
@@ -581,59 +582,62 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	smp_rmb();
 
 	ret = -EFAULT;
-	is_ci = false;
-	pfn = 0;
 	page = NULL;
-	pte_size = PAGE_SIZE;
 	writing = (dsisr & DSISR_ISSTORE) != 0;
 	/* If writing != 0, then the HPTE must allow writing, if we get here */
 	write_ok = writing;
 	hva = gfn_to_hva_memslot(memslot, gfn);
-	npages = get_user_pages_fast(hva, 1, writing ? FOLL_WRITE : 0, pages);
-	if (npages < 1) {
-		/* Check if it's an I/O mapping */
-		down_read(&current->mm->mmap_sem);
-		vma = find_vma(current->mm, hva);
-		if (vma && vma->vm_start <= hva && hva + psize <= vma->vm_end &&
-		    (vma->vm_flags & VM_PFNMAP)) {
-			pfn = vma->vm_pgoff +
-				((hva - vma->vm_start) >> PAGE_SHIFT);
-			pte_size = psize;
-			is_ci = pte_ci(__pte((pgprot_val(vma->vm_page_prot))));
-			write_ok = vma->vm_flags & VM_WRITE;
-		}
-		up_read(&current->mm->mmap_sem);
-		if (!pfn)
-			goto out_put;
+
+	/*
+	 * Do a fast check first, since __gfn_to_pfn_memslot doesn't
+	 * do it with !atomic && !async, which is how we call it.
+	 * We always ask for write permission since the common case
+	 * is that the page is writable.
+	 */
+	if (__get_user_pages_fast(hva, 1, 1, &page) = 1) {
+		write_ok = true;
 	} else {
-		page = pages[0];
-		pfn = page_to_pfn(page);
-		if (PageHuge(page)) {
-			page = compound_head(page);
-			pte_size <<= compound_order(page);
-		}
-		/* if the guest wants write access, see if that is OK */
-		if (!writing && hpte_is_writable(r)) {
-			pte_t *ptep, pte;
-			unsigned long flags;
-			/*
-			 * We need to protect against page table destruction
-			 * hugepage split and collapse.
-			 */
-			local_irq_save(flags);
-			ptep = find_current_mm_pte(current->mm->pgd,
-						   hva, NULL, NULL);
-			if (ptep) {
-				pte = kvmppc_read_update_linux_pte(ptep, 1);
-				if (__pte_write(pte))
-					write_ok = 1;
-			}
-			local_irq_restore(flags);
+		/* Call KVM generic code to do the slow-path check */
+		pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL,
+					   writing, &write_ok);
+		if (is_error_noslot_pfn(pfn))
+			return -EFAULT;
+		page = NULL;
+		if (pfn_valid(pfn)) {
+			page = pfn_to_page(pfn);
+			if (PageReserved(page))
+				page = NULL;
 		}
 	}
 
+	/*
+	 * Read the PTE from the process' radix tree and use that
+	 * so we get the shift and attribute bits.
+	 */
+	local_irq_disable();
+	ptep = __find_linux_pte(vcpu->arch.pgdir, hva, NULL, &shift);
+	/*
+	 * If the PTE disappeared temporarily due to a THP
+	 * collapse, just return and let the guest try again.
+	 */
+	if (!ptep) {
+		local_irq_enable();
+		if (page)
+			put_page(page);
+		return RESUME_GUEST;
+	}
+	pte = *ptep;
+	local_irq_enable();
+	hpa = pte_pfn(pte) << PAGE_SHIFT;
+	pte_size = PAGE_SIZE;
+	if (shift)
+		pte_size = 1ul << shift;
+	is_ci = pte_ci(pte);
+
 	if (psize > pte_size)
 		goto out_put;
+	if (pte_size > psize)
+		hpa |= hva & (pte_size - psize);
 
 	/* Check WIMG vs. the actual page we're accessing */
 	if (!hpte_cache_flags_ok(r, is_ci)) {
@@ -647,14 +651,13 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	}
 
 	/*
-	 * Set the HPTE to point to pfn.
-	 * Since the pfn is at PAGE_SIZE granularity, make sure we
+	 * Set the HPTE to point to hpa.
+	 * Since the hpa is at PAGE_SIZE granularity, make sure we
 	 * don't mask out lower-order bits if psize < PAGE_SIZE.
 	 */
 	if (psize < PAGE_SIZE)
 		psize = PAGE_SIZE;
-	r = (r & HPTE_R_KEY_HI) | (r & ~(HPTE_R_PP0 - psize)) |
-					((pfn << PAGE_SHIFT) & ~(psize - 1));
+	r = (r & HPTE_R_KEY_HI) | (r & ~(HPTE_R_PP0 - psize)) | hpa;
 	if (hpte_is_writable(r) && !write_ok)
 		r = hpte_make_readonly(r);
 	ret = RESUME_GUEST;
@@ -719,20 +722,13 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
 	asm volatile("ptesync" : : : "memory");
 	preempt_enable();
 	if (page && hpte_is_writable(r))
-		SetPageDirty(page);
+		set_page_dirty_lock(page);
 
  out_put:
 	trace_kvm_page_fault_exit(vcpu, hpte, ret);
 
-	if (page) {
-		/*
-		 * We drop pages[0] here, not page because page might
-		 * have been set to the head page of a compound, but
-		 * we have to drop the reference on the correct tail
-		 * page to match the get inside gup()
-		 */
-		put_page(pages[0]);
-	}
+	if (page)
+		put_page(page);
 	return ret;
 
  out_unlock:
Signature · 1 line
2.13.6
#2 lore raw

PATCH 2/23 KVM: PPC: Book3S HV: Increment mmu_notifier_seq when modifying radix pte rc bits

In reply to Suraj Jitindar Singh

The kvm mmu_notifier_seq is used to communicate that a page mapping has changed to code which is using that information in constructing a different but reliant page mapping. For example when constructing a mapping for a nested guest it is used to detect when the guest mapping has changed, which would render the nested guest mapping invalid.

When running nested guests it is important that the rc bits are kept in sync between the 2 ptes on the host in which they exist; the pte for the guest, and the pte for the nested guest. This is done when inserting the nested pte in __kvmhv_nested_page_fault_radix() by reducing the rc bits being set in the nested pte to those already set in the guest pte. And when setting the bits in the nested pte in response to an interrupt in kvmhv_handle_nested_set_rc_radix() the same bits are also set in the guest pte, with the bits not set in the nested pte if this fails.

When the host wants to remove rc bits from the guest pte in kvm_radix_test_clear_dirty(), if first removes then from the guest pte and then from any corresponding nested ptes which map the same guest page. This means that there is a window between which the rc bits could get out of sync between the two ptes as they might have been seen as set in the guest pte and thus updated in the nested pte assuming as such, while the host might be in the process of removing those rc bits leading to an inconsistency.

In kvm_radix_test_clear_dirty() the mmu_lock spin lock is held across removing the rc bits from the guest and nested pte, and the same is done across updating the rc bits in the guest and nested pte in kvmhv_handle_nested_set_rc_radix() and so there is no window for them to get out of sync in this case. However when constructing the pte in __kvmhv_nested_page_fault_radix() we drop the mmu_lock spin lock between reading the guest pte and inserting the nested pte, presenting a window for them to get out of sync. This is because the rc bits could have been observed as set in the guest pte and set in the nested pte accordingly, however in the mean time the rc bits in the guest pte could have been cleared, and since the nested pte wasn't yet inserted there is no way for the kvm_radix_test_clear_dirty() function to clear them and so an inconsistency can arise.

To avoid the possibility of the rc bits getting out of sync, increment the mmu_notifier_seq in kvm_radix_test_clear_dirty() under the mmu_lock when clearing rc bits. This means that when inserting the nested pte in __kvmhv_nested_page_fault_radix() we will bail out and retry when we see that the mmu_seq differs indicating that the guest pte has changed.

Fixes ae59a7e1945b ("KVM: PPC: Book3S HV: Keep rc bits in shadow pgtable in sync with host")
---
 arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 ++
 1 file changed, 2 insertions(+)
arch/powerpc/kvm/book3s_64_mmu_radix.c +2 -0
index 2d415c36a61d..310d8dde9a48 100644
@@ -1044,6 +1044,8 @@ static int kvm_radix_test_clear_dirty(struct kvm *kvm,
 		kvmhv_update_nest_rmap_rc_list(kvm, rmapp, _PAGE_DIRTY, 0,
 					       old & PTE_RPN_MASK,
 					       1UL << shift);
+		/* Notify anyone trying to map the page that it has changed */
+		kvm->mmu_notifier_seq++;
 		spin_unlock(&kvm->mmu_lock);
 	}
 	return ret;
Signature · 1 line
2.13.6
#3 lore raw

PATCH 3/23 KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested guests

In reply to Suraj Jitindar Singh

Don't allow hpt (hash page table) guests to act as guest hypervisors and thus be able to run nested guests. There is currently no support for this, if a nested guest is to be run it must be run at the lowest level. Explicitly disallow hash guests from enabling the nested kvm-hv capability at the hypervisor level.

---
 arch/powerpc/kvm/book3s_hv.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +5 -1
index cde3f5a4b3e4..ce960301bfaa 100644
@@ -5336,8 +5336,12 @@ static int kvmhv_enable_nested(struct kvm *kvm)
 		return -ENODEV;
 
 	/* kvm = NULL means the caller is testing if the capability exists */
-	if (kvm)
+	if (kvm) {
+		/* Only radix guests can act as nested hv and thus run guests */
+		if (!kvm_is_radix(kvm))
+			return -1;
 		kvm->arch.nested_enable = true;
+	}
 	return 0;
 }
 
Signature · 1 line
2.13.6
#4 lore raw

Re: [PATCH 03/23] KVM: PPC: Book3S HV: Nested: Don't allow hash guests to run nested guests

Quoted 16 lines · Suraj Jitindar Singh

Don't allow hpt (hash page table) guests to act as guest hypervisors and thus be able to run nested guests. There is currently no support for this, if a nested guest is to be run it must be run at the lowest level. Explicitly disallow hash guests from enabling the nested kvm-hv capability at the hypervisor level.

---
 arch/powerpc/kvm/book3s_hv.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +0 -0
index cde3f5a4b3e4..ce960301bfaa 100644

arch/powerpc/kvm/book3s_hv.c static int kvmhv_enable_nested(struct kvm *kvm) · line 5344

Quoted 10 lines
arch/powerpc/kvm/book3s_hv.c +5 -1
@@ -5336,8 +5336,12 @@ static int kvmhv_enable_nested(struct kvm *kvm)
 		return -ENODEV;
 
 	/* kvm = NULL means the caller is testing if the capability exists */
-	if (kvm)
+	if (kvm) {
+		/* Only radix guests can act as nested hv and thus run guests */
+		if (!kvm_is_radix(kvm))
+			return -1;
 		kvm->arch.nested_enable = true;
+	}

I don't think this is necessary, and is possibly undesirable, since a guest can switch between HPT and radix mode. In fact if a guest in HPT mode tries to do any of the hcalls for managing nested guests, it will get errors, because we have this:

static inline bool nesting_enabled(struct kvm *kvm)

{
	return kvm->arch.nested_enable && kvm_is_radix(kvm);
}

and H_SET_PARTITION_TABLE, H_ENTER_NESTED, etc. all return H_FUNCTION if nested_enabled() is false. (This is as the code is today without your patch). Furthermore, kvmppc_switch_mmu_to_hpt() does this:

	if (nesting_enabled(kvm))
		kvmhv_release_all_nested(kvm);

So I think it is all covered already without your patch.

Paul.

#5 lore raw

PATCH 4/23 KVM: PPC: Book3S HV: Handle making H_ENTER_NESTED hcall in a separate function

In reply to Suraj Jitindar Singh

A pseries guest (that is a guest acting as a guest hypervisor) which would like to run it's own guest (a nested guest) must perform this by invoking the top level hypervisor via the H_ENTER_NESTED hcall.

Separate out the code which handles calling this into a separate function for code clarity and readability.

No functional change.

---
 arch/powerpc/kvm/book3s_hv.c | 88 ++++++++++++++++++++++++--------------------
 1 file changed, 49 insertions(+), 39 deletions(-)
arch/powerpc/kvm/book3s_hv.c +49 -39
index ce960301bfaa..4901738a3c31 100644
@@ -3367,7 +3367,55 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc)
 }
 
 /*
+ * Handle making the H_ENTER_NESTED hcall if we're pseries.
+ */
+static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit,
+				     unsigned long lpcr)
+{
+	/* call our hypervisor to load up HV regs and go */
+	struct hv_guest_state hvregs;
+	/* we need to save/restore host & guest psscr since L0 doesn't for us */
+	unsigned long host_psscr;
+	int trap;
+
+	host_psscr = mfspr(SPRN_PSSCR_PR);
+	mtspr(SPRN_PSSCR_PR, vcpu->arch.psscr);
+	kvmhv_save_hv_regs(vcpu, &hvregs);
+	hvregs.lpcr = lpcr;
+	vcpu->arch.regs.msr = vcpu->arch.shregs.msr;
+	hvregs.version = HV_GUEST_STATE_VERSION;
+	if (vcpu->arch.nested) {
+		hvregs.lpid = vcpu->arch.nested->shadow_lpid;
+		hvregs.vcpu_token = vcpu->arch.nested_vcpu_id;
+	} else {
+		hvregs.lpid = vcpu->kvm->arch.lpid;
+		hvregs.vcpu_token = vcpu->vcpu_id;
+	}
+	hvregs.hdec_expiry = time_limit;
+	trap = plpar_hcall_norets(H_ENTER_NESTED, __pa(&hvregs),
+				  __pa(&vcpu->arch.regs));
+	kvmhv_restore_hv_return_state(vcpu, &hvregs);
+	vcpu->arch.shregs.msr = vcpu->arch.regs.msr;
+	vcpu->arch.shregs.dar = mfspr(SPRN_DAR);
+	vcpu->arch.shregs.dsisr = mfspr(SPRN_DSISR);
+	vcpu->arch.psscr = mfspr(SPRN_PSSCR_PR);
+	mtspr(SPRN_PSSCR_PR, host_psscr);
+
+	/* H_CEDE has to be handled now, not later */
+	if (trap = BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested &&
+	    kvmppc_get_gpr(vcpu, 3) = H_CEDE) {
+		kvmppc_nested_cede(vcpu);
+		trap = 0;
+	}
+
+	return trap;
+}
+
+/*
  * Load up hypervisor-mode registers on P9.
+ * This is only called on baremetal (powernv) systems i.e. where
+ * CPU_FTR_HVMODE is set. This is only used for radix guests, however that
+ * radix guest may be a direct guest of this hypervisor or a nested guest.
  */
 static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,
 				     unsigned long lpcr)
@@ -3569,45 +3617,7 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,
 	mtspr(SPRN_DEC, vcpu->arch.dec_expires - mftb());
 
 	if (kvmhv_on_pseries()) {
-		/*
-		 * We need to save and restore the guest visible part of the
-		 * psscr (i.e. using SPRN_PSSCR_PR) since the hypervisor
-		 * doesn't do this for us. Note only required if pseries since
-		 * this is done in kvmhv_load_hv_regs_and_go() below otherwise.
-		 */
-		unsigned long host_psscr;
-		/* call our hypervisor to load up HV regs and go */
-		struct hv_guest_state hvregs;
-
-		host_psscr = mfspr(SPRN_PSSCR_PR);
-		mtspr(SPRN_PSSCR_PR, vcpu->arch.psscr);
-		kvmhv_save_hv_regs(vcpu, &hvregs);
-		hvregs.lpcr = lpcr;
-		vcpu->arch.regs.msr = vcpu->arch.shregs.msr;
-		hvregs.version = HV_GUEST_STATE_VERSION;
-		if (vcpu->arch.nested) {
-			hvregs.lpid = vcpu->arch.nested->shadow_lpid;
-			hvregs.vcpu_token = vcpu->arch.nested_vcpu_id;
-		} else {
-			hvregs.lpid = vcpu->kvm->arch.lpid;
-			hvregs.vcpu_token = vcpu->vcpu_id;
-		}
-		hvregs.hdec_expiry = time_limit;
-		trap = plpar_hcall_norets(H_ENTER_NESTED, __pa(&hvregs),
-					  __pa(&vcpu->arch.regs));
-		kvmhv_restore_hv_return_state(vcpu, &hvregs);
-		vcpu->arch.shregs.msr = vcpu->arch.regs.msr;
-		vcpu->arch.shregs.dar = mfspr(SPRN_DAR);
-		vcpu->arch.shregs.dsisr = mfspr(SPRN_DSISR);
-		vcpu->arch.psscr = mfspr(SPRN_PSSCR_PR);
-		mtspr(SPRN_PSSCR_PR, host_psscr);
-
-		/* H_CEDE has to be handled now, not later */
-		if (trap = BOOK3S_INTERRUPT_SYSCALL && !vcpu->arch.nested &&
-		    kvmppc_get_gpr(vcpu, 3) = H_CEDE) {
-			kvmppc_nested_cede(vcpu);
-			trap = 0;
-		}
+		trap = kvmhv_pseries_enter_guest(vcpu, time_limit, lpcr);
 	} else {
 		trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr);
 	}
Signature · 1 line
2.13.6
#6 lore raw

PATCH 5/23 KVM: PPC: Book3S HV: Enable calling kvmppc_hpte_hv_fault in virtual mode

In reply to Suraj Jitindar Singh

The function kvmppc_hpte_hv_fault() is used to search a hpt (hash page table) for an existing entry given a slb entry, a fault code and a fault address.

Currently this function is only called in real mode. Modify this function so that it can be called in virtual mode and add a function parameter used to specify if the function is being called from real mode or not.

---
 arch/powerpc/include/asm/kvm_ppc.h      | 3 ++-
 arch/powerpc/kvm/book3s_hv_rm_mmu.c     | 9 +++++++--
 arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++
 3 files changed, 11 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +2 -1
index 2484e6a8f5ca..2c4d659cf8bb 100644
@@ -688,7 +688,8 @@ long kvmppc_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,
 long kvmppc_rm_h_page_init(struct kvm_vcpu *vcpu, unsigned long flags,
 			   unsigned long dest, unsigned long src);
 long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,
-                          unsigned long slb_v, unsigned int status, bool data);
+                          unsigned long slb_v, unsigned int status,
+			  bool data, bool is_realmode);
 unsigned long kvmppc_rm_h_xirr(struct kvm_vcpu *vcpu);
 unsigned long kvmppc_rm_h_xirr_x(struct kvm_vcpu *vcpu);
 unsigned long kvmppc_rm_h_ipoll(struct kvm_vcpu *vcpu, unsigned long server);
arch/powerpc/kvm/book3s_hv_rm_mmu.c +7 -2
index 63e0ce91e29d..9f7ad4eaa528 100644
@@ -1182,9 +1182,12 @@ EXPORT_SYMBOL(kvmppc_hv_find_lock_hpte);
  * -1 to pass the fault up to host kernel mode code, -2 to do that
  * and also load the instruction word (for MMIO emulation),
  * or 0 if we should make the guest retry the access.
+ * For a nested hypervisor, this will be called in virtual mode
+ * (is_realmode = false) and should be called with preemption disabled.
  */
 long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,
-			  unsigned long slb_v, unsigned int status, bool data)
+			  unsigned long slb_v, unsigned int status,
+			  bool data, bool is_realmode)
 {
 	struct kvm *kvm = vcpu->kvm;
 	long int index;
@@ -1222,7 +1225,9 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,
 			v = hpte_new_to_old_v(v, r);
 			r = hpte_new_to_old_r(r);
 		}
-		rev = real_vmalloc_addr(&kvm->arch.hpt.rev[index]);
+		rev = &kvm->arch.hpt.rev[index];
+		if (is_realmode)
+			rev = real_vmalloc_addr(rev);
 		gr = rev->guest_rpte;
 
 		unlock_hpte(hpte, orig_v);
arch/powerpc/kvm/book3s_hv_rmhandlers.S +2 -0
index 337e64468d78..54e1864d4702 100644
@@ -2066,6 +2066,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
 	/* Search the hash table. */
 	mr	r3, r9			/* vcpu pointer */
 	li	r7, 1			/* data fault */
+	li	r8, 1			/* is real mode */
 	bl	kvmppc_hpte_hv_fault
 	ld	r9, HSTATE_KVM_VCPU(r13)
 	ld	r10, VCPU_PC(r9)
@@ -2158,6 +2159,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
 	mr	r4, r10
 	mr	r6, r11
 	li	r7, 0			/* instruction fault */
+	li	r8, 1			/* is real mode */
 	bl	kvmppc_hpte_hv_fault
 	ld	r9, HSTATE_KVM_VCPU(r13)
 	ld	r10, VCPU_PC(r9)
Signature · 1 line
2.13.6
#7 lore raw

PATCH 6/23 KVM: PPC: Book3S HV: Allow hpt manipulation hcalls to be called in virtual mode

In reply to Suraj Jitindar Singh

The hcalls H_ENTER, H_REMOVE, H_READ, H_CLEAR_MOD, H_CLEAR_READ, H_PROTECT and H_BULK_REMOVE are used by a guest to call into the hypervisor in order to control manipulation of it's hpt (hash page table) by the hypervisor on it'the guest's behalf.

Currently the functions which handle these hcalls are only called in real mode from the kvm exit path in book3s_hv_rmhandlers.c.

Modify the functions which handle these hcalls so that they can be called in real mode and call them from the virtual mode hcall handling function kvmppc_pseries_do_hcall() if we're a pseries machine. There is no need to call these functions again on powernv as they were already called from real mode.

---
 arch/powerpc/include/asm/kvm_book3s.h |  16 ++++-
 arch/powerpc/kvm/book3s_64_mmu_hv.c   |  10 +---
 arch/powerpc/kvm/book3s_hv.c          |  60 +++++++++++++++++++
 arch/powerpc/kvm/book3s_hv_rm_mmu.c   | 107 +++++++++++++++++++++++++---------
 4 files changed, 159 insertions(+), 34 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +15 -1
index 506e4df2d730..6d0bb22dc637 100644
@@ -252,9 +252,23 @@ extern void kvmppc_unpin_guest_page(struct kvm *kvm, void *addr,
 extern long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
 			long pte_index, unsigned long pteh, unsigned long ptel,
 			pgd_t *pgdir, bool realmode, unsigned long *idx_ret);
+extern long kvmppc_virtmode_do_h_enter(struct kvm *kvm, unsigned long flags,
+				       long pte_index, unsigned long pteh,
+				       unsigned long ptel,
+				       unsigned long *pte_idx_ret);
 extern long kvmppc_do_h_remove(struct kvm *kvm, unsigned long flags,
 			unsigned long pte_index, unsigned long avpn,
-			unsigned long *hpret);
+			bool realmode, unsigned long *hpret);
+extern long kvmppc_do_h_read(struct kvm_vcpu *vcpu, unsigned long flags,
+			     unsigned long pte_index, bool realmode);
+extern long kvmppc_do_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,
+				  unsigned long pte_index, bool realmode);
+extern long kvmppc_do_h_clear_ref(struct kvm_vcpu *vcpu, unsigned long flags,
+				  unsigned long pte_index, bool realmode);
+extern long kvmppc_do_h_protect(struct kvm_vcpu *vcpu, unsigned long flags,
+				unsigned long pte_index, unsigned long avpn,
+				unsigned long va, bool realmode);
+extern long kvmppc_do_h_bulk_remove(struct kvm_vcpu *vcpu, bool realmode);
 extern long kvmppc_hv_get_dirty_log_hpt(struct kvm *kvm,
 			struct kvm_memory_slot *memslot, unsigned long *map);
 extern void kvmppc_harvest_vpa_dirty(struct kvmppc_vpa *vpa,
arch/powerpc/kvm/book3s_64_mmu_hv.c +3 -7
index a485bb018193..ab97b6bcf226 100644
@@ -42,10 +42,6 @@
 	do { } while (0)
 #endif
 
-static long kvmppc_virtmode_do_h_enter(struct kvm *kvm, unsigned long flags,
-				long pte_index, unsigned long pteh,
-				unsigned long ptel, unsigned long *pte_idx_ret);
-
 struct kvm_resize_hpt {
 	/* These fields read-only after init */
 	struct kvm *kvm;
@@ -287,7 +283,7 @@ static void kvmppc_mmu_book3s_64_hv_reset_msr(struct kvm_vcpu *vcpu)
 	kvmppc_set_msr(vcpu, msr);
 }
 
-static long kvmppc_virtmode_do_h_enter(struct kvm *kvm, unsigned long flags,
+long kvmppc_virtmode_do_h_enter(struct kvm *kvm, unsigned long flags,
 				long pte_index, unsigned long pteh,
 				unsigned long ptel, unsigned long *pte_idx_ret)
 {
@@ -1907,7 +1903,7 @@ static ssize_t kvm_htab_write(struct file *file, const char __user *buf,
 			nb += HPTE_SIZE;
 
 			if (be64_to_cpu(hptp[0]) & (HPTE_V_VALID | HPTE_V_ABSENT))
-				kvmppc_do_h_remove(kvm, 0, i, 0, tmp);
+				kvmppc_do_h_remove(kvm, 0, i, 0, false, tmp);
 			err = -EIO;
 			ret = kvmppc_virtmode_do_h_enter(kvm, H_EXACT, i, v, r,
 							 tmp);
@@ -1937,7 +1933,7 @@ static ssize_t kvm_htab_write(struct file *file, const char __user *buf,
 
 		for (j = 0; j < hdr.n_invalid; ++j) {
 			if (be64_to_cpu(hptp[0]) & (HPTE_V_VALID | HPTE_V_ABSENT))
-				kvmppc_do_h_remove(kvm, 0, i, 0, tmp);
+				kvmppc_do_h_remove(kvm, 0, i, 0, false, tmp);
 			++i;
 			hptp += 2;
 		}
arch/powerpc/kvm/book3s_hv.c +60 -0

66 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv_rm_mmu.c +81 -26

241 lines not on this page · raw message

Signature · 1 line
2.13.6
#8 lore raw

PATCH 7/23 KVM: PPC: Book3S HV: Make kvmppc_invalidate_hpte() take lpid not a kvm struct

In reply to Suraj Jitindar Singh

The function kvmppc_invalidate_hpte() is used to invalidate and perform the tlbies for a given pte in a hpt (hash page table). Currently the function takes a kvm struct as an argument, however the only member of this struct that it accesses in the lpid field. Modify this function to take an lpid argument in place of the kvm struct.

No functional change.

---
 arch/powerpc/include/asm/kvm_book3s.h |  2 +-
 arch/powerpc/kvm/book3s_64_mmu_hv.c   |  6 +++---
 arch/powerpc/kvm/book3s_hv_rm_mmu.c   | 22 ++++++++++++----------
 3 files changed, 16 insertions(+), 14 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -1
index 6d0bb22dc637..c69eeb4e176c 100644
@@ -241,7 +241,7 @@ extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev,
 			unsigned long *rmap, long pte_index, int realmode);
 extern void kvmppc_update_dirty_map(const struct kvm_memory_slot *memslot,
 			unsigned long gfn, unsigned long psize);
-extern void kvmppc_invalidate_hpte(struct kvm *kvm, __be64 *hptep,
+extern void kvmppc_invalidate_hpte(unsigned int lpid, __be64 *hptep,
 			unsigned long pte_index);
 void kvmppc_clear_ref_hpte(struct kvm *kvm, __be64 *hptep,
 			unsigned long pte_index);
arch/powerpc/kvm/book3s_64_mmu_hv.c +3 -3
index ab97b6bcf226..bbb23b3f8bb9 100644
@@ -701,7 +701,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,
 		/* HPTE was previously valid, so we need to invalidate it */
 		unlock_rmap(rmap);
 		hptep[0] |= cpu_to_be64(HPTE_V_ABSENT);
-		kvmppc_invalidate_hpte(kvm, hptep, index);
+		kvmppc_invalidate_hpte(kvm->arch.lpid, hptep, index);
 		/* don't lose previous R and C bits */
 		r |= be64_to_cpu(hptep[1]) & (HPTE_R_R | HPTE_R_C);
 	} else {
@@ -829,7 +829,7 @@ static void kvmppc_unmap_hpte(struct kvm *kvm, unsigned long i,
 	if ((be64_to_cpu(hptep[0]) & HPTE_V_VALID) &&
 	    hpte_rpn(ptel, psize) = gfn) {
 		hptep[0] |= cpu_to_be64(HPTE_V_ABSENT);
-		kvmppc_invalidate_hpte(kvm, hptep, i);
+		kvmppc_invalidate_hpte(kvm->arch.lpid, hptep, i);
 		hptep[1] &= ~cpu_to_be64(HPTE_R_KEY_HI | HPTE_R_KEY_LO);
 		/* Harvest R and C */
 		rcbits = be64_to_cpu(hptep[1]) & (HPTE_R_R | HPTE_R_C);
@@ -1094,7 +1094,7 @@ static int kvm_test_clear_dirty_npages(struct kvm *kvm, unsigned long *rmapp)
 
 		/* need to make it temporarily absent so C is stable */
 		hptep[0] |= cpu_to_be64(HPTE_V_ABSENT);
-		kvmppc_invalidate_hpte(kvm, hptep, i);
+		kvmppc_invalidate_hpte(kvm->arch.lpid, hptep, i);
 		v = be64_to_cpu(hptep[0]);
 		r = be64_to_cpu(hptep[1]);
 		if (r & HPTE_R_C) {
arch/powerpc/kvm/book3s_hv_rm_mmu.c +12 -10
index bd31d36332a8..1d26d509aaf6 100644
@@ -440,7 +440,7 @@ static inline int is_mmio_hpte(unsigned long v, unsigned long r)
 		(HPTE_R_KEY_HI | HPTE_R_KEY_LO));
 }
 
-static void do_tlbies(struct kvm *kvm, unsigned long *rbvalues,
+static void do_tlbies(unsigned int lpid, unsigned long *rbvalues,
 		      long npages, int global, bool need_sync)
 {
 	long i;
@@ -455,7 +455,7 @@ static void do_tlbies(struct kvm *kvm, unsigned long *rbvalues,
 			asm volatile("ptesync" : : : "memory");
 		for (i = 0; i < npages; ++i) {
 			asm volatile(PPC_TLBIE_5(%0,%1,0,0,0) : :
-				     "r" (rbvalues[i]), "r" (kvm->arch.lpid));
+				     "r" (rbvalues[i]), "r" (lpid));
 		}
 
 		if (cpu_has_feature(CPU_FTR_P9_TLBIE_BUG)) {
@@ -465,7 +465,7 @@ static void do_tlbies(struct kvm *kvm, unsigned long *rbvalues,
 			 */
 			asm volatile("ptesync": : :"memory");
 			asm volatile(PPC_TLBIE_5(%0,%1,0,0,0) : :
-				     "r" (rbvalues[0]), "r" (kvm->arch.lpid));
+				     "r" (rbvalues[0]), "r" (lpid));
 		}
 
 		asm volatile("eieio; tlbsync; ptesync" : : : "memory");
@@ -516,7 +516,8 @@ long kvmppc_do_h_remove(struct kvm *kvm, unsigned long flags,
 	if (v & HPTE_V_VALID) {
 		hpte[0] &= ~cpu_to_be64(HPTE_V_VALID);
 		rb = compute_tlbie_rb(v, pte_r, pte_index);
-		do_tlbies(kvm, &rb, 1, global_invalidates(kvm), true);
+		do_tlbies(kvm->arch.lpid, &rb, 1, global_invalidates(kvm),
+			  true);
 		/*
 		 * The reference (R) and change (C) bits in a HPT
 		 * entry can be set by hardware at any time up until
@@ -652,7 +653,7 @@ long kvmppc_do_h_bulk_remove(struct kvm_vcpu *vcpu, bool realmode)
 			break;
 
 		/* Now that we've collected a batch, do the tlbies */
-		do_tlbies(kvm, tlbrb, n, global, true);
+		do_tlbies(kvm->arch.lpid, tlbrb, n, global, true);
 
 		/* Read PTE low words after tlbie to get final R/C values */
 		for (k = 0; k < n; ++k) {
@@ -736,7 +737,8 @@ long kvmppc_do_h_protect(struct kvm_vcpu *vcpu, unsigned long flags,
 			rb = compute_tlbie_rb(v, r, pte_index);
 			hpte[0] = cpu_to_be64((pte_v & ~HPTE_V_VALID) |
 					      HPTE_V_ABSENT);
-			do_tlbies(kvm, &rb, 1, global_invalidates(kvm), true);
+			do_tlbies(kvm->arch.lpid, &rb, 1,
+				  global_invalidates(kvm), true);
 			/* Don't lose R/C bit updates done by hardware */
 			r |= be64_to_cpu(hpte[1]) & (HPTE_R_R | HPTE_R_C);
 			hpte[1] = cpu_to_be64(r);
@@ -898,7 +900,7 @@ long kvmppc_do_h_clear_mod(struct kvm_vcpu *vcpu, unsigned long flags,
 	if (v & HPTE_V_VALID) {
 		/* need to make it temporarily absent so C is stable */
 		hpte[0] |= cpu_to_be64(HPTE_V_ABSENT);
-		kvmppc_invalidate_hpte(kvm, hpte, pte_index);
+		kvmppc_invalidate_hpte(kvm->arch.lpid, hpte, pte_index);
 		r = be64_to_cpu(hpte[1]);
 		gr |= r & (HPTE_R_R | HPTE_R_C);
 		if (r & HPTE_R_C) {
@@ -1064,7 +1066,7 @@ long kvmppc_rm_h_page_init(struct kvm_vcpu *vcpu, unsigned long flags,
 	return ret;
 }
 
-void kvmppc_invalidate_hpte(struct kvm *kvm, __be64 *hptep,
+void kvmppc_invalidate_hpte(unsigned int lpid, __be64 *hptep,
 			unsigned long pte_index)
 {
 	unsigned long rb;
@@ -1078,7 +1080,7 @@ void kvmppc_invalidate_hpte(struct kvm *kvm, __be64 *hptep,
 		hp1 = hpte_new_to_old_r(hp1);
 	}
 	rb = compute_tlbie_rb(hp0, hp1, pte_index);
-	do_tlbies(kvm, &rb, 1, 1, true);
+	do_tlbies(lpid, &rb, 1, 1, true);
 }
 EXPORT_SYMBOL_GPL(kvmppc_invalidate_hpte);
 
@@ -1099,7 +1101,7 @@ void kvmppc_clear_ref_hpte(struct kvm *kvm, __be64 *hptep,
 	rbyte = (be64_to_cpu(hptep[1]) & ~HPTE_R_R) >> 8;
 	/* modify only the second-last byte, which contains the ref bit */
 	*((char *)hptep + 14) = rbyte;
-	do_tlbies(kvm, &rb, 1, 1, false);
+	do_tlbies(kvm->arch.lpid, &rb, 1, 1, false);
 }
 EXPORT_SYMBOL_GPL(kvmppc_clear_ref_hpte);
 
Signature · 1 line
2.13.6
#9 lore raw

PATCH 8/23 KVM: PPC: Book3S HV: Nested: Allow pseries hypervisor to run hpt nested guest

In reply to Suraj Jitindar Singh

Allow a pseries guest hypervisor (must be a radix guest) to run a hpt (hash page table) nested guest.

Modify the entry path such that a pseries hypervisor will always use the kvmhv_run_single_vcpu() function to enter a guest which will result in it calling H_ENTER_NESTED. Also modify the API to H_ENTER_NESTED to add a version 2 which adds a slb pointer to the argument list which provides the slb state to be used to run the nested guest.

Also modify the exit path such that a pseries hypervisor will call kvmppc_hpte_hv_fault() when handling a page fault which would normally be called from real mode in book3s_hv_rmhandlers.c on a powernv hypervisor. This is required for subsequent functions which are invoked to handle the page fault. Also save the slb state on guest exit and don't zero slb_max.

Modify kvm_vm_ioctl_get_smmu_info_hv() such that only 4k and 64k page size support is reported to the guest. This is to ensure that we can maintain a 1-to-1 mapping between the guest hpt and the shadow hpt for simplicity (this could be relaxed later with appropriate support) since a 16M page would likely have to be broken into multiple smaller pages since the radix guest is likely to at most be backed by 2M pages.

Modify do_tlbies() such that a pseries hypervisor will make the H_TLB_INVALIDATE hcall to notify it's hypervisor of the invalidation of partition scoped translation information which is required to keep the shadow hpt in sync.

Finally allow a pseries hypervisor to run a nested hpt guest by reporting the KVM_CAP_PPC_MMU_HASH_V3 capability and allowing handling of kvmhv_configure_mmu().

---
 arch/powerpc/include/asm/hvcall.h     | 36 --------------
 arch/powerpc/include/asm/kvm_book3s.h |  2 +
 arch/powerpc/include/asm/kvm_host.h   | 55 +++++++++++++++++++++
 arch/powerpc/kvm/book3s_hv.c          | 90 ++++++++++++++++++++++++++++-------
 arch/powerpc/kvm/book3s_hv_nested.c   | 22 ++++++++-
 arch/powerpc/kvm/book3s_hv_rm_mmu.c   | 20 ++++++++
 arch/powerpc/kvm/powerpc.c            |  3 +-
 7 files changed, 173 insertions(+), 55 deletions(-)
arch/powerpc/include/asm/hvcall.h +0 -36
index 11112023e327..19afab30c7d0 100644
@@ -481,42 +481,6 @@ struct h_cpu_char_result {
 	u64 behaviour;
 };
 
-/* Register state for entering a nested guest with H_ENTER_NESTED */
-struct hv_guest_state {
-	u64 version;		/* version of this structure layout */
-	u32 lpid;
-	u32 vcpu_token;
-	/* These registers are hypervisor privileged (at least for writing) */
-	u64 lpcr;
-	u64 pcr;
-	u64 amor;
-	u64 dpdes;
-	u64 hfscr;
-	s64 tb_offset;
-	u64 dawr0;
-	u64 dawrx0;
-	u64 ciabr;
-	u64 hdec_expiry;
-	u64 purr;
-	u64 spurr;
-	u64 ic;
-	u64 vtb;
-	u64 hdar;
-	u64 hdsisr;
-	u64 heir;
-	u64 asdr;
-	/* These are OS privileged but need to be set late in guest entry */
-	u64 srr0;
-	u64 srr1;
-	u64 sprg[4];
-	u64 pidr;
-	u64 cfar;
-	u64 ppr;
-};
-
-/* Latest version of hv_guest_state structure */
-#define HV_GUEST_STATE_VERSION	1
-
 #endif /* __ASSEMBLY__ */
 #endif /* __KERNEL__ */
 #endif /* _ASM_POWERPC_HVCALL_H */
arch/powerpc/include/asm/kvm_book3s.h +2 -0
index c69eeb4e176c..40218e81b75f 100644
@@ -317,6 +317,8 @@ long kvmhv_do_nested_tlbie(struct kvm_vcpu *vcpu);
 int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu,
 			  u64 time_limit, unsigned long lpcr);
 void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr);
+void kvmhv_save_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
+void kvmhv_restore_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
 void kvmhv_restore_hv_return_state(struct kvm_vcpu *vcpu,
 				   struct hv_guest_state *hr);
 long int kvmhv_nested_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu);
arch/powerpc/include/asm/kvm_host.h +55 -0

61 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv.c +74 -16

150 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv_nested.c +21 -1
index 735e0ac6f5b2..68d492e8861e 100644
@@ -51,6 +51,16 @@ void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr)
 	hr->ppr = vcpu->arch.ppr;
 }
 
+void kvmhv_save_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp)
+{
+	int i;
+
+	for (i = 0; i < 64; i++)
+		slbp->slb[i] = vcpu->arch.slb[i];
+	slbp->slb_max = vcpu->arch.slb_max;
+	slbp->slb_nr = vcpu->arch.slb_nr;
+}
+
 static void byteswap_pt_regs(struct pt_regs *regs)
 {
 	unsigned long *addr = (unsigned long *) regs;
@@ -169,6 +179,16 @@ static void restore_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr)
 	vcpu->arch.ppr = hr->ppr;
 }
 
+void kvmhv_restore_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp)
+{
+	int i;
+
+	for (i = 0; i < 64; i++)
+		vcpu->arch.slb[i] = slbp->slb[i];
+	vcpu->arch.slb_max = slbp->slb_max;
+	vcpu->arch.slb_nr = slbp->slb_nr;
+}
+
 void kvmhv_restore_hv_return_state(struct kvm_vcpu *vcpu,
 				   struct hv_guest_state *hr)
 {
@@ -239,7 +259,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 		return H_PARAMETER;
 	if (kvmppc_need_byteswap(vcpu))
 		byteswap_hv_regs(&l2_hv);
-	if (l2_hv.version != HV_GUEST_STATE_VERSION)
+	if (l2_hv.version != 1)
 		return H_P2;
 
 	regs_ptr = kvmppc_get_gpr(vcpu, 5);
arch/powerpc/kvm/book3s_hv_rm_mmu.c +20 -0
index 1d26d509aaf6..53fe51d04d78 100644
@@ -446,6 +446,25 @@ static void do_tlbies(unsigned int lpid, unsigned long *rbvalues,
 	long i;
 
 	/*
+	 * Handle the case where we're running as a nested hypervisor and so
+	 * have to make an hcall to handle invalidations for us.
+	 */
+	if (kvmhv_on_pseries()) {
+		unsigned long rc, ric = 0, prs = 0, r = 0;
+
+		for (i = 0; i < npages; i++) {
+			rc = plpar_hcall_norets(H_TLB_INVALIDATE,
+						H_TLBIE_P1_ENC(ric, prs, r),
+						lpid, rbvalues[i]);
+			if (rc)
+				pr_err("KVM: HPT TLB page invalidation hcall failed"
+					", rc=%ld\n", rc);
+		}
+
+		return;
+	}
+
+	/*
 	 * We use the POWER9 5-operand versions of tlbie and tlbiel here.
 	 * Since we are using RIC=0 PRS=0 R=0, and P7/P8 tlbiel ignores
 	 * the RS field, this is backwards-compatible with P7 and P8.
@@ -1355,3 +1374,4 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,
 
 	return -1;		/* send fault up to host kernel mode */
 }
+EXPORT_SYMBOL_GPL(kvmppc_hpte_hv_fault);
arch/powerpc/kvm/powerpc.c +1 -2
index 3e566c2e6066..b74f794873cd 100644
@@ -604,8 +604,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 		r = !!(hv_enabled && radix_enabled());
 		break;
 	case KVM_CAP_PPC_MMU_HASH_V3:
-		r = !!(hv_enabled && cpu_has_feature(CPU_FTR_ARCH_300) &&
-		       cpu_has_feature(CPU_FTR_HVMODE));
+		r = !!(hv_enabled && cpu_has_feature(CPU_FTR_ARCH_300));
 		break;
 	case KVM_CAP_PPC_NESTED_HV:
 		r = !!(hv_enabled && kvmppc_hv_ops->enable_nested &&
Signature · 1 line
2.13.6
#10 lore raw

PATCH 9/23 KVM: PPC: Book3S HV: Nested: Improve comments and naming of nest rmap functions

In reply to Suraj Jitindar Singh

The nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot.

Improve the naming of some of these functions such that it's clearer what they do, the functions with remove in the name remove the rmap _and_ perform an invalidation so rename them invalidate to reflect this.

kvmhv_insert_nest_rmap() takes a kvm struct as an argument which is unused, so remove this argument.

Additionally improve the function comments and add information about which locks must be held for clarity.

No functional change.

---
 arch/powerpc/include/asm/kvm_book3s_64.h |  4 +--
 arch/powerpc/kvm/book3s_64_mmu_radix.c   |  8 +++---
 arch/powerpc/kvm/book3s_hv_nested.c      | 49 +++++++++++++++++++++++---------
 3 files changed, 42 insertions(+), 19 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +2 -2
index bb7c8cc77f1a..bec78f15e2f5 100644
@@ -624,12 +624,12 @@ extern int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
 			     unsigned long gpa, unsigned int level,
 			     unsigned long mmu_seq, unsigned int lpid,
 			     unsigned long *rmapp, struct rmap_nested **n_rmap);
-extern void kvmhv_insert_nest_rmap(struct kvm *kvm, unsigned long *rmapp,
+extern void kvmhv_insert_nest_rmap(unsigned long *rmapp,
 				   struct rmap_nested **n_rmap);
 extern void kvmhv_update_nest_rmap_rc_list(struct kvm *kvm, unsigned long *rmapp,
 					   unsigned long clr, unsigned long set,
 					   unsigned long hpa, unsigned long nbytes);
-extern void kvmhv_remove_nest_rmap_range(struct kvm *kvm,
+extern void kvmhv_invalidate_nest_rmap_range(struct kvm *kvm,
 				const struct kvm_memory_slot *memslot,
 				unsigned long gpa, unsigned long hpa,
 				unsigned long nbytes);
arch/powerpc/kvm/book3s_64_mmu_radix.c +4 -4
index 310d8dde9a48..48b844d33dc9 100644
@@ -405,7 +405,7 @@ void kvmppc_unmap_pte(struct kvm *kvm, pte_t *pte, unsigned long gpa,
 
 	gpa &= ~(page_size - 1);
 	hpa = old & PTE_RPN_MASK;
-	kvmhv_remove_nest_rmap_range(kvm, memslot, gpa, hpa, page_size);
+	kvmhv_invalidate_nest_rmap_range(kvm, memslot, gpa, hpa, page_size);
 
 	if ((old & _PAGE_DIRTY) && memslot->dirty_bitmap)
 		kvmppc_update_dirty_map(memslot, gfn, page_size);
@@ -643,7 +643,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
 		}
 		kvmppc_radix_set_pte_at(kvm, gpa, (pte_t *)pud, pte);
 		if (rmapp && n_rmap)
-			kvmhv_insert_nest_rmap(kvm, rmapp, n_rmap);
+			kvmhv_insert_nest_rmap(rmapp, n_rmap);
 		ret = 0;
 		goto out_unlock;
 	}
@@ -695,7 +695,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
 		}
 		kvmppc_radix_set_pte_at(kvm, gpa, pmdp_ptep(pmd), pte);
 		if (rmapp && n_rmap)
-			kvmhv_insert_nest_rmap(kvm, rmapp, n_rmap);
+			kvmhv_insert_nest_rmap(rmapp, n_rmap);
 		ret = 0;
 		goto out_unlock;
 	}
@@ -721,7 +721,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte,
 	}
 	kvmppc_radix_set_pte_at(kvm, gpa, ptep, pte);
 	if (rmapp && n_rmap)
-		kvmhv_insert_nest_rmap(kvm, rmapp, n_rmap);
+		kvmhv_insert_nest_rmap(rmapp, n_rmap);
 	ret = 0;
 
  out_unlock:
arch/powerpc/kvm/book3s_hv_nested.c +36 -13

100 lines not on this page · raw message

Signature · 1 line
2.13.6
#11 lore raw

PATCH 10/23 KVM: PPC: Book3S HV: Nested: Increase gpa field in nest rmap to 46 bits

In reply to Suraj Jitindar Singh

The nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot.

Increase the size of the gpa (guest physical address) field in the nested rmap entry to 46 bits such that it can be reused to store a nested hpt (hash page table) guest entry where this field will hold the hpt index (which can be up to 46 bits).

Additionally introduce helper functions to access these bit fields for simplicity.

No functional change.

---
 arch/powerpc/include/asm/kvm_book3s_64.h |  5 ++--
 arch/powerpc/kvm/book3s_hv_nested.c      | 41 ++++++++++++++++++++++++--------
 2 files changed, 34 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +3 -2
index bec78f15e2f5..ef6af64a4451 100644
@@ -50,12 +50,13 @@ struct kvm_nested_guest {
 /*
  * We define a nested rmap entry as a single 64-bit quantity
  * 0xFFF0000000000000	12-bit lpid field
- * 0x000FFFFFFFFFF000	40-bit guest 4k page frame number
+ * 0x000FFFFFFFFFFFC0	46-bit guest page frame number (radix) or hpt index
  * 0x0000000000000001	1-bit  single entry flag
  */
 #define RMAP_NESTED_LPID_MASK		0xFFF0000000000000UL
 #define RMAP_NESTED_LPID_SHIFT		(52)
-#define RMAP_NESTED_GPA_MASK		0x000FFFFFFFFFF000UL
+#define RMAP_NESTED_GPA_MASK		0x000FFFFFFFFFFFC0UL
+#define RMAP_NESTED_GPA_SHIFT		(6)
 #define RMAP_NESTED_IS_SINGLE_ENTRY	0x0000000000000001UL
 
 /* Structure for a nested guest rmap entry */
arch/powerpc/kvm/book3s_hv_nested.c +31 -10

72 lines not on this page · raw message

Signature · 1 line
2.13.6
#12 lore raw

PATCH 11/23 KVM: PPC: Book3S HV: Nested: Remove single nest rmap entries

In reply to Suraj Jitindar Singh

The nested rmap entries are used to track nested pages which map a given guest page such that that information can be retrieved from the guest memslot. These entries are stored in the guest memslot as a singly linked list (llist) with the next pointer of the last entry in the list used to store a "single entry" to save having to add another list entry.

This approach while saving a small amount of memory significantly complicates the list handling. For simplicity and code clarity remove the existence of these "single entries" and always insert another list entry to hold the final value.

No functional change.

---
 arch/powerpc/include/asm/kvm_book3s_64.h | 65 ++++++++++----------------------
 arch/powerpc/kvm/book3s_hv_nested.c      | 51 ++++++++++---------------
 2 files changed, 39 insertions(+), 77 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +19 -46
index ef6af64a4451..410e609efd37 100644
@@ -48,66 +48,39 @@ struct kvm_nested_guest {
 };
 
 /*
+ * We use nested rmap entries to store information so that we can find pte
+ * entries in the shadow page table of a nested guest when the host is modifying
+ * a pte for a l1 guest page. For a radix nested guest this is the nested
+ * gpa which can then be used to walk the radix shadow page table to find a
+ * pte. For a hash nested guest this is an index into the shadow hpt which can
+ * be used to find a pte. Irrespective the lpid of the nested guest is stored.
+ *
+ * These entries are stored in a rmap_nested struct. There may be multiple
+ * entries for a single l1 guest page since that guest may have multiple nested
+ * guests and map the same page into more than 1, or a single nested guest may
+ * map the same l1 guest page with multiple hpt entries. To accommodate this
+ * the rmap_nested entries are linked together in a singly linked list with the
+ * corresponding rmap entry in the rmap array of the memslot containing the
+ * head pointer of the linked list (or NULL if list is empty).
+ */
+
+/*
  * We define a nested rmap entry as a single 64-bit quantity
  * 0xFFF0000000000000	12-bit lpid field
  * 0x000FFFFFFFFFFFC0	46-bit guest page frame number (radix) or hpt index
- * 0x0000000000000001	1-bit  single entry flag
+ * 0x000000000000003F	6-bit unused field
  */
 #define RMAP_NESTED_LPID_MASK		0xFFF0000000000000UL
 #define RMAP_NESTED_LPID_SHIFT		(52)
 #define RMAP_NESTED_GPA_MASK		0x000FFFFFFFFFFFC0UL
 #define RMAP_NESTED_GPA_SHIFT		(6)
-#define RMAP_NESTED_IS_SINGLE_ENTRY	0x0000000000000001UL
 
 /* Structure for a nested guest rmap entry */
 struct rmap_nested {
 	struct llist_node list;
-	u64 rmap;
+	u64 rmap;			/* layout defined above */
 };
 
-/*
- * for_each_nest_rmap_safe - iterate over the list of nested rmap entries
- *			     safe against removal of the list entry or NULL list
- * @pos:	a (struct rmap_nested *) to use as a loop cursor
- * @node:	pointer to the first entry
- *		NOTE: this can be NULL
- * @rmapp:	an (unsigned long *) in which to return the rmap entries on each
- *		iteration
- *		NOTE: this must point to already allocated memory
- *
- * The nested_rmap is a llist of (struct rmap_nested) entries pointed to by the
- * rmap entry in the memslot. The list is always terminated by a "single entry"
- * stored in the list element of the final entry of the llist. If there is ONLY
- * a single entry then this is itself in the rmap entry of the memslot, not a
- * llist head pointer.
- *
- * Note that the iterator below assumes that a nested rmap entry is always
- * non-zero.  This is true for our usage because the LPID field is always
- * non-zero (zero is reserved for the host).
- *
- * This should be used to iterate over the list of rmap_nested entries with
- * processing done on the u64 rmap value given by each iteration. This is safe
- * against removal of list entries and it is always safe to call free on (pos).
- *
- * e.g.
- * struct rmap_nested *cursor;
- * struct llist_node *first;
- * unsigned long rmap;
- * for_each_nest_rmap_safe(cursor, first, &rmap) {
- *	do_something(rmap);
- *	free(cursor);
- * }
- */
-#define for_each_nest_rmap_safe(pos, node, rmapp)			       \
-	for ((pos) = llist_entry((node), typeof(*(pos)), list);		       \
-	     (node) &&							       \
-	     (*(rmapp) = ((RMAP_NESTED_IS_SINGLE_ENTRY & ((u64) (node))) ?     \
-			  ((u64) (node)) : ((pos)->rmap))) &&		       \
-	     (((node) = ((RMAP_NESTED_IS_SINGLE_ENTRY & ((u64) (node))) ?      \
-			 ((struct llist_node *) ((pos) = NULL)) :	       \
-			 (pos)->list.next)), true);			       \
-	     (pos) = llist_entry((node), typeof(*(pos)), list))
-
 struct kvm_nested_guest *kvmhv_get_nested(struct kvm *kvm, int l1_lpid,
 					  bool create);
 void kvmhv_put_nested(struct kvm_nested_guest *gp);
arch/powerpc/kvm/book3s_hv_nested.c +20 -31

91 lines not on this page · raw message

Signature · 1 line
2.13.6
#13 lore raw

PATCH 12/23 KVM: PPC: Book3S HV: Nested: add kvmhv_remove_all_nested_rmap_lpid()

In reply to Suraj Jitindar Singh

Nested rmap entries are used to store a reverse mapping from a L1 guest page in a kvm memslot to a nested guest pte.

Implement a function to remove all nest rmap entries of a given lpid from all of the memslots for a given guest.

---
 arch/powerpc/kvm/book3s_hv_nested.c | 47 +++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
arch/powerpc/kvm/book3s_hv_nested.c +47 -0
index c76e499437ee..58a5de2aa2af 100644
@@ -819,6 +819,53 @@ void kvmhv_insert_nest_rmap(unsigned long *rmapp, struct rmap_nested **n_rmap)
 	*n_rmap = NULL;
 }
 
+/* called with kvm->mmu_lock held */
+static void kvmhv_remove_nested_rmap_lpid(unsigned long *rmapp, int l1_lpid)
+{
+	struct llist_node **next = &(((struct llist_head *) rmapp)->first);
+	u64 match = lpid_to_n_rmap(l1_lpid);
+
+	while (*next) {
+		struct llist_node *entry = (*next);
+		struct rmap_nested *n_rmap = llist_entry(entry, typeof(*n_rmap),
+							 list);
+
+		if (kvmhv_n_rmap_is_equal(match, n_rmap->rmap,
+					  RMAP_NESTED_LPID_MASK)) {
+			*next = entry->next;
+			kfree(n_rmap);
+		} else {
+			next = &(entry->next);
+		}
+	}
+}
+
+/*
+ * caller must hold gp->tlb_lock
+ * For a given nested lpid, remove all of the rmap entries which match that
+ * nest lpid. Note that no invalidation/tlbie is done for the entries, it is
+ * assumed that the caller will perform an lpid wide invalidation after calling
+ * this function.
+ */
+static void kvmhv_remove_all_nested_rmap_lpid(struct kvm *kvm, int l1_lpid)
+{
+	struct kvm_memory_slot *memslot;
+
+	kvm_for_each_memslot(memslot, kvm_memslots(kvm)) {
+		unsigned long page;
+
+		for (page = 0; page < memslot->npages; page++) {
+			unsigned long *rmapp;
+
+			spin_lock(&kvm->mmu_lock);
+			rmapp = &memslot->arch.rmap[page];
+			if (*rmapp) /* Are there any rmap entries? */
+				kvmhv_remove_nested_rmap_lpid(rmapp, l1_lpid);
+			spin_unlock(&kvm->mmu_lock);
+		}
+	}
+}
+
 /*
  * called with kvm->mmu_lock held
  * Given a single rmap entry, update the rc bits in the corresponding shadow
Signature · 1 line
2.13.6
#14 lore raw

PATCH 13/23 KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup

In reply to Suraj Jitindar Singh

Add the infrastructure to book3s_hv_nested.c to allow a nested hpt (hash page table) guest to be setup. As this patch doesn't add the capability of creating or removing mmu translations return H_PARAMETER when an attempt to actually run a nested hpt guest is made.

Add fields to the nested guest struct to store the hpt and the vrma slb entry.

Update kvmhv_update_ptbl_cache() to determine when a nested guest is switching from radix to hpt or hpt to radix and perform the required setup. A page table (radix) or hpt (hash) must be allocated with any existing table being freed and the radix field in the nested guest struct being updated under the mmu_lock (this means that when holding the mmu_lock the radix field can be tested and the existance of the correct type of page table guaranteed). Also remove all of the nest rmap entries which belong to this nested guest since a nested rmap entry is specific to whether the nested guest is hash or radix.

When a nested guest is initially created or when the partition table entry is empty we assume a radix guest since it is much less expensive to allocate a radix page table compared to a hpt.

The hpt which is allocated in the hypervisor for the nested guest (called the shadow hpt) is identical in size to the one allocated in the guest hypervisor to ensure a 1-to-1 mapping between page table entries. This simplifies handling of the entries however this requirement could be relaxed in future if support was added.

Introduce a hash nested_page_fault function to be envoked when the nested guest which experiences a page fault is hash, returns -EINVAL for now. Also return -EINVAL when handling the H_TLB_INVALIDATE hcall. Also lacking support for the hypervisor paging out a guest page which has been mapped through to a nested guest. These 3 portions of functionality added in proceeding patches.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |   2 +
 arch/powerpc/include/asm/book3s/64/mmu.h      |   9 +
 arch/powerpc/include/asm/kvm_book3s_64.h      |   4 +-
 arch/powerpc/kvm/book3s_hv_nested.c           | 255 ++++++++++++++++++++++----
 4 files changed, 235 insertions(+), 35 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +2 -0
index 15b75005bc34..c04e37b2c30d 100644
@@ -45,12 +45,14 @@
 #define SLB_VSID_KP		ASM_CONST(0x0000000000000400)
 #define SLB_VSID_N		ASM_CONST(0x0000000000000200) /* no-execute */
 #define SLB_VSID_L		ASM_CONST(0x0000000000000100)
+#define SLB_VSID_L_SHIFT	8
 #define SLB_VSID_C		ASM_CONST(0x0000000000000080) /* class */
 #define SLB_VSID_LP		ASM_CONST(0x0000000000000030)
 #define SLB_VSID_LP_00		ASM_CONST(0x0000000000000000)
 #define SLB_VSID_LP_01		ASM_CONST(0x0000000000000010)
 #define SLB_VSID_LP_10		ASM_CONST(0x0000000000000020)
 #define SLB_VSID_LP_11		ASM_CONST(0x0000000000000030)
+#define SLB_VSID_LP_SHIFT	4
 #define SLB_VSID_LLP		(SLB_VSID_L|SLB_VSID_LP)
 
 #define SLB_VSID_KERNEL		(SLB_VSID_KP)
arch/powerpc/include/asm/book3s/64/mmu.h +9 -0
index 23b83d3593e2..8c02e40f1125 100644
@@ -62,6 +62,7 @@ struct patb_entry {
 extern struct patb_entry *partition_tb;
 
 /* Bits in patb0 field */
+/* Radix */
 #define PATB_HR		(1UL << 63)
 #define RPDB_MASK	0x0fffffffffffff00UL
 #define RPDB_SHIFT	(1UL << 8)
@@ -70,6 +71,14 @@ extern struct patb_entry *partition_tb;
 #define RTS2_SHIFT	5		/* bottom 3 bits of radix tree size */
 #define RTS2_MASK	(7UL << RTS2_SHIFT)
 #define RPDS_MASK	0x1f		/* root page dir. size field */
+/* Hash */
+#define PATB_HTABORG	0x0ffffffffffc0000UL	/* hpt base */
+#define PATB_PS		0xe0			/* page size */
+#define PATB_PS_L	0x80
+#define PATB_PS_L_SHIFT	7
+#define PATB_PS_LP	0x60
+#define PATB_PS_LP_SHIFT	5
+#define PATB_HTABSIZE	0x1f			/* hpt size */
 
 /* Bits in patb1 field */
 #define PATB_GR		(1UL << 63)	/* guest uses radix; must match HR */
arch/powerpc/include/asm/kvm_book3s_64.h +3 -1
index 410e609efd37..c874ab3a037e 100644
@@ -35,7 +35,9 @@ struct kvm_nested_guest {
 	struct kvm *l1_host;		/* L1 VM that owns this nested guest */
 	int l1_lpid;			/* lpid L1 guest thinks this guest is */
 	int shadow_lpid;		/* real lpid of this nested guest */
-	pgd_t *shadow_pgtable;		/* our page table for this guest */
+	pgd_t *shadow_pgtable;		/* page table for this guest if radix */
+	struct kvm_hpt_info shadow_hpt;	/* hpt for this guest if hash */
+	u64 vrma_slb_v;			/* vrma slb for this guest if hash */
 	u64 l1_gr_to_hr;		/* L1's addr of part'n-scoped table */
 	u64 process_table;		/* process table entry for this guest */
 	long refcnt;			/* number of pointers to this struct */
arch/powerpc/kvm/book3s_hv_nested.c +221 -34

430 lines not on this page · raw message

Signature · 1 line
2.13.6
#15 lore raw

Re: [PATCH 13/23] KVM: PPC: Book3S HV: Nested: Infrastructure for nested hpt guest setup

Quoted 28 lines · Suraj Jitindar Singh

Add the infrastructure to book3s_hv_nested.c to allow a nested hpt (hash page table) guest to be setup. As this patch doesn't add the capability of creating or removing mmu translations return H_PARAMETER when an attempt to actually run a nested hpt guest is made.

Add fields to the nested guest struct to store the hpt and the vrma slb entry.

Update kvmhv_update_ptbl_cache() to determine when a nested guest is switching from radix to hpt or hpt to radix and perform the required setup. A page table (radix) or hpt (hash) must be allocated with any existing table being freed and the radix field in the nested guest struct being updated under the mmu_lock (this means that when holding the mmu_lock the radix field can be tested and the existance of the correct type of page table guaranteed). Also remove all of the nest rmap entries which belong to this nested guest since a nested rmap entry is specific to whether the nested guest is hash or radix.

When a nested guest is initially created or when the partition table entry is empty we assume a radix guest since it is much less expensive to allocate a radix page table compared to a hpt.

The hpt which is allocated in the hypervisor for the nested guest (called the shadow hpt) is identical in size to the one allocated in the guest hypervisor to ensure a 1-to-1 mapping between page table entries. This simplifies handling of the entries however this requirement could be relaxed in future if support was added.

Quoted 8 lines

Introduce a hash nested_page_fault function to be envoked when the nested guest which experiences a page fault is hash, returns -EINVAL for now. Also return -EINVAL when handling the H_TLB_INVALIDATE hcall. Also lacking support for the hypervisor paging out a guest page which has been mapped through to a nested guest. These 3 portions of functionality added in proceeding patches.

Small nit below...

Quoted 13 lines
arch/powerpc/kvm/book3s_hv_nested.c +13 -0
+/* Caller must hold gp->tlb_lock */
+static int kvmhv_switch_to_radix_nested(struct kvm_nested_guest *gp)
+{
+	struct kvm *kvm = gp->l1_host;
+	pgd_t *pgtable;
+
+	/* try to allocate a radix tree */
+	pgtable = pgd_alloc(kvm->mm);
+	if (!pgtable) {
+		pr_err_ratelimited("KVM: Couldn't alloc nested radix tree\n");
+		return -ENOMEM;
+	}
+

arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 642

Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10 -0
+	/* mmu_lock protects shadow_hpt & radix in nested guest struct */
+	spin_lock(&kvm->mmu_lock);
+	kvmppc_free_hpt(&gp->shadow_hpt);
+	gp->radix = 1;
+	gp->shadow_pgtable = pgtable;
+	spin_unlock(&kvm->mmu_lock);
+
+	/* remove all nested rmap entries and perform global invalidation */
+	kvmhv_remove_all_nested_rmap_lpid(kvm, gp->l1_lpid);
+	kvmhv_flush_lpid(gp->shadow_lpid, gp->radix);

Shouldn't this flush be done using the old value of gp->radix, i.e. 0? Both because we want to flush the old translations for the guest, and because we haven't changed the partition table entry for the guest at this point, so it still says HPT.

Quoted 20 lines
arch/powerpc/kvm/book3s_hv_nested.c +20 -0
+
+	return 0;
+}
+
+/* Caller must hold gp->tlb_lock */
+static int kvmhv_switch_to_hpt_nested(struct kvm_nested_guest *gp, int order)
+{
+	struct kvm *kvm = gp->l1_host;
+	struct kvm_hpt_info info;
+	int rc;
+
+	/* try to allocate an hpt */
+	rc = kvmppc_allocate_hpt(&info, order);
+	if (rc) {
+		pr_err_ratelimited("KVM: Couldn't alloc nested hpt\n");
+		return rc;
+	}
+
+	/* mmu_lock protects shadow_pgtable & radix in nested guest struct */
+	spin_lock(&kvm->mmu_lock);

arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 672

Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10 -0
+	kvmppc_free_pgtable_radix(kvm, gp->shadow_pgtable, gp->shadow_lpid);
+	pgd_free(kvm->mm, gp->shadow_pgtable);
+	gp->shadow_pgtable = NULL;
+	gp->radix = 0;
+	gp->shadow_hpt = info;
+	spin_unlock(&kvm->mmu_lock);
+
+	/* remove all nested rmap entries and perform global invalidation */
+	kvmhv_remove_all_nested_rmap_lpid(kvm, gp->l1_lpid);
+	kvmhv_flush_lpid(gp->shadow_lpid, gp->radix);

Similarly, shouldn't this be a radix flush?

arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_copy_tofrom_guest_nested(struct kvm_vcpu *vcpu) · line 645

Quoted 3 lines
arch/powerpc/kvm/book3s_hv_nested.c +3 -0
+
+	return 0;
+}

Paul.

#16 lore raw

PATCH 14/23 KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest

In reply to Suraj Jitindar Singh

A version 2 of the H_ENTER_NESTED hcall was added with an argument to specify the slb entries which should be used to run the nested guest.

Add support for this version of the hcall structures to kvmhv_enter_nested_guest() and context switch the slb when the nested guest being run is a hpt (hash page table) guest.

---
 arch/powerpc/kvm/book3s_hv_nested.c | 84 ++++++++++++++++++++++++++++++++++---
 1 file changed, 79 insertions(+), 5 deletions(-)
arch/powerpc/kvm/book3s_hv_nested.c +79 -5
index 82690eafee77..883f8896ed60 100644
@@ -104,6 +104,28 @@ static void byteswap_hv_regs(struct hv_guest_state *hr)
 	hr->ppr = swab64(hr->ppr);
 }
 
+static void byteswap_guest_slb(struct guest_slb *slbp)
+{
+	int i;
+
+	for (i = 0; i < 64; i++) {
+		slbp->slb[i].esid = swab64(slbp->slb[i].esid);
+		slbp->slb[i].vsid = swab64(slbp->slb[i].vsid);
+		slbp->slb[i].orige = swab64(slbp->slb[i].orige);
+		slbp->slb[i].origv = swab64(slbp->slb[i].origv);
+		slbp->slb[i].valid = swab32(slbp->slb[i].valid);
+		slbp->slb[i].Ks = swab32(slbp->slb[i].Ks);
+		slbp->slb[i].Kp = swab32(slbp->slb[i].Kp);
+		slbp->slb[i].nx = swab32(slbp->slb[i].nx);
+		slbp->slb[i].large = swab32(slbp->slb[i].large);
+		slbp->slb[i].tb = swab32(slbp->slb[i].tb);
+		slbp->slb[i].class = swab32(slbp->slb[i].class);
+		/* base_page_size is u8 thus no need to byteswap */
+	}
+	slbp->slb_max = swab64(slbp->slb_max);
+	slbp->slb_nr = swab64(slbp->slb_nr);
+}
+
 static void save_hv_return_state(struct kvm_vcpu *vcpu, int trap,
 				 struct hv_guest_state *hr)
 {
@@ -238,12 +260,13 @@ static void kvmhv_nested_mmio_needed(struct kvm_vcpu *vcpu, u64 regs_ptr)
 
 long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 {
-	long int err, r;
+	long int err, r, ret = H_SUCCESS;
 	struct kvm_nested_guest *l2;
 	struct pt_regs l2_regs, saved_l1_regs;
 	struct hv_guest_state l2_hv, saved_l1_hv;
+	struct guest_slb *l2_slb = NULL, *saved_l1_slb = NULL;
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
-	u64 hv_ptr, regs_ptr;
+	u64 hv_ptr, regs_ptr, slb_ptr = 0UL;
 	u64 hdec_exp;
 	s64 delta_purr, delta_spurr, delta_ic, delta_vtb;
 	u64 mask;
@@ -261,7 +284,9 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 		return H_PARAMETER;
 	if (kvmppc_need_byteswap(vcpu))
 		byteswap_hv_regs(&l2_hv);
-	if (l2_hv.version != 1)
+	/* Do we support the guest version of the argument structures */
+	if ((l2_hv.version > HV_GUEST_STATE_MAX_VERSION) ||
+			(l2_hv.version < HV_GUEST_STATE_MIN_VERSION))
 		return H_P2;
 
 	regs_ptr = kvmppc_get_gpr(vcpu, 5);
@@ -296,6 +321,9 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 			return H_PARAMETER;
 	} else {
 		return H_PARAMETER;
+		/* must be at least V2 to support hpt guest */
+		if (l2_hv.version < 2)
+			return H_PARAMETER;
 		/* hpt doesn't support gtse or uprt and required vpm */
 		if ((l2_hv.lpcr & LPCR_HR) || (l2_hv.lpcr & LPCR_GTSE) ||
 					      (l2_hv.lpcr & LPCR_UPRT) ||
@@ -307,6 +335,26 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	vcpu->arch.regs.msr = vcpu->arch.shregs.msr;
 	saved_l1_regs = vcpu->arch.regs;
 	kvmhv_save_hv_regs(vcpu, &saved_l1_hv);
+	/* if running hpt then context switch the slb in the vcpu struct */
+	if (!radix) {
+		slb_ptr = kvmppc_get_gpr(vcpu, 6);
+		l2_slb = kzalloc(sizeof(*l2_slb), GFP_KERNEL);
+		saved_l1_slb = kzalloc(sizeof(*saved_l1_slb), GFP_KERNEL);
+
+		if ((!l2_slb) || (!saved_l1_slb)) {
+			ret = H_HARDWARE;
+			goto out_free;
+		}
+		err = kvm_vcpu_read_guest(vcpu, slb_ptr, l2_slb,
+					  sizeof(struct guest_slb));
+		if (err) {
+			ret = H_PARAMETER;
+			goto out_free;
+		}
+		if (kvmppc_need_byteswap(vcpu))
+			byteswap_guest_slb(l2_slb);
+		kvmhv_save_guest_slb(vcpu, saved_l1_slb);
+	}
 
 	/* convert TB values/offsets to host (L0) values */
 	hdec_exp = l2_hv.hdec_expiry - vc->tb_offset;
@@ -323,6 +371,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	lpcr = (vc->lpcr & ~mask) | (l2_hv.lpcr & mask);
 	sanitise_hv_regs(vcpu, &l2_hv);
 	restore_hv_regs(vcpu, &l2_hv);
+	if (!radix)
+		kvmhv_restore_guest_slb(vcpu, l2_slb);
 
 	vcpu->arch.ret = RESUME_GUEST;
 	vcpu->arch.trap = 0;
@@ -332,8 +382,11 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 			r = RESUME_HOST;
 			break;
 		}
-		r = kvmhv_run_single_vcpu(vcpu->arch.kvm_run, vcpu, hdec_exp,
-					  lpcr);
+		if (radix)
+			r = kvmhv_run_single_vcpu(vcpu->arch.kvm_run, vcpu,
+						  hdec_exp, lpcr);
+		else
+			r = RESUME_HOST; /* XXX TODO hpt entry path */
 	} while (is_kvmppc_resume_guest(r));
 
 	/* save L2 state for return */
@@ -344,6 +397,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	delta_ic = vcpu->arch.ic - l2_hv.ic;
 	delta_vtb = vc->vtb - l2_hv.vtb;
 	save_hv_return_state(vcpu, vcpu->arch.trap, &l2_hv);
+	if (!radix)
+		kvmhv_save_guest_slb(vcpu, l2_slb);
 
 	/* restore L1 state */
 	vcpu->arch.nested = NULL;
@@ -354,6 +409,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 		vcpu->arch.shregs.msr |= MSR_TS_S;
 	vc->tb_offset = saved_l1_hv.tb_offset;
 	restore_hv_regs(vcpu, &saved_l1_hv);
+	if (!radix)
+		kvmhv_restore_guest_slb(vcpu, saved_l1_slb);
 	vcpu->arch.purr += delta_purr;
 	vcpu->arch.spurr += delta_spurr;
 	vcpu->arch.ic += delta_ic;
@@ -363,9 +420,21 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 
 	/* copy l2_hv_state and regs back to guest */
 	if (kvmppc_need_byteswap(vcpu)) {
+		if (!radix)
+			byteswap_guest_slb(l2_slb);
 		byteswap_hv_regs(&l2_hv);
 		byteswap_pt_regs(&l2_regs);
 	}
+	if (!radix) {
+		err = kvm_vcpu_write_guest(vcpu, slb_ptr, l2_slb,
+					   sizeof(struct guest_slb));
+		if (err) {
+			ret = H_AUTHORITY;
+			goto out_free;
+		}
+		kfree(l2_slb);
+		kfree(saved_l1_slb);
+	}
 	err = kvm_vcpu_write_guest(vcpu, hv_ptr, &l2_hv,
 				   sizeof(struct hv_guest_state));
 	if (err)
@@ -384,6 +453,11 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	}
 
 	return vcpu->arch.trap;
+
+out_free:
+	kfree(l2_slb);
+	kfree(saved_l1_slb);
+	return ret;
 }
 
 long kvmhv_nested_init(void)
Signature · 1 line
2.13.6
#17 lore raw

Re: [PATCH 14/23] KVM: PPC: Book3S HV: Nested: Context switch slb for nested hpt guest

Quoted 8 lines · Suraj Jitindar Singh

A version 2 of the H_ENTER_NESTED hcall was added with an argument to specify the slb entries which should be used to run the nested guest.

Add support for this version of the hcall structures to kvmhv_enter_nested_guest() and context switch the slb when the nested guest being run is a hpt (hash page table) guest.

Question below...

Quoted 13 lines
arch/powerpc/kvm/book3s_hv_nested.c +9 -0
@@ -307,6 +335,26 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	vcpu->arch.regs.msr = vcpu->arch.shregs.msr;
 	saved_l1_regs = vcpu->arch.regs;
 	kvmhv_save_hv_regs(vcpu, &saved_l1_hv);
+	/* if running hpt then context switch the slb in the vcpu struct */
+	if (!radix) {
+		slb_ptr = kvmppc_get_gpr(vcpu, 6);
+		l2_slb = kzalloc(sizeof(*l2_slb), GFP_KERNEL);
+		saved_l1_slb = kzalloc(sizeof(*saved_l1_slb), GFP_KERNEL);
+
+		if ((!l2_slb) || (!saved_l1_slb)) {
+			ret = H_HARDWARE;
+			goto out_free;

arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) · line 356

Quoted 10 lines
arch/powerpc/kvm/book3s_hv_nested.c +10 -0
+		}
+		err = kvm_vcpu_read_guest(vcpu, slb_ptr, l2_slb,
+					  sizeof(struct guest_slb));
+		if (err) {
+			ret = H_PARAMETER;
+			goto out_free;
+		}
+		if (kvmppc_need_byteswap(vcpu))
+			byteswap_guest_slb(l2_slb);
+		kvmhv_save_guest_slb(vcpu, saved_l1_slb);

Why are we bothering to save the SLB state of the L1 guest, which has to be a radix guest? Won't the L1 SLB state always just have 0 entries?

arch/powerpc/kvm/book3s_hv_nested.c long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu) · line 413

Quoted 6 lines
arch/powerpc/kvm/book3s_hv_nested.c +2 -0
@@ -354,6 +409,8 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 		vcpu->arch.shregs.msr |= MSR_TS_S;
 	vc->tb_offset = saved_l1_hv.tb_offset;
 	restore_hv_regs(vcpu, &saved_l1_hv);
+	if (!radix)
+		kvmhv_restore_guest_slb(vcpu, saved_l1_slb);

Likewise here can't we just set vcpu->arch.slb_max and vcpu->arch.slb_nr to zero?

Paul.

#18 lore raw

PATCH 15/23 KVM: PPC: Book3S HV: Store lpcr and hdec_exp in the vcpu struct

In reply to Suraj Jitindar Singh

When running a single vcpu with kvmhv_run_single_vcpu() the lpcr and hypervisor decrementer expiry are passed as function arguments. When running a vcore with kvmppc_run_vcpu() the lpcr is taken from the vcore and there is no need to consider the hypervisor decrementer expiry as it only applies when running a nested guest.

These fields will need to be accessed in the guest entry path in book3s_hv_rmhandlers.S when running a nested hpt (hash page table) guest. To allow for this store the lpcr and hdec_exp in the vcpu struct.

No functional change.

---
 arch/powerpc/include/asm/kvm_book3s.h |  3 +--
 arch/powerpc/include/asm/kvm_host.h   |  2 ++
 arch/powerpc/kvm/book3s_hv.c          | 40 +++++++++++++++++------------------
 arch/powerpc/kvm/book3s_hv_nested.c   | 10 ++++-----
 4 files changed, 27 insertions(+), 28 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -2
index 40218e81b75f..e1dc1872e453 100644
@@ -314,8 +314,7 @@ void kvmhv_set_ptbl_entry(unsigned int lpid, u64 dw0, u64 dw1);
 void kvmhv_release_all_nested(struct kvm *kvm);
 long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu);
 long kvmhv_do_nested_tlbie(struct kvm_vcpu *vcpu);
-int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu,
-			  u64 time_limit, unsigned long lpcr);
+int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu);
 void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr);
 void kvmhv_save_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
 void kvmhv_restore_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
arch/powerpc/include/asm/kvm_host.h +2 -0
index bad09c213be6..b092701951ee 100644
@@ -793,10 +793,12 @@ struct kvm_vcpu_arch {
 
 	u32 online;
 
+	unsigned long lpcr;
 	/* For support of nested guests */
 	struct kvm_nested_guest *nested;
 	u32 nested_vcpu_id;
 	gpa_t nested_io_gpr;
+	u64 hdec_exp;
 #endif
 
 #ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING
arch/powerpc/kvm/book3s_hv.c +19 -21
index be72bc6b4cd5..8407071d5e22 100644
@@ -3429,8 +3429,7 @@ static noinline void kvmppc_run_core(struct kvmppc_vcore *vc)
 /*
  * Handle making the H_ENTER_NESTED hcall if we're pseries.
  */
-static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit,
-				     unsigned long lpcr)
+static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit)
 {
 	/* call our hypervisor to load up HV regs and go */
 	struct hv_guest_state hvregs;
@@ -3454,7 +3453,7 @@ static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit,
 	host_psscr = mfspr(SPRN_PSSCR_PR);
 	mtspr(SPRN_PSSCR_PR, vcpu->arch.psscr);
 	kvmhv_save_hv_regs(vcpu, &hvregs);
-	hvregs.lpcr = lpcr;
+	hvregs.lpcr = vcpu->arch.lpcr;
 	vcpu->arch.regs.msr = vcpu->arch.shregs.msr;
 	if (vcpu->arch.nested) {
 		hvregs.lpid = vcpu->arch.nested->shadow_lpid;
@@ -3536,8 +3535,7 @@ static int kvmhv_pseries_enter_guest(struct kvm_vcpu *vcpu, u64 time_limit,
  * CPU_FTR_HVMODE is set. This is only used for radix guests, however that
  * radix guest may be a direct guest of this hypervisor or a nested guest.
  */
-static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,
-				     unsigned long lpcr)
+static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit)
 {
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
 	s64 hdec;
@@ -3594,7 +3592,7 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,
 
 	mtspr(SPRN_AMOR, ~0UL);
 
-	mtspr(SPRN_LPCR, lpcr);
+	mtspr(SPRN_LPCR, vcpu->arch.lpcr);
 	isync();
 
 	kvmppc_xive_push_vcpu(vcpu);
@@ -3666,8 +3664,7 @@ static int kvmhv_load_hv_regs_and_go(struct kvm_vcpu *vcpu, u64 time_limit,
  * Virtual-mode guest entry for POWER9 and later when the host and
  * guest are both using the radix MMU.  The LPIDR has already been set.
  */
-int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,
-			 unsigned long lpcr)
+int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu)
 {
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
 	unsigned long host_dscr = mfspr(SPRN_DSCR);
@@ -3675,7 +3672,7 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,
 	unsigned long host_iamr = mfspr(SPRN_IAMR);
 	unsigned long host_amr = mfspr(SPRN_AMR);
 	s64 dec;
-	u64 tb;
+	u64 tb, time_limit;
 	int trap, save_pmu;
 
 	dec = mfspr(SPRN_DEC);
@@ -3683,8 +3680,8 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,
 	if (dec < 512)
 		return BOOK3S_INTERRUPT_HV_DECREMENTER;
 	local_paca->kvm_hstate.dec_expires = dec + tb;
-	if (local_paca->kvm_hstate.dec_expires < time_limit)
-		time_limit = local_paca->kvm_hstate.dec_expires;
+	time_limit = min_t(u64, local_paca->kvm_hstate.dec_expires,
+				vcpu->arch.hdec_exp);
 
 	vcpu->arch.ceded = 0;
 
@@ -3736,15 +3733,16 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit,
 	mtspr(SPRN_DEC, vcpu->arch.dec_expires - mftb());
 
 	if (kvmhv_on_pseries()) {
-		trap = kvmhv_pseries_enter_guest(vcpu, time_limit, lpcr);
+		trap = kvmhv_pseries_enter_guest(vcpu, time_limit);
 	} else {
-		trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit, lpcr);
+		trap = kvmhv_load_hv_regs_and_go(vcpu, time_limit);
 	}
 
 	if (kvm_is_radix(vcpu->kvm))
 		vcpu->arch.slb_max = 0;
 	dec = mfspr(SPRN_DEC);
-	if (!(lpcr & LPCR_LD)) /* Sign extend if not using large decrementer */
+	/* Sign extend if not using large decrementer */
+	if (!(vcpu->arch.lpcr & LPCR_LD))
 		dec = (s32) dec;
 	tb = mftb();
 	vcpu->arch.dec_expires = dec + tb;
@@ -4145,9 +4143,7 @@ static int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 	return vcpu->arch.ret;
 }
 
-int kvmhv_run_single_vcpu(struct kvm_run *kvm_run,
-			  struct kvm_vcpu *vcpu, u64 time_limit,
-			  unsigned long lpcr)
+int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)
 {
 	int trap, r, pcpu;
 	int srcu_idx, lpid;
@@ -4206,7 +4202,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run,
 		}
 		if (test_bit(BOOK3S_IRQPRIO_EXTERNAL,
 			     &vcpu->arch.pending_exceptions))
-			lpcr |= LPCR_MER;
+			vcpu->arch.lpcr |= LPCR_MER;
 	} else if (vcpu->arch.pending_exceptions ||
 		   vcpu->arch.doorbell_request ||
 		   xive_interrupt_pending(vcpu)) {
@@ -4242,7 +4238,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run,
 	/* Tell lockdep that we're about to enable interrupts */
 	trace_hardirqs_on();
 
-	trap = kvmhv_p9_guest_entry(vcpu, time_limit, lpcr);
+	trap = kvmhv_p9_guest_entry(vcpu);
 	vcpu->arch.trap = trap;
 
 	trace_hardirqs_off();
@@ -4399,6 +4395,9 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)
 	vcpu->arch.state = KVMPPC_VCPU_BUSY_IN_HOST;
 
 	do {
+		/* update vcpu->arch.lpcr in case a previous loop modified it */
+		vcpu->arch.lpcr = vcpu->arch.vcore->lpcr;
+		vcpu->arch.hdec_exp = ~(u64)0;
 		/*
 		 * The early POWER9 chips that can't mix radix and HPT threads
 		 * on the same core also need the workaround for the problem
@@ -4412,8 +4411,7 @@ static int kvmppc_vcpu_run_hv(struct kvm_run *run, struct kvm_vcpu *vcpu)
 		if (kvmhv_on_pseries() || (kvm->arch.threads_indep &&
 					   kvm_is_radix(kvm) &&
 					   !no_mixing_hpt_and_radix))
-			r = kvmhv_run_single_vcpu(run, vcpu, ~(u64)0,
-						  vcpu->arch.vcore->lpcr);
+			r = kvmhv_run_single_vcpu(run, vcpu);
 		else
 			r = kvmppc_run_vcpu(run, vcpu);
 
arch/powerpc/kvm/book3s_hv_nested.c +5 -5
index 883f8896ed60..f80491e9ff97 100644
@@ -267,7 +267,6 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	struct guest_slb *l2_slb = NULL, *saved_l1_slb = NULL;
 	struct kvmppc_vcore *vc = vcpu->arch.vcore;
 	u64 hv_ptr, regs_ptr, slb_ptr = 0UL;
-	u64 hdec_exp;
 	s64 delta_purr, delta_spurr, delta_ic, delta_vtb;
 	u64 mask;
 	unsigned long lpcr;
@@ -357,7 +356,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	}
 
 	/* convert TB values/offsets to host (L0) values */
-	hdec_exp = l2_hv.hdec_expiry - vc->tb_offset;
+	vcpu->arch.hdec_exp = l2_hv.hdec_expiry - vc->tb_offset;
 	vc->tb_offset += l2_hv.tb_offset;
 
 	/* set L1 state to L2 state */
@@ -377,14 +376,15 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 	vcpu->arch.ret = RESUME_GUEST;
 	vcpu->arch.trap = 0;
 	do {
-		if (mftb() >= hdec_exp) {
+		if (mftb() >= vcpu->arch.hdec_exp) {
 			vcpu->arch.trap = BOOK3S_INTERRUPT_HV_DECREMENTER;
 			r = RESUME_HOST;
 			break;
 		}
+		/* update vcpu->arch.lpcr in case a previous loop modified it */
+		vcpu->arch.lpcr = lpcr;
 		if (radix)
-			r = kvmhv_run_single_vcpu(vcpu->arch.kvm_run, vcpu,
-						  hdec_exp, lpcr);
+			r = kvmhv_run_single_vcpu(vcpu->arch.kvm_run, vcpu);
 		else
 			r = RESUME_HOST; /* XXX TODO hpt entry path */
 	} while (is_kvmppc_resume_guest(r));
Signature · 1 line
2.13.6
#19 lore raw

PATCH 16/23 KVM: PPC: Book3S HV: Nested: Make kvmppc_run_vcpu() entry path nested capable

In reply to Suraj Jitindar Singh

In order to run a hpt (hash page table) guest the kvm entry path used must enter real mode before loading up the guest mmu state. Currently the only path which does this is calling kvmppc_run_vcpu() which then uses the entry path in book3s_hv_rmhandlers.S and until now this path didn't accomodate running a nested guest.

Have the nested hpt guest entry path call kvmppc_run_vcpu() and modify the entry path in book3s_hv_rmhandlers.S to be able to run a nested guest.

For the entry path this means loading the smaller of the guest hypervisor decrementer and the host decrementer into the host decrementer since we want control back when either expires. Additionally the correct LPID and LPCR must be loaded, and the guest slb entries must be restored. When checking if an interrupt can be injected to the guest in kvmppc_guest_entry_inject_int() we return -1 if entering a nested guest while there is something pending for the L1 guest to indicate that the nested guest shouldn't be entered and control should be passed back to the L1 guest.

On the exit path we must save the guest slb entries to be returned to the L1 guest hypervisor. Additionally the correct vrma_slb_v entry must be loaded for kvmppc_hpte_hv_fault() if the guest was in real mode. The correct hpt must be used in kvmppc_hpte_hv_fault(). And the correct handle_exit function must be called depending on whether a nested guest was being run or not.

---
 arch/powerpc/include/asm/kvm_asm.h      |   5 ++
 arch/powerpc/include/asm/kvm_book3s.h   |   3 +-
 arch/powerpc/include/asm/kvm_ppc.h      |   2 +-
 arch/powerpc/kernel/asm-offsets.c       |   5 ++
 arch/powerpc/kvm/book3s_64_mmu_hv.c     |   2 +-
 arch/powerpc/kvm/book3s_hv.c            |  55 +++++++-------
 arch/powerpc/kvm/book3s_hv_builtin.c    |  33 ++++++---
 arch/powerpc/kvm/book3s_hv_interrupts.S |  25 ++++++-
 arch/powerpc/kvm/book3s_hv_nested.c     |   2 +-
 arch/powerpc/kvm/book3s_hv_rm_mmu.c     |  80 +++++++++++++++------
 arch/powerpc/kvm/book3s_hv_rmhandlers.S | 124 ++++++++++++++++++++++----------
 arch/powerpc/kvm/book3s_xive.h          |  15 ++++
 12 files changed, 252 insertions(+), 99 deletions(-)
arch/powerpc/include/asm/kvm_asm.h +5 -0
index 635fb154b33f..83bfd74ce67c 100644
@@ -104,6 +104,11 @@
  * completely in the guest.
  */
 #define BOOK3S_INTERRUPT_HV_RM_HARD	0x5555
+/*
+ * Special trap used when running a nested guest to communicate that control
+ * should be passed back to the L1 guest. e.g. Because interrupt pending
+ */
+#define BOOK3S_INTERRUPT_HV_NEST_EXIT	0x5556
 
 #define BOOK3S_IRQPRIO_SYSTEM_RESET		0
 #define BOOK3S_IRQPRIO_DATA_SEGMENT		1
arch/powerpc/include/asm/kvm_book3s.h +2 -1
index e1dc1872e453..f13dab096dad 100644
@@ -158,7 +158,7 @@ extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu);
 extern int kvmppc_book3s_hv_page_fault(struct kvm_run *run,
 			struct kvm_vcpu *vcpu, unsigned long addr,
 			unsigned long status);
-extern long kvmppc_hv_find_lock_hpte(struct kvm *kvm, gva_t eaddr,
+extern long kvmppc_hv_find_lock_hpte(struct kvm_hpt_info *hpt, gva_t eaddr,
 			unsigned long slb_v, unsigned long valid);
 extern int kvmppc_hv_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu,
 			unsigned long gpa, gva_t ea, int is_store);
@@ -315,6 +315,7 @@ void kvmhv_release_all_nested(struct kvm *kvm);
 long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu);
 long kvmhv_do_nested_tlbie(struct kvm_vcpu *vcpu);
 int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu);
+int kvmppc_run_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu);
 void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr);
 void kvmhv_save_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
 void kvmhv_restore_guest_slb(struct kvm_vcpu *vcpu, struct guest_slb *slbp);
arch/powerpc/include/asm/kvm_ppc.h +1 -1
index 2c4d659cf8bb..46bbdc38b2c5 100644
@@ -697,7 +697,7 @@ int kvmppc_rm_h_ipi(struct kvm_vcpu *vcpu, unsigned long server,
                     unsigned long mfrr);
 int kvmppc_rm_h_cppr(struct kvm_vcpu *vcpu, unsigned long cppr);
 int kvmppc_rm_h_eoi(struct kvm_vcpu *vcpu, unsigned long xirr);
-void kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu);
+int kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu);
 
 /*
  * Host-side operations we want to set up while running in real
arch/powerpc/kernel/asm-offsets.c +5 -0
index 4ccb6b3a7fbd..7652ad430aab 100644
@@ -511,9 +511,14 @@ int main(void)
 	OFFSET(VCPU_VPA, kvm_vcpu, arch.vpa.pinned_addr);
 	OFFSET(VCPU_VPA_DIRTY, kvm_vcpu, arch.vpa.dirty);
 	OFFSET(VCPU_HEIR, kvm_vcpu, arch.emul_inst);
+	OFFSET(VCPU_LPCR, kvm_vcpu, arch.lpcr);
 	OFFSET(VCPU_NESTED, kvm_vcpu, arch.nested);
+	OFFSET(VCPU_NESTED_LPID, kvm_nested_guest, shadow_lpid);
+	OFFSET(VCPU_NESTED_RADIX, kvm_nested_guest, radix);
+	OFFSET(VCPU_NESTED_VRMA_SLB_V, kvm_nested_guest, vrma_slb_v);
 	OFFSET(VCPU_CPU, kvm_vcpu, cpu);
 	OFFSET(VCPU_THREAD_CPU, kvm_vcpu, arch.thread_cpu);
+	OFFSET(VCPU_HDEC_EXP, kvm_vcpu, arch.hdec_exp);
 #endif
 #ifdef CONFIG_PPC_BOOK3S
 	OFFSET(VCPU_PURR, kvm_vcpu, arch.purr);
arch/powerpc/kvm/book3s_64_mmu_hv.c +1 -1
index bbb23b3f8bb9..2b30b48dce49 100644
@@ -361,7 +361,7 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
 
 	preempt_disable();
 	/* Find the HPTE in the hash table */
-	index = kvmppc_hv_find_lock_hpte(kvm, eaddr, slb_v,
+	index = kvmppc_hv_find_lock_hpte(&kvm->arch.hpt, eaddr, slb_v,
 					 HPTE_V_VALID | HPTE_V_ABSENT);
 	if (index < 0) {
 		preempt_enable();
arch/powerpc/kvm/book3s_hv.c +29 -26

109 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv_builtin.c +24 -9
index 7c1909657b55..049c3111b530 100644
@@ -759,26 +759,40 @@ void kvmhv_p9_restore_lpcr(struct kvm_split_mode *sip)
  * Is there a PRIV_DOORBELL pending for the guest (on POWER9)?
  * Can we inject a Decrementer or a External interrupt?
  */
-void kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu)
+int kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu)
 {
 	int ext;
 	unsigned long vec = 0;
-	unsigned long lpcr;
+	unsigned long old_lpcr, lpcr;
+
+#ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE
+	/*
+	 * Don't enter a nested guest if there is something pending for this
+	 * vcpu for the l1 guest. Return -1 to indicate this.
+	 */
+	if (vcpu->arch.nested && (vcpu->arch.pending_exceptions ||
+				  vcpu->arch.prodded ||
+				  vcpu->arch.doorbell_request ||
+				  xive_interrupt_pending(vcpu)))
+		return -1;
+#endif
 
 	/* Insert EXTERNAL bit into LPCR at the MER bit position */
 	ext = (vcpu->arch.pending_exceptions >> BOOK3S_IRQPRIO_EXTERNAL) & 1;
-	lpcr = mfspr(SPRN_LPCR);
-	lpcr |= ext << LPCR_MER_SH;
-	mtspr(SPRN_LPCR, lpcr);
-	isync();
+	old_lpcr = mfspr(SPRN_LPCR);
+	lpcr = old_lpcr | (ext << LPCR_MER_SH);
+	if (lpcr != old_lpcr) {
+		mtspr(SPRN_LPCR, lpcr);
+		isync();
+	}
 
 	if (vcpu->arch.shregs.msr & MSR_EE) {
 		if (ext) {
 			vec = BOOK3S_INTERRUPT_EXTERNAL;
 		} else {
-			long int dec = mfspr(SPRN_DEC);
+			s64 dec = mfspr(SPRN_DEC);
 			if (!(lpcr & LPCR_LD))
-				dec = (int) dec;
+				dec = (s32) dec;
 			if (dec < 0)
 				vec = BOOK3S_INTERRUPT_DECREMENTER;
 		}
@@ -795,12 +809,13 @@ void kvmppc_guest_entry_inject_int(struct kvm_vcpu *vcpu)
 		vcpu->arch.shregs.msr = msr;
 	}
 
-	if (vcpu->arch.doorbell_request) {
+	if (cpu_has_feature(CPU_FTR_ARCH_300) && vcpu->arch.doorbell_request) {
 		mtspr(SPRN_DPDES, 1);
 		vcpu->arch.vcore->dpdes = 1;
 		smp_wmb();
 		vcpu->arch.doorbell_request = 0;
 	}
+	return 0;
 }
 
 static void flush_guest_tlb(struct kvm *kvm)
arch/powerpc/kvm/book3s_hv_interrupts.S +22 -3
index 63fd81f3039d..624f9951731d 100644
@@ -58,10 +58,20 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S)
 	/*
 	 * Put whatever is in the decrementer into the
 	 * hypervisor decrementer.
+	 * If running a nested guest then put the lower of the host decrementer
+	 * and the guest hypervisor decrementer into the hypervisor decrementer
+	 * since we want control back from the nested guest when either expires.
 	 */
 BEGIN_FTR_SECTION
 	ld	r5, HSTATE_KVM_VCORE(r13)
-	ld	r6, VCORE_KVM(r5)
+	ld	r6, HSTATE_KVM_VCPU(r13)
+	cmpdi   cr1, r6, 0              /* Do we actually have a vcpu? */
+	beq     cr1, 33f
+	ld      r7, VCPU_NESTED(r6)
+	cmpdi   cr1, r7, 0              /* Do we have a nested guest? */
+	beq     cr1, 33f
+	ld      r10, VCPU_HDEC_EXP(r6)  /* If so load the hdec expiry */
+33:	ld	r6, VCORE_KVM(r5)
 	ld	r9, KVM_HOST_LPCR(r6)
 	andis.	r9, r9, LPCR_LD@h
 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
@@ -72,8 +82,17 @@ BEGIN_FTR_SECTION
 	bne	32f
 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
 	extsw	r8,r8
-32:	mtspr	SPRN_HDEC,r8
-	add	r8,r8,r7
+BEGIN_FTR_SECTION
+32:	beq     cr1, 34f		/* did we load hdec expiry above? */
+	subf    r10, r7, r10		/* r10 = guest_hdec = hdec_exp - tb */
+	cmpd    r8, r10			/* host decrementer < hdec? */
+	ble     34f
+	mtspr   SPRN_HDEC, r10		/* put guest_hdec into the hv decr */
+	b       35f
+34:
+END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
+	mtspr	SPRN_HDEC,r8		/* put host decr into hv decr */
+35:	add	r8,r8,r7
 	std	r8,HSTATE_DECEXP(r13)
 
 	/* Jump to partition switch code */
arch/powerpc/kvm/book3s_hv_nested.c +1 -1
index f80491e9ff97..54d6ff0bee5b 100644
@@ -386,7 +386,7 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 		if (radix)
 			r = kvmhv_run_single_vcpu(vcpu->arch.kvm_run, vcpu);
 		else
-			r = RESUME_HOST; /* XXX TODO hpt entry path */
+			r = kvmppc_run_vcpu(vcpu->arch.kvm_run, vcpu);
 	} while (is_kvmppc_resume_guest(r));
 
 	/* save L2 state for return */
arch/powerpc/kvm/book3s_hv_rm_mmu.c +59 -21

162 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv_rmhandlers.S +88 -36

227 lines not on this page · raw message

arch/powerpc/kvm/book3s_xive.h +15 -0
index 50494d0ee375..d6f10d7ec4d2 100644
@@ -283,5 +283,20 @@ int kvmppc_xive_attach_escalation(struct kvm_vcpu *vcpu, u8 prio,
 				  bool single_escalation);
 struct kvmppc_xive *kvmppc_xive_get_device(struct kvm *kvm, u32 type);
 
+static inline bool xive_interrupt_pending(struct kvm_vcpu *vcpu)
+{
+        if (!xics_on_xive())
+                return false;
+        return vcpu->arch.irq_pending || vcpu->arch.xive_saved_state.pipr <
+                vcpu->arch.xive_saved_state.cppr;
+}
+
+#else /* !CONFIG_KVM_XICS */
+
+static inline bool xive_interrupt_pending(struct kvm_vcpu *vcpu)
+{
+        return false;
+}
+
 #endif /* CONFIG_KVM_XICS */
 #endif /* _KVM_PPC_BOOK3S_XICS_H */
Signature · 1 line
2.13.6
#20 lore raw

PATCH 17/23 KVM: PPC: Book3S HV: Nested: Rename kvmhv_xlate_addr_nested_radix

In reply to Suraj Jitindar Singh

Rename kvmhv_xlate_addr_nested() to kvmhv_xlate_addr_nested_radix() for clarity since there will need to be a hash function added.

Additionally if we can't permit an access since a nested guest is writing to a page which has the KVM_MEM_READONLY bit set in the memslot then give the nested guest an interrupt, there's not much else which can be done in this case. For now this is the only place where an interrupt is injected directly to the nested guest by the L0 hypervisor without involving the L1 guest hypervisor.

---
 arch/powerpc/kvm/book3s.c           |  1 +
 arch/powerpc/kvm/book3s_hv_nested.c | 81 +++++++++++++++++++++++++++----------
 2 files changed, 61 insertions(+), 21 deletions(-)
arch/powerpc/kvm/book3s.c +1 -0
index 9524d92bc45d..ff6586f1ed6f 100644
@@ -140,6 +140,7 @@ void kvmppc_inject_interrupt(struct kvm_vcpu *vcpu, int vec, u64 flags)
 	kvmppc_set_pc(vcpu, kvmppc_interrupt_offset(vcpu) + vec);
 	vcpu->arch.mmu.reset_msr(vcpu);
 }
+EXPORT_SYMBOL_GPL(kvmppc_inject_interrupt);
 
 static int kvmppc_book3s_vec2irqprio(unsigned int vec)
 {
arch/powerpc/kvm/book3s_hv_nested.c +60 -21

158 lines not on this page · raw message

Signature · 1 line
2.13.6
#21 lore raw

PATCH 18/23 KVM: PPC: Book3S HV: Separate out hashing from kvmppc_hv_find_lock_hpte()

In reply to Suraj Jitindar Singh

Separate out the hashing function from kvmppc_hv_find_lock_hpte() as kvmppc_hv_get_hash_value() to allow this function to be reused and prevent code duplication.

No functional change.

---
 arch/powerpc/kvm/book3s_hv_rm_mmu.c | 60 ++++++++++++++++++++++---------------
 1 file changed, 36 insertions(+), 24 deletions(-)
arch/powerpc/kvm/book3s_hv_rm_mmu.c +36 -24
index a939782d8a5e..c8a379a6f533 100644
@@ -1162,6 +1162,40 @@ static struct mmio_hpte_cache_entry *
 	return &vcpu->arch.mmio_cache.entry[index];
 }
 
+/*
+ * Given an effective address and a slb entry, work out the hash and the
+ * virtual page number
+ */
+unsigned long kvmppc_hv_get_hash_value(struct kvm_hpt_info *hpt, gva_t eaddr,
+				       unsigned long slb_v, unsigned long *avpn,
+				       unsigned int *pshift_p)
+{
+	unsigned long hash, somask, vsid;
+	unsigned int pshift = 12;
+
+	if (slb_v & SLB_VSID_L)
+		pshift = slb_base_page_shift[(slb_v & SLB_VSID_LP) >> 4];
+	if (slb_v & SLB_VSID_B_1T) {
+		somask = (1UL << 40) - 1;
+		vsid = (slb_v & ~SLB_VSID_B) >> SLB_VSID_SHIFT_1T;
+		vsid ^= vsid << 25;
+	} else {
+		somask = (1UL << 28) - 1;
+		vsid = (slb_v & ~SLB_VSID_B) >> SLB_VSID_SHIFT;
+	}
+	hash = (vsid ^ ((eaddr & somask) >> pshift)) & kvmppc_hpt_mask(hpt);
+	*avpn = slb_v & ~(somask >> 16);	/* also includes B */
+	*avpn |= (eaddr & somask) >> 16;
+
+	if (pshift >= 24)
+		*avpn &= ~((1UL << (pshift - 16)) - 1);
+	else
+		*avpn &= ~0x7fUL;
+	*pshift_p = pshift;
+
+	return hash;
+}
+
 /* When called from virtmode, this func should be protected by
  * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK
  * can trigger deadlock issue.
@@ -1171,39 +1205,17 @@ long kvmppc_hv_find_lock_hpte(struct kvm_hpt_info *hpt, gva_t eaddr,
 {
 	unsigned int i;
 	unsigned int pshift;
-	unsigned long somask;
-	unsigned long vsid, hash;
-	unsigned long avpn;
 	__be64 *hpte;
-	unsigned long mask, val;
+	unsigned long hash, mask, val;
 	unsigned long v, r, orig_v;
 
 	/* Get page shift, work out hash and AVPN etc. */
 	mask = SLB_VSID_B | HPTE_V_AVPN | HPTE_V_SECONDARY;
-	val = 0;
-	pshift = 12;
+	hash = kvmppc_hv_get_hash_value(hpt, eaddr, slb_v, &val, &pshift);
 	if (slb_v & SLB_VSID_L) {
 		mask |= HPTE_V_LARGE;
 		val |= HPTE_V_LARGE;
-		pshift = slb_base_page_shift[(slb_v & SLB_VSID_LP) >> 4];
-	}
-	if (slb_v & SLB_VSID_B_1T) {
-		somask = (1UL << 40) - 1;
-		vsid = (slb_v & ~SLB_VSID_B) >> SLB_VSID_SHIFT_1T;
-		vsid ^= vsid << 25;
-	} else {
-		somask = (1UL << 28) - 1;
-		vsid = (slb_v & ~SLB_VSID_B) >> SLB_VSID_SHIFT;
 	}
-	hash = (vsid ^ ((eaddr & somask) >> pshift)) & kvmppc_hpt_mask(hpt);
-	avpn = slb_v & ~(somask >> 16);	/* also includes B */
-	avpn |= (eaddr & somask) >> 16;
-
-	if (pshift >= 24)
-		avpn &= ~((1UL << (pshift - 16)) - 1);
-	else
-		avpn &= ~0x7fUL;
-	val |= avpn;
 
 	for (;;) {
 		hpte = (__be64 *)(hpt->virt + (hash << 7));
Signature · 1 line
2.13.6
#22 lore raw

PATCH 19/23 KVM: PPC: Book3S HV: Nested: Implement nested hpt mmu translation

In reply to Suraj Jitindar Singh

Implement the insertion of nested hpt (hash page table) translations in to the shadow hpt. The shadow hpt is used to store ptes (page table entries) which provide the translation from nested guest virtual address to host real address. The translation from nested guest effective address to virtual address is provided by the slb which the nested guest can manage itself directly.

In order to construct this translation the hash page table in L1 memory must first be searched to provide a translation from nested guest virtual address to L1 guest real address. If no such entry is found then an interrupt is provided to the L1 guest hypervisor so that it can insert an entry.

The L1 guest real address is then translated to a host real address through the radix page tables for the L1 guest, with an entry created if one doesn't already exist.

The rc bits are then set for the pte in the L1 guest memory and the host radix pte for the L1 guest since these will be set by the hardware in the shadow pte automatically and so no interrupt will be provided to ensure these can be kept in sync. In fact the rc bits in the shadow hpt are set by software here. The c (change) bit is only set if the nested guest is writing, otherwise the page is mapped read only so that we can fault on a write to set the change bit and upgrade the write permissions.

The combination of the pte permissions is applied to the entry inserted and a nest rmap entry inserted.

Since we may come in with an existing entry when we just need to upgrade the write permissions on a page (in which case the index was found and stored in kvmppc_hpte_hv_fault()) we check for this case and just load the guest rpte entry from the rev map rather than having to look it up in L1 guest memory again.

This doesn't support the invalidation of translations by either the L0 or L1 hypervisors, this functionality is added in proceeding patches.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |   2 +
 arch/powerpc/include/asm/kvm_book3s.h         |   4 +
 arch/powerpc/include/asm/kvm_book3s_64.h      |   9 +
 arch/powerpc/kvm/book3s_hv_nested.c           | 385 +++++++++++++++++++++++++-
 arch/powerpc/kvm/book3s_hv_rm_mmu.c           |   6 +-
 5 files changed, 404 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +2 -0
index c04e37b2c30d..f33dcb84a0bf 100644
@@ -90,6 +90,8 @@
 #define HPTE_R_KEY_HI		ASM_CONST(0x3000000000000000)
 #define HPTE_R_KEY_BIT0		ASM_CONST(0x2000000000000000)
 #define HPTE_R_KEY_BIT1		ASM_CONST(0x1000000000000000)
+#define HPTE_R_B		ASM_CONST(0x0c00000000000000)
+#define HPTE_R_B_1T		ASM_CONST(0x0400000000000000)
 #define HPTE_R_RPN_SHIFT	12
 #define HPTE_R_RPN		ASM_CONST(0x0ffffffffffff000)
 #define HPTE_R_RPN_3_0		ASM_CONST(0x01fffffffffff000)
arch/powerpc/include/asm/kvm_book3s.h +4 -0
index f13dab096dad..b43d7f348712 100644
@@ -158,6 +158,10 @@ extern void kvmppc_mmu_flush_segments(struct kvm_vcpu *vcpu);
 extern int kvmppc_book3s_hv_page_fault(struct kvm_run *run,
 			struct kvm_vcpu *vcpu, unsigned long addr,
 			unsigned long status);
+extern unsigned long kvmppc_hv_get_hash_value(struct kvm_hpt_info *hpt,
+					      gva_t eaddr, unsigned long slb_v,
+					      unsigned long *avpn,
+					      unsigned int *pshift_p);
 extern long kvmppc_hv_find_lock_hpte(struct kvm_hpt_info *hpt, gva_t eaddr,
 			unsigned long slb_v, unsigned long valid);
 extern int kvmppc_hv_emulate_mmio(struct kvm_run *run, struct kvm_vcpu *vcpu,
arch/powerpc/include/asm/kvm_book3s_64.h +9 -0
index c874ab3a037e..0db673501110 100644
@@ -369,6 +369,15 @@ static inline unsigned long hpte_make_readonly(unsigned long ptel)
 	return ptel;
 }
 
+static inline unsigned long hpte_make_writable(unsigned long ptel)
+{
+	if ((ptel & HPTE_R_PP0) || (ptel & HPTE_R_PP) = PP_RWXX)
+		ptel = (ptel & ~HPTE_R_PPP) | PP_RWXX;
+	else
+		ptel = (ptel & ~HPTE_R_PP) | PP_RWRW;
+	return ptel;
+}
+
 static inline bool hpte_cache_flags_ok(unsigned long hptel, bool is_ci)
 {
 	unsigned int wimg = hptel & HPTE_R_WIMG;
arch/powerpc/kvm/book3s_hv_nested.c +384 -1

402 lines not on this page · raw message

arch/powerpc/kvm/book3s_hv_rm_mmu.c +5 -1
index c8a379a6f533..3c01957acb0e 100644
@@ -1195,6 +1195,7 @@ unsigned long kvmppc_hv_get_hash_value(struct kvm_hpt_info *hpt, gva_t eaddr,
 
 	return hash;
 }
+EXPORT_SYMBOL_GPL(kvmppc_hv_get_hash_value);
 
 /* When called from virtmode, this func should be protected by
  * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK
@@ -1291,8 +1292,11 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,
 
 	hpt = &kvm->arch.hpt;
 	nested = vcpu->arch.nested;
-	if (nested)
+	if (nested) {
 		hpt = &nested->shadow_hpt;
+		/* reuse fault_gpa field to save slb for nested pgfault funcn */
+		vcpu->arch.fault_gpa = slb_v;
+	}
 
 	/* For protection fault, expect to find a valid HPTE */
 	valid = HPTE_V_VALID;
Signature · 1 line
2.13.6
#23 lore raw

PATCH 20/23 KVM: PPC: Book3S HV: Nested: Handle tlbie hcall for nested hpt guest

In reply to Suraj Jitindar Singh

The tlbie instruction is used to invalidate caching of translation information derived from the partition and process scoped page tables. This instruction is hypervisor privileged when specifying partition scoped translations to be invalidated. Thus this interface is paravirtualised with the H_TLB_INVALIDATE hcall which is used by a pseries hypervisor to perform partition scoped invalidations. This is then handled in the hypervisor in kvmhv_emulate_priv_tlbie().

When handling this hcall in the hypervisor it is necessary to invalidate caching of partition scoped translations which are in the shadow page table (radix) or shadow hpt (hash page table). This functionality already exists for radix, so implement it for a hash guest.

LPID wide invalidations are already handled and don't differ from the radix case. However the case where a single tlb entry corresponding to a virtual address is being invalidated needs to be implemented here. The information to find the entry is provided as an hcall argument and is expected to match the layout of rb specified for the tlbie instruction.

The rb value provides an abbreviated virtual address, base and actual page size and segment size to be used to search for the corresponding tlb entry. A hash is computed and this is used to find the pteg which needs to be searched. However since only 64 bits of the virtual address are supplied and depending on the segment and hpt size up to all 78 bits of the virtual address may be needed to compute the hash we have to mask out the bits which can't be determined and iterate through all possible combinations looking for a matching entry in all of the ptegs which are addressed by all of the possible hash values. Although there is in theory a 1-to-1 relationship between ptes in the shadow hpt and the hpt maintained by the guest hypervisor we need to invalidate all matches since they can't be differentiated.

When a matching entry is found it is invalidated if it was valid and the corresponding tlbie is issued by the hypervisor, irrespective the pte is then zeroed. An optimisation here would be to just make the pte absent and extend the rev map to store the host real doubleword since it is still valid.

Note: ric = 3 (cluster bombs) are not supported even though the ISA technically allows for them, their encoding is implemenatation dependant and linux doesn't use them.

---
 arch/powerpc/include/asm/book3s/64/mmu-hash.h |  11 +
 arch/powerpc/kvm/book3s_hv_nested.c           | 293 ++++++++++++++++++++++----
 2 files changed, 258 insertions(+), 46 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu-hash.h +11 -0
index f33dcb84a0bf..70f4545fecbb 100644
@@ -129,6 +129,17 @@
 #define TLBIEL_INVAL_SET_MASK	0xfff000	/* set number to inval. */
 #define TLBIEL_INVAL_SET_SHIFT	12
 
+/* Fields in the rb registers for the tlbie instruction */
+#define TLBIE_RB_AVA_4K		ASM_CONST(0xfffffffffffff000)
+#define TLBIE_RB_AVA_L		ASM_CONST(0xfffffffffff00000)
+#define TLBIE_RB_LP		ASM_CONST(0x00000000000ff000)
+#define TLBIE_RB_B		ASM_CONST(0x0000000000000300)
+#define TLBIE_RB_B_1T		ASM_CONST(0x0000000000000100)
+#define TLBIE_RB_B_SHIFT	50	/* Shift to match the pte location */
+#define TLBIE_RB_AVAL		ASM_CONST(0x00000000000000fe)
+#define TLBIE_RB_AVAL_SHIFT	12
+#define TLBIE_RB_L		ASM_CONST(0x0000000000000001)
+
 #define POWER7_TLB_SETS		128	/* # sets in POWER7 TLB */
 #define POWER8_TLB_SETS		512	/* # sets in POWER8 TLB */
 #define POWER9_TLB_SETS_HASH	256	/* # sets in POWER9 TLB Hash mode */
arch/powerpc/kvm/book3s_hv_nested.c +247 -46

372 lines not on this page · raw message

Signature · 1 line
2.13.6
#24 lore raw

PATCH 21/23 KVM: PPC: Book3S HV: Nested: Implement nest rmap invalidations for hpt guests

In reply to Suraj Jitindar Singh

The nest rmap is used to store a reverse mapping from the (L1) guest real address back to a pte in the shadow page table which maps it. This is used when the host is modifying a L1 guest pte (either invalidating it or modifying the rc bits) to make the necessary changes to the ptes in the shadow tables which map that L1 guest page. This is already implemented for a nested radix guest where the rmap entry stores the gpa (guest physical address) of the nested pte which can be used to traverse the shadow page table and find any matching ptes. Implement this nested rmap invalidation for nested hpt (hash page table) guests.

We reuse the nest rmap structure that already exists for radix nested guests for nested hpt guests. Instead of storing the gpa the hpt index of the pte is stored. This means that a pte in the shadow hpt can be uniquely identified by the nest rmap. As with the radix case we check that the same host page is being addressed to detect if this is a stale rmap entry, in which case we skip the invalidation.

When the host is invalidating a mapping for a L1 guest page use the nest rmap to find any shadow ptes in the shadow hpt which map that page and invalidate then, also invalidate any caching of the entry. A future optimisation would be to make the pte absend so that we can avoid having to lookup the guest rpte the next time an entry is faulted in.

When the host is clearing rc bits for a mapping for a L1 guest page use the nest rmap to find any shadow ptes in the shadow hpt which map that page and invalidate them as in the above case for invalidating a L1 guest page. It is not sufficient to clear the rc bits in the shadow pte since hardware can set them again without software intervention, so the mapping must be made invalid so that we will take a page fault and can ensure that the rc bits stay in sync in the page fault handler.

---
 arch/powerpc/kvm/book3s_hv_nested.c | 114 +++++++++++++++++++++++++++---------
 1 file changed, 85 insertions(+), 29 deletions(-)
arch/powerpc/kvm/book3s_hv_nested.c +85 -29
index 57add167115e..90788a52b298 100644
@@ -25,6 +25,9 @@ static struct patb_entry *pseries_partition_tb;
 static void kvmhv_update_ptbl_cache(struct kvm_nested_guest *gp);
 static void kvmhv_remove_all_nested_rmap_lpid(struct kvm *kvm, int lpid);
 static void kvmhv_free_memslot_nest_rmap(struct kvm_memory_slot *free);
+static void kvmhv_invalidate_shadow_pte_hash(struct kvm_hpt_info *hpt,
+					     unsigned int lpid, __be64 *hptep,
+					     unsigned long index);
 
 void kvmhv_save_hv_regs(struct kvm_vcpu *vcpu, struct hv_guest_state *hr)
 {
@@ -1135,30 +1138,57 @@ static void kvmhv_update_nest_rmap_rc(struct kvm *kvm, u64 n_rmap,
 				      unsigned long hpa, unsigned long mask)
 {
 	struct kvm_nested_guest *gp;
-	unsigned long gpa;
-	unsigned int shift, lpid;
-	pte_t *ptep;
+	unsigned int lpid;
 
-	gpa = n_rmap_to_gpa(n_rmap);
 	lpid = n_rmap_to_lpid(n_rmap);;
 	gp = kvmhv_find_nested(kvm, lpid);
 	if (!gp)
 		return;
 
-	/* Find the pte */
-	if (gp->radix)
-		ptep = __find_linux_pte(gp->shadow_pgtable, gpa, NULL, &shift);
-	else
-		ptep = NULL;	/* XXX TODO */
 	/*
-	 * If the pte is present and the pfn is still the same, update the pte.
-	 * If the pfn has changed then this is a stale rmap entry, the nested
-	 * gpa actually points somewhere else now, and there is nothing to do.
-	 * XXX A future optimisation would be to remove the rmap entry here.
+	 * Find the pte, and ensure it's valid and still points to the same
+	 * host page. If the pfn has changed then this is a stale rmap entry,
+	 * the shadow pte actually points somewhere else now, and there is
+	 * nothing to do. Otherwise clear the requested rc bits from the shadow
+	 * pte and perform the appropriate cache invalidation.
+	 * XXX A future optimisation would be to remove the rmap entry
 	 */
-	if (ptep && pte_present(*ptep) && ((pte_val(*ptep) & mask) = hpa)) {
-		__radix_pte_update(ptep, clr, set);
-		kvmppc_radix_tlbie_page(kvm, gpa, shift, lpid);
+	if (gp->radix) {
+		unsigned long gpa = n_rmap_to_gpa(n_rmap);
+		unsigned int shift;
+		pte_t *ptep;
+
+		ptep = __find_linux_pte(gp->shadow_pgtable, gpa, NULL, &shift);
+		/* pte present and still points to the same host page? */
+		if (ptep && pte_present(*ptep) && ((pte_val(*ptep) & mask) =
+						   hpa)) {
+			__radix_pte_update(ptep, clr, set);
+			kvmppc_radix_tlbie_page(kvm, gpa, shift, lpid);
+		}
+	 } else {
+		unsigned long v, r, index = n_rmap_to_index(n_rmap);
+		__be64 *hptep = (__be64 *)(gp->shadow_hpt.virt + (index << 4));
+
+		preempt_disable();
+		while (!try_lock_hpte(hptep, HPTE_V_HVLOCK))
+			cpu_relax();
+		v = be64_to_cpu(hptep[0]) & ~HPTE_V_HVLOCK;
+		r = be64_to_cpu(hptep[1]);
+
+		/*
+		 * It's not enough to just clear the rc bits here since the
+		 * hardware can just set them again transparently, we need to
+		 * make the pte invalid so that an attempt to access the page
+		 * will invoke the page fault handler and we can ensure
+		 * consistency across the rc bits in the various ptes.
+		 */
+		if ((v & HPTE_V_VALID) && ((r & mask) = hpa))
+			kvmhv_invalidate_shadow_pte_hash(&gp->shadow_hpt,
+							 gp->shadow_lpid, hptep,
+							 index);
+		else	/* Leave pte unchanged */
+			__unlock_hpte(hptep, v);
+		preempt_enable();
 	}
 }
 
@@ -1179,7 +1209,7 @@ void kvmhv_update_nest_rmap_rc_list(struct kvm *kvm, unsigned long *rmapp,
 	if ((clr | set) & ~(_PAGE_DIRTY | _PAGE_ACCESSED))
 		return;
 
-	mask = PTE_RPN_MASK & ~(nbytes - 1);
+	mask = HPTE_R_RPN_3_0 & ~(nbytes - 1);
 	hpa &= mask;
 
 	llist_for_each_entry(cursor, head->first, list)
@@ -1195,24 +1225,50 @@ static void kvmhv_invalidate_nest_rmap(struct kvm *kvm, u64 n_rmap,
 				       unsigned long hpa, unsigned long mask)
 {
 	struct kvm_nested_guest *gp;
-	unsigned long gpa;
-	unsigned int shift, lpid;
-	pte_t *ptep;
+	unsigned int lpid;
 
-	gpa = n_rmap_to_gpa(n_rmap);
 	lpid = n_rmap_to_lpid(n_rmap);;
 	gp = kvmhv_find_nested(kvm, lpid);
 	if (!gp)
 		return;
 
-	/* Find and invalidate the pte */
-	if (gp->radix)
+	/*
+	 * Find the pte, and ensure it's valid and still points to the same
+	 * host page. If the pfn has changed then this is a stale rmap entry,
+	 * the shadow pte actually points somewhere else now, and there is
+	 * nothing to do. Otherwise invalidate the shadow pte and perform the
+	 * appropriate cache invalidation.
+	 */
+	if (gp->radix) {
+		unsigned long gpa = n_rmap_to_gpa(n_rmap);
+		unsigned int shift;
+		pte_t *ptep;
+
 		ptep = __find_linux_pte(gp->shadow_pgtable, gpa, NULL, &shift);
-	else
-		ptep = NULL;	/* XXX TODO */
-	/* Don't spuriously invalidate ptes if the pfn has changed */
-	if (ptep && pte_present(*ptep) && ((pte_val(*ptep) & mask) = hpa))
-		kvmppc_unmap_pte(kvm, ptep, gpa, shift, NULL, gp->shadow_lpid);
+		/* pte present and still points to the same host page? */
+		if (ptep && pte_present(*ptep) && ((pte_val(*ptep) & mask) =
+						   hpa))
+			kvmppc_unmap_pte(kvm, ptep, gpa, shift, NULL,
+					 gp->shadow_lpid);
+	} else {
+		unsigned long v, r, index = n_rmap_to_index(n_rmap);
+		__be64 *hptep = (__be64 *)(gp->shadow_hpt.virt + (index << 4));
+
+		preempt_disable();
+		while (!try_lock_hpte(hptep, HPTE_V_HVLOCK))
+			cpu_relax();
+		v = be64_to_cpu(hptep[0]) & ~HPTE_V_HVLOCK;
+		r = be64_to_cpu(hptep[1]);
+
+		/* Invalidate existing pte if valid and host addr matches */
+		if ((v & HPTE_V_VALID) && ((r & mask) = hpa))
+			kvmhv_invalidate_shadow_pte_hash(&gp->shadow_hpt,
+							 gp->shadow_lpid, hptep,
+							 index);
+		else	/* Leave pte unchanged */
+			__unlock_hpte(hptep, v);
+		preempt_enable();
+	}
 }
 
 /*
@@ -1252,7 +1308,7 @@ void kvmhv_invalidate_nest_rmap_range(struct kvm *kvm,
 	gfn = (gpa >> PAGE_SHIFT) - memslot->base_gfn;
 	end_gfn = gfn + (nbytes >> PAGE_SHIFT);
 
-	addr_mask = PTE_RPN_MASK & ~(nbytes - 1);
+	addr_mask = HPTE_R_RPN_3_0 & ~(nbytes - 1);
 	hpa &= addr_mask;
 
 	for (; gfn < end_gfn; gfn++) {
Signature · 1 line
2.13.6
#25 lore raw

PATCH 22/23 KVM: PPC: Book3S HV: Nested: Enable nested hpt guests

In reply to Suraj Jitindar Singh

Allow nested hpt (hash page table) guests to be run.

---
 arch/powerpc/kvm/book3s_hv_nested.c | 1 -
 1 file changed, 1 deletion(-)
arch/powerpc/kvm/book3s_hv_nested.c +0 -1
index 90788a52b298..bc7b411793be 100644
@@ -330,7 +330,6 @@ long kvmhv_enter_nested_guest(struct kvm_vcpu *vcpu)
 					       (l2_hv.lpcr & LPCR_VPM1))
 			return H_PARAMETER;
 	} else {
-		return H_PARAMETER;
 		/* must be at least V2 to support hpt guest */
 		if (l2_hv.version < 2)
 			return H_PARAMETER;
Signature · 1 line
2.13.6
#26 lore raw

PATCH 23/23 KVM: PPC: Book3S HV: Add nested hpt pte information to debugfs

In reply to Suraj Jitindar Singh

The radix entry in the vm debugfs folder is used to export debug information about the ptes contained in the page table of a radix guest. There is a corrsponding htab entry which provides the same pte information for a hpt (hash page table) guest.

When a radix guest is running a nested guest this entry also provides information about the shadow ptes in the shadow page table.

Add the same information for a nested hpt guest running under a radix guest to this entry. The format followed is the same as that for the htab entry for a hpt guest.

---
 arch/powerpc/kvm/book3s_64_mmu_radix.c | 157 +++++++++++++++++++++------------
 1 file changed, 101 insertions(+), 56 deletions(-)
arch/powerpc/kvm/book3s_64_mmu_radix.c +101 -56
index 48b844d33dc9..8ab9d487c9e8 100644
@@ -1188,6 +1188,8 @@ static int debugfs_radix_release(struct inode *inode, struct file *file)
 	return 0;
 }
 
+#define HPTE_SIZE      (2 * sizeof(unsigned long))
+
 static ssize_t debugfs_radix_read(struct file *file, char __user *buf,
 				 size_t len, loff_t *ppos)
 {
@@ -1197,6 +1199,7 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf,
 	struct kvm *kvm;
 	unsigned long gpa;
 	pgd_t *pgt;
+	struct kvm_hpt_info *hpt;
 	struct kvm_nested_guest *nested;
 	pgd_t pgd, *pgdp;
 	pud_t pud, *pudp;
@@ -1234,84 +1237,126 @@ static ssize_t debugfs_radix_read(struct file *file, char __user *buf,
 	gpa = p->gpa;
 	nested = NULL;
 	pgt = NULL;
+	hpt = NULL;
 	while (len != 0 && p->lpid >= 0) {
-		if (gpa >= RADIX_PGTABLE_RANGE) {
-			gpa = 0;
-			pgt = NULL;
-			if (nested) {
-				kvmhv_put_nested(nested);
-				nested = NULL;
-			}
-			p->lpid = kvmhv_nested_next_lpid(kvm, p->lpid);
-			p->hdr = 0;
-			if (p->lpid < 0)
-				break;
-		}
-		if (!pgt) {
+		if (!pgt && !hpt) {
 			if (p->lpid = 0) {
 				pgt = kvm->arch.pgtable;
 			} else {
 				nested = kvmhv_get_nested(kvm, p->lpid, false);
 				if (!nested) {
-					gpa = RADIX_PGTABLE_RANGE;
+					p->lpid = kvmhv_nested_next_lpid(kvm,
+							p->lpid);
 					continue;
 				}
-				pgt = nested->shadow_pgtable;
+				if (nested->radix)
+					pgt = nested->shadow_pgtable;
+				else
+					hpt = &nested->shadow_hpt;
 			}
 		}
+		if ((pgt && (gpa >= RADIX_PGTABLE_RANGE)) || (hpt &&
+					(gpa >= kvmppc_hpt_npte(hpt)))) {
+			gpa = 0;
+			pgt = NULL;
+			hpt = NULL;
+			if (nested) {
+				kvmhv_put_nested(nested);
+				nested = NULL;
+			}
+			p->lpid = kvmhv_nested_next_lpid(kvm, p->lpid);
+			p->hdr = 0;
+			continue;
+		}
 		n = 0;
 		if (!p->hdr) {
 			if (p->lpid > 0)
 				n = scnprintf(p->buf, sizeof(p->buf),
-					      "\nNested LPID %d: ", p->lpid);
-			n += scnprintf(p->buf + n, sizeof(p->buf) - n,
-				      "pgdir: %lx\n", (unsigned long)pgt);
+					      "\nNested LPID %d: \n", p->lpid);
+			if (pgt)
+				n += scnprintf(p->buf + n, sizeof(p->buf) - n,
+					      "RADIX:\npgdir: %lx\n",
+					      (unsigned long)pgt);
+			else
+				n += scnprintf(p->buf + n, sizeof(p->buf) - n,
+					       "HASH:\nnpte: %lx\n",
+					       kvmppc_hpt_npte(hpt));
 			p->hdr = 1;
 			goto copy;
 		}
 
-		pgdp = pgt + pgd_index(gpa);
-		pgd = READ_ONCE(*pgdp);
-		if (!(pgd_val(pgd) & _PAGE_PRESENT)) {
-			gpa = (gpa & PGDIR_MASK) + PGDIR_SIZE;
-			continue;
-		}
+		if (pgt) {
+			pgdp = pgt + pgd_index(gpa);
+			pgd = READ_ONCE(*pgdp);
+			if (!(pgd_val(pgd) & _PAGE_PRESENT)) {
+				gpa = (gpa & PGDIR_MASK) + PGDIR_SIZE;
+				continue;
+			}
 
-		pudp = pud_offset(&pgd, gpa);
-		pud = READ_ONCE(*pudp);
-		if (!(pud_val(pud) & _PAGE_PRESENT)) {
-			gpa = (gpa & PUD_MASK) + PUD_SIZE;
-			continue;
-		}
-		if (pud_val(pud) & _PAGE_PTE) {
-			pte = pud_val(pud);
-			shift = PUD_SHIFT;
-			goto leaf;
-		}
+			pudp = pud_offset(&pgd, gpa);
+			pud = READ_ONCE(*pudp);
+			if (!(pud_val(pud) & _PAGE_PRESENT)) {
+				gpa = (gpa & PUD_MASK) + PUD_SIZE;
+				continue;
+			}
+			if (pud_val(pud) & _PAGE_PTE) {
+				pte = pud_val(pud);
+				shift = PUD_SHIFT;
+				goto leaf;
+			}
 
-		pmdp = pmd_offset(&pud, gpa);
-		pmd = READ_ONCE(*pmdp);
-		if (!(pmd_val(pmd) & _PAGE_PRESENT)) {
-			gpa = (gpa & PMD_MASK) + PMD_SIZE;
-			continue;
-		}
-		if (pmd_val(pmd) & _PAGE_PTE) {
-			pte = pmd_val(pmd);
-			shift = PMD_SHIFT;
-			goto leaf;
-		}
+			pmdp = pmd_offset(&pud, gpa);
+			pmd = READ_ONCE(*pmdp);
+			if (!(pmd_val(pmd) & _PAGE_PRESENT)) {
+				gpa = (gpa & PMD_MASK) + PMD_SIZE;
+				continue;
+			}
+			if (pmd_val(pmd) & _PAGE_PTE) {
+				pte = pmd_val(pmd);
+				shift = PMD_SHIFT;
+				goto leaf;
+			}
 
-		ptep = pte_offset_kernel(&pmd, gpa);
-		pte = pte_val(READ_ONCE(*ptep));
-		if (!(pte & _PAGE_PRESENT)) {
-			gpa += PAGE_SIZE;
-			continue;
-		}
-		shift = PAGE_SHIFT;
-	leaf:
-		n = scnprintf(p->buf, sizeof(p->buf),
+			ptep = pte_offset_kernel(&pmd, gpa);
+			pte = pte_val(READ_ONCE(*ptep));
+			if (!(pte & _PAGE_PRESENT)) {
+				gpa += PAGE_SIZE;
+				continue;
+			}
+			shift = PAGE_SHIFT;
+		leaf:
+			n = scnprintf(p->buf, sizeof(p->buf),
 			      " %lx: %lx %d\n", gpa, pte, shift);
-		gpa += 1ul << shift;
+			gpa += 1ul << shift;
+		} else { /* hpt */
+			__be64 *hptp = (__be64 *)(hpt->virt + (gpa * HPTE_SIZE));
+			unsigned long v, hr, gr;
+
+			if (!(be64_to_cpu(hptp[0]) & (HPTE_V_VALID |
+							HPTE_V_ABSENT))) {
+				gpa++;
+				continue;
+			}
+			/* lock the HPTE so it's stable and read it */
+			preempt_disable();
+			while (!try_lock_hpte(hptp, HPTE_V_HVLOCK))
+				cpu_relax();
+			v = be64_to_cpu(hptp[0]) & ~HPTE_V_HVLOCK;
+			hr = be64_to_cpu(hptp[1]);
+			gr = hpt->rev[gpa].guest_rpte;
+			unlock_hpte(hptp, v);
+			preempt_enable();
+
+			if (!(v & (HPTE_V_VALID | HPTE_V_ABSENT))) {
+				gpa++;
+				continue;
+			}
+
+			n = scnprintf(p->buf, sizeof(p->buf),
+				      "%6lx %.16lx %.16lx %.16lx\n",
+				      gpa, v, hr, gr);
+			gpa++;
+		}
 	copy:
 		p->chars_left = n;
 		if (n > len)
Signature · 1 line
2.13.6