KVM: PPC: Migration bugfixes for HV KVM
Subsystems (3)
cover KVM: PPC: Migration bugfixes for HV KVM
This series of two patches fixes a couple of bugs in live migration under HV-style KVM on PPC. They only touch PPC/HV code and add no new APIs. The patches are against Alex Graf's kvm-ppc-next branch. I would like them to go in for 3.10.
Paul.
arch/powerpc/include/asm/kvm_book3s.h | 3 +- arch/powerpc/include/asm/kvm_book3s_64.h | 13 ++++ arch/powerpc/include/asm/kvm_host.h | 2 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 120 +++++++++++++++++++++++++----- arch/powerpc/kvm/book3s_hv.c | 30 +++++--- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 11 --- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 + 8 files changed, 143 insertions(+), 41 deletions(-)
PATCH 1/2 KVM: PPC: Book3S HV: Make HPT reading code notice R/C bit changes
At present, the code that determines whether a HPT entry has changed, and thus needs to be sent to userspace when it is copying the HPT, doesn't consider a hardware update to the reference and change bits (R and C) in the HPT entries to constitute a change that needs to be sent to userspace. This adds code to check for changes in R and C when we are scanning the HPT to find changed entries, and adds code to set the changed flag for the HPTE when we update the R and C bits in the guest view of the HPTE.
Since we now need to set the HPTE changed flag in book3s_64_mmu_hv.c as well as book3s_hv_rm_mmu.c, we move the note_hpte_modification() function into kvm_book3s_64.h.
Current Linux guest kernels don't use the hardware updates of R and C in the HPT, so this change won't affect them. Linux (or other) kernels might in future want to use the R and C bits and have them correctly transferred across when a guest is migrated, so it is better to correct this deficiency.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 13 +++++++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 59 +++++++++++++++++++++++++----- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 11 ------ 3 files changed, 63 insertions(+), 20 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +13 -0
@@ -268,4 +268,17 @@ static inline int is_vrma_hpte(unsigned long hpte_v) (HPTE_V_1TB_SEG | (VRMA_VSID << (40 - 16))); } +#ifdef CONFIG_KVM_BOOK3S_64_HV +/* +* Note modification of an HPTE; set the HPTE modified bit +* if anyone is interested. +*/ +static inline void note_hpte_modification (struct kvm *kvm, +struct revmap_entry *rev) +{ +if (atomic_read (&kvm->arch.hpte_mod_interest)) + rev->guest_rpte |=HPTE_GR_MODIFIED ; +} +#endif /* CONFIG_KVM_BOOK3S_64_HV */ +#endif /* __ASM_KVM_BOOK3S_64_H__ */
arch/powerpc/kvm/book3s_64_mmu_hv.c +50 -9
110 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +0 -11
@@ -97,17 +97,6 @@ void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev, }EXPORT_SYMBOL_GPL (kvmppc_add_revmap_chain); -/* -* Note modification of an HPTE; set the HPTE modified bit -* if anyone is interested. -*/ -static inline void note_hpte_modification (struct kvm *kvm, -struct revmap_entry *rev) -{ -if (atomic_read (&kvm->arch.hpte_mod_interest)) - rev->guest_rpte |=HPTE_GR_MODIFIED ; -} -/* Remove this HPTE from the chain for a real page */ static void remove_revmap_chain(struct kvm *kvm, long pte_index, struct revmap_entry *rev,
Signature · 1 line
1.7.10.4
PATCH 2/2 KVM: PPC: Book3S HV: Report VPA and DTL modifications in dirty map
In reply to Paul MackerrasAt present, the KVM_GET_DIRTY_LOG ioctl doesn't report modifications done by the host to the virtual processor areas (VPAs) and dispatch trace logs (DTLs) registered by the guest. This is because those modifications are done either in real mode or in the host kernel context, and in neither case does the access go through the guest's HPT, and thus no change (C) bit gets set in the guest's HPT.
However, the changes done by the host do need to be tracked so that the modified pages get transferred when doing live migration. In order to track these modifications, this adds a dirty flag to the struct representing the VPA/DTL areas, and arranges to set the flag when the VPA/DTL gets modified by the host. Then, when we are collecting the dirty log, we also check the dirty flags for the VPA and DTL for each vcpu and set the relevant bit in the dirty log if necessary. Doing this also means we now need to keep track of the guest physical address of the VPA/DTL areas.
So as not to lose track of modifications to a VPA/DTL area when it gets unregistered, or when a new area gets registered in its place, we need to transfer the dirty state to the rmap chain. This adds code to kvmppc_unpin_guest_page() to do that if the area was dirty. To simplify that code, we now require that all VPA, DTL and SLB shadow buffer areas fit within a single host page. Guests already comply with this requirement because pHyp requires that these areas not cross a 4k boundary.
--- arch/powerpc/include/asm/kvm_book3s.h | 3 +- arch/powerpc/include/asm/kvm_host.h | 2 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 61 ++++++++++++++++++++++++++----- arch/powerpc/kvm/book3s_hv.c | 30 +++++++++------ arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 ++ 6 files changed, 80 insertions(+), 21 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +2 -1
@@ -156,7 +156,8 @@ void kvmppc_clear_ref_hpte(struct kvm *kvm, unsigned long *hptep, unsigned long pte_index);extern void *kvmppc_pin_guest_page (struct kvm *kvm,unsigned long addr,unsigned long *nb_ret); -extern void kvmppc_unpin_guest_page (struct kvm *kvm,void *addr); +extern void kvmppc_unpin_guest_page (struct kvm *kvm,void *addr, +unsigned long gpa,bool dirty);extern long kvmppc_virtmode_h_enter (struct kvm_vcpu *vcpu,unsigned long flags,long pte_index,unsigned long pteh,unsigned long ptel); extern long kvmppc_do_h_enter(struct kvm *kvm, unsigned long flags,
arch/powerpc/include/asm/kvm_host.h +2 -0
@@ -301,11 +301,13 @@ struct kvmppc_vcore {* that a guest can register. */ struct kvmppc_vpa { +unsigned long gpa;/* Current guest phys addr */ void *pinned_addr;/* Address in kernel linear mapping */ void *pinned_end;/* End of region */ unsigned long next_gpa;/* Guest phys addr for update */ unsigned long len;/* Number of bytes required */ u8 update_pending;/* 1 => update pinned_addr from next_gpa */ +bool dirty;/* true => area has been modified by kernel */ };struct kvmppc_pte {
arch/powerpc/kernel/asm-offsets.c +1 -0
@@ -477,6 +477,7 @@ int main(void) DEFINE(VCPU_DSISR, offsetof(struct kvm_vcpu, arch.shregs.dsisr)); DEFINE(VCPU_DAR, offsetof(struct kvm_vcpu, arch.shregs.dar)); DEFINE(VCPU_VPA, offsetof(struct kvm_vcpu, arch.vpa.pinned_addr)); + DEFINE(VCPU_VPA_DIRTY, offsetof(struct kvm_vcpu, arch.vpa.dirty));#endif #ifdef CONFIG_PPC_BOOK3S DEFINE (VCPU_VCPUID , offsetof(struct kvm_vcpu , vcpu_id));
arch/powerpc/kvm/book3s_64_mmu_hv.c +52 -9
101 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +19 -11
@@ -259,7 +259,7 @@ static unsigned long do_h_register_vpa(struct kvm_vcpu *vcpu, len = ((struct reg_vpa *)va)->length.hword; else len = ((struct reg_vpa *)va)->length.word; -kvmppc_unpin_guest_page (kvm, va); +kvmppc_unpin_guest_page (kvm, va, vpa, false);/* Check length */ if (len > nb || len <sizeof (struct reg_vpa ))
@@ -359,13 +359,13 @@ static void kvmppc_update_vpa(struct kvm_vcpu *vcpu, struct kvmppc_vpa *vpap) va =NULL ; nb =0 ;if (gpa) - va =kvmppc_pin_guest_page (kvm, vpap->next_gpa, &nb); + va =kvmppc_pin_guest_page (kvm, gpa, &nb);spin_lock (&vcpu->arch.vpa_update_lock);if (gpa == vpap->next_gpa)break ;/* sigh... unpin that one and try again */ if (va) -kvmppc_unpin_guest_page (kvm, va); +kvmppc_unpin_guest_page (kvm, va, gpa, false); } vpap->update_pending =0 ;
@@ -375,12 +375,15 @@ static void kvmppc_update_vpa(struct kvm_vcpu *vcpu, struct kvmppc_vpa *vpap)* has changed the mappings underlying guest memory, * so unregister the region. */ -kvmppc_unpin_guest_page (kvm, va); +kvmppc_unpin_guest_page (kvm, va, gpa, false); va =NULL ; }if (vpap->pinned_addr) -kvmppc_unpin_guest_page (kvm, vpap->pinned_addr); +kvmppc_unpin_guest_page (kvm, vpap->pinned_addr, vpap->gpa, + vpap->dirty); + vpap->gpa = gpa; vpap->pinned_addr = va; + vpap->dirty = false;if (va) vpap->pinned_end = va + vpap->len; }
@@ -472,6 +475,7 @@ static void kvmppc_create_dtl_entry(struct kvm_vcpu *vcpu,/* order writing *dt vs. writing vpa->dtl_idx */ smp_wmb (); vpa->dtl_idx = ++vcpu->arch.dtl_index; + vcpu->arch.dtl.dirty = true; }int kvmppc_pseries_do_hcall (struct kvm_vcpu *vcpu)
@@ -913,15 +917,19 @@ out:return ERR_PTR (err); } +static void unpin_vpa (struct kvm *kvm,struct kvmppc_vpa *vpa) +{ +if (vpa->pinned_addr) +kvmppc_unpin_guest_page (kvm, vpa->pinned_addr, vpa->gpa, + vpa->dirty); +} +void kvmppc_core_vcpu_free (struct kvm_vcpu *vcpu) {spin_lock (&vcpu->arch.vpa_update_lock); -if (vcpu->arch.dtl.pinned_addr) -kvmppc_unpin_guest_page (vcpu->kvm, vcpu->arch.dtl.pinned_addr); -if (vcpu->arch.slb_shadow.pinned_addr) -kvmppc_unpin_guest_page (vcpu->kvm, vcpu->arch.slb_shadow.pinned_addr); -if (vcpu->arch.vpa.pinned_addr) -kvmppc_unpin_guest_page (vcpu->kvm, vcpu->arch.vpa.pinned_addr); +unpin_vpa (vcpu->kvm, &vcpu->arch.dtl); +unpin_vpa (vcpu->kvm, &vcpu->arch.slb_shadow); +unpin_vpa (vcpu->kvm, &vcpu->arch.vpa);spin_unlock (&vcpu->arch.vpa_update_lock);kvm_vcpu_uninit (vcpu);kmem_cache_free (kvm_vcpu_cache, vcpu);
arch/powerpc/kvm/book3s_hv_rmhandlers.S +4 -0
@@ -260,6 +260,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_206) lwz r5, LPPACA_YIELDCOUNT(r3) addi r5, r5, 1 stw r5, LPPACA_YIELDCOUNT(r3) + li r6, 1 + stb r6, VCPU_VPA_DIRTY(r4) 25: /* Load up DAR and DSISR */ ld r5, VCPU_DAR(r4)
@@ -1018,6 +1020,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_206) lwz r3, LPPACA_YIELDCOUNT(r8) addi r3, r3, 1 stw r3, LPPACA_YIELDCOUNT(r8) + li r3, 1 + stb r3, VCPU_VPA_DIRTY(r9) 25: /* Save PMU registers if requested */ /* r8 and cr0.eq are live here */
Signature · 1 line
1.7.10.4
Alexander Graf
AppliedQuoted 4 lines · Paul Mackerras
This series of two patches fixes a couple of bugs in live migration under HV-style KVM on PPC. They only touch PPC/HV code and add no new APIs. The patches are against Alex Graf's kvm-ppc-next branch. I would like them to go in for 3.10.
Thanks, applied both to kvm-ppc-queue.
Alex
Quoted 16 lines
Paul.
arch/powerpc/include/asm/kvm_book3s.h | 3 +- arch/powerpc/include/asm/kvm_book3s_64.h | 13 ++++ arch/powerpc/include/asm/kvm_host.h | 2 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 120 +++++++++++++++++++++++++----- arch/powerpc/kvm/book3s_hv.c | 30 +++++--- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 11 --- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 + 8 files changed, 143 insertions(+), 41 deletions(-)--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html