KVM: PPC: Update Book3S HV memory handling
Subsystems (3)
v3 cover KVM: PPC: Update Book3S HV memory handling
This series of patches updates the Book3S-HV KVM code that manages the guest hashed page table (HPT) to enable several things:
* MMIO emulation and MMIO pass-through
* Use of small pages (4kB or 64kB, depending on config) to back the guest memory
* Pageable guest memory - i.e. backing pages can be removed from the guest and reinstated on demand, using the MMU notifier mechanism
* Guests can be given read-only access to pages even though they think they have mapped them read/write. When they try to write to them their access is upgraded to read/write. This allows KSM to share pages between guests.
On PPC970 we have no way to get DSIs and ISIs to come to the hypervisor, so we can't do MMIO emulation or pageable guest memory. On POWER7 we set the VPM1 bit in the LPCR to make all DSIs and ISIs come to the hypervisor (host) as HDSIs or HISIs.
This code is working well in my tests. The sporadic crashes that I was seeing earlier are fixed by the second patch in the series. Somewhat to my surprise, when I implemented the last patch in the series I started to see KSM coalescing pages without any further effort on my part -- my tests were on a machine with Fedora 16 installed, and it has ksmtuned running by default.
This series is on top of Alex Graf's kvm-ppc-next branch. The first patch in my series fixes a bug in one of the patches in that branch ("KVM: PPC: booke: Improve timer register emulation").
These patches only touch arch/powerpc except for patch 12, which adds a couple of barriers to allow mmu_notifier_retry() to be used outside of the kvm->mmu_lock.
Paul.
PATCH v3 1/14 KVM: PPC: Make wakeups work again for Book3S HV guests
When commit f43fdc15fa ("KVM: PPC: booke: Improve timer register emulation") factored out some code in arch/powerpc/kvm/powerpc.c into a new helper function, kvm_vcpu_kick(), an error crept in which causes Book3s HV guest vcpus to stall. This fixes it. On POWER7 machines, guest vcpus are grouped together into virtual CPU cores that share a single waitqueue, so it's important to use vcpu->arch.wqp rather than &vcpu->wq.
--- arch/powerpc/kvm/powerpc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/powerpc/kvm/powerpc.c +1 -1
@@ -561,7 +561,7 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)int cpu = vcpu->cpu; me =get_cpu (); -if (waitqueue_active (&vcpu->wq)) { +if (waitqueue_active (vcpu->arch.wqp)) {wake_up_interruptible (vcpu->arch.wqp); vcpu->stat.halt_wakeup++; }else if (cpu != me && cpu !=-1 ) {
Signature · 1 line
1.7.7.3
PATCH v3 2/14 KVM: PPC: Move kvm_vcpu_ioctl_[gs]et_one_reg down to platform-specific code
In reply to Paul MackerrasThis moves the get/set_one_reg implementation down from powerpc.c into booke.c, book3s_pr.c and book3s_hv.c. This avoids #ifdefs in C code, but more importantly, it fixes a bug on Book3s HV where we were accessing beyond the end of the kvm_vcpu struct (via the to_book3s() macro) and corrupting memory, causing random crashes and file corruption.
On Book3s HV we only accept setting the HIOR to zero, since the guest runs in supervisor mode and its vectors are never offset from zero.
--- arch/powerpc/include/asm/kvm_ppc.h | 3 ++ arch/powerpc/kvm/book3s_hv.c | 33 ++++++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_pr.c | 33 ++++++++++++++++++++++++++++++ arch/powerpc/kvm/booke.c | 10 +++++++++ arch/powerpc/kvm/powerpc.c | 39 ------------------------------------ 5 files changed, 79 insertions(+), 39 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +3 -0
@@ -176,6 +176,9 @@ int kvmppc_core_set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs);void kvmppc_get_sregs_ivor (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs);int kvmppc_set_sregs_ivor (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs); +int kvm_vcpu_ioctl_get_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg); +int kvm_vcpu_ioctl_set_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg); +void kvmppc_set_pid (struct kvm_vcpu *vcpu,u32 pid);#ifdef CONFIG_KVM_BOOK3S_64_HV
arch/powerpc/kvm/book3s_hv.c +33 -0
@@ -392,6 +392,39 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,return 0 ; } +int kvm_vcpu_ioctl_get_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +int r = -EINVAL ; + +switch (reg->id) { +case KVM_ONE_REG_PPC_HIOR : + reg->u.reg64 =0 ; + r =0 ; +break ; +default : +break ; + } + +return r; +} + +int kvm_vcpu_ioctl_set_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +int r = -EINVAL ; + +switch (reg->id) { +case KVM_ONE_REG_PPC_HIOR : +/* Only allow this to be set to zero */ +if (reg->u.reg64 ==0 ) + r =0 ; +break ; +default : +break ; + } + +return r; +} +int kvmppc_core_check_processor_compat (void ) {if (cpu_has_feature (CPU_FTR_HVMODE ))
arch/powerpc/kvm/book3s_pr.c +33 -0
@@ -863,6 +863,39 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,return 0 ; } +int kvm_vcpu_ioctl_get_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +int r = -EINVAL ; + +switch (reg->id) { +case KVM_ONE_REG_PPC_HIOR : + reg->u.reg64 =to_book3s (vcpu)->hior; + r =0 ; +break ; +default : +break ; + } + +return r; +} + +int kvm_vcpu_ioctl_set_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +int r = -EINVAL ; + +switch (reg->id) { +case KVM_ONE_REG_PPC_HIOR : +to_book3s (vcpu)->hior = reg->u.reg64; +to_book3s (vcpu)->hior_explicit = true; + r =0 ; +break ; +default : +break ; + } + +return r; +} +int kvmppc_core_check_processor_compat (void ) {return 0 ;
arch/powerpc/kvm/booke.c +10 -0
@@ -887,6 +887,16 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,return kvmppc_core_set_sregs (vcpu, sregs); } +int kvm_vcpu_ioctl_get_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +return -EINVAL ; +} + +int kvm_vcpu_ioctl_set_one_reg (struct kvm_vcpu *vcpu,struct kvm_one_reg *reg) +{ +return -EINVAL ; +} +int kvm_arch_vcpu_ioctl_get_fpu (struct kvm_vcpu *vcpu,struct kvm_fpu *fpu) {return -ENOTSUPP ;
arch/powerpc/kvm/powerpc.c +0 -39
@@ -624,45 +624,6 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,return r; } -static int kvm_vcpu_ioctl_get_one_reg (struct kvm_vcpu *vcpu, -struct kvm_one_reg *reg) -{ -int r = -EINVAL ; - -switch (reg->id) { -#ifdef CONFIG_PPC_BOOK3S -case KVM_ONE_REG_PPC_HIOR : - reg->u.reg64 =to_book3s (vcpu)->hior; - r =0 ; -break ; -#endif -default : -break ; - } - -return r; -} - -static int kvm_vcpu_ioctl_set_one_reg (struct kvm_vcpu *vcpu, -struct kvm_one_reg *reg) -{ -int r = -EINVAL ; - -switch (reg->id) { -#ifdef CONFIG_PPC_BOOK3S -case KVM_ONE_REG_PPC_HIOR : -to_book3s (vcpu)->hior = reg->u.reg64; -to_book3s (vcpu)->hior_explicit = true; - r =0 ; -break ; -#endif -default : -break ; - } - -return r; -} -int kvm_arch_vcpu_ioctl_get_mpstate (struct kvm_vcpu *vcpu,struct kvm_mp_state *mp_state) {
Signature · 1 line
1.7.7.3
PATCH v3 3/14 KVM: PPC: Keep a record of HV guest view of hashed page table entries
In reply to Paul MackerrasThis adds an array that parallels the guest hashed page table (HPT), that is, it has one entry per HPTE, used to store the guest's view of the second doubleword of the corresponding HPTE. The first doubleword in the HPTE is the same as the guest's idea of it, so we don't need to store a copy, but the second doubleword in the HPTE has the real page number rather than the guest's logical page number. This allows us to remove the back_translate() and reverse_xlate() functions.
This "reverse mapping" array is vmalloc'd, meaning that to access it in real mode we have to walk the kernel's page tables explicitly. That is done by the new real_vmalloc_addr() function. (In fact this returns an address in the linear mapping, so the result is usable both in real mode and in virtual mode.)
There are also some minor cleanups here: moving the definitions of HPT_ORDER etc. to a header file and defining HPT_NPTE for HPT_NPTEG << 3.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 8 +++ arch/powerpc/include/asm/kvm_host.h | 10 ++++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 44 +++++++++++---- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 87 ++++++++++++++++++------------ 4 files changed, 103 insertions(+), 46 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +8 -0
@@ -35,6 +35,14 @@ static inline void svcpu_put(struct kvmppc_book3s_shadow_vcpu *svcpu)#define SPAPR_TCE_SHIFT 12 +#ifdef CONFIG_KVM_BOOK3S_64_HV +/* For now use fixed-size 16MB page table */ +#define HPT_ORDER 24 +#define HPT_NPTEG (1ul << (HPT_ORDER - 7))/* 128B per pteg */ +#define HPT_NPTE (HPT_NPTEG << 3)/* 8 PTEs per PTEG */ +#define HPT_HASH_MASK (HPT_NPTEG - 1) +#endif +static inline unsigned long compute_tlbie_rb (unsigned long v,unsigned long r,unsigned long pte_index) {
arch/powerpc/include/asm/kvm_host.h +10 -0
@@ -166,9 +166,19 @@ struct kvmppc_rma_info {atomic_t use_count; }; +/* +* The reverse mapping array has one entry for each HPTE, +* which stores the guest's view of the second word of the HPTE +* (including the guest physical address of the mapping). +*/ +struct revmap_entry { +unsigned long guest_rpte; +}; +struct kvm_arch {#ifdef CONFIG_KVM_BOOK3S_64_HV unsigned long hpt_virt; +struct revmap_entry *revmap;unsigned long ram_npages;unsigned long ram_psize;unsigned long ram_porder;
arch/powerpc/kvm/book3s_64_mmu_hv.c +32 -12
108 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +53 -34
166 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 4/14 KVM: PPC: Keep page physical addresses in per-slot arrays
In reply to Paul MackerrasThis allocates an array for each memory slot that is added to store the physical addresses of the pages in the slot. This array is vmalloc'd and accessed in kvmppc_h_enter using real_vmalloc_addr(). This allows us to remove the ram_pginfo field from the kvm_arch struct, and removes the 64GB guest RAM limit that we had.
We use the low-order bits of the array entries to store a flag indicating that we have done get_page on the corresponding page, and therefore need to call put_page when we are finished with the page. Currently this is set for all pages except those in our special RMO regions.
--- arch/powerpc/include/asm/kvm_host.h | 9 ++- arch/powerpc/kvm/book3s_64_mmu_hv.c | 18 +++--- arch/powerpc/kvm/book3s_hv.c | 114 +++++++++++++++++------------------ arch/powerpc/kvm/book3s_hv_rm_mmu.c | 41 +++++++++++- 4 files changed, 107 insertions(+), 75 deletions(-)
arch/powerpc/include/asm/kvm_host.h +6 -3
@@ -38,6 +38,7 @@#define KVM_MEMORY_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4 +#define KVM_MEM_SLOTS_NUM (KVM_MEMORY_SLOTS + KVM_PRIVATE_MEM_SLOTS)#ifdef CONFIG_KVM_MMIO #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
@@ -175,25 +176,27 @@ struct revmap_entry {unsigned long guest_rpte; }; +/* Low-order bits in kvm->arch.slot_phys[][] */ +#define KVMPPC_GOT_PAGE 0x80 +struct kvm_arch {#ifdef CONFIG_KVM_BOOK3S_64_HV unsigned long hpt_virt;struct revmap_entry *revmap; -unsigned long ram_npages;unsigned long ram_psize;unsigned long ram_porder; -struct kvmppc_pginfo *ram_pginfo;unsigned int lpid;unsigned int host_lpid;unsigned long host_lpcr;unsigned long sdr1;unsigned long host_sdr1;int tlbie_lock; -int n_rma_pages;unsigned long lpcr;unsigned long rmor;struct kvmppc_rma_info *rma;struct list_head spapr_tce_tables; +unsigned long *slot_phys[KVM_MEM_SLOTS_NUM ]; +int slot_npages[KVM_MEM_SLOTS_NUM ];unsigned short last_vcpu[NR_CPUS ];struct kvmppc_vcore *vcores[KVM_MAX_VCORES ];#endif /* CONFIG_KVM_BOOK3S_64_HV */
arch/powerpc/kvm/book3s_64_mmu_hv.c +9 -9
@@ -98,16 +98,16 @@ void kvmppc_free_hpt(struct kvm *kvm)void kvmppc_map_vrma (struct kvm *kvm,struct kvm_userspace_memory_region *mem) {unsigned long i; -unsigned long npages = kvm->arch.ram_npages; -unsigned long pfn; +unsigned long npages; +unsigned long pa;unsigned long *hpte;unsigned long hash;unsigned long porder = kvm->arch.ram_porder;struct revmap_entry *rev; -struct kvmppc_pginfo *pginfo = kvm->arch.ram_pginfo; +unsigned long *physp; -if (!pginfo) -return ; + physp = kvm->arch.slot_phys[mem->slot]; + npages = kvm->arch.slot_npages[mem->slot];/* VRMA can't be > 1TB */ if (npages >1ul << (40 - porder))
@@ -117,9 +117,10 @@ void kvmppc_map_vrma(struct kvm *kvm, struct kvm_userspace_memory_region *mem) npages =HPT_NPTEG ;for (i =0 ; i < npages; ++i) { - pfn = pginfo[i].pfn; -if (!pfn) + pa = physp[i]; +if (!pa)break ; + pa &=PAGE_MASK ;/* can't use hpt_hash since va > 64 bits */ hash = (i ^ (VRMA_VSID ^ (VRMA_VSID <<25 ))) &HPT_HASH_MASK ;/*
@@ -131,8 +132,7 @@ void kvmppc_map_vrma(struct kvm *kvm, struct kvm_userspace_memory_region *mem) hash = (hash <<3 ) +7 ; hpte = (unsigned long *) (kvm->arch.hpt_virt + (hash <<4 ));/* HPTE low word - RPN, protection, etc. */ - hpte[1 ] = (pfn <<PAGE_SHIFT ) |HPTE_R_R |HPTE_R_C | -HPTE_R_M |PP_RWXX ; + hpte[1 ] = pa |HPTE_R_R |HPTE_R_C |HPTE_R_M |PP_RWXX ;smp_wmb (); hpte[0 ] =HPTE_V_1TB_SEG | (VRMA_VSID << (40 -16 )) | (i << (VRMA_PAGE_ORDER - 16)) | HPTE_V_BOLTED |
arch/powerpc/kvm/book3s_hv.c +55 -59
209 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +37 -4
65 lines not on this page · raw message
Signature · 1 line
1.7.7.3
Re: [PATCH v3 04/14] KVM: PPC: Keep page physical addresses in per-slot arrays
Quoted 343 lines · Paul Mackerras
This allocates an array for each memory slot that is added to store the physical addresses of the pages in the slot. This array is vmalloc'd and accessed in
kvmppc_h_enterusingreal_vmalloc_addr(). This allows us to remove theram_pginfofield from thekvm_archstruct, and removes the 64GB guest RAM limit that we had.We use the low-order bits of the array entries to store a flag indicating that we have done
get_pageon the corresponding page, and therefore need to callput_pagewhen we are finished with the page. Currently this is set for all pages except those in our special RMO regions.Signed-off-by Paul Mackerras <[email protected]>---
arch/powerpc/include/asm/kvm_host.h | 9 ++- arch/powerpc/kvm/book3s_64_mmu_hv.c | 18 +++--- arch/powerpc/kvm/book3s_hv.c | 114 +++++++++++++++++------------------ arch/powerpc/kvm/book3s_hv_rm_mmu.c | 41 +++++++++++- 4 files changed, 107 insertions(+), 75 deletions(-)arch/powerpc/include/asm/kvm_host.h +6
-3@@ -38,6 +38,7 @@#define KVM_MEMORY_SLOTS 32/* memory slots that does not exposed to userspace */ #define KVM_PRIVATE_MEM_SLOTS 4 +#define KVM_MEM_SLOTS_NUM (KVM_MEMORY_SLOTS + KVM_PRIVATE_MEM_SLOTS)#ifdef CONFIG_KVM_MMIO #define KVM_COALESCED_MMIO_PAGE_OFFSET 1@@ -175,25 +176,27 @@ struct revmap_entry {unsigned long guest_rpte; }; +/* Low-order bits in kvm->arch.slot_phys[][] */ +#define KVMPPC_GOT_PAGE 0x80 +struct kvm_arch {#ifdef CONFIG_KVM_BOOK3S_64_HV unsigned long hpt_virt;struct revmap_entry *revmap; -unsigned long ram_npages;unsigned long ram_psize;unsigned long ram_porder; -struct kvmppc_pginfo *ram_pginfo;unsigned int lpid;unsigned int host_lpid;unsigned long host_lpcr;unsigned long sdr1;unsigned long host_sdr1;int tlbie_lock; -int n_rma_pages;unsigned long lpcr;unsigned long rmor;struct kvmppc_rma_info *rma;struct list_head spapr_tce_tables; +unsigned long *slot_phys[KVM_MEM_SLOTS_NUM ]; +int slot_npages[KVM_MEM_SLOTS_NUM ];unsigned short last_vcpu[NR_CPUS ];struct kvmppc_vcore *vcores[KVM_MAX_VCORES ];#endif /* CONFIG_KVM_BOOK3S_64_HV */ arch/powerpc/kvm/book3s_64_mmu_hv.c +9
-9@@ -98,16 +98,16 @@ void kvmppc_free_hpt(struct kvm *kvm)void kvmppc_map_vrma (struct kvm *kvm,struct kvm_userspace_memory_region *mem) {unsigned long i; -unsigned long npages = kvm->arch.ram_npages; -unsigned long pfn; +unsigned long npages; +unsigned long pa;unsigned long *hpte;unsigned long hash;unsigned long porder = kvm->arch.ram_porder;struct revmap_entry *rev; -struct kvmppc_pginfo *pginfo = kvm->arch.ram_pginfo; +unsigned long *physp; -if (!pginfo) -return ; + physp = kvm->arch.slot_phys[mem->slot]; + npages = kvm->arch.slot_npages[mem->slot];/* VRMA can't be > 1TB */ if (npages >1ul << (40 - porder))@@ -117,9 +117,10 @@ void kvmppc_map_vrma(struct kvm *kvm, struct kvm_userspace_memory_region *mem) npages =HPT_NPTEG ;for (i =0 ; i < npages; ++i) { - pfn = pginfo[i].pfn; -if (!pfn) + pa = physp[i]; +if (!pa)break ; + pa &=PAGE_MASK ;/* can't use hpt_hash since va > 64 bits */ hash = (i ^ (VRMA_VSID ^ (VRMA_VSID <<25 ))) &HPT_HASH_MASK ;/* @@ -131,8 +132,7 @@ void kvmppc_map_vrma(struct kvm *kvm, struct kvm_userspace_memory_region *mem) hash = (hash <<3 ) +7 ; hpte = (unsigned long *) (kvm->arch.hpt_virt + (hash <<4 ));/* HPTE low word - RPN, protection, etc. */ - hpte[1 ] = (pfn <<PAGE_SHIFT ) |HPTE_R_R |HPTE_R_C | -HPTE_R_M |PP_RWXX ; + hpte[1 ] = pa |HPTE_R_R |HPTE_R_C |HPTE_R_M |PP_RWXX ;smp_wmb (); hpte[0 ] =HPTE_V_1TB_SEG | (VRMA_VSID << (40 -16 )) | (i << (VRMA_PAGE_ORDER - 16)) | HPTE_V_BOLTED |arch/powerpc/kvm/book3s_hv.c +55
-59@@ -50,14 +50,6 @@#include <linux/vmalloc.h> #include <linux/highmem.h> -/* -* For now, limit memory to 64GB and require it to be large pages. -* This value is chosen because it makes the ram_pginfo array be -* 64kB in size, which is about as large as we want to be trying -* to allocate with kmalloc. -*/ -#define MAX_MEM_ORDER 36 -#define LARGE_PAGE_ORDER 24/* 16MB pages */ /* #define EXIT_DEBUG */ @@ -147,10 +139,12 @@ static unsigned long do_h_register_vpa(struct kvm_vcpu *vcpu, unsigned long vcpuid, unsigned long vpa) {struct kvm *kvm = vcpu->kvm; -unsigned long pg_index, ra, len; +unsigned long gfn, pg_index, ra, len;unsigned long pg_offset;void *va;struct kvm_vcpu *tvcpu; +struct kvm_memory_slot *memslot; +unsigned long *physp; tvcpu =kvmppc_find_vcpu (kvm, vcpuid);if (!tvcpu)@@ -164,14 +158,20 @@ static unsigned long do_h_register_vpa(struct kvm_vcpu *vcpu,if (vpa &0x7f )return H_PARAMETER ;/* registering new area; convert logical addr to real */ - pg_index = vpa >> kvm->arch.ram_porder; - pg_offset = vpa & (kvm->arch.ram_psize -1 ); -if (pg_index >= kvm->arch.ram_npages) + gfn = vpa >>PAGE_SHIFT ; + memslot =gfn_to_memslot (kvm, gfn); +if (!memslot || !(memslot->flags &KVM_MEMSLOT_INVALID )) +return H_PARAMETER ; + physp = kvm->arch.slot_phys[memslot->id]; +if (!physp)return H_PARAMETER ; -if (kvm->arch.ram_pginfo[pg_index].pfn =0 ) + pg_index = (gfn - memslot->base_gfn) >> + (kvm->arch.ram_porder -PAGE_SHIFT ); + pg_offset = vpa & (kvm->arch.ram_psize -1 ); + ra = physp[pg_index]; +if (!ra)return H_PARAMETER ; - ra = kvm->arch.ram_pginfo[pg_index].pfn <<PAGE_SHIFT ; - ra |= pg_offset; + ra = (ra &PAGE_MASK ) | pg_offset; va =__va (ra);if (flags <=1 ) len = *(unsigned short *)(va +4 );@@ -1108,12 +1108,11 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm, struct kvm_userspace_memory_region *mem) {unsigned long psize, porder; -unsigned long i, npages, totalpages; -unsigned long pg_ix; -struct kvmppc_pginfo *pginfo; +unsigned long i, npages;unsigned long hva;struct kvmppc_rma_info *ri =NULL ;struct page *page; +unsigned long *phys;/* For now, only allow 16MB pages */ porder =LARGE_PAGE_ORDER ;@@ -1125,20 +1124,21 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm,return -EINVAL ; } +/* Allocate a slot_phys array */ npages = mem->memory_size >> porder; - totalpages = (mem->guest_phys_addr + mem->memory_size) >> porder; - -/* More memory than we have space to track? */ -if (totalpages > (1ul << (MAX_MEM_ORDER -LARGE_PAGE_ORDER ))) -return -EINVAL ; + phys = kvm->arch.slot_phys[mem->slot]; +if (!phys) { + phys =vzalloc (npages *sizeof (unsigned long )); +if (!phys) +return -ENOMEM ; + kvm->arch.slot_phys[mem->slot] = phys; + kvm->arch.slot_npages[mem->slot] = npages; + }/* Do we already have an RMA registered? */ if (mem->guest_phys_addr =0 && kvm->arch.rma)return -EINVAL ; -if (totalpages > kvm->arch.ram_npages) - kvm->arch.ram_npages = totalpages; -/* Is this one of our preallocated RMAs? */ if (mem->guest_phys_addr =0 ) {struct vm_area_struct *vma;@@ -1171,7 +1171,6 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm, }atomic_inc (&ri->use_count); kvm->arch.rma = ri; - kvm->arch.n_rma_pages = rma_size >> porder;/* Update LPCR and RMOR */ lpcr = kvm->arch.lpcr;@@ -1195,12 +1194,9 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm, ri->base_pfn << PAGE_SHIFT, rma_size, lpcr); } - pg_ix = mem->guest_phys_addr >> porder; - pginfo = kvm->arch.ram_pginfo + pg_ix; -for (i =0 ; i < npages; ++i, ++pg_ix) { -if (ri && pg_ix < kvm->arch.n_rma_pages) { - pginfo[i].pfn = ri->base_pfn + - (pg_ix << (porder -PAGE_SHIFT )); +for (i =0 ; i < npages; ++i) { +if (ri && i < ri->npages) { + phys[i] = (ri->base_pfn <<PAGE_SHIFT ) + (i << porder);continue ; } hva = mem->userspace_addr + (i << porder);@@ -1216,7 +1212,7 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm, hva, compound_order(page)); goto err; } - pginfo[i].pfn =page_to_pfn (page); + phys[i] = (page_to_pfn (page) <<PAGE_SHIFT ) |KVMPPC_GOT_PAGE ; }return 0 ;@@ -1225,6 +1221,28 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm,return -EINVAL ; } +static void unpin_slot (struct kvm *kvm,int slot_id) +{ +unsigned long *physp; +unsigned long j, npages, pfn; +struct page *page; + + physp = kvm->arch.slot_phys[slot_id]; + npages = kvm->arch.slot_npages[slot_id]; +if (physp) { +for (j =0 ; j < npages; j++) { +if (!(physp[j] &KVMPPC_GOT_PAGE )) +continue ; + pfn = physp[j] >>PAGE_SHIFT ; + page =pfn_to_page (pfn); +SetPageDirty (page); +put_page (page); + } +vfree (physp); + kvm->arch.slot_phys[slot_id] =NULL ; + } +} +void kvmppc_core_commit_memory_region (struct kvm *kvm,struct kvm_userspace_memory_region *mem) {@@ -1236,8 +1254,6 @@ void kvmppc_core_commit_memory_region(struct kvm *kvm,int kvmppc_core_init_vm (struct kvm *kvm) {long r; -unsigned long npages =1ul << (MAX_MEM_ORDER -LARGE_PAGE_ORDER ); -long err = -ENOMEM ;unsigned long lpcr;/* Allocate hashed page table */ @@ -1247,19 +1263,9 @@ int kvmppc_core_init_vm(struct kvm *kvm)INIT_LIST_HEAD (&kvm->arch.spapr_tce_tables); - kvm->arch.ram_pginfo =kzalloc (npages *sizeof (struct kvmppc_pginfo ), -GFP_KERNEL ); -if (!kvm->arch.ram_pginfo) { -pr_err ("kvmppc_core_init_vm: couldn't alloc %lu bytes\n" , - npages *sizeof (struct kvmppc_pginfo )); - goto out_free; - } - - kvm->arch.ram_npages =0 ; kvm->arch.ram_psize =1ul <<LARGE_PAGE_ORDER ; kvm->arch.ram_porder =LARGE_PAGE_ORDER ; kvm->arch.rma =NULL ; - kvm->arch.n_rma_pages =0 ; kvm->arch.host_sdr1 =mfspr (SPRN_SDR1 );@@ -1282,25 +1288,15 @@ int kvmppc_core_init_vm(struct kvm *kvm) kvm->arch.lpcr = lpcr;return 0 ; - - out_free: -kvmppc_free_hpt (kvm); -return err; }void kvmppc_core_destroy_vm (struct kvm *kvm) { -struct kvmppc_pginfo *pginfo;unsigned long i; -if (kvm->arch.ram_pginfo) { - pginfo = kvm->arch.ram_pginfo; - kvm->arch.ram_pginfo =NULL ; -for (i = kvm->arch.n_rma_pages; i < kvm->arch.ram_npages; ++i) -if (pginfo[i].pfn) -put_page (pfn_to_page (pginfo[i].pfn)); -kfree (pginfo); - } +for (i =0 ; i <KVM_MEM_SLOTS_NUM ; i++) +unpin_slot (kvm, i); +if (kvm->arch.rma) {kvm_release_rma (kvm->arch.rma); kvm->arch.rma =NULL ;arch/powerpc/kvm/book3s_hv_rm_mmu.c +0
-0@@ -20,6 +20,25 @@#include <asm/synch.h>
arch/powerpc/kvm/book3s_hv_rm_mmu.c line 30
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_rm_mmu.c +8
-0#include <asm/ppc-opcode.h> +/* +* Since this file is built in even if KVM is a module, we need +* a local copy of this function for the case where kvm_main.c is +* modular. +*/ +static struct kvm_memory_slot *builtin_gfn_to_memslot (struct kvm *kvm, +gfn_t gfn) +{
Shouldn't this rather be in a header file then? I'd rather not have this code duplicated. Please follow up with a patch to merge this copy and the real one into something in a header file.
Alex
PATCH v3 5/14 KVM: PPC: Add an interface for pinning guest pages in Book3s HV guests
In reply to Paul MackerrasThis adds two new functions, kvmppc_pin_guest_page() and kvmppc_unpin_guest_page(), and uses them to pin the guest pages where the guest has registered areas of memory for the hypervisor to update, (i.e. the per-cpu virtual processor areas, SLB shadow buffers and dispatch trace logs) and then unpin them when they are no longer required.
Although it is not strictly necessary to pin the pages at this point, since all guest pages are already pinned, later commits in this series will mean that guest pages aren't all pinned.
--- arch/powerpc/include/asm/kvm_book3s.h | 3 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 38 ++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 67 ++++++++++++++++++--------------- 3 files changed, 78 insertions(+), 30 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +3 -0
@@ -140,6 +140,9 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat,extern void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr);extern int kvmppc_emulate_paired_single (struct kvm_run *run,struct kvm_vcpu *vcpu);extern pfn_t kvmppc_gfn_to_pfn (struct kvm_vcpu *vcpu,gfn_t gfn); +extern void *kvmppc_pin_guest_page (struct kvm *kvm,unsigned long addr, +unsigned long *nb_ret); +extern void kvmppc_unpin_guest_page (struct kvm *kvm,void *addr);extern void kvmppc_entry_trampoline (void );extern void kvmppc_hv_entry_trampoline (void );
arch/powerpc/kvm/book3s_64_mmu_hv.c +38 -0
@@ -184,6 +184,44 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,return -ENOENT ; } +void *kvmppc_pin_guest_page (struct kvm *kvm,unsigned long gpa, +unsigned long *nb_ret) +{ +struct kvm_memory_slot *memslot; +unsigned long gfn = gpa >>PAGE_SHIFT ; +struct page *page; +unsigned long offset; +unsigned long pfn, pa; +unsigned long *physp; + + memslot =gfn_to_memslot (kvm, gfn); +if (!memslot || (memslot->flags &KVM_MEMSLOT_INVALID )) +return NULL ; + physp = kvm->arch.slot_phys[memslot->id]; +if (!physp) +return NULL ; + physp += (gfn - memslot->base_gfn) >> + (kvm->arch.ram_porder -PAGE_SHIFT ); + pa = *physp; +if (!pa) +return NULL ; + pfn = pa >>PAGE_SHIFT ; + page =pfn_to_page (pfn); +get_page (page); + offset = gpa & (kvm->arch.ram_psize -1 ); +if (nb_ret) + *nb_ret = kvm->arch.ram_psize - offset; +return page_address (page) + offset; +} + +void kvmppc_unpin_guest_page (struct kvm *kvm,void *va) +{ +struct page *page =virt_to_page (va); + + page =compound_head (page); +put_page (page); +} +void kvmppc_mmu_book3s_hv_init (struct kvm_vcpu *vcpu) {struct kvmppc_mmu *mmu = &vcpu->arch.mmu;
arch/powerpc/kvm/book3s_hv.c +37 -30
123 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 6/14 KVM: PPC: Make the H_ENTER hcall more reliable
In reply to Paul MackerrasAt present, our implementation of H_ENTER only makes one try at locking each slot that it looks at, and doesn't even retry the ldarx/stdcx. atomic update sequence that it uses to attempt to lock the slot. Thus it can return the H_PTEG_FULL error unnecessarily, particularly when the H_EXACT flag is set, meaning that the caller wants a specific PTEG slot.
This improves the situation by making a second pass when no free HPTE slot is found, where we spin until we succeed in locking each slot in turn and then check whether it is full while we hold the lock. If the second pass fails, then we return H_PTEG_FULL.
This also moves lock_hpte to a header file (since later commits in this series will need to use it from other source files) and renames it to try_lock_hpte, which is a somewhat less misleading name.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 25 ++++++++++++ arch/powerpc/kvm/book3s_hv_rm_mmu.c | 63 ++++++++++++++++-------------- 2 files changed, 59 insertions(+), 29 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +25 -0
@@ -43,6 +43,31 @@ static inline void svcpu_put(struct kvmppc_book3s_shadow_vcpu *svcpu)#define HPT_HASH_MASK (HPT_NPTEG - 1)#endif +/* +* We use a lock bit in HPTE dword 0 to synchronize updates and +* accesses to each HPTE, and another bit to indicate non-present +* HPTEs. +*/ +#define HPTE_V_HVLOCK 0x40UL + +static inline long try_lock_hpte (unsigned long *hpte,unsigned long bits) +{ +unsigned long tmp, old; + + asmvolatile (" ldarx %0,0,%2\n" +" and. %1,%0,%3\n" +" bne 2f\n" +" ori %0,%0,%4\n" +" stdcx. %0,0,%2\n" +" beq+ 2f\n" +" li %1,%3\n" +"2: isync" + :"=&r" (tmp),"=&r" (old) + :"r" (hpte),"r" (bits),"i" (HPTE_V_HVLOCK ) + :"cc" ,"memory" ); +return old =0 ; +} +static inline unsigned long compute_tlbie_rb (unsigned long v,unsigned long r,unsigned long pte_index) {
arch/powerpc/kvm/book3s_hv_rm_mmu.c +34 -29
105 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 7/14 KVM: PPC: Only get pages when actually needed, not in prepare_memory_region()
In reply to Paul MackerrasThis removes the code from kvmppc_core_prepare_memory_region() that looked up the VMA for the region being added and called hva_to_page to get the pfns for the memory. We have no guarantee that there will be anything mapped there at the time of the KVM_SET_USER_MEMORY_REGION ioctl call; userspace can do that ioctl and then map memory into the region later.
Instead we defer looking up the pfn for each memory page until it is needed, which generally means when the guest does an H_ENTER hcall on the page. Since we can't call get_user_pages in real mode, if we don't already have the pfn for the page, kvmppc_h_enter() will return H_TOO_HARD and we then call kvmppc_virtmode_h_enter() once we get back to kernel context. That calls kvmppc_get_guest_page() to get the pfn for the page, and then calls back to kvmppc_h_enter() to redo the HPTE insertion.
When the first vcpu starts executing, we need to have the RMO or VRMA region mapped so that the guest's real mode accesses will work. Thus we now have a check in kvmppc_vcpu_run() to see if the RMO/VRMA is set up and if not, call kvmppc_hv_setup_rma(). It checks if the memslot starting at guest physical 0 now has RMO memory mapped there; if so it sets it up for the guest, otherwise on POWER7 it sets up the VRMA. The function that does that, kvmppc_map_vrma, is now a bit simpler, as it calls kvmppc_virtmode_h_enter instead of creating the HPTE itself.
Since we are now potentially updating entries in the slot_phys[] arrays from multiple vcpu threads, we now have a spinlock protecting those updates to ensure that we don't lose track of any references to pages.
--- arch/powerpc/include/asm/kvm_book3s.h | 4 + arch/powerpc/include/asm/kvm_book3s_64.h | 12 ++ arch/powerpc/include/asm/kvm_host.h | 2 + arch/powerpc/include/asm/kvm_ppc.h | 4 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 130 +++++++++++++--- arch/powerpc/kvm/book3s_hv.c | 244 +++++++++++++++++------------- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 54 ++++---- 7 files changed, 290 insertions(+), 160 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +4 -0
@@ -143,6 +143,10 @@ extern pfn_t kvmppc_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn);extern void *kvmppc_pin_guest_page (struct kvm *kvm,unsigned long addr,unsigned long *nb_ret);extern void kvmppc_unpin_guest_page (struct kvm *kvm,void *addr); +extern long kvmppc_virtmode_h_enter (struct kvm_vcpu *vcpu,unsigned long flags, +long pte_index,unsigned long pteh,unsigned long ptel); +extern long kvmppc_h_enter (struct kvm_vcpu *vcpu,unsigned long flags, +long pte_index,unsigned long pteh,unsigned long ptel);extern void kvmppc_entry_trampoline (void );extern void kvmppc_hv_entry_trampoline (void );
arch/powerpc/include/asm/kvm_book3s_64.h +12 -0
@@ -101,4 +101,16 @@ static inline unsigned long compute_tlbie_rb(unsigned long v, unsigned long r, return rb; } +static inline unsigned long hpte_page_size (unsigned long h,unsigned long l) +{ +/* only handle 4k, 64k and 16M pages for now */ +if (!(h &HPTE_V_LARGE )) +return 1ul <<12 ;/* 4k page */ +if ((l &0xf000 ) ==0x1000 &&cpu_has_feature (CPU_FTR_ARCH_206 )) +return 1ul <<16 ;/* 64k page */ +if ((l &0xff000 ) ==0 ) +return 1ul <<24 ;/* 16M page */ +return 0 ;/* error */ +} +#endif /* __ASM_KVM_BOOK3S_64_H__ */
arch/powerpc/include/asm/kvm_host.h +2 -0
@@ -194,7 +194,9 @@ struct kvm_arch {unsigned long lpcr;unsigned long rmor;struct kvmppc_rma_info *rma; +int rma_setup_done;struct list_head spapr_tce_tables; +spinlock_t slot_phys_lock;unsigned long *slot_phys[KVM_MEM_SLOTS_NUM ];int slot_npages[KVM_MEM_SLOTS_NUM ];unsigned short last_vcpu[NR_CPUS ];
arch/powerpc/include/asm/kvm_ppc.h +2 -2
@@ -121,8 +121,8 @@ extern long kvmppc_alloc_hpt(struct kvm *kvm);extern void kvmppc_free_hpt (struct kvm *kvm);extern long kvmppc_prepare_vrma (struct kvm *kvm,struct kvm_userspace_memory_region *mem); -extern void kvmppc_map_vrma (struct kvm *kvm, -struct kvm_userspace_memory_region *mem); +extern void kvmppc_map_vrma (struct kvm_vcpu *vcpu, +struct kvm_memory_slot *memslot);extern int kvmppc_pseries_do_hcall (struct kvm_vcpu *vcpu);extern long kvm_vm_ioctl_create_spapr_tce (struct kvm *kvm,struct kvm_create_spapr_tce *args);
arch/powerpc/kvm/book3s_64_mmu_hv.c +106 -24
167 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +137 -107
315 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +27 -27
95 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 8/14 KVM: PPC: Allow use of small pages to back Book3S HV guests
In reply to Paul MackerrasThis relaxes the requirement that the guest memory be provided as 16MB huge pages, allowing it to be provided as normal memory, i.e. in pages of PAGE_SIZE bytes (4k or 64k). To allow this, we index the kvm->arch.slot_phys[] arrays with a small page index, even if huge pages are being used, and use the low-order 5 bits of each entry to store the order of the enclosing page with respect to normal pages, i.e. log_2(enclosing_page_size / PAGE_SIZE).
--- arch/powerpc/include/asm/kvm_book3s_64.h | 10 +++ arch/powerpc/include/asm/kvm_host.h | 3 +- arch/powerpc/include/asm/kvm_ppc.h | 2 +- arch/powerpc/include/asm/reg.h | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 122 ++++++++++++++++++++---------- arch/powerpc/kvm/book3s_hv.c | 57 ++++++++------ arch/powerpc/kvm/book3s_hv_rm_mmu.c | 6 +- 7 files changed, 132 insertions(+), 69 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +10 -0
@@ -113,4 +113,14 @@ static inline unsigned long hpte_page_size(unsigned long h, unsigned long l) return 0; /* error */ } +static inline bool slot_is_aligned (struct kvm_memory_slot *memslot, +unsigned long pagesize) +{ +unsigned long mask = (pagesize >>PAGE_SHIFT ) -1 ; + +if (pagesize <=PAGE_SIZE ) +return 1 ; +return !(memslot->base_gfn & mask) && !(memslot->npages & mask); +} +#endif /* __ASM_KVM_BOOK3S_64_H__ */
arch/powerpc/include/asm/kvm_host.h +1 -2
@@ -177,14 +177,13 @@ struct revmap_entry { };/* Low-order bits in kvm->arch.slot_phys[][] */ +#define KVMPPC_PAGE_ORDER_MASK 0x1f#define KVMPPC_GOT_PAGE 0x80struct kvm_arch {#ifdef CONFIG_KVM_BOOK3S_64_HV unsigned long hpt_virt;struct revmap_entry *revmap; -unsigned long ram_psize; -unsigned long ram_porder;unsigned int lpid;unsigned int host_lpid;unsigned long host_lpcr;
arch/powerpc/include/asm/kvm_ppc.h +1 -1
@@ -122,7 +122,7 @@ extern void kvmppc_free_hpt(struct kvm *kvm);extern long kvmppc_prepare_vrma (struct kvm *kvm,struct kvm_userspace_memory_region *mem);extern void kvmppc_map_vrma (struct kvm_vcpu *vcpu, -struct kvm_memory_slot *memslot); +struct kvm_memory_slot *memslot,unsigned long porder);extern int kvmppc_pseries_do_hcall (struct kvm_vcpu *vcpu);extern long kvm_vm_ioctl_create_spapr_tce (struct kvm *kvm,struct kvm_create_spapr_tce *args);
arch/powerpc/include/asm/reg.h +1 -0
@@ -237,6 +237,7 @@#define LPCR_ISL (1ul << (63-2))#define LPCR_VC_SH (63-2)#define LPCR_DPFD_SH (63-11) +#define LPCR_VRMASD (0x1ful << (63-16))#define LPCR_VRMA_L (1ul << (63-12))#define LPCR_VRMA_LP0 (1ul << (63-15))#define LPCR_VRMA_LP1 (1ul << (63-16))
arch/powerpc/kvm/book3s_64_mmu_hv.c +83 -39
206 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +31 -26
123 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +5 -1
@@ -77,6 +77,10 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags, memslot =builtin_gfn_to_memslot (kvm, gfn);if (!(memslot && !(memslot->flags &KVM_MEMSLOT_INVALID )))return H_PARAMETER ; + +/* Check if the requested page fits entirely in the memslot. */ +if (!slot_is_aligned (memslot, psize)) +return H_PARAMETER ; slot_fn = gfn - memslot->base_gfn; physp = kvm->arch.slot_phys[memslot->id];
@@ -88,9 +92,9 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags, pa = *physp;if (!pa)return H_TOO_HARD ; + pte_size =PAGE_SIZE << (pa &KVMPPC_PAGE_ORDER_MASK ); pa &=PAGE_MASK ; - pte_size = kvm->arch.ram_psize;if (pte_size < psize)return H_PARAMETER ;if (pa && pte_size > psize)
Signature · 1 line
1.7.7.3
PATCH v3 9/14 KVM: PPC: Allow I/O mappings in memory slots
In reply to Paul MackerrasThis provides for the case where userspace maps an I/O device into the address range of a memory slot using a VM_PFNMAP mapping. In that case, we work out the pfn from vma->vm_pgoff, and record the cache enable bits from vma->vm_page_prot in two low-order bits in the slot_phys array entries. Then, in kvmppc_h_enter() we check that the cache bits in the HPTE that the guest wants to insert match the cache bits in the slot_phys array entry. However, we do allow the guest to create what it thinks is a non-cacheable or write-through mapping to memory that is actually cacheable, so that we can use normal system memory as part of an emulated device later on. In that case the actual HPTE we insert is a cacheable HPTE.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 26 ++++++++++++ arch/powerpc/include/asm/kvm_host.h | 2 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 65 ++++++++++++++++++++---------- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 15 +++++- 4 files changed, 84 insertions(+), 24 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +26 -0
@@ -113,6 +113,32 @@ static inline unsigned long hpte_page_size(unsigned long h, unsigned long l)return 0 ;/* error */ } +static inline int hpte_cache_flags_ok (unsigned long ptel,unsigned long io_type) +{ +unsigned int wimg = ptel &HPTE_R_WIMG ; + +/* Handle SAO */ +if (wimg = (HPTE_R_W |HPTE_R_I |HPTE_R_M ) && +cpu_has_feature (CPU_FTR_ARCH_206 )) + wimg =HPTE_R_M ; + +if (!io_type) +return wimg =HPTE_R_M ; + +return (wimg & (HPTE_R_W |HPTE_R_I )) = io_type; +} + +/* Return HPTE cache control bits corresponding to Linux pte bits */ +static inline unsigned long hpte_cache_bits (unsigned long pte_val) +{ +#if _PAGE_NO_CACHE = HPTE_R_I && _PAGE_WRITETHRU = HPTE_R_W +return pte_val & (HPTE_R_W |HPTE_R_I ); +#else +return ((pte_val & _PAGE_NO_CACHE) ?HPTE_R_I :0 ) + + ((pte_val & _PAGE_WRITETHRU) ?HPTE_R_W :0 ); +#endif +} +static inline bool slot_is_aligned (struct kvm_memory_slot *memslot,unsigned long pagesize) {
arch/powerpc/include/asm/kvm_host.h +2 -0
@@ -178,6 +178,8 @@ struct revmap_entry {/* Low-order bits in kvm->arch.slot_phys[][] */ #define KVMPPC_PAGE_ORDER_MASK 0x1f +#define KVMPPC_PAGE_NO_CACHE HPTE_R_I/* 0x20 */ +#define KVMPPC_PAGE_WRITETHRU HPTE_R_W/* 0x40 */ #define KVMPPC_GOT_PAGE 0x80struct kvm_arch {
arch/powerpc/kvm/book3s_64_mmu_hv.c +44 -21
98 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rm_mmu.c +12 -3
@@ -65,6 +65,7 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags,unsigned long g_ptel = ptel;struct kvm_memory_slot *memslot;unsigned long *physp, pte_size; +unsigned long is_io;bool realmode = vcpu->arch.vcore->vcore_state =VCORE_RUNNING ; psize =hpte_page_size (pteh, ptel);
@@ -92,6 +93,7 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags, pa = *physp;if (!pa)return H_TOO_HARD ; + is_io = pa & (HPTE_R_I |HPTE_R_W ); pte_size =PAGE_SIZE << (pa &KVMPPC_PAGE_ORDER_MASK ); pa &=PAGE_MASK ;
@@ -104,9 +106,16 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags, ptel |= pa;/* Check WIMG */ -if ((ptel &HPTE_R_WIMG ) !=HPTE_R_M && - (ptel &HPTE_R_WIMG ) != (HPTE_R_W |HPTE_R_I |HPTE_R_M )) -return H_PARAMETER ; +if (!hpte_cache_flags_ok (ptel, is_io)) { +if (is_io) +return H_PARAMETER ; +/* +* Allow guest to map emulated device memory as +* uncacheable, but actually make it cacheable. +*/ + ptel &= ~(HPTE_R_W |HPTE_R_I |HPTE_R_G ); + ptel |=HPTE_R_M ; + } pteh &= ~0x60UL ; pteh |=HPTE_V_VALID ;
Signature · 1 line
1.7.7.3
PATCH v3 10/14 KVM: PPC: Maintain a doubly-linked list of guest HPTEs for each gfn
In reply to Paul MackerrasThis expands the reverse mapping array to contain two links for each HPTE which are used to link together HPTEs that correspond to the same guest logical page. Each circular list of HPTEs is pointed to by the rmap array entry for the guest logical page, pointed to by the relevant memslot. Links are 32-bit HPT entry indexes rather than full 64-bit pointers, to save space. We use 3 of the remaining 32 bits in the rmap array entries as a lock bit, a referenced bit and a present bit (the present bit is needed since HPTE index 0 is valid). The bit lock for the rmap chain nests inside the HPTE lock bit.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 18 ++++++ arch/powerpc/include/asm/kvm_host.h | 17 ++++++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 84 +++++++++++++++++++++++++++++- 3 files changed, 117 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +18 -0
@@ -113,6 +113,11 @@ static inline unsigned long hpte_page_size(unsigned long h, unsigned long l)return 0 ;/* error */ } +static inline unsigned long hpte_rpn (unsigned long ptel,unsigned long psize) +{ +return ((ptel &HPTE_R_RPN ) & ~(psize -1 )) >>PAGE_SHIFT ; +} +static inline int hpte_cache_flags_ok (unsigned long ptel,unsigned long io_type) {unsigned int wimg = ptel &HPTE_R_WIMG ;
@@ -139,6 +144,19 @@ static inline unsigned long hpte_cache_bits(unsigned long pte_val)#endif } +static inline void lock_rmap (unsigned long *rmap) +{ +do { +while (test_bit (KVMPPC_RMAP_LOCK_BIT , rmap)) +cpu_relax (); + }while (test_and_set_bit_lock (KVMPPC_RMAP_LOCK_BIT , rmap)); +} + +static inline void unlock_rmap (unsigned long *rmap) +{ +__clear_bit_unlock (KVMPPC_RMAP_LOCK_BIT , rmap); +} +static inline bool slot_is_aligned (struct kvm_memory_slot *memslot,unsigned long pagesize) {
arch/powerpc/include/asm/kvm_host.h +16 -1
@@ -170,12 +170,27 @@ struct kvmppc_rma_info {/* * The reverse mapping array has one entry for each HPTE, * which stores the guest's view of the second word of the HPTE -* (including the guest physical address of the mapping). +* (including the guest physical address of the mapping), +* plus forward and backward pointers in a doubly-linked ring +* of HPTEs that map the same host page. The pointers in this +* ring are 32-bit HPTE indexes, to save space. */ struct revmap_entry {unsigned long guest_rpte; +unsigned int forw, back; }; +/* +* We use the top bit of each memslot->rmap entry as a lock bit, +* and bit 32 as a present flag. The bottom 32 bits are the +* index in the guest HPT of a HPTE that points to the page. +*/ +#define KVMPPC_RMAP_LOCK_BIT 63 +#define KVMPPC_RMAP_REF_BIT 33 +#define KVMPPC_RMAP_REFERENCED (1ul << KVMPPC_RMAP_REF_BIT) +#define KVMPPC_RMAP_PRESENT 0x100000000ul +#define KVMPPC_RMAP_INDEX 0xfffffffful +/* Low-order bits in kvm->arch.slot_phys[][] */ #define KVMPPC_PAGE_ORDER_MASK 0x1f#define KVMPPC_PAGE_NO_CACHE HPTE_R_I/* 0x20 */
arch/powerpc/kvm/book3s_hv_rm_mmu.c +83 -1
126 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 11/14 KVM: PPC: Implement MMIO emulation support for Book3S HV guests
In reply to Paul MackerrasThis provides the low-level support for MMIO emulation in Book3S HV guests. When the guest tries to map a page which is not covered by any memslot, that page is taken to be an MMIO emulation page. Instead of inserting a valid HPTE, we insert an HPTE that has the valid bit clear but another hypervisor software-use bit set, which we call HPTE_V_ABSENT, to indicate that this is an absent page. An absent page is treated much like a valid page as far as guest hcalls (H_ENTER, H_REMOVE, H_READ etc.) are concerned, except of course that an absent HPTE doesn't need to be invalidated with tlbie since it was never valid as far as the hardware is concerned.
When the guest accesses a page for which there is an absent HPTE, it will take a hypervisor data storage interrupt (HDSI) since we now set the VPM1 bit in the LPCR. Our HDSI handler for HPTE-not-present faults looks up the hash table and if it finds an absent HPTE mapping the requested virtual address, will switch to kernel mode and handle the fault in kvmppc_book3s_hv_page_fault(), which at present just calls kvmppc_hv_emulate_mmio() to set up the MMIO emulation.
This is based on an earlier patch by Benjamin Herrenschmidt, but since heavily reworked.
--- arch/powerpc/include/asm/kvm_book3s.h | 5 + arch/powerpc/include/asm/kvm_book3s_64.h | 26 +++ arch/powerpc/include/asm/kvm_host.h | 5 + arch/powerpc/include/asm/mmu-hash64.h | 2 +- arch/powerpc/include/asm/ppc-opcode.h | 4 +- arch/powerpc/include/asm/reg.h | 1 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kernel/exceptions-64s.S | 8 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 228 +++++++++++++++++++++++++-- arch/powerpc/kvm/book3s_hv.c | 21 ++- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 262 ++++++++++++++++++++++++++---- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 127 ++++++++++++--- 12 files changed, 607 insertions(+), 83 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +5 -0
@@ -121,6 +121,11 @@ extern void kvmppc_mmu_book3s_hv_init(struct kvm_vcpu *vcpu);extern int kvmppc_mmu_map_page (struct kvm_vcpu *vcpu,struct kvmppc_pte *pte);extern int kvmppc_mmu_map_segment (struct kvm_vcpu *vcpu,ulong eaddr);extern void kvmppc_mmu_flush_segments (struct kvm_vcpu *vcpu); +extern int kvmppc_book3s_hv_page_fault (struct kvm_run *run, +struct kvm_vcpu *vcpu,unsigned long addr, +unsigned long status); +extern long kvmppc_hv_find_lock_hpte (struct kvm *kvm,gva_t eaddr, +unsigned long slb_v,unsigned long valid);extern void kvmppc_mmu_hpte_cache_map (struct kvm_vcpu *vcpu,struct hpte_cache *pte);extern struct hpte_cache *kvmppc_mmu_hpte_cache_next (struct kvm_vcpu *vcpu);
arch/powerpc/include/asm/kvm_book3s_64.h +26 -0
@@ -43,12 +43,15 @@ static inline void svcpu_put(struct kvmppc_book3s_shadow_vcpu *svcpu)#define HPT_HASH_MASK (HPT_NPTEG - 1)#endif +#define VRMA_VSID 0x1ffffffUL/* 1TB VSID reserved for VRMA */ +/* * We use a lock bit in HPTE dword 0 to synchronize updates and * accesses to each HPTE, and another bit to indicate non-present * HPTEs. */ #define HPTE_V_HVLOCK 0x40UL +#define HPTE_V_ABSENT 0x20ULstatic inline long try_lock_hpte (unsigned long *hpte,unsigned long bits) {
@@ -144,6 +147,29 @@ static inline unsigned long hpte_cache_bits(unsigned long pte_val)#endif } +static inline bool hpte_read_permission (unsigned long pp,unsigned long key) +{ +if (key) +return PP_RWRX <= pp && pp <=PP_RXRX ; +return 1 ; +} + +static inline bool hpte_write_permission (unsigned long pp,unsigned long key) +{ +if (key) +return pp =PP_RWRW ; +return pp <=PP_RWRW ; +} + +static inline int hpte_get_skey_perm (unsigned long hpte_r,unsigned long amr) +{ +unsigned long skey; + + skey = ((hpte_r &HPTE_R_KEY_HI ) >>57 ) | + ((hpte_r &HPTE_R_KEY_LO ) >>9 ); +return (amr >> (62 -2 * skey)) &3 ; +} +static inline void lock_rmap (unsigned long *rmap) { do {
arch/powerpc/include/asm/kvm_host.h +5 -0
@@ -210,6 +210,7 @@ struct kvm_arch {unsigned long lpcr;unsigned long rmor;struct kvmppc_rma_info *rma; +unsigned long vrma_slb_v;int rma_setup_done;struct list_head spapr_tce_tables;spinlock_t slot_phys_lock;
@@ -452,6 +453,10 @@ struct kvm_vcpu_arch {#ifdef CONFIG_KVM_BOOK3S_64_HV struct kvm_vcpu_arch_shared shregs; +unsigned long pgfault_addr; +long pgfault_index; +unsigned long pgfault_hpte[2 ]; +struct list_head run_list;struct task_struct *run_task;struct kvm_run *kvm_run;
arch/powerpc/include/asm/mmu-hash64.h +1 -1
@@ -108,11 +108,11 @@ extern char initial_stab[];#define HPTE_V_VRMA_MASK ASM_CONST(0x4001ffffff000000)/* Values for PP (assumes Ks=0, Kp=1) */ -/* pp0 will always be 0 for linux */ #define PP_RWXX 0/* Supervisor read/write, User none */ #define PP_RWRX 1/* Supervisor read/write, User read */ #define PP_RWRW 2/* Supervisor read/write, User read/write */ #define PP_RXRX 3/* Supervisor read, User read */ +#define PP_RXXX (HPTE_R_PP0 | 2)/* Supervisor read, user none */ #ifndef __ASSEMBLY__
arch/powerpc/include/asm/ppc-opcode.h +3 -1
@@ -45,6 +45,7 @@#define PPC_INST_MFSPR_DSCR_MASK 0xfc1fffff#define PPC_INST_MTSPR_DSCR 0x7c1103a6#define PPC_INST_MTSPR_DSCR_MASK 0xfc1fffff +#define PPC_INST_SLBFEE 0x7c0007a7#define PPC_INST_STRING 0x7c00042a#define PPC_INST_STRING_MASK 0xfc0007fe
@@ -183,7 +184,8 @@ __PPC_RS(t) | __PPC_RA(a) | __PPC_RB(b))#define PPC_ERATSX_DOT (t, a, w) stringify_in_c(.long PPC_INST_ERATSX_DOT | \ __PPC_RS(t) | __PPC_RA(a) | __PPC_RB(b)) - +#define PPC_SLBFEE_DOT (t, b) stringify_in_c(.long PPC_INST_SLBFEE | \ + __PPC_RT(t) | __PPC_RB(b))/* * Define what the VSX XX1 form instructions will look like, then add
arch/powerpc/include/asm/reg.h +1 -0
@@ -216,6 +216,7 @@#define DSISR_ISSTORE 0x02000000/* access was a store */ #define DSISR_DABRMATCH 0x00400000/* hit data breakpoint */ #define DSISR_NOSEGMENT 0x00200000/* STAB/SLB miss */ +#define DSISR_KEYFAULT 0x00200000/* Key fault */ #define SPRN_TBRL 0x10C/* Time Base Read Lower Register (user, R/O) */ #define SPRN_TBRU 0x10D/* Time Base Read Upper Register (user, R/O) */ #define SPRN_TBWL 0x11C/* Time Base Lower Register (super, R/W) */
arch/powerpc/kernel/asm-offsets.c +1 -0
@@ -454,6 +454,7 @@ int main(void) DEFINE(KVM_LAST_VCPU, offsetof(struct kvm, arch.last_vcpu)); DEFINE(KVM_LPCR, offsetof(struct kvm, arch.lpcr)); DEFINE(KVM_RMOR, offsetof(struct kvm, arch.rmor)); + DEFINE(KVM_VRMA_SLB_V, offsetof(struct kvm, arch.vrma_slb_v)); DEFINE(VCPU_DSISR, offsetof(struct kvm_vcpu, arch.shregs.dsisr)); DEFINE(VCPU_DAR, offsetof(struct kvm_vcpu, arch.shregs.dar));#endif
arch/powerpc/kernel/exceptions-64s.S +4 -4
@@ -100,14 +100,14 @@ data_access_not_stab: END_MMU_FTR_SECTION_IFCLR(MMU_FTR_SLB) #endif EXCEPTION_PROLOG_PSERIES(PACA_EXGEN, data_access_common, EXC_STD, - KVMTEST_PR, 0x300) + KVMTEST, 0x300) . = 0x380 .globl data_access_slb_pSeries data_access_slb_pSeries: HMT_MEDIUM SET_SCRATCH0(r13) - EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST_PR, 0x380) + EXCEPTION_PROLOG_1(PACA_EXSLB, KVMTEST, 0x380) std r3,PACA_EXSLB+EX_R3(r13) mfspr r3,SPRN_DAR #ifdef __DISABLED__
@@ -329,8 +329,8 @@ do_stab_bolted_pSeries: EXCEPTION_PROLOG_PSERIES_1(.do_stab_bolted, EXC_STD) #endif /* CONFIG_POWER4_ONLY */ - KVM_HANDLER_PR_SKIP(PACA_EXGEN, EXC_STD, 0x300) - KVM_HANDLER_PR_SKIP(PACA_EXSLB, EXC_STD, 0x380) + KVM_HANDLER_SKIP(PACA_EXGEN, EXC_STD, 0x300) + KVM_HANDLER_SKIP(PACA_EXSLB, EXC_STD, 0x380) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x400) KVM_HANDLER_PR(PACA_EXSLB, EXC_STD, 0x480) KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0x900)
arch/powerpc/kvm/book3s_64_mmu_hv.c +218 -10
252 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +12 -9
@@ -326,19 +326,18 @@ static int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,break ; }/* -* We get these next two if the guest does a bad real-mode access, -* as we have enabled VRMA (virtualized real mode area) mode in the -* LPCR. We just generate an appropriate DSI/ISI to the guest. +* We get this if the guest accesses a page which it thinks +* it has mapped but which is not actually present, because +* it is for an emulated I/O device. +* Any other HDSI interrupt has been handled already. */ case BOOK3S_INTERRUPT_H_DATA_STORAGE : - vcpu->arch.shregs.dsisr = vcpu->arch.fault_dsisr; - vcpu->arch.shregs.dar = vcpu->arch.fault_dar; -kvmppc_inject_interrupt (vcpu,BOOK3S_INTERRUPT_DATA_STORAGE ,0 ); - r =RESUME_GUEST ; + r =kvmppc_book3s_hv_page_fault (run, vcpu, + vcpu->arch.fault_dar, vcpu->arch.fault_dsisr);break ;case BOOK3S_INTERRUPT_H_INST_STORAGE :kvmppc_inject_interrupt (vcpu,BOOK3S_INTERRUPT_INST_STORAGE , -0x08000000 ); + vcpu->arch.shregs.msr &0x58000000 ); r =RESUME_GUEST ;break ;/*
@@ -1228,6 +1227,8 @@ static int kvmppc_hv_setup_rma(struct kvm_vcpu *vcpu)/* Update VRMASD field in the LPCR */ senc =slb_pgsize_encoding (psize); + kvm->arch.vrma_slb_v = senc |SLB_VSID_B_1T | + (VRMA_VSID <<SLB_VSID_SHIFT_1T ); lpcr = kvm->arch.lpcr & ~LPCR_VRMASD ; lpcr |= senc << (LPCR_VRMASD_SH -4 ); kvm->arch.lpcr = lpcr;
@@ -1324,7 +1325,9 @@ int kvmppc_core_init_vm(struct kvm *kvm) kvm->arch.host_lpcr = lpcr =mfspr (SPRN_LPCR ); lpcr &=LPCR_PECE |LPCR_LPES ; lpcr |= (4UL <<LPCR_DPFD_SH ) |LPCR_HDICE | -LPCR_VPM0 |LPCR_VRMA_L ; +LPCR_VPM0 |LPCR_VPM1 ; + kvm->arch.vrma_slb_v =SLB_VSID_B_1T | + (VRMA_VSID <<SLB_VSID_SHIFT_1T ); } kvm->arch.lpcr = lpcr;
arch/powerpc/kvm/book3s_hv_rm_mmu.c +226 -36
378 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rmhandlers.S +105 -22
169 lines not on this page · raw message
Signature · 1 line
1.7.7.3
PATCH v3 12/14 KVM: Add barriers to allow mmu_notifier_retry to be used locklessly
In reply to Paul MackerrasThis adds an smp_wmb in kvm_mmu_notifier_invalidate_range_end() and an smp_rmb in mmu_notifier_retry() so that mmu_notifier_retry() will give the correct answer when called without kvm->mmu_lock being held. PowerPC Book3S HV KVM wants to use a bitlock per guest page rather than a single global spinlock in order to improve the scalability of updates to the guest MMU hashed page table, and so needs this.
--- include/linux/kvm_host.h | 14 +++++++++----- virt/kvm/kvm_main.c | 6 +++--- 2 files changed, 12 insertions(+), 8 deletions(-)
include/linux/kvm_host.h +9 -5
@@ -700,12 +700,16 @@ static inline int mmu_notifier_retry(struct kvm_vcpu *vcpu, unsigned long mmu_seif (unlikely (vcpu->kvm->mmu_notifier_count))return 1 ;/* -* Both reads happen under the mmu_lock and both values are -* modified under mmu_lock, so there's no need of smb_rmb() -* here in between, otherwise mmu_notifier_count should be -* read before mmu_notifier_seq, see -* mmu_notifier_invalidate_range_end write side. +* Ensure the read of mmu_notifier_count happens before the read +* of mmu_notifier_seq. This interacts with the smp_wmb() in +* mmu_notifier_invalidate_range_end to make sure that the caller +* either sees the old (non-zero) value of mmu_notifier_count or +* the new (incremented) value of mmu_notifier_seq. +* PowerPC Book3s HV KVM calls this under a per-page lock +* rather than under kvm->mmu_lock, for scalability, so +* can't rely on kvm->mmu_lock to keep things ordered. */ +smp_rmb ();if (vcpu->kvm->mmu_notifier_seq != mmu_seq)return 1 ;return 0 ;
virt/kvm/kvm_main.c +3 -3
@@ -357,11 +357,11 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,* been freed. */ kvm->mmu_notifier_seq++; +smp_wmb ();/* * The above sequence increase must be visible before the -* below count decrease but both values are read by the kvm -* page fault under mmu_lock spinlock so we don't need to add -* a smb_wmb() here in between the two. +* below count decrease, which is ensured by the smp_wmb above +* in conjunction with the smp_rmb in mmu_notifier_retry(). */ kvm->mmu_notifier_count--;spin_unlock (&kvm->mmu_lock);
Signature · 1 line
1.7.7.3
Re: [PATCH v3 12/14] KVM: Add barriers to allow mmu_notifier_retry to be used locklessly
Quoted 8 lines · Paul Mackerras
This adds an
smp_wmbinkvm_mmu_notifier_invalidate_range_end()and ansmp_rmbinmmu_notifier_retry()so thatmmu_notifier_retry()will give the correct answer when called withoutkvm->mmu_lockbeing held. PowerPC Book3S HV KVM wants to use a bitlock per guest page rather than a single global spinlock in order to improve the scalability of updates to the guest MMU hashed page table, and so needs this.Signed-off-by Paul Mackerras <[email protected]>
Avi, mind to ack?
Alex
Quoted 53 lines
---
include/linux/kvm_host.h| 14 +++++++++-----virt/kvm/kvm_main.c| 6 +++---2 files changed, 12 insertions(+), 8 deletions(-)include/linux/kvm_host.h +9
-5@@ -700,12 +700,16 @@ static inline int mmu_notifier_retry(struct kvm_vcpu *vcpu, unsigned long mmu_seif (unlikely (vcpu->kvm->mmu_notifier_count))return 1 ;/* -* Both reads happen under the mmu_lock and both values are -* modified under mmu_lock, so there's no need of smb_rmb() -* here in between, otherwise mmu_notifier_count should be -* read before mmu_notifier_seq, see -* mmu_notifier_invalidate_range_end write side. +* Ensure the read of mmu_notifier_count happens before the read +* of mmu_notifier_seq. This interacts with the smp_wmb() in +* mmu_notifier_invalidate_range_end to make sure that the caller +* either sees the old (non-zero) value of mmu_notifier_count or +* the new (incremented) value of mmu_notifier_seq. +* PowerPC Book3s HV KVM calls this under a per-page lock +* rather than under kvm->mmu_lock, for scalability, so +* can't rely on kvm->mmu_lock to keep things ordered. */ +smp_rmb ();if (vcpu->kvm->mmu_notifier_seq != mmu_seq)return 1 ;return 0 ;virt/kvm/kvm_main.c +3
-3@@ -357,11 +357,11 @@ static void kvm_mmu_notifier_invalidate_range_end(struct mmu_notifier *mn,* been freed. */ kvm->mmu_notifier_seq++; +smp_wmb ();/* * The above sequence increase must be visible before the -* below count decrease but both values are read by the kvm -* page fault under mmu_lock spinlock so we don't need to add -* a smb_wmb() here in between the two. +* below count decrease, which is ensured by the smp_wmb above +* in conjunction with the smp_rmb in mmu_notifier_retry(). */ kvm->mmu_notifier_count--;spin_unlock (&kvm->mmu_lock);Signature · 1 line
1.7.7.3
Avi Kivity
Acked-byRe: [PATCH v3 12/14] KVM: Add barriers to allow mmu_notifier_retry to be used locklessly
Quoted 9 lines · Alexander Graf
Quoted 7 lines · Paul Mackerras
This adds an
smp_wmbinkvm_mmu_notifier_invalidate_range_end()and ansmp_rmbinmmu_notifier_retry()so thatmmu_notifier_retry()will give the correct answer when called withoutkvm->mmu_lockbeing held. PowerPC Book3S HV KVM wants to use a bitlock per guest page rather than a single global spinlock in order to improve the scalability of updates to the guest MMU hashed page table, and so needs this.
Quoted 4 lines
Quoted 1 line
Signed-off-by Paul Mackerras <[email protected]>Avi, mind to ack?
Signature · 1 line
error compiling committee.c: too many arguments to function
PATCH v3 13/14 KVM: PPC: Implement MMU notifiers for Book3S HV guests
In reply to Paul MackerrasThis adds the infrastructure to enable us to page out pages underneath a Book3S HV guest, on processors that support virtualized partition memory, that is, POWER7. Instead of pinning all the guest's pages, we now look in the host userspace Linux page tables to find the mapping for a given guest page. Then, if the userspace Linux PTE gets invalidated, kvm_unmap_hva() gets called for that address, and we replace all the guest HPTEs that refer to that page with absent HPTEs, i.e. ones with the valid bit clear and the HPTE_V_ABSENT bit set, which will cause an HDSI when the guest tries to access them. Finally, the page fault handler is extended to reinstantiate the guest HPTE when the guest tries to access a page which has been paged out.
Since we can't intercept the guest DSI and ISI interrupts on PPC970, we still have to pin all the guest pages on PPC970. We have a new flag, kvm->arch.using_mmu_notifiers, that indicates whether we can page guest pages out. If it is not set, the MMU notifier callbacks do nothing and everything operates as before.
--- arch/powerpc/include/asm/kvm_book3s.h | 4 + arch/powerpc/include/asm/kvm_book3s_64.h | 31 ++++ arch/powerpc/include/asm/kvm_host.h | 16 ++ arch/powerpc/include/asm/reg.h | 3 + arch/powerpc/kvm/Kconfig | 1 + arch/powerpc/kvm/book3s_64_mmu_hv.c | 290 +++++++++++++++++++++++++++--- arch/powerpc/kvm/book3s_hv.c | 25 ++-- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 140 +++++++++++--- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 49 +++++ arch/powerpc/kvm/powerpc.c | 3 + arch/powerpc/mm/hugetlbpage.c | 2 + 11 files changed, 499 insertions(+), 65 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +4 -0
@@ -145,6 +145,10 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat,extern void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr);extern int kvmppc_emulate_paired_single (struct kvm_run *run,struct kvm_vcpu *vcpu);extern pfn_t kvmppc_gfn_to_pfn (struct kvm_vcpu *vcpu,gfn_t gfn); +extern void kvmppc_add_revmap_chain (struct kvm *kvm,struct revmap_entry *rev, +unsigned long *rmap,long pte_index,int realmode); +extern void kvmppc_invalidate_hpte (struct kvm *kvm,unsigned long *hptep, +unsigned long pte_index);extern void *kvmppc_pin_guest_page (struct kvm *kvm,unsigned long addr,unsigned long *nb_ret);extern void kvmppc_unpin_guest_page (struct kvm *kvm,void *addr);
arch/powerpc/include/asm/kvm_book3s_64.h +31 -0
@@ -136,6 +136,37 @@ static inline int hpte_cache_flags_ok(unsigned long ptel, unsigned long io_type)return (wimg & (HPTE_R_W |HPTE_R_I )) == io_type; } +/* +* Lock and read a linux PTE. If it's present and writable, atomically +* set dirty and referenced bits and return the PTE, otherwise return 0. +*/ +static inline pte_t kvmppc_read_update_linux_pte (pte_t *p) +{ +pte_t pte, tmp; + +/* wait until _PAGE_BUSY is clear then set it atomically */ + __asm__ __volatile__ ( +"1: ldarx %0,0,%3\n" +" andi. %1,%0,%4\n" +" bne- 1b\n" +" ori %1,%0,%4\n" +" stdcx. %1,0,%3\n" +" bne- 1b" + :"=&r" (pte),"=&r" (tmp),"=m" (*p) + :"r" (p),"i" (_PAGE_BUSY) + :"cc" ); + +if (pte_present (pte)) { + pte =pte_mkyoung (pte); +if (pte_write (pte)) + pte =pte_mkdirty (pte); + } + + *p = pte;/* clears _PAGE_BUSY */ + +return pte; +} +/* Return HPTE cache control bits corresponding to Linux pte bits */ static inline unsigned long hpte_cache_bits (unsigned long pte_val) {
arch/powerpc/include/asm/kvm_host.h +16 -0
@@ -32,6 +32,7 @@#include <linux/atomic.h> #include <asm/kvm_asm.h> #include <asm/processor.h> +#include <asm/page.h> #define KVM_MAX_VCPUS NR_CPUS#define KVM_MAX_VCORES NR_CPUS
@@ -44,6 +45,19 @@#define KVM_COALESCED_MMIO_PAGE_OFFSET 1#endif +#ifdef CONFIG_KVM_BOOK3S_64_HV +#include <linux/mmu_notifier.h> + +#define KVM_ARCH_WANT_MMU_NOTIFIER + +struct kvm ; +extern int kvm_unmap_hva (struct kvm *kvm,unsigned long hva); +extern int kvm_age_hva (struct kvm *kvm,unsigned long hva); +extern int kvm_test_age_hva (struct kvm *kvm,unsigned long hva); +extern void kvm_set_spte_hva (struct kvm *kvm,unsigned long hva,pte_t pte); + +#endif +/* We don't currently support large pages. */ #define KVM_HPAGE_GFN_SHIFT (x) 0#define KVM_NR_PAGE_SIZES 1
@@ -212,6 +226,7 @@ struct kvm_arch {struct kvmppc_rma_info *rma;unsigned long vrma_slb_v;int rma_setup_done; +int using_mmu_notifiers;struct list_head spapr_tce_tables;spinlock_t slot_phys_lock;unsigned long *slot_phys[KVM_MEM_SLOTS_NUM ];
@@ -460,6 +475,7 @@ struct kvm_vcpu_arch {struct list_head run_list;struct task_struct *run_task;struct kvm_run *kvm_run; +pgd_t *pgdir;#endif };
arch/powerpc/include/asm/reg.h +3 -0
@@ -495,6 +495,9 @@#define SPRN_SPRG7 0x117/* Special Purpose Register General 7 */ #define SPRN_SRR0 0x01A/* Save/Restore Register 0 */ #define SPRN_SRR1 0x01B/* Save/Restore Register 1 */ +#define SRR1_ISI_NOPT 0x40000000/* ISI: Not found in hash */ +#define SRR1_ISI_N_OR_G 0x10000000/* ISI: Access is no-exec or G */ +#define SRR1_ISI_PROT 0x08000000/* ISI: Other protection fault */ #define SRR1_WAKEMASK 0x00380000/* reason for wakeup */ #define SRR1_WAKESYSERR 0x00300000/* System error */ #define SRR1_WAKEEE 0x00200000/* External interrupt */
arch/powerpc/kvm/Kconfig +1 -0
@@ -69,6 +69,7 @@ config KVM_BOOK3S_64 config KVM_BOOK3S_64_HV bool "KVM support for POWER7 and PPC970 using hypervisor mode in host" depends on KVM_BOOK3S_64 + select MMU_NOTIFIER ---help--- Support running unmodified book3s_64 guest kernels in virtual machines on POWER7 and PPC970 processors that have
arch/powerpc/kvm/book3s_64_mmu_hv.c +266 -24
355 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv.c +14 -11
@@ -326,19 +326,19 @@ static int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,break ; }/* -* We get this if the guest accesses a page which it thinks -* it has mapped but which is not actually present, because -* it is for an emulated I/O device. -* Any other HDSI interrupt has been handled already. +* We get these next two if the guest accesses a page which it thinks +* it has mapped but which is not actually present, either because +* it is for an emulated I/O device or because the corresonding +* host page has been paged out. Any other HDSI/HISI interrupts +* have been handled already. */ case BOOK3S_INTERRUPT_H_DATA_STORAGE : r =kvmppc_book3s_hv_page_fault (run, vcpu, vcpu->arch.fault_dar, vcpu->arch.fault_dsisr);break ;case BOOK3S_INTERRUPT_H_INST_STORAGE : -kvmppc_inject_interrupt (vcpu,BOOK3S_INTERRUPT_INST_STORAGE , - vcpu->arch.shregs.msr &0x58000000 ); - r =RESUME_GUEST ; + r =kvmppc_book3s_hv_page_fault (run, vcpu, +kvmppc_get_pc (vcpu),0 );break ;/* * This occurs if the guest executes an illegal instruction.
@@ -900,6 +900,7 @@ int kvmppc_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu)flush_altivec_to_thread (current);flush_vsx_to_thread (current); vcpu->arch.wqp = &vcpu->arch.vcore->wq; + vcpu->arch.pgdir = current->mm->pgd; do { r = kvmppc_run_vcpu(run, vcpu);
@@ -1123,9 +1124,9 @@ int kvmppc_core_prepare_memory_region(struct kvm *kvm,unsigned long *phys;/* Allocate a slot_phys array */ - npages = mem->memory_size >>PAGE_SHIFT ; phys = kvm->arch.slot_phys[mem->slot]; -if (!phys) { +if (!kvm->arch.using_mmu_notifiers && !phys) { + npages = mem->memory_size >>PAGE_SHIFT ; phys =vzalloc (npages *sizeof (unsigned long ));if (!phys)return -ENOMEM ;
@@ -1331,6 +1332,7 @@ int kvmppc_core_init_vm(struct kvm *kvm) } kvm->arch.lpcr = lpcr; + kvm->arch.using_mmu_notifiers = !!cpu_has_feature (CPU_FTR_ARCH_206 );spin_lock_init (&kvm->arch.slot_phys_lock);return 0 ; }
@@ -1339,8 +1341,9 @@ void kvmppc_core_destroy_vm(struct kvm *kvm) {unsigned long i; -for (i =0 ; i <KVM_MEM_SLOTS_NUM ; i++) -unpin_slot (kvm, i); +if (!kvm->arch.using_mmu_notifiers) +for (i =0 ; i <KVM_MEM_SLOTS_NUM ; i++) +unpin_slot (kvm, i);if (kvm->arch.rma) {kvm_release_rma (kvm->arch.rma);
arch/powerpc/kvm/book3s_hv_rm_mmu.c +110 -30
254 lines not on this page · raw message
arch/powerpc/kvm/book3s_hv_rmhandlers.S +49 -0
67 lines not on this page · raw message
arch/powerpc/kvm/powerpc.c +3 -0
@@ -244,6 +244,9 @@ int kvm_dev_ioctl_check_extension(long ext)if (cpu_has_feature (CPU_FTR_ARCH_201 )) r =2 ;break ; +case KVM_CAP_SYNC_MMU : + r =cpu_has_feature (CPU_FTR_ARCH_206 ) ?1 :0 ; +break ;#endif default : r =0 ;
arch/powerpc/mm/hugetlbpage.c +2 -0
@@ -12,6 +12,7 @@#include <linux/io.h> #include <linux/slab.h> #include <linux/hugetlb.h> +#include <linux/export.h> #include <linux/of_fdt.h> #include <linux/memblock.h> #include <linux/bootmem.h>
@@ -102,6 +103,7 @@ pte_t *find_linux_pte_or_hugepte(pgd_t *pgdir, unsigned long ea, unsigned *shift *shift =hugepd_shift (*hpdp);return hugepte_offset (hpdp, ea, pdshift); } +EXPORT_SYMBOL_GPL (find_linux_pte_or_hugepte);pte_t *huge_pte_offset (struct mm_struct *mm,unsigned long addr) {
Signature · 1 line
1.7.7.3
PATCH v3 14/14 KVM: PPC: Allow for read-only pages backing a Book3S HV guest
In reply to Paul MackerrasWith this, if a guest does an H_ENTER with a read/write HPTE on a page which is currently read-only, we make the actual HPTE inserted be a read-only version of the HPTE. We now intercept protection faults as well as HPTE not found faults, and for a protection fault we work out whether it should be reflected to the guest (e.g. because the guest HPTE didn't allow write access to usermode) or handled by switching to kernel context and calling kvmppc_book3s_hv_page_fault, which will then request write access to the page and update the actual HPTE.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 20 +++++++++++++- arch/powerpc/kvm/book3s_64_mmu_hv.c | 39 +++++++++++++++++++++++++++-- arch/powerpc/kvm/book3s_hv_rm_mmu.c | 32 +++++++++++++++++------- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 +- 4 files changed, 78 insertions(+), 17 deletions(-)
arch/powerpc/include/asm/kvm_book3s_64.h +18 -2
@@ -121,6 +121,22 @@ static inline unsigned long hpte_rpn(unsigned long ptel, unsigned long psize)return ((ptel &HPTE_R_RPN ) & ~(psize -1 )) >>PAGE_SHIFT ; } +static inline int hpte_is_writable (unsigned long ptel) +{ +unsigned long pp = ptel & (HPTE_R_PP0 |HPTE_R_PP ); + +return pp !=PP_RXRX && pp !=PP_RXXX ; +} + +static inline unsigned long hpte_make_readonly (unsigned long ptel) +{ +if ((ptel &HPTE_R_PP0 ) || (ptel &HPTE_R_PP ) ==PP_RWXX ) + ptel = (ptel & ~HPTE_R_PP ) |PP_RXXX ; +else + ptel |=PP_RXRX ; +return ptel; +} +static inline int hpte_cache_flags_ok (unsigned long ptel,unsigned long io_type) {unsigned int wimg = ptel &HPTE_R_WIMG ;
@@ -140,7 +156,7 @@ static inline int hpte_cache_flags_ok(unsigned long ptel, unsigned long io_type)* Lock and read a linux PTE. If it's present and writable, atomically * set dirty and referenced bits and return the PTE, otherwise return 0. */ -static inline pte_t kvmppc_read_update_linux_pte (pte_t *p) +static inline pte_t kvmppc_read_update_linux_pte (pte_t *p,int writing) {pte_t pte, tmp;
@@ -158,7 +174,7 @@ static inline pte_t kvmppc_read_update_linux_pte(pte_t *p)if (pte_present (pte)) { pte =pte_mkyoung (pte); -if (pte_write (pte)) +if (writing &&pte_write (pte)) pte =pte_mkdirty (pte); }
arch/powerpc/kvm/book3s_64_mmu_hv.c +36 -3
@@ -503,6 +503,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,struct page *page, *pages[1 ];long index, ret, npages;unsigned long is_io; +unsigned int writing, write_ok;struct vm_area_struct *vma;/*
@@ -553,8 +554,11 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, pfn =0 ; page =NULL ; pte_size =PAGE_SIZE ; + writing = (dsisr &DSISR_ISSTORE ) !=0 ; +/* If writing != 0, then the HPTE must allow writing, if we get here */ + write_ok = writing; hva =gfn_to_hva_memslot (memslot, gfn); - npages =get_user_pages_fast (hva,1 ,1 , pages); + npages =get_user_pages_fast (hva,1 , writing, pages);if (npages <1 ) {/* Check if it's an I/O mapping */ down_read (¤t->mm->mmap_sem);
@@ -565,6 +569,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, ((hva - vma->vm_start) >>PAGE_SHIFT ); pte_size = psize; is_io =hpte_cache_bits (pgprot_val (vma->vm_page_prot)); + write_ok = vma->vm_flags &VM_WRITE ; }up_read (¤t->mm->mmap_sem);if (!pfn)
@@ -575,6 +580,24 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, page =compound_head (page); pte_size <<=compound_order (page); } +/* if the guest wants write access, see if that is OK */ +if (!writing &&hpte_is_writable (r)) { +pte_t *ptep, pte; + +/* +* We need to protect against page table destruction +* while looking up and updating the pte. +*/ +rcu_read_lock_sched (); + ptep =find_linux_pte_or_hugepte (current->mm->pgd, + hva,NULL ); +if (ptep &&pte_present (*ptep)) { + pte =kvmppc_read_update_linux_pte (ptep,1 ); +if (pte_write (pte)) + write_ok =1 ; + } +rcu_read_unlock_sched (); + } pfn =page_to_pfn (page); }
@@ -595,6 +618,8 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,/* Set the HPTE to point to pfn */ r = (r & ~(HPTE_R_PP0 - pte_size)) | (pfn <<PAGE_SHIFT ); +if (hpte_is_writable (r) && !write_ok) + r =hpte_make_readonly (r); ret =RESUME_GUEST ;preempt_disable ();while (!try_lock_hpte (hptep,HPTE_V_HVLOCK ))
@@ -614,14 +639,22 @@ int kvmppc_book3s_hv_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu,unlock_rmap (rmap); goto out_unlock; } -kvmppc_add_revmap_chain (kvm, rev, rmap, index,0 ); + +if (hptep[0 ] &HPTE_V_VALID ) { +/* HPTE was previously valid, so we need to invalidate it */ +unlock_rmap (rmap); + hptep[0 ] |=HPTE_V_ABSENT ; +kvmppc_invalidate_hpte (kvm, hptep, index); + }else { +kvmppc_add_revmap_chain (kvm, rev, rmap, index,0 ); + } hptep[1 ] = r;eieio (); hptep[0 ] = hpte[0 ]; asmvolatile ("ptesync" : : :"memory" );preempt_enable (); -if (page) +if (page &&hpte_is_writable (r))SetPageDirty (page); out_put:
arch/powerpc/kvm/book3s_hv_rm_mmu.c +22 -10
@@ -120,7 +120,7 @@ static void remove_revmap_chain(struct kvm *kvm, long pte_index, }static pte_t lookup_linux_pte (struct kvm_vcpu *vcpu,unsigned long hva, -unsigned long *pte_sizep) +int writing,unsigned long *pte_sizep) {pte_t *ptep;unsigned long ps = *pte_sizep;
@@ -137,7 +137,7 @@ static pte_t lookup_linux_pte(struct kvm_vcpu *vcpu, unsigned long hva,return __pte (0 );if (!pte_present (*ptep))return __pte (0 ); -return kvmppc_read_update_linux_pte (ptep); +return kvmppc_read_update_linux_pte (ptep, writing); } long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags,
@@ -154,12 +154,14 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags,unsigned long is_io;unsigned long *rmap;pte_t pte; +unsigned int writing;unsigned long mmu_seq;bool realmode = vcpu->arch.vcore->vcore_state ==VCORE_RUNNING ; psize =hpte_page_size (pteh, ptel);if (!psize)return H_PARAMETER ; + writing =hpte_is_writable (ptel); pteh &= ~(HPTE_V_HVLOCK |HPTE_V_ABSENT |HPTE_V_VALID );/* used later to detect if we might have been invalidated */
@@ -208,8 +210,11 @@ long kvmppc_h_enter(struct kvm_vcpu *vcpu, unsigned long flags,/* Look up the Linux PTE for the backing page */ pte_size = psize; - pte =lookup_linux_pte (vcpu, hva, &pte_size); + pte =lookup_linux_pte (vcpu, hva, writing, &pte_size);if (pte_present (pte)) { +if (writing && !pte_write (pte)) +/* make the actual HPTE be read-only */ + ptel =hpte_make_readonly (ptel); is_io =hpte_cache_bits (pte_val (pte)); pa =pte_pfn (pte) <<PAGE_SHIFT ; }
@@ -678,7 +683,9 @@ EXPORT_SYMBOL(kvmppc_hv_find_lock_hpte);/* * Called in real mode to check whether an HPTE not found fault -* is due to accessing a paged-out page or an emulated MMIO page. +* is due to accessing a paged-out page or an emulated MMIO page, +* or if a protection fault is due to accessing a page that the +* guest wanted read/write access to but which we made read-only. * Returns a possibly modified status (DSISR) value if not * (i.e. pass the interrupt to the guest), * -1 to pass the fault up to host kernel mode code, -2 to do that
@@ -696,12 +703,17 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr,struct revmap_entry *rev;unsigned long pp, key; - valid =HPTE_V_VALID |HPTE_V_ABSENT ; +/* For protection fault, expect to find a valid HPTE */ + valid =HPTE_V_VALID ; +if (status &DSISR_NOHPTE ) + valid |=HPTE_V_ABSENT ; index =kvmppc_hv_find_lock_hpte (kvm, addr, slb_v, valid); -if (index <0 ) -return status;/* there really was no HPTE */ - +if (index <0 ) { +if (status &DSISR_NOHPTE ) +return status;/* there really was no HPTE */ +return 0 ;/* for prot fault, HPTE disappeared */ + } hpte = (unsigned long *)(kvm->arch.hpt_virt + (index <<4 )); v = hpte[0 ] & ~HPTE_V_HVLOCK ; r = hpte[1 ];
@@ -712,8 +724,8 @@ long kvmppc_hpte_hv_fault(struct kvm_vcpu *vcpu, unsigned long addr, asmvolatile ("lwsync" : : :"memory" ); hpte[0 ] = v; -/* If the HPTE is valid by now, retry the instruction */ -if (v &HPTE_V_VALID ) +/* For not found, if the HPTE is valid by now, retry the instruction */ +if ((status &DSISR_NOHPTE ) && (v &HPTE_V_VALID ))return 0 ;/* Check access permissions to the page */
arch/powerpc/kvm/book3s_hv_rmhandlers.S +2 -2
@@ -1111,8 +1111,8 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_201) kvmppc_hdsi: mfspr r4, SPRN_HDAR mfspr r6, SPRN_HDSISR - /* HPTE not found fault? */ - andis. r0, r6, DSISR_NOHPTE@h + /* HPTE not found fault or protection fault? */ + andis. r0, r6, (DSISR_NOHPTE | DSISR_PROTFAULT)@h beq 1f /* if not, send it to the guest */ andi. r0, r11, MSR_DR /* data relocation enabled? */ beq 3f
Signature · 1 line
1.7.7.3
Alexander Graf
AppliedQuoted 28 lines · Paul Mackerras
This series of patches updates the Book3S-HV KVM code that manages the guest hashed page table (HPT) to enable several things:
* MMIO emulation and MMIO pass-through
* Use of small pages (4kB or 64kB, depending on config) to back the guest memory
* Pageable guest memory - i.e. backing pages can be removed from the guest and reinstated on demand, using the MMU notifier mechanism
* Guests can be given read-only access to pages even though they think they have mapped them read/write. When they try to write to them their access is upgraded to read/write. This allows KSM to share pages between guests.
On PPC970 we have no way to get DSIs and ISIs to come to the hypervisor, so we can't do MMIO emulation or pageable guest memory. On POWER7 we set the VPM1 bit in the LPCR to make all DSIs and ISIs come to the hypervisor (host) as HDSIs or HISIs.
This code is working well in my tests. The sporadic crashes that I was seeing earlier are fixed by the second patch in the series. Somewhat to my surprise, when I implemented the last patch in the series I started to see KSM coalescing pages without any further effort on my part -- my tests were on a machine with Fedora 16 installed, and it has ksmtuned running by default.
Quoted 7 lines
This series is on top of Alex Graf's kvm-ppc-next branch. The first patch in my series fixes a bug in one of the patches in that branch ("KVM: PPC: booke: Improve timer register emulation").
These patches only touch
arch/powerpcexcept for patch 12, which adds a couple of barriers to allowmmu_notifier_retry()to be used outside of thekvm->mmu_lock.
Thanks, applied all to kvm-ppc-next, awaiting the one follow-up patch though.
Alex