KVM: PPC: Implement passthrough of emulated devices for nested guests
Subsystems (3)
cover KVM: PPC: Implement passthrough of emulated devices for nested guests
This patch series allows for emulated devices to be passed through to nested guests, irrespective of at which level the device is being emulated.
Note that the emulated device must be using dma, not virtio.
For example, passing through an emulated e1000:
1. Emulate the device at L(n) for L(n+1)
qemu-system-ppc64 -netdev type=user,id=net0 -device e1000,netdev=net0
2. Assign the VFIO-PCI driver at L(n+1)
echo 0000:00:00.0 > /sys/bus/pci/drivers/e1000/unbind echo 0000:00:00.0 > /sys/bus/pci/drivers/vfio-pci/bind chmod 666 /dev/vfio/0
3. Pass the device through from L(n+1) to L(n+2)
qemu-system-ppc64 -device vfio-pci,host=0000:00:00.0
4. L(n+2) can now access the device which will be emulated at L(n)
Suraj Jitindar Singh (8):
KVM: PPC: Only report KVM_CAP_SPAPR_TCE_VFIO on powernv machines
KVM: PPC: Book3S HV: Add function kvmhv_vcpu_is_radix()
KVM: PPC: Book3S HV: Implement functions to access quadrants 1 & 2
KVM: PPC: Add load_from_eaddr and store_to_eaddr to the kvmppc_ops
struct
KVM: PPC: Update kvmppc_st and kvmppc_ld to use quadrants
KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an L2
guest
KVM: PPC: Introduce new hcall H_COPY_TOFROM_GUEST to access quadrants
1 & 2
KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an L3
guest
arch/powerpc/include/asm/hvcall.h | 1 + arch/powerpc/include/asm/kvm_book3s.h | 10 ++- arch/powerpc/include/asm/kvm_book3s_64.h | 13 ++++ arch/powerpc/include/asm/kvm_host.h | 3 + arch/powerpc/include/asm/kvm_ppc.h | 4 ++ arch/powerpc/kernel/exceptions-64s.S | 9 +++ arch/powerpc/kvm/book3s_64_mmu_radix.c | 97 ++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 58 ++++++++++++++-- arch/powerpc/kvm/book3s_hv_nested.c | 114 +++++++++++++++++++++++++++++-- arch/powerpc/kvm/powerpc.c | 28 +++++++- arch/powerpc/mm/fault.c | 1 + 11 files changed, 323 insertions(+), 15 deletions(-)
Signature · 1 line
2.13.6
PATCH 1/1 KVM: PPC: Book3S PR: Set hflag to indicate that POWER9 supports 1T segments
When booting a kvm-pr guest on a POWER9 machine the following message is observed:
"qemu-system-ppc64: KVM does not support 1TiB segments which guest expects"
This is because the guest is expecting to be able to use 1T segments however we don't indicate support for it. This is because we don't set the BOOK3S_HFLAG_MULTI_PGSIZE flag in the hflags in kvmppc_set_pvr_pr() on POWER9.
POWER9 does indeed have support for 1T segments, so add a case for POWER9 to the switch statement to ensure it is set.
--- arch/powerpc/kvm/book3s_pr.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/kvm/book3s_pr.c +1 -0
@@ -587,6 +587,7 @@ void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr)case PVR_POWER8 :case PVR_POWER8E :case PVR_POWER8NVL : +case PVR_POWER9 : vcpu->arch.hflags |=BOOK3S_HFLAG_MULTI_PGSIZE |BOOK3S_HFLAG_NEW_TLBIE ;break ;
Signature · 1 line
2.13.6
Michael Ellerman
reviewerRe: [PATCH] KVM: PPC: Book3S PR: Set hflag to indicate that POWER9 supports 1T segments
Quoted 11 lines · Suraj Jitindar Singh
When booting a kvm-pr guest on a POWER9 machine the following message is observed:
"qemu-system-ppc64: KVM does not support 1TiB segments which guest expects"This is because the guest is expecting to be able to use 1T segments however we don't indicate support for it. This is because we don't set the
BOOK3S_HFLAG_MULTI_PGSIZEflag in the hflags inkvmppc_set_pvr_pr()on POWER9.POWER9 does indeed have support for 1T segments, so add a case for POWER9 to the switch statement to ensure it is set.
If this just checked mmu_has_feature(MMU_FTR_1T_SEGMENT) it would have been right in this case, and we'd also never have to update it in future.
Any reason not to?
cheers
Quoted 14 lines
arch/powerpc/kvm/book3s_pr.c +1
-0@@ -587,6 +587,7 @@ void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr)case PVR_POWER8 :case PVR_POWER8E :case PVR_POWER8NVL : +case PVR_POWER9 : vcpu->arch.hflags |=BOOK3S_HFLAG_MULTI_PGSIZE |BOOK3S_HFLAG_NEW_TLBIE ;break ;Signature · 1 line
2.13.6
Re: [PATCH] KVM: PPC: Book3S PR: Set hflag to indicate that POWER9 supports 1T segments
Quoted 11 lines · Michael Ellerman
Quoted 9 lines · Suraj Jitindar Singh
When booting a kvm-pr guest on a POWER9 machine the following message is observed:
"qemu-system-ppc64: KVM does not support 1TiB segments which guest expects"This is because the guest is expecting to be able to use 1T segments however we don't indicate support for it. This is because we don't set the
BOOK3S_HFLAG_MULTI_PGSIZEflag in the hflags inkvmppc_set_pvr_pr()on POWER9.
Quoted 8 lines
Quoted 2 lines
POWER9 does indeed have support for 1T segments, so add a case for POWER9 to the switch statement to ensure it is set.
If this just checked
mmu_has_feature(MMU_FTR_1T_SEGMENT)it would have been right in this case, and we'd also never have to update it in future.Any reason not to?
PR KVM can emulate a different processor from the host. Checking mmu_has_feature() will tell you about the host, not the CPU being emulated.
Paul.
Re: [PATCH] KVM: PPC: Book3S PR: Set hflag to indicate that POWER9 supports 1T segments
In reply to Suraj Jitindar SinghQuoted 9 lines · Suraj Jitindar Singh
When booting a kvm-pr guest on a POWER9 machine the following message is observed:
"qemu-system-ppc64: KVM does not support 1TiB segments which guest expects"This is because the guest is expecting to be able to use 1T segments however we don't indicate support for it. This is because we don't set the
BOOK3S_HFLAG_MULTI_PGSIZEflag in the hflags inkvmppc_set_pvr_pr()on POWER9.
Quoted 4 lines
POWER9 does indeed have support for 1T segments, so add a case for POWER9 to the switch statement to ensure it is set.
Signed-off-by Suraj Jitindar Singh <[email protected]>
Thanks, patch applied to my kvm-ppc-next branch.
Paul.
PATCH 1/8 KVM: PPC: Only report KVM_CAP_SPAPR_TCE_VFIO on powernv machines
In reply to Suraj Jitindar SinghThe kvm capability KVM_CAP_SPAPR_TCE_VFIO is used to indicate the availability of in kernel tce acceleration for vfio. However it is currently the case that this is only available on a powernv machine, not for a pseries machine.
Thus make this capability dependent on having the cpu feature CPU_FTR_HVMODE.
--- arch/powerpc/kvm/powerpc.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/kvm/powerpc.c +5 -1
@@ -496,6 +496,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)int r;/* Assume we're using HV mode when the HV module is loaded */ int hv_enabled = kvmppc_hv_ops ?1 :0 ; +int kvm_on_pseries = !cpu_has_feature (CPU_FTR_HVMODE );if (kvm) {/*
@@ -543,8 +544,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)#ifdef CONFIG_PPC_BOOK3S_64 case KVM_CAP_SPAPR_TCE :case KVM_CAP_SPAPR_TCE_64 : -/* fallthrough */ + r =1 ; +break ;case KVM_CAP_SPAPR_TCE_VFIO : + r = !kvm_on_pseries; +break ;case KVM_CAP_PPC_RTAS :case KVM_CAP_PPC_FIXUP_HCALL :case KVM_CAP_PPC_ENABLE_HCALL :
Signature · 1 line
2.13.6
PATCH 2/8 KVM: PPC: Book3S HV: Add function kvmhv_vcpu_is_radix()
In reply to Suraj Jitindar SinghThere exists a function kvm_is_radix() which is used to determine if a kvm instance is using the radix mmu. However this only applies to the first level (L1) guest. Add a function kvmhv_vcpu_is_radix() which can be used to determine if the current execution context of the vcpu is radix, accounting for if the vcpu is running a nested guest.
Currently all nested guests must be radix but this may change in the future.
--- arch/powerpc/include/asm/kvm_book3s_64.h | 13 +++++++++++++ arch/powerpc/kvm/book3s_hv_nested.c | 1 + 2 files changed, 14 insertions(+)
arch/powerpc/include/asm/kvm_book3s_64.h +13 -0
@@ -55,6 +55,7 @@ struct kvm_nested_guest {cpumask_t need_tlb_flush;cpumask_t cpu_in_guest;short prev_cpu[NR_CPUS ]; +u8 radix;/* is this nested guest radix */ };/*
@@ -150,6 +151,18 @@ static inline bool kvm_is_radix(struct kvm *kvm) return kvm->arch.radix; } +static inline bool kvmhv_vcpu_is_radix (struct kvm_vcpu *vcpu) +{ +bool radix; + +if (vcpu->arch.nested) + radix = vcpu->arch.nested->radix; +else + radix =kvm_is_radix (vcpu->kvm); + +return radix; +} +#define KVM_DEFAULT_HPT_ORDER 24/* 16MB HPT by default */ #endif
arch/powerpc/kvm/book3s_hv_nested.c +1 -0
@@ -480,6 +480,7 @@ struct kvm_nested_guest *kvmhv_alloc_nested(struct kvm *kvm, unsigned int lpid)if (shadow_lpid <0 ) goto out_free2; gp->shadow_lpid = shadow_lpid; + gp->radix =1 ;memset (gp->prev_cpu,-1 ,sizeof (gp->prev_cpu));
Signature · 1 line
2.13.6
PATCH 3/8 KVM: PPC: Book3S HV: Implement functions to access quadrants 1 & 2
In reply to Suraj Jitindar SinghThe POWER9 radix mmu has the concept of quadrants. The quadrant number is the two high bits of the effective address and determines the fully qualified address to be used for the translation. The fully qualified address consists of the effective lpid, the effective pid and the effective address. This gives then 4 possible quadrants 0, 1, 2, and 3.
When accessing these quadrants the fully qualified address is obtained as follows:
Quadrant | Hypervisor | Guest
--------------------------------------------------------------------------
| EA[0:1] = 0xb00 | EA[0:1] = 0xb00 0 | effLPID = 0 | effLPID = LPIDR | effPID = PIDR | effPID = PIDR -------------------------------------------------------------------------- | EA[0:1] = 0xb01 | 1 | effLPID = LPIDR | Invalid Access | effPID = PIDR | -------------------------------------------------------------------------- | EA[0:1] = 0xb10 | 2 | effLPID = LPIDR | Invalid Access | effPID = 0 | -------------------------------------------------------------------------- | EA[0:1] = 0xb11 | EA[0:1] = 0xb11 3 | effLPID = 0 | effLPID = LPIDR | effPID = 0 | effPID = 0
--------------------------------------------------------------------------
In the Guest;
Quadrant 3 is normally used to address the operating system since this uses effPID=0 and effLPID=LPIDR, meaning the PID register doesn't need to be switched.
Quadrant 0 is normally used to address user space since the effLPID and effPID are taken from the corresponding registers.
In the Host;
Quadrant 0 and 3 are used as above, however the effLPID is always 0 to address the host.
Quadrants 1 and 2 can be used by the host to address guest memory using a guest effective address. Since the effLPID comes from the LPID register, the host loads the LPID of the guest it would like to access (and the PID of the process) and can perform accesses to a guest effective address.
This means quadrant 1 can be used to address the guest user space and quadrant 2 can be used to address the guest operating system from the hypervisor, using a guest effective address.
Access to the quadrants can cause a Hypervisor Data Storage Interrupt (HDSI) due to being unable to perform partition scoped translation. Previously this could only be generated from a guest and so the code path expects us to take the KVM trampoline in the interrupt handler. This is no longer the case so we modify the handler to call bad_page_fault() to check if we were expecting this fault so we can handle it gracefully and just return with an error code. In the hash mmu case we still raise an unknown exception since quadrants aren't defined for the hash mmu.
--- arch/powerpc/include/asm/kvm_book3s.h | 4 ++ arch/powerpc/kernel/exceptions-64s.S | 9 ++++ arch/powerpc/kvm/book3s_64_mmu_radix.c | 97 ++++++++++++++++++++++++++++++++++ arch/powerpc/mm/fault.c | 1 + 4 files changed, 111 insertions(+)
arch/powerpc/include/asm/kvm_book3s.h +4 -0
@@ -188,6 +188,10 @@ extern int kvmppc_book3s_hcall_implemented(struct kvm *kvm, unsigned long hc);extern int kvmppc_book3s_radix_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned long ea,unsigned long dsisr); +extern long kvmhv_copy_from_guest_radix (struct kvm_vcpu *vcpu,gva_t eaddr, +void *to,unsigned long n); +extern long kvmhv_copy_to_guest_radix (struct kvm_vcpu *vcpu,gva_t eaddr, +void *from,unsigned long n);extern int kvmppc_mmu_walk_radix_tree (struct kvm_vcpu *vcpu,gva_t eaddr,struct kvmppc_pte *gpte,u64 root,u64 *pte_ret_p);
arch/powerpc/kernel/exceptions-64s.S +9 -0
@@ -995,7 +995,16 @@ EXC_COMMON_BEGIN(h_data_storage_common) bl save_nvgprs RECONCILE_IRQ_STATE(r10, r11) addi r3,r1,STACK_FRAME_OVERHEAD +BEGIN_MMU_FTR_SECTION + ld r4,PACA_EXGEN+EX_DAR(r13) + lwz r5,PACA_EXGEN+EX_DSISR(r13) + std r4,_DAR(r1) + std r5,_DSISR(r1) + li r5,SIGSEGV + bl bad_page_fault +MMU_FTR_SECTION_ELSE bl unknown_exception +ALT_MMU_FTR_SECTION_END_IFSET(MMU_FTR_TYPE_RADIX) b ret_from_except
arch/powerpc/kvm/book3s_64_mmu_radix.c +97 -0
103 lines not on this page · raw message
arch/powerpc/mm/fault.c +1 -0
@@ -636,6 +636,7 @@ void bad_page_fault(struct pt_regs *regs, unsigned long address, int sig)switch (TRAP (regs)) {case 0x300 :case 0x380 : +case 0xe00 :printk (KERN_ALERT "Unable to handle kernel paging request for " "data at address 0x%08lx\n" , regs->dar);break ;
Signature · 1 line
2.13.6
PATCH 4/8 KVM: PPC: Add load_from_eaddr and store_to_eaddr to the kvmppc_ops struct
In reply to Suraj Jitindar SinghThe kvmppc_ops struct is used to store function pointers to kvm implementation specific functions.
Introduce two new functions load_from_eaddr and store_to_eaddr to be used to load from and store to a guest effective address respectively.
Also implement these for the kvm-hv module. If we are using the radix mmu then we can call the functions to access quadrant 1 and 2.
--- arch/powerpc/include/asm/kvm_ppc.h | 4 ++++ arch/powerpc/kvm/book3s_hv.c | 40 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
arch/powerpc/include/asm/kvm_ppc.h +4 -0
@@ -326,6 +326,10 @@ struct kvmppc_ops { unsigned long flags);void (*giveup_ext)(struct kvm_vcpu *vcpu,ulong msr);int (*enable_nested)(struct kvm *kvm); +int (*load_from_eaddr)(struct kvm_vcpu *vcpu,ulong *eaddr,void *ptr, +int size); +int (*store_to_eaddr)(struct kvm_vcpu *vcpu,ulong *eaddr,void *ptr, +int size); };extern struct kvmppc_ops *kvmppc_hv_ops;
arch/powerpc/kvm/book3s_hv.c +40 -0
@@ -5213,6 +5213,44 @@ static int kvmhv_enable_nested(struct kvm *kvm)return 0 ; } +static int kvmhv_load_from_eaddr (struct kvm_vcpu *vcpu,ulong *eaddr,void *ptr, +int size) +{ +int rc = -EINVAL ; + +if (kvmhv_vcpu_is_radix (vcpu)) { + rc =kvmhv_copy_from_guest_radix (vcpu, *eaddr, ptr, size); + +if (rc >0 ) + rc = -EINVAL ; + } + +/* For now quadrants are the only way to access nested guest memory */ +if (rc && vcpu->arch.nested) + rc = -EAGAIN ; + +return rc; +} + +static int kvmhv_store_to_eaddr (struct kvm_vcpu *vcpu,ulong *eaddr,void *ptr, +int size) +{ +int rc = -EINVAL ; + +if (kvmhv_vcpu_is_radix (vcpu)) { + rc =kvmhv_copy_to_guest_radix (vcpu, *eaddr, ptr, size); + +if (rc >0 ) + rc = -EINVAL ; + } + +/* For now quadrants are the only way to access nested guest memory */ +if (rc && vcpu->arch.nested) + rc = -EAGAIN ; + +return rc; +} +static struct kvmppc_ops kvm_ops_hv = { .get_sregs = kvm_arch_vcpu_ioctl_get_sregs_hv, .set_sregs = kvm_arch_vcpu_ioctl_set_sregs_hv,
@@ -5253,6 +5291,8 @@ static struct kvmppc_ops kvm_ops_hv = { .get_rmmu_info = kvmhv_get_rmmu_info, .set_smt_mode = kvmhv_set_smt_mode, .enable_nested = kvmhv_enable_nested, + .load_from_eaddr = kvmhv_load_from_eaddr, + .store_to_eaddr = kvmhv_store_to_eaddr, };static int kvm_init_subcore_bitmap (void )
Signature · 1 line
2.13.6
PATCH 5/8 KVM: PPC: Update kvmppc_st and kvmppc_ld to use quadrants
In reply to Suraj Jitindar SinghThe functions kvmppc_st and kvmppc_ld are used to access guest memory from the host using a guest effective address. They do so by translating through the process table to obtain a guest real address and then using kvm_read_guest or kvm_write_guest to make the access with the guest real address.
This method of access however only works for L1 guests and will give the incorrect results for a nested guest.
We can however use the store_to_eaddr and load_from_eaddr kvmppc_ops to perform the access for a nested guesti (and a L1 guest). So attempt this method first and fall back to the old method if this fails and we aren't running a nested guest.
At this stage there is no fall back method to perform the access for a nested guest and this is left as a future improvement. For now we will return to the nested guest and rely on the fact that a translation should be faulted in before retrying the access.
--- arch/powerpc/kvm/powerpc.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
arch/powerpc/kvm/powerpc.c +16 -2
@@ -331,10 +331,17 @@ int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, {ulong mp_pa = vcpu->arch.magic_page_pa &KVM_PAM &PAGE_MASK ;struct kvmppc_pte pte; -int r; +int r = -EINVAL ; vcpu->stat.st++; +if (vcpu->kvm->arch.kvm_ops && vcpu->kvm->arch.kvm_ops->store_to_eaddr) + r = vcpu->kvm->arch.kvm_ops->store_to_eaddr (vcpu, eaddr, ptr, + size); + +if ((!r) || (r == -EAGAIN )) +return r; + r =kvmppc_xlate (vcpu, *eaddr, data ?XLATE_DATA :XLATE_INST ,XLATE_WRITE , &pte);if (r <0 )
@@ -367,10 +374,17 @@ int kvmppc_ld(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, {ulong mp_pa = vcpu->arch.magic_page_pa &KVM_PAM &PAGE_MASK ;struct kvmppc_pte pte; -int rc; +int rc = -EINVAL ; vcpu->stat.ld++; +if (vcpu->kvm->arch.kvm_ops && vcpu->kvm->arch.kvm_ops->load_from_eaddr) + rc = vcpu->kvm->arch.kvm_ops->load_from_eaddr (vcpu, eaddr, ptr, + size); + +if ((!rc) || (rc == -EAGAIN )) +return rc; + rc =kvmppc_xlate (vcpu, *eaddr, data ?XLATE_DATA :XLATE_INST ,XLATE_READ , &pte);if (rc)
Signature · 1 line
2.13.6
PATCH 6/8 KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an L2 guest
In reply to Suraj Jitindar SinghAllow for a device which is being emulated at L0 (the host) for an L1 guest to be passed through to a nested (L2) guest.
The existing kvmppc_hv_emulate_mmio function can be used here. The main challenge is that for a load the result must be stored into the L2 gpr, not an L1 gpr as would normally be the case after going out to qemu to complete the operation. This presents a challenge as at this point the L2 gpr state has been written back into L1 memory.
To work around this we store the address in L1 memory of the L2 gpr where the result of the load is to be stored and use the new io_gpr value KVM_MMIO_REG_NESTED_GPR to indicate that this is a nested load for which completion must be done when returning back into the kernel. Then in kvmppc_complete_mmio_load() the resultant value is written into L1 memory at the location of the indicated L2 gpr.
Note that we don't currently let an L1 guest emulate a device for an L2 guest which is then passed through to an L3 guest.
--- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/include/asm/kvm_host.h | 3 +++ arch/powerpc/kvm/book3s_hv.c | 12 ++++++---- arch/powerpc/kvm/book3s_hv_nested.c | 43 ++++++++++++++++++++++++++++++----- arch/powerpc/kvm/powerpc.c | 4 ++++ 5 files changed, 53 insertions(+), 11 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -1
@@ -311,7 +311,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu,void kvmhv_save_hv_regs (struct kvm_vcpu *vcpu,struct hv_guest_state *hr);void kvmhv_restore_hv_return_state (struct kvm_vcpu *vcpu,struct hv_guest_state *hr); -long int kvmhv_nested_page_fault (struct kvm_vcpu *vcpu); +long int kvmhv_nested_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu);void kvmppc_giveup_fac (struct kvm_vcpu *vcpu,ulong fac);
arch/powerpc/include/asm/kvm_host.h +3 -0
@@ -793,6 +793,7 @@ struct kvm_vcpu_arch {/* For support of nested guests */ struct kvm_nested_guest *nested;u32 nested_vcpu_id; +gpa_t nested_io_gpr;#endif #ifdef CONFIG_KVM_BOOK3S_HV_EXIT_TIMING
@@ -827,6 +828,8 @@ struct kvm_vcpu_arch {#define KVM_MMIO_REG_FQPR 0x00c0#define KVM_MMIO_REG_VSX 0x0100#define KVM_MMIO_REG_VMX 0x0180 +#define KVM_MMIO_REG_NESTED_GPR 0xffc0 +#define __KVM_HAVE_ARCH_WQP#define __KVM_HAVE_CREATE_DEVICE
arch/powerpc/kvm/book3s_hv.c +8 -4
@@ -984,6 +984,10 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)if (ret ==H_INTERRUPT ) {kvmppc_set_gpr (vcpu,3 ,0 );return -EINTR ; + }else if (ret ==H_TOO_HARD ) { +kvmppc_set_gpr (vcpu,3 ,0 ); + vcpu->arch.hcall_needed =0 ; +return RESUME_HOST ; }break ;case H_TLB_INVALIDATE :
@@ -1335,7 +1339,7 @@ static int kvmppc_handle_exit_hv(struct kvm_run *run, struct kvm_vcpu *vcpu,return r; } -static int kvmppc_handle_nested_exit (struct kvm_vcpu *vcpu) +static int kvmppc_handle_nested_exit (struct kvm_run *run,struct kvm_vcpu *vcpu) {int r;int srcu_idx;
@@ -1393,7 +1397,7 @@ static int kvmppc_handle_nested_exit(struct kvm_vcpu *vcpu)*/ case BOOK3S_INTERRUPT_H_DATA_STORAGE : srcu_idx =srcu_read_lock (&vcpu->kvm->srcu); - r =kvmhv_nested_page_fault (vcpu); + r =kvmhv_nested_page_fault (run, vcpu);srcu_read_unlock (&vcpu->kvm->srcu, srcu_idx);break ;case BOOK3S_INTERRUPT_H_INST_STORAGE :
@@ -1403,7 +1407,7 @@ static int kvmppc_handle_nested_exit(struct kvm_vcpu *vcpu)if (vcpu->arch.shregs.msr &HSRR1_HISI_WRITE ) vcpu->arch.fault_dsisr |=DSISR_ISSTORE ; srcu_idx =srcu_read_lock (&vcpu->kvm->srcu); - r =kvmhv_nested_page_fault (vcpu); + r =kvmhv_nested_page_fault (run, vcpu);srcu_read_unlock (&vcpu->kvm->srcu, srcu_idx);break ;
@@ -4058,7 +4062,7 @@ int kvmhv_run_single_vcpu(struct kvm_run *kvm_run,if (!nested) r =kvmppc_handle_exit_hv (kvm_run, vcpu, current);else - r =kvmppc_handle_nested_exit (vcpu); + r =kvmppc_handle_nested_exit (kvm_run, vcpu); } vcpu->arch.ret = r;
arch/powerpc/kvm/book3s_hv_nested.c +37 -6
78 lines not on this page · raw message
arch/powerpc/kvm/powerpc.c +4 -0
@@ -1210,6 +1210,10 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu,kvmppc_set_vmx_byte (vcpu, gpr);break ;#endif +case KVM_MMIO_REG_NESTED_GPR : +kvm_vcpu_write_guest (vcpu, vcpu->arch.nested_io_gpr, &gpr, +sizeof (gpr)); +break ;default :BUG (); }
Signature · 1 line
2.13.6
PATCH 7/8 KVM: PPC: Introduce new hcall H_COPY_TOFROM_GUEST to access quadrants 1 & 2
In reply to Suraj Jitindar SinghA guest cannot access quadrants 1 or 2 as this would result in an exception. Thus introduce the hcall H_COPY_TOFROM_GUEST to be used by a guest when it wants to perform an access to quadrants 1 or 2, for example when it wants to access memory for one of its nested guests.
Also provide an implementation for the kvm-hv module.
--- arch/powerpc/include/asm/hvcall.h | 1 + arch/powerpc/include/asm/kvm_book3s.h | 4 ++ arch/powerpc/kvm/book3s_64_mmu_radix.c | 7 ++-- arch/powerpc/kvm/book3s_hv.c | 6 ++- arch/powerpc/kvm/book3s_hv_nested.c | 75 ++++++++++++++++++++++++++++++++++ 5 files changed, 89 insertions(+), 4 deletions(-)
arch/powerpc/include/asm/hvcall.h +1 -0
@@ -335,6 +335,7 @@#define H_SET_PARTITION_TABLE 0xF800#define H_ENTER_NESTED 0xF804#define H_TLB_INVALIDATE 0xF808 +#define H_COPY_TOFROM_GUEST 0xF80C/* Values for 2nd argument to H_SET_MODE */ #define H_SET_MODE_RESOURCE_SET_CIABR 1
arch/powerpc/include/asm/kvm_book3s.h +4 -0
@@ -188,6 +188,9 @@ extern int kvmppc_book3s_hcall_implemented(struct kvm *kvm, unsigned long hc);extern int kvmppc_book3s_radix_page_fault (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned long ea,unsigned long dsisr); +extern unsigned long __kvmhv_copy_tofrom_guest_radix (int lpid,int pid, +gva_t eaddr,void *to,void *from, +unsigned long n);extern long kvmhv_copy_from_guest_radix (struct kvm_vcpu *vcpu,gva_t eaddr,void *to,unsigned long n); extern long kvmhv_copy_to_guest_radix(struct kvm_vcpu *vcpu, gva_t eaddr,
@@ -302,6 +305,7 @@ long kvmhv_nested_init(void);void kvmhv_nested_exit (void );void kvmhv_vm_nested_init (struct kvm *kvm);long kvmhv_set_partition_table (struct kvm_vcpu *vcpu); +long kvmhv_copy_tofrom_guest_nested (struct kvm_vcpu *vcpu);void kvmhv_set_ptbl_entry (unsigned int lpid,u64 dw0,u64 dw1);void kvmhv_release_all_nested (struct kvm *kvm);long kvmhv_enter_nested_guest (struct kvm_vcpu *vcpu);
arch/powerpc/kvm/book3s_64_mmu_radix.c +4 -3
@@ -29,9 +29,9 @@*/ static int p9_supported_radix_bits[4 ] = {5 ,9 ,9 ,13 }; -static unsigned long __kvmhv_copy_tofrom_guest_radix (int lpid,int pid, -gva_t eaddr,void *to,void *from, -unsigned long n) +unsigned long __kvmhv_copy_tofrom_guest_radix (int lpid,int pid, +gva_t eaddr,void *to,void *from, +unsigned long n) {unsigned long quadrant, ret = n;int old_pid, old_lpid;
@@ -82,6 +82,7 @@ static unsigned long __kvmhv_copy_tofrom_guest_radix(int lpid, int pid,return ret; } +EXPORT_SYMBOL_GPL (__kvmhv_copy_tofrom_guest_radix);static long kvmhv_copy_tofrom_guest_radix (struct kvm_vcpu *vcpu,gva_t eaddr,void *to,void *from,unsigned long n)
arch/powerpc/kvm/book3s_hv.c +5 -1
@@ -995,7 +995,11 @@ int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu)if (nesting_enabled (vcpu->kvm)) ret =kvmhv_do_nested_tlbie (vcpu);break ; - +case H_COPY_TOFROM_GUEST : + ret =H_FUNCTION ; +if (nesting_enabled (vcpu->kvm)) + ret =kvmhv_copy_tofrom_guest_nested (vcpu); +break ;default :return RESUME_HOST ; }
arch/powerpc/kvm/book3s_hv_nested.c +75 -0
81 lines not on this page · raw message
Signature · 1 line
2.13.6
PATCH 8/8 KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an L3 guest
In reply to Suraj Jitindar SinghPreviously when a device was being emulated by an L1 guest for an L2 guest, that device couldn't then be passed through to an L3 guest. This was because the L1 guest had no method for accessing L3 memory.
The hcall H_COPY_TOFROM_GUEST provides this access. Thus this setup for passthrough can now be allowed.
--- arch/powerpc/kvm/book3s_64_mmu_radix.c | 9 ++++----- arch/powerpc/kvm/book3s_hv_nested.c | 5 ----- 2 files changed, 4 insertions(+), 10 deletions(-)
arch/powerpc/kvm/book3s_64_mmu_radix.c +4 -5
@@ -37,11 +37,10 @@ unsigned long __kvmhv_copy_tofrom_guest_radix(int lpid, int pid,int old_pid, old_lpid;bool is_load = !!to; -/* Can't access quadrants 1 or 2 in non-HV mode */ -if (kvmhv_on_pseries ()) { -/* TODO h-call */ -return -EPERM ; - } +/* Can't access quadrants 1 or 2 in non-HV mode, call the HV to do it */ +if (kvmhv_on_pseries ()) +return plpar_hcall_norets (H_COPY_TOFROM_GUEST , lpid, pid, eaddr, + to, from, n); quadrant =1 ;if (!pid)
arch/powerpc/kvm/book3s_hv_nested.c +0 -5
@@ -1284,11 +1284,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_run *run, }/* passthrough of emulated MMIO case */ -if (kvmhv_on_pseries ()) { -pr_err ("emulated MMIO passthrough?\n" ); -return -EINVAL ; - } -return kvmppc_hv_emulate_mmio (run, vcpu, gpa, ea, writing); }if (memslot->flags &KVM_MEM_READONLY ) {
Signature · 1 line
2.13.6
Quoted 18 lines · Suraj Jitindar Singh
This patch series allows for emulated devices to be passed through to nested
guests, irrespective of at which level the device is being emulated.Note that the emulated device must be using dma, not virtio.
For example, passing through an emulated e1000:
1. Emulate the device at L(n) for L(n+1)
qemu-system-ppc64 -netdev type=user,id=net0 -device e1000,netdev=net0
2. Assign the VFIO-PCI driver at L(n+1)
echo 0000:00:00.0 >
/sys/bus/pci/drivers/e1000/unbindecho 0000:00:00.0 >/sys/bus/pci/drivers/vfio-pci/bindchmod 666/dev/vfio/0
Quoted 5 lines
3. Pass the device through from L(n+1) to L(n+2)
qemu-system-ppc64 -device vfio-pci,host=0000:00:00.0
4. L(n+2) can now access the device which will be emulated at L(n)
Note,
[PATCH] KVM: PPC: Book3S PR: Set hflag to indicate that POWER9 supports 1T segments
is not supposed to be part of this series
Quoted 33 lines
Suraj Jitindar Singh (8):
KVM: PPC: Only reportKVM_CAP_SPAPR_TCE_VFIOon powernv machines
KVM: PPC: Book3S HV: Add functionkvmhv_vcpu_is_radix()
KVM: PPC: Book3S HV: Implement functions to access quadrants 1 & 2
KVM: PPC: Addload_from_eaddrandstore_to_eaddrto thekvmppc_ops
struct
KVM: PPC: Updatekvmppc_standkvmppc_ldto use quadrants
KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an
L2
guest
KVM: PPC: Introduce new hcallH_COPY_TOFROM_GUESTto access
quadrants1 & 2KVM: PPC: Book3S HV: Allow passthrough of an emulated device to an
L3
guestarch/powerpc/include/asm/hvcall.h | 1 + arch/powerpc/include/asm/kvm_book3s.h | 10 ++- arch/powerpc/include/asm/kvm_book3s_64.h | 13 ++++ arch/powerpc/include/asm/kvm_host.h | 3 + arch/powerpc/include/asm/kvm_ppc.h | 4 ++ arch/powerpc/kernel/exceptions-64s.S | 9 +++ arch/powerpc/kvm/book3s_64_mmu_radix.c | 97 ++++++++++++++++++++++++++ arch/powerpc/kvm/book3s_hv.c | 58 ++++++++++++++-- arch/powerpc/kvm/book3s_hv_nested.c | 114 +++++++++++++++++++++++++++++-- arch/powerpc/kvm/powerpc.c | 28 +++++++- arch/powerpc/mm/fault.c | 1 + 11 files changed, 323 insertions(+), 15 deletions(-)