KVM: PPC: Virtualize Gekko guests
Subsystems (3)
cover KVM: PPC: Virtualize Gekko guests
In an effort to get KVM on PPC more useful for other userspace users than Qemu, I figured it'd be a nice idea to implement virtualization of the Gekko CPU.
The Gekko is the CPU used in the GameCube. In a slightly more modern fashion it lives on in the Wii today.
Using this patch set and a modified version of Dolphin, I was able to virtualize simple GameCube demos on a 970MP system.
As always, while getting this to run I stumbled across several broken parts and fixed them as they came up. So expect some bug fixes in this patch set too.
Alexander Graf (18):
KVM: PPC: Add QPR registers
KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs
KVM: PPC: Teach MMIO Signedness
KVM: PPC: Add AGAIN type for emulation return
KVM: PPC: Add hidden flag for paired singles
KVM: PPC: Add Gekko SPRs
KVM: PPC: Combine extension interrupt handlers
KVM: PPC: Preload FPU when possible
KVM: PPC: Fix typo in book3s_32 debug code
KVM: PPC: Implement mtsr instruction emulation
KVM: PPC: Make software load/store return eaddr
KVM: PPC: Make ext giveup non-static
KVM: PPC: Add helpers to call FPU instructions
KVM: PPC: Fix error in BAT assignment
KVM: PPC: Add helpers to modify ppc fields
KVM: PPC: Enable program interrupt to do MMIO
KVM: PPC: Reserve a chunk of memory for opcodes
KVM: PPC: Implement Paired Single emulation
arch/powerpc/include/asm/kvm.h | 7 + arch/powerpc/include/asm/kvm_asm.h | 1 + arch/powerpc/include/asm/kvm_book3s.h | 8 +- arch/powerpc/include/asm/kvm_fpu.h | 45 + arch/powerpc/include/asm/kvm_host.h | 6 + arch/powerpc/include/asm/kvm_ppc.h | 43 +- arch/powerpc/include/asm/reg.h | 10 + arch/powerpc/kernel/ppc_ksyms.c | 2 + arch/powerpc/kvm/Makefile | 2 + arch/powerpc/kvm/book3s.c | 132 +++- arch/powerpc/kvm/book3s_32_mmu.c | 2 +- arch/powerpc/kvm/book3s_64_emulate.c | 94 ++- arch/powerpc/kvm/book3s_paired_singles.c | 1356 ++++++++++++++++++++++++++++++ arch/powerpc/kvm/emulate.c | 18 +- arch/powerpc/kvm/fpu.S | 77 ++ arch/powerpc/kvm/powerpc.c | 56 ++- 16 files changed, 1821 insertions(+), 38 deletions(-) create mode 100644 arch/powerpc/include/asm/kvm_fpu.h create mode 100644 arch/powerpc/kvm/book3s_paired_singles.c create mode 100644 arch/powerpc/kvm/fpu.S
PATCH 1/18 KVM: PPC: Add QPR registers
The Gekko has GPRs, SPRs and FPRs like normal PowerPC codes, but it also has QPRs which are basically single precision only FPU registers that get used when in paired single mode.
The following patches depend on them being around, so let's add the definitions early.
--- arch/powerpc/include/asm/kvm_host.h | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)
arch/powerpc/include/asm/kvm_host.h +5 -0
@@ -186,6 +186,11 @@ struct kvm_vcpu_arch {u64 vsr[32 ];#endif +#ifdef CONFIG_PPC_BOOK3S +/* For Gekko paired singles */ +u32 qpr[32 ]; +#endif +ulong pc;ulong ctr;ulong lr;
Signature · 1 line
1.6.0.2
A message that is not in the archive (sent off-list or not yet received): 1265298925-31954-1-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org
PATCH 2/18 KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs
Right now MMIO access can only happen for GPRs and is at most 32 bit wide. That's actually enough for almost all types of hardware out there.
Unfortunately, the guest I was using used FPU writes to MMIO regions, so it ended up writing 64 bit MMIOs using FPRs and QPRs.
So let's add code to handle those odd cases too.
--- arch/powerpc/include/asm/kvm.h | 7 +++++++ arch/powerpc/include/asm/kvm_ppc.h | 2 +- arch/powerpc/kvm/powerpc.c | 24 ++++++++++++++++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/kvm.h +7 -0
@@ -77,4 +77,11 @@ struct kvm_debug_exit_arch {struct kvm_guest_debug_arch { }; +#define REG_MASK 0x001f +#define REG_EXT_MASK 0xffe0 +#define REG_GPR 0x0000 +#define REG_FPR 0x0020 +#define REG_QPR 0x0040 +#define REG_FQPR 0x0060 +#endif /* __LINUX_KVM_POWERPC_H */
arch/powerpc/include/asm/kvm_ppc.h +1 -1
@@ -49,7 +49,7 @@ extern int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu, unsigned int rt, unsigned int bytes, int is_bigendian);extern int kvmppc_handle_store (struct kvm_run *run,struct kvm_vcpu *vcpu, -u32 val,unsigned int bytes,int is_bigendian); +u64 val,unsigned int bytes,int is_bigendian);extern int kvmppc_emulate_instruction (struct kvm_run *run,struct kvm_vcpu *vcpu);
arch/powerpc/kvm/powerpc.c +22 -2
@@ -277,7 +277,7 @@ static void kvmppc_complete_dcr_load(struct kvm_vcpu *vcpu,static void kvmppc_complete_mmio_load (struct kvm_vcpu *vcpu,struct kvm_run *run) { -ulong gpr; +u64 gpr;if (run->mmio.len >sizeof (gpr)) {printk (KERN_ERR "bad MMIO length: %d\n" , run->mmio.len);
@@ -286,6 +286,7 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu,if (vcpu->arch.mmio_is_bigendian) {switch (run->mmio.len) { +case 8 : gpr = *(u64 *)run->mmio.data;break ;case 4 : gpr = *(u32 *)run->mmio.data;break ;case 2 : gpr = *(u16 *)run->mmio.data;break ;case 1 : gpr = *(u8 *)run->mmio.data;break ;
@@ -300,6 +301,24 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu, }kvmppc_set_gpr (vcpu, vcpu->arch.io_gpr, gpr); + +switch (vcpu->arch.io_gpr ®_EXT_MASK ) { +case REG_GPR : +kvmppc_set_gpr (vcpu, vcpu->arch.io_gpr, gpr); +break ; +case REG_FPR : + vcpu->arch.fpr[vcpu->arch.io_gpr ®_MASK ] = gpr; +break ; +case REG_QPR : + vcpu->arch.qpr[vcpu->arch.io_gpr ®_MASK ] = gpr; +break ; +case REG_FQPR : + vcpu->arch.fpr[vcpu->arch.io_gpr ®_MASK ] = gpr; + vcpu->arch.qpr[vcpu->arch.io_gpr ®_MASK ] = gpr; +break ; +default : +BUG (); + } } int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
@@ -323,7 +342,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu, }int kvmppc_handle_store (struct kvm_run *run,struct kvm_vcpu *vcpu, -u32 val,unsigned int bytes,int is_bigendian) +u64 val,unsigned int bytes,int is_bigendian) {void *data = run->mmio.data;
@@ -341,6 +360,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,/* Store the value at the lowest bytes in 'data'. */ if (is_bigendian) {switch (bytes) { +case 8 : *(u64 *)data = val;break ;case 4 : *(u32 *)data = val;break ;case 2 : *(u16 *)data = val;break ;case 1 : *(u8 *)data = val;break ;
Signature · 1 line
1.6.0.2
A message that is not in the archive (sent off-list or not yet received): 1265298925-31954-3-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org
Re: [PATCH 02/18] KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs
Quoted 20 lines · Alexander Graf
Right now MMIO access can only happen for GPRs and is at most 32 bit wide. That's actually enough for almost all types of hardware out there.
Unfortunately, the guest I was using used FPU writes to MMIO regions, so it ended up writing 64 bit MMIOs using FPRs and QPRs.
So let's add code to handle those odd cases too.
Signed-off-by Alexander Graf<[email protected]>--- arch/powerpc/include/asm/kvm.h | 7 +++++++ arch/powerpc/include/asm/kvm_ppc.h | 2 +- arch/powerpc/kvm/powerpc.c | 24 ++++++++++++++++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-)arch/powerpc/include/asm/kvm.h +0
-0
arch/powerpc/include/asm/kvm.h struct kvm_debug_exit_arch { · line 86
Quoted 10 lines
arch/powerpc/include/asm/kvm.h +6
-0@@ -77,4 +77,11 @@ struct kvm_debug_exit_arch {struct kvm_guest_debug_arch { }; +#define REG_MASK 0x001f +#define REG_EXT_MASK 0xffe0 +#define REG_GPR 0x0000 +#define REG_FPR 0x0020 +#define REG_QPR 0x0040 +#define REG_FQPR 0x0060
These names seem too generic to belong in asm/kvm.h - some application could use the same names. Please add a KVM_ prefix.
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 02/18] KVM: PPC: Enable MMIO to do 64 bits, fprs and qprs
Am 07.02.2010 um 13:29 schrieb Avi Kivity <[email protected]>:
Quoted 24 lines · Alexander Graf
Quoted 23 lines
Right now MMIO access can only happen for GPRs and is at most 32 bit wide. That's actually enough for almost all types of hardware out there.
Unfortunately, the guest I was using used FPU writes to MMIO regions, so it ended up writing 64 bit MMIOs using FPRs and QPRs.
So let's add code to handle those odd cases too.
Signed-off-by Alexander Graf<[email protected]>--- arch/powerpc/include/asm/kvm.h | 7 +++++++ arch/powerpc/include/asm/kvm_ppc.h | 2 +- arch/powerpc/kvm/powerpc.c | 24 ++++++++++++++++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-)arch/powerpc/include/asm/kvm.h +0
-0@@ -77,4 +77,11 @@ struct kvm_debug_exit_arch {struct kvm_guest_debug_arch { };
Quoted 9 lines
arch/powerpc/include/asm/kvm.h struct kvm_debug_exit_arch {
Quoted 7 lines
arch/powerpc/include/asm/kvm.h +6
-0+#define REG_MASK 0x001f +#define REG_EXT_MASK 0xffe0 +#define REG_GPR 0x0000 +#define REG_FPR 0x0020 +#define REG_QPR 0x0040 +#define REG_FQPR 0x0060These names seem too generic to belong in
asm/kvm.h- some application could use the same names. Please add aKVM_prefix.
Yes, will do.
Alex
PATCH 3/18 KVM: PPC: Teach MMIO Signedness
In reply to a message not in the archiveThe guest I was trying to get to run uses the LHA and LHAU instructions. Those instructions basically do a load, but also sign extend the result.
Since we need to fill our registers by hand when doing MMIO, we also need to sign extend manually.
This patch implements sign extended MMIO and the LHA(U) instructions.
--- arch/powerpc/include/asm/kvm_host.h | 1 + arch/powerpc/include/asm/kvm_ppc.h | 3 +++ arch/powerpc/kvm/emulate.c | 14 ++++++++++++++ arch/powerpc/kvm/powerpc.c | 32 ++++++++++++++++++++++++++++++++ 4 files changed, 50 insertions(+), 0 deletions(-)
arch/powerpc/include/asm/kvm_host.h +1 -0
@@ -268,6 +268,7 @@ struct kvm_vcpu_arch {u8 io_gpr;/* GPR used as IO source/target */ u8 mmio_is_bigendian; +u8 mmio_sign_extend;u8 dcr_needed;u8 dcr_is_write;
arch/powerpc/include/asm/kvm_ppc.h +3 -0
@@ -48,6 +48,9 @@ extern void kvmppc_dump_vcpu(struct kvm_vcpu *vcpu);extern int kvmppc_handle_load (struct kvm_run *run,struct kvm_vcpu *vcpu,unsigned int rt,unsigned int bytes,int is_bigendian); +extern int kvmppc_handle_loads (struct kvm_run *run,struct kvm_vcpu *vcpu, +unsigned int rt,unsigned int bytes, +int is_bigendian);extern int kvmppc_handle_store (struct kvm_run *run,struct kvm_vcpu *vcpu,u64 val,unsigned int bytes,int is_bigendian);
arch/powerpc/kvm/emulate.c +14 -0
@@ -62,6 +62,8 @@#define OP_STBU 39#define OP_LHZ 40#define OP_LHZU 41 +#define OP_LHA 42 +#define OP_LHAU 43#define OP_STH 44#define OP_STHU 45
@@ -450,6 +452,18 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)kvmppc_set_gpr (vcpu, ra, vcpu->arch.paddr_accessed);break ; +case OP_LHA : + rt =get_rt (inst); + emulated =kvmppc_handle_loads (run, vcpu, rt,2 ,1 ); +break ; + +case OP_LHAU : + ra =get_ra (inst); + rt =get_rt (inst); + emulated =kvmppc_handle_loads (run, vcpu, rt,2 ,1 ); +kvmppc_set_gpr (vcpu, ra, vcpu->arch.paddr_accessed); +break ; +case OP_STH : rs =get_rs (inst); emulated = kvmppc_handle_store(run, vcpu,
arch/powerpc/kvm/powerpc.c +32 -0
@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu, } } +if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr &0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ; +#endif +case 2 : +if (gpr &0x8000 ) + gpr |=0xffffffffffff0000ULL ; +break ; +case 1 : +if (gpr &0x80 ) + gpr |=0xffffffffffffff00ULL ; +break ; + } + } +kvmppc_set_gpr (vcpu, vcpu->arch.io_gpr, gpr);switch (vcpu->arch.io_gpr ®_EXT_MASK ) {
@@ -337,10 +356,23 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu, vcpu->arch.mmio_is_bigendian = is_bigendian; vcpu->mmio_needed =1 ; vcpu->mmio_is_write =0 ; + vcpu->arch.mmio_sign_extend =0 ;return EMULATE_DO_MMIO ; } +/* Same as above, but sign extends */ +int kvmppc_handle_loads (struct kvm_run *run,struct kvm_vcpu *vcpu, +unsigned int rt,unsigned int bytes,int is_bigendian) +{ +int r; + + r =kvmppc_handle_load (run, vcpu, rt, bytes, is_bigendian); + vcpu->arch.mmio_sign_extend =1 ; + +return r; +} +int kvmppc_handle_store (struct kvm_run *run,struct kvm_vcpu *vcpu,u64 val,unsigned int bytes,int is_bigendian) {
Signature · 1 line
1.6.0.2
A message that is not in the archive (sent off-list or not yet received): 1265298925-31954-4-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
Quoted 10 lines · Alexander Graf
The guest I was trying to get to run uses the LHA and LHAU instructions. Those instructions basically do a load, but also sign extend the result.
Since we need to fill our registers by hand when doing MMIO, we also need to sign extend manually.
This patch implements sign extended MMIO and the LHA(U) instructions.
arch/powerpc/kvm/powerpc.c +0
-0@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu, }
arch/powerpc/kvm/powerpc.c static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu, · line 310
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +7
-0} +if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr&0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ;
Wouldn't
gpr = (s64)(gpr << 32) >> 32;
work? Not sure if >> is guaranteed to sign extend.
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
Am 07.02.2010 um 13:32 schrieb Avi Kivity <[email protected]>:
Quoted 25 lines · Alexander Graf
Quoted 11 lines
The guest I was trying to get to run uses the LHA and LHAU instructions. Those instructions basically do a load, but also sign extend the result.
Since we need to fill our registers by hand when doing MMIO, we also need to sign extend manually.
This patch implements sign extended MMIO and the LHA(U) instructions.
arch/powerpc/kvm/powerpc.c +0
-0@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu, }arch/powerpc/kvm/powerpc.c static void kvmppc_complete_mmio_load(struct · line 311
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +7
-0} +if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr&0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ;Wouldn't
Quoted 3 lines
gpr = (s64)(gpr << 32) >> 32;
work? Not sure if >> is guaranteed to sign extend.
Not sure either. The code as is is rather obvious imho, so I wouldn't want to replace it with anything that's even remotely magical.
Alex
A message that is not in the archive (sent off-list or not yet received): 3CEF000F-1751-4E65-A08A-C71B2CE8DAEE-l3A5Bk7waGM@public.gmane.org
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
Quoted 9 lines · Alexander Graf
Quoted 9 lines
Quoted 8 lines
arch/powerpc/kvm/powerpc.c +7
-0+if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr&0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ;
Quoted 9 lines
Quoted 5 lines
Wouldn't
gpr = (s64)(gpr << 32) >> 32;
work? Not sure if >> is guaranteed to sign extend.
Not sure either. The code as is is rather obvious imho, so I wouldn't want to replace it with anything that's even remotely magical.
That's fair.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
In reply to a message not in the archiveQuoted 24 lines · Avi Kivity
Quoted 10 lines · Alexander Graf
The guest I was trying to get to run uses the LHA and LHAU instructions. Those instructions basically do a load, but also sign extend the result.
Since we need to fill our registers by hand when doing MMIO, we also need to sign extend manually.
This patch implements sign extended MMIO and the LHA(U) instructions.
arch/powerpc/kvm/powerpc.c +0
-0@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu,arch/powerpc/kvm/powerpc.c static void kvmppc_complete_mmio_load(struct · line 310
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +7
-0} } +if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr&0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ;Wouldn't
Quoted 3 lines
gpr = (s64)(gpr << 32) >> 32;work? Not sure if >> is guaranteed to sign extend.
It's technically implementation dependent but I don't know of an implementation that doesn't sign extend.
Regards,
Anthony Liguori
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
Am 07.02.2010 um 17:27 schrieb Anthony Liguori <[email protected]>:
Quoted 25 lines · Avi Kivity
Quoted 24 lines · Alexander Graf
Quoted 10 lines
The guest I was trying to get to run uses the LHA and LHAU instructions. Those instructions basically do a load, but also sign extend the result.
Since we need to fill our registers by hand when doing MMIO, we also need to sign extend manually.
This patch implements sign extended MMIO and the LHA(U) instructions.
arch/powerpc/kvm/powerpc.c +0
-0@@ -300,6 +300,25 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu,arch/powerpc/kvm/powerpc.c static void kvmppc_complete_mmio_load(struct · line 310
Quoted 10 lines
arch/powerpc/kvm/powerpc.c +7
-0} } +if (vcpu->arch.mmio_sign_extend) { +switch (run->mmio.len) { +#ifdef CONFIG_PPC64 +case 4 : +if (gpr&0x80000000 ) + gpr |=0xffffffff00000000ULL ; +break ;Wouldn't
Quoted 5 lines
Quoted 3 lines
gpr = (s64)(gpr << 32) >> 32;
work? Not sure if >> is guaranteed to sign extend.
It's technically implementation dependent but I don't know of an implementation that doesn't sign extend.
Hrm, would
gpr = (s64)(s32)gpr;
work? :)
Alex
A message that is not in the archive (sent off-list or not yet received): 1CA08386-21CA-4B4F-A1E6-56C4DE584BA6-l3A5Bk7waGM@public.gmane.org
Re: [PATCH 03/18] KVM: PPC: Teach MMIO Signedness
Quoted 8 lines · Alexander Graf
Quoted 1 line
It's technically implementation dependent but I don't know of an implementation that doesn't sign extend.
Hrm, would
gpr = (s64)(s32)gpr;
work? :)
Yes. Integer promotion does guarantee sign extension.
Regards,
Anthony Liguori
Quoted 2 lines
Alex
PATCH 4/18 KVM: PPC: Add AGAIN type for emulation return
In reply to a message not in the archiveEmulation of an instruction can have different outcomes. It can succeed, fail, require MMIO, do funky BookE stuff - or it can just realize something's odd and will be fixed the next time around.
Exactly that is what EMULATE_AGAIN means. Using that flag we can now tell the caller that nothing happened, but we still want to go back to the guest and see what happens next time we come around.
--- arch/powerpc/include/asm/kvm_ppc.h | 1 + arch/powerpc/kvm/book3s.c | 3 +++ arch/powerpc/kvm/emulate.c | 4 +++- 3 files changed, 7 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +1 -0
@@ -37,6 +37,7 @@ enum emulation_result {EMULATE_DO_MMIO ,/* kvm_run filled with MMIO request */ EMULATE_DO_DCR ,/* kvm_run filled with DCR request */ EMULATE_FAIL ,/* can't emulate this instruction */ + EMULATE_AGAIN, /* something went wrong. go again */ };extern int __kvmppc_vcpu_run (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu);
arch/powerpc/kvm/book3s.c +3 -0
@@ -788,6 +788,9 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,case EMULATE_DONE : r =RESUME_GUEST_NV ;break ; +case EMULATE_AGAIN : + r =RESUME_GUEST ; +break ;case EMULATE_FAIL :printk (KERN_CRIT "%s: emulation at %lx failed (%08x)\n" , __func__, vcpu->arch.pc, vcpu->arch.last_inst);
arch/powerpc/kvm/emulate.c +3 -1
@@ -486,7 +486,9 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)if (emulated ==EMULATE_FAIL ) { emulated =kvmppc_core_emulate_op (run, vcpu, inst, &advance); -if (emulated ==EMULATE_FAIL ) { +if (emulated ==EMULATE_AGAIN ) { + advance =0 ; + }else if (emulated ==EMULATE_FAIL ) { advance =0 ;printk (KERN_ERR "Couldn't emulate instruction 0x%08x " "(op %d xop %d)\n" , inst,get_op (inst),get_xop (inst));
Signature · 1 line
1.6.0.2
PATCH 5/18 KVM: PPC: Add hidden flag for paired singles
In reply to a message not in the archiveThe Gekko implements an extension called paired singles. When the guest wants to use that extension, we need to make sure we're not running the host FPU, because all FPU instructions need to get emulated to accomodate for additional operations that occur.
This patch adds an hflag to track if we're in paired single mode or not.
--- arch/powerpc/include/asm/kvm_asm.h | 1 + arch/powerpc/kvm/book3s.c | 4 ++++ 2 files changed, 5 insertions(+), 0 deletions(-)
arch/powerpc/include/asm/kvm_asm.h +1 -0
@@ -88,6 +88,7 @@#define BOOK3S_HFLAG_DCBZ32 0x1#define BOOK3S_HFLAG_SLB 0x2 +#define BOOK3S_HFLAG_PAIRED_SINGLE 0x4#define RESUME_FLAG_NV (1<<0)/* Reload guest nonvolatile state? */ #define RESUME_FLAG_HOST (1<<1)/* Resume host? */
arch/powerpc/kvm/book3s.c +4 -0
@@ -638,6 +638,10 @@ static int kvmppc_handle_ext(struct kvm_vcpu *vcpu, unsigned int exit_nr,u64 *thread_fpr = (u64 *)t->fpr;int i; +/* When we have paired singles, we emulate in software */ +if (vcpu->arch.hflags &BOOK3S_HFLAG_PAIRED_SINGLE ) +return RESUME_GUEST ; +if (!(vcpu->arch.msr & msr)) {kvmppc_book3s_queue_irqprio (vcpu, exit_nr);return RESUME_GUEST ;
Signature · 1 line
1.6.0.2
PATCH 6/18 KVM: PPC: Add Gekko SPRs
In reply to a message not in the archiveThe Gekko has some SPR values that differ from other PPC core values and also some additional ones.
Let's add support for them in our mfspr/mtspr emulator.
--- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/include/asm/reg.h | 10 +++++ arch/powerpc/kvm/book3s_64_emulate.c | 70 +++++++++++++++++++++++++++++++++ 3 files changed, 81 insertions(+), 0 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -0
@@ -82,6 +82,7 @@ struct kvmppc_vcpu_book3s {struct kvmppc_bat ibat[8 ];struct kvmppc_bat dbat[8 ];u64 hid[6 ]; +u64 gqr[8 ];int slb_nr;u64 sdr1;u64 dsisr;
arch/powerpc/include/asm/reg.h +10 -0
@@ -293,10 +293,12 @@#define HID1_ABE (1<<10)/* 7450 Address Broadcast Enable */ #define HID1_PS (1<<16)/* 750FX PLL selection */ #define SPRN_HID2 0x3F8/* Hardware Implementation Register 2 */ +#define SPRN_HID2_GEKKO 0x398/* Gekko HID2 Register */ #define SPRN_IABR 0x3F2/* Instruction Address Breakpoint Register */ #define SPRN_IABR2 0x3FA/* 83xx */ #define SPRN_IBCR 0x135/* 83xx Insn Breakpoint Control Reg */ #define SPRN_HID4 0x3F4/* 970 HID4 */ +#define SPRN_HID4_GEKKO 0x3F3/* Gekko HID4 */ #define SPRN_HID5 0x3F6/* 970 HID5 */ #define SPRN_HID6 0x3F9/* BE HID 6 */ #define HID6_LB (0x0F<<12)/* Concurrent Large Page Modes */
@@ -465,6 +467,14 @@#define SPRN_VRSAVE 0x100/* Vector Register Save Register */ #define SPRN_XER 0x001/* Fixed Point Exception Register */ +#define SPRN_MMCR0_GEKKO 0x3B8/* Gekko Monitor Mode Control Register 0 */ +#define SPRN_MMCR1_GEKKO 0x3BC/* Gekko Monitor Mode Control Register 1 */ +#define SPRN_PMC1_GEKKO 0x3B9/* Gekko Performance Monitor Control 1 */ +#define SPRN_PMC2_GEKKO 0x3BA/* Gekko Performance Monitor Control 2 */ +#define SPRN_PMC3_GEKKO 0x3BD/* Gekko Performance Monitor Control 3 */ +#define SPRN_PMC4_GEKKO 0x3BE/* Gekko Performance Monitor Control 4 */ +#define SPRN_WPAR_GEKKO 0x399/* Gekko Write Pipe Address Register */ +#define SPRN_SCOMC 0x114/* SCOM Access Control */ #define SPRN_SCOMD 0x115/* SCOM Access DATA */
arch/powerpc/kvm/book3s_64_emulate.c +70 -0
114 lines not on this page · raw message
Signature · 1 line
1.6.0.2
PATCH 12/18 KVM: PPC: Make ext giveup non-static
In reply to a message not in the archiveWe need to call the ext giveup handlers from code outside of book3s.c. So let's make it non-static.
--- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/book3s.c | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -0
@@ -120,6 +120,7 @@ extern int kvmppc_st(struct kvm_vcpu *vcpu, ulong *eaddr, int size, void *ptr, bextern void kvmppc_book3s_queue_irqprio (struct kvm_vcpu *vcpu,unsigned int vec);extern void kvmppc_set_bat (struct kvm_vcpu *vcpu,struct kvmppc_bat *bat,bool upper,u32 val); +extern void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr);extern u32 kvmppc_trampoline_lowmem;extern u32 kvmppc_trampoline_enter;
arch/powerpc/kvm/book3s.c +1 -2
@@ -35,7 +35,6 @@/* #define EXIT_DEBUG_SIMPLE */ /* #define DEBUG_EXT */ -static void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr);static int kvmppc_handle_ext (struct kvm_vcpu *vcpu,unsigned int exit_nr,ulong msr);
@@ -597,7 +596,7 @@ static inline int get_fpr_index(int i) }/* Give up external provider (FPU, Altivec, VSX) */ -static void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr) +void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr) {struct thread_struct *t = ¤t->thread;u64 *vcpu_fpr = vcpu->arch.fpr;
Signature · 1 line
1.6.0.2
PATCH 14/18 KVM: PPC: Fix error in BAT assignment
In reply to a message not in the archiveBATs didn't work. Well, they did, but only up to BAT3. As soon as we came to BAT4 the offset calculation was screwed up and we ended up overwriting BAT0-3.
Fortunately, Linux hasn't been using BAT4+. It's still a good idea to write correct code though.
--- arch/powerpc/kvm/book3s_64_emulate.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/kvm/book3s_64_emulate.c +2 -2
@@ -233,13 +233,13 @@ static void kvmppc_write_bat(struct kvm_vcpu *vcpu, int sprn, u32 val) bat = &vcpu_book3s->ibat[(sprn -SPRN_IBAT0U ) /2 ];break ; case SPRN_IBAT4U ... SPRN_IBAT7L: - bat = &vcpu_book3s->ibat[(sprn -SPRN_IBAT4U ) /2 ]; + bat = &vcpu_book3s->ibat[4 + ((sprn -SPRN_IBAT4U ) /2 )];break ; case SPRN_DBAT0U ... SPRN_DBAT3L: bat = &vcpu_book3s->dbat[(sprn -SPRN_DBAT0U ) /2 ];break ; case SPRN_DBAT4U ... SPRN_DBAT7L: - bat = &vcpu_book3s->dbat[(sprn -SPRN_DBAT4U ) /2 ]; + bat = &vcpu_book3s->dbat[4 + ((sprn -SPRN_DBAT4U ) /2 )];break ;default :BUG ();
Signature · 1 line
1.6.0.2
PATCH 15/18 KVM: PPC: Add helpers to modify ppc fields
In reply to a message not in the archiveThe PowerPC specification always lists bits from MSB to LSB. That is really confusing when you're trying to write C code, because it fits in pretty badly with the normal (1 << xx) schemes.
So I came up with some nice wrappers that allow to get and set fields in a u64 with bit numbers exactly as given in the spec. That makes the code in KVM and the spec easier comparable.
--- arch/powerpc/include/asm/kvm_ppc.h | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +33 -0
@@ -103,6 +103,39 @@ extern void kvmppc_booke_exit(void);extern void kvmppc_core_destroy_mmu (struct kvm_vcpu *vcpu); +/* +* Cuts out inst bits with ordering according to spec. +* That means the leftmost bit is zero. All given bits are included. +*/ +static inline u32 kvmppc_get_field (u64 inst,int msb,int lsb) +{ +u32 r; +u32 mask; + +BUG_ON (msb > lsb); + + mask = (1 << (lsb - msb +1 )) -1 ; + r = (inst >> (63 - lsb)) & mask; + +return r; +} + +/* +* Replaces inst bits with ordering according to spec. +*/ +static inline u32 kvmppc_set_field (u64 inst,int msb,int lsb,int value) +{ +u32 r; +u32 mask; + +BUG_ON (msb > lsb); + + mask = ((1 << (lsb - msb +1 )) -1 ) << (63 - lsb); + r = (inst & ~mask) | ((value << (63 - lsb)) & mask); + +return r; +} +#ifdef CONFIG_PPC_BOOK3S /* We assume we're always acting on the current vcpu */
Signature · 1 line
1.6.0.2
PATCH 16/18 KVM: PPC: Enable program interrupt to do MMIO
In reply to a message not in the archiveWhen we get a program interrupt we usually don't expect it to perform an MMIO operation. But why not? When we emulate paired singles, we can end up loading or storing to an MMIO address - and the handling of those happens in the program interrupt handler.
So let's teach the program interrupt handler how to deal with EMULATE_MMIO.
--- arch/powerpc/kvm/book3s.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
arch/powerpc/kvm/book3s.c +4 -0
@@ -840,6 +840,10 @@ program_interrupt:kvmppc_core_queue_program (vcpu, flags); r =RESUME_GUEST ;break ; +case EMULATE_DO_MMIO : + run->exit_reason =KVM_EXIT_MMIO ; + r =RESUME_HOST_NV ; +break ;default :BUG (); }
Signature · 1 line
1.6.0.2
Quoted 7 lines · Alexander Graf
In an effort to get KVM on PPC more useful for other userspace users than Qemu, I figured it'd be a nice idea to implement virtualization of the Gekko CPU.
The Gekko is the CPU used in the GameCube. In a slightly more modern fashion it lives on in the Wii today.
Quoted 7 lines
Using this patch set and a modified version of Dolphin, I was able to virtualize simple GameCube demos on a 970MP system.
As always, while getting this to run I stumbled across several broken parts and fixed them as they came up. So expect some bug fixes in this patch set too.
This is halfway into emulation rather than virtualization. What does performance look like when running fpu intensive applications?
I might have missed it, but I didn't see the KVM_CAP and save/restore support for this.
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Am 07.02.2010 um 13:54 schrieb Avi Kivity <[email protected]>:
Quoted 11 lines · Alexander Graf
Quoted 10 lines
In an effort to get KVM on PPC more useful for other userspace users than Qemu, I figured it'd be a nice idea to implement virtualization of the Gekko CPU.
The Gekko is the CPU used in the GameCube. In a slightly more modern fashion it lives on in the Wii today.
Using this patch set and a modified version of Dolphin, I was able to virtualize simple GameCube demos on a 970MP system.
Quoted 6 lines
Quoted 4 lines
As always, while getting this to run I stumbled across several broken parts and fixed them as they came up. So expect some bug fixes in this patch set too.
This is halfway into emulation rather than virtualization. What does performance look like when running fpu intensive applications?
It is for the FPU. It is not for whatever runs on the CPU.
I haven't benchmarked things so far,
The only two choices I have to get this running is in-kernel emulation or userspace emulation. According to how x86 deals with things I suppose full state transition to userspace and continuing emulation there isn't considered a good idea. So I went with in-kernel.
Quoted 2 lines
I might have missed it, but I didn't see the
KVM_CAPand save/restore support for this.
Ah, cap again. Right. Mind if I send an patch on top of the set?
As far as save/restore goes, the ioctl to get/set fprs isn't even implemented (yet)! We're really off full state migration to/from userspace yet.
Alex
Quoted 22 lines · Alexander Graf
Am 07.02.2010 um 13:54 schrieb Avi Kivity <[email protected]>:
Quoted 12 lines · Alexander Graf
Quoted 11 lines
In an effort to get KVM on PPC more useful for other userspace users than
Qemu, I figured it'd be a nice idea to implement virtualization of the Gekko CPU.The Gekko is the CPU used in the GameCube. In a slightly more modern fashion it lives on in the Wii today.
Using this patch set and a modified version of Dolphin, I was able to virtualize simple GameCube demos on a 970MP system.
Quoted 7 lines
Quoted 4 lines
As always, while getting this to run I stumbled across several broken parts and fixed them as they came up. So expect some bug fixes in this patch set too.
This is halfway into emulation rather than virtualization. What does performance look like when running fpu intensive applications?
Quoted 8 lines
It is for the FPU. It is not for whatever runs on the CPU.
I haven't benchmarked things so far,
The only two choices I have to get this running is in-kernel emulation or userspace emulation. According to how x86 deals with things I suppose full state transition to userspace and continuing emulation there isn't considered a good idea. So I went with in-kernel.
It's not a good idea for the kernel either, if it happens all the time. If a typical Gekko application uses the fpu and the emulated instructions intensively, performance will suck badly (as in: qemu/tcg will be faster).
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 32 lines · Avi Kivity
Quoted 22 lines · Alexander Graf
Am 07.02.2010 um 13:54 schrieb Avi Kivity <[email protected]>:
Quoted 12 lines · Alexander Graf
Quoted 11 lines
In an effort to get KVM on PPC more useful for other userspace users than
Qemu, I figured it'd be a nice idea to implement virtualization of the Gekko CPU.The Gekko is the CPU used in the GameCube. In a slightly more modern fashion it lives on in the Wii today.
Using this patch set and a modified version of Dolphin, I was able to virtualize simple GameCube demos on a 970MP system.
Quoted 7 lines
Quoted 4 lines
As always, while getting this to run I stumbled across several broken parts and fixed them as they came up. So expect some bug fixes in this patch set too.
This is halfway into emulation rather than virtualization. What does performance look like when running fpu intensive applications?
Quoted 8 lines
It is for the FPU. It is not for whatever runs on the CPU.
I haven't benchmarked things so far,
The only two choices I have to get this running is in-kernel emulation or userspace emulation. According to how x86 deals with things I suppose full state transition to userspace and continuing emulation there isn't considered a good idea. So I went with in-kernel.
Quoted 5 lines
It's not a good idea for the kernel either, if it happens all the time. If a typical Gekko application uses the fpu and the emulated instructions intensively, performance will suck badly (as in: qemu/tcg will be faster).
Yeah, I haven't really gotten far enough to run full-blown guests yet. So far I'm on demos and they look pretty good.
But as far as intercept speed goes - I just tried running this little piece of code in kvmctl:
.global _start_start:
li r3, 42
mtsprg 0, r3
mfsprg r4, 0
b _start
and measured the amount of exits I get on my test machine:
processor : 0
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)
--->
exits 1811108
I have no idea how we manage to get that many exits, but apparently we are. So I'm less concerned about the speed of the FPU rerouting at the moment.
If it really gets unusably slow, I'd rather binary patch the guest on the fly in KVM according to rules set by the userspace client. But we'll get there when it turns out to be too slow. For now I'd rather like to have something working at all and then improve speed :-).
Alex
Quoted 27 lines · Alexander Graf
Quoted 7 lines
It's not a good idea for the kernel either, if it happens all the time. If a typical Gekko application uses the fpu and the emulated instructions intensively, performance will suck badly (as in: qemu/tcg will be faster).
Yeah, I haven't really gotten far enough to run full-blown guests yet. So far I'm on demos and they look pretty good.
But as far as intercept speed goes - I just tried running this little piece of code in kvmctl:
.global
_start_start:
li r3, 42
mtsprg 0, r3
mfsprg r4, 0
b_startand measured the amount of exits I get on my test machine:
processor : 0
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)
Quoted 8 lines
--->
exits 1811108
I have no idea how we manage to get that many exits, but apparently we are. So I'm less concerned about the speed of the FPU rerouting at the moment.
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Quoted 2 lines
If it really gets unusably slow, I'd rather binary patch the guest on the fly in KVM according to rules set by the userspace client.
Is that even possible? Do those register-pair instructions and registers map 1:1 to 970 instructions and registers?
Quoted 4 lines
But we'll
get there when it turns out to be too slow. For now I'd rather like to have something working at all and then improve speed :-).
Well, I want to see the light at the end of the tunnel first. Adding code is easy, ripping it out later not so much.
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 30 lines · Avi Kivity
Quoted 29 lines · Alexander Graf
Quoted 7 lines
It's not a good idea for the kernel either, if it happens all the time. If a typical Gekko application uses the fpu and the emulated instructions intensively, performance will suck badly (as in: qemu/tcg will be faster).
Yeah, I haven't really gotten far enough to run full-blown guests yet. So far I'm on demos and they look pretty good.
But as far as intercept speed goes - I just tried running this little piece of code in kvmctl:
.global
_start_start:
li r3, 42
mtsprg 0, r3
mfsprg r4, 0
b_startand measured the amount of exits I get on my test machine:
processor : 0
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)--->
Quoted 8 lines
Quoted 6 lines
exits 1811108
I have no idea how we manage to get that many exits, but apparently we are. So I'm less concerned about the speed of the FPU rerouting at the moment.
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
True.
Quoted 5 lines
Quoted 3 lines
If it really gets unusably slow, I'd rather binary patch the guest on the fly in KVM according to rules set by the userspace client.
Is that even possible? Do those register-pair instructions and registers map 1:1 to 970 instructions and registers?
Almost. Basically all I need to do is execute 2 FPU instructions instead of one for single instructions and paired single special instructions. So if I could patch the instruction to jump to some shared memory page, it'd become fast. At least as long as I figure out how to make sure we run with FP=0 in normal code, but with FP=1 in the special page ;).
Quoted 7 lines
Quoted 5 lines
But we'll
get there when it turns out to be too slow. For now I'd rather like to have something working at all and then improve speed :-).Well, I want to see the light at the end of the tunnel first. Adding code is easy, ripping it out later not so much.
Hum, so you suggest I get some real application running properly first so we can evaluate if it's fast enough?
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 87CEECB5-107A-46EB-89F5-1E1F92AC22AA-l3A5Bk7waGM@public.gmane.org
Quoted 7 lines · Alexander Graf
Quoted 5 lines
Quoted 3 lines
If it really gets unusably slow, I'd rather binary patch the guest on the fly in KVM according to rules set by the userspace client.
Is that even possible? Do those register-pair instructions and registers map 1:1 to 970 instructions and registers?
Almost. Basically all I need to do is execute 2 FPU instructions instead of one for single instructions and paired single special instructions. So if I could patch the instruction to jump to some shared memory page, it'd become fast. At least as long as I figure out how to make sure we run with FP=0 in normal code, but with FP=1 in the special page ;).
How do you locate a free virtual address to poke your shared memory page into? What if the guest kernel instantiates it later?
Aren't direct jumps limited in their offset?
What if an exception happens in the shared memory page?
Patching is hard, let's go shopping.
Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
But we'll
get there when it turns out to be too slow. For now I'd rather like to have something working at all and then improve speed :-).Well, I want to see the light at the end of the tunnel first. Adding code is easy, ripping it out later not so much.
Hum, so you suggest I get some real application running properly first so we can evaluate if it's fast enough?
Yes, a real application typical for whatever use case you envision for Gekko emulation (can you shed a few words on that please).
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Alexander Graf
Quoted 5 lines
Quoted 3 lines
If it really gets unusably slow, I'd rather binary patch the guest on the fly in KVM according to rules set by the userspace client.
Is that even possible? Do those register-pair instructions and registers map 1:1 to 970 instructions and registers?
Almost. Basically all I need to do is execute 2 FPU instructions instead of one for single instructions and paired single special instructions. So if I could patch the instruction to jump to some shared memory page, it'd become fast. At least as long as I figure out how to make sure we run with FP=0 in normal code, but with FP=1 in the special page ;).
How do you locate a free virtual address to poke your shared memory page into?
Most applications use the same virtual memory layout. I'm 100% confident we can make the region not collide with those. For applications actually doing memory mapping themselves, there is Linux, where we know the layout too, and one or two special applications. But I think it's feasible. At least for 99.9% of the cases.
Quoted 1 line
What if the guest kernel instantiates it later?
Well, then we're screwed and need to fall back to trapping and emulating like my patch does now. I guess we could blacklist those guests.
Quoted 1 line
Aren't direct jumps limited in their offset?
Yes. We can do an absolute branch do negative addresses, effectively jumping to 0xffffffff - x whereas x is 15 bits & ~3 IIRC. That's definitely enough for at least a shared page for registers and a jump table :-).
Quoted 1 line
What if an exception happens in the shared memory page?
Well, then the guest kernel needs to be gracious. I'm fairly sure it is ;-). It doesn't make sense to examine on which ip an interrupt occured.
Quoted 1 line
Patching is hard, let's go shopping.
Yay :)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
But we'll
get there when it turns out to be too slow. For now I'd rather like to have something working at all and then improve speed :-).
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Well, I want to see the light at the end of the tunnel first. Adding code is easy, ripping it out later not so much.
Hum, so you suggest I get some real application running properly first so we can evaluate if it's fast enough?
Yes, a real application typical for whatever use case you envision for Gekko emulation (can you shed a few words on that please).
I did mention Dolphin, right?
Basically I envision that this is the easiest way to do PR for KVM on PPC. Releasing this properly will instantly raise awareness and thus potentially increase our user base by a lot. IMHO it'd also help KVM in general, keeping it in the news.
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): A5BC5A7E-D45B-4BAF-804A-B364810F50DA-l3A5Bk7waGM@public.gmane.org
Quoted 8 lines · Alexander Graf
Quoted 5 lines
Quoted 3 lines
Hum, so you suggest I get some real application running properly first so we can evaluate if it's fast enough?
Yes, a real application typical for whatever use case you envision for Gekko emulation (can you shed a few words on that please).
I did mention Dolphin, right?
Must have missed it.
Quoted 4 lines
Basically I envision that this is the easiest way to do PR for KVM on PPC. Releasing this properly will instantly raise awareness and thus potentially increase our user base by a lot. IMHO it'd also help KVM in general, keeping it in the news.
To me it seems the intersection of gamers with ppc desktop linux owners would be rather small.
I'm not opposed to merging special use cases (esp. as you're doing all of the work AND are responsible for maintainance), but I would like to be sure that it doesn't end up unusable.
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Avi Kivity
Quoted 8 lines · Alexander Graf
Quoted 5 lines
Quoted 3 lines
Hum, so you suggest I get some real application running properly first so we can evaluate if it's fast enough?
Yes, a real application typical for whatever use case you envision for Gekko emulation (can you shed a few words on that please).
I did mention Dolphin, right?
Quoted 8 lines
Must have missed it.
Quoted 4 lines
Basically I envision that this is the easiest way to do PR for KVM on PPC. Releasing this properly will instantly raise awareness and thus potentially increase our user base by a lot. IMHO it'd also help KVM in general, keeping it in the news.
To me it seems the intersection of gamers with ppc desktop linux owners would be rather small.
There are no ppc desktop linux owners left. Well - almost none. There's only servers and gamers.
Quoted 1 line
I'm not opposed to merging special use cases (esp. as you're doing all of the work AND are responsible for maintainance), but I would like to be sure that it doesn't end up unusable.
Yep :-). I'll try and see how far I can get on getting something real running. Then I get a feeling for how fast this whole approach is.
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 939C8633-1B2C-4888-B1C1-357DF1C56CE6-l3A5Bk7waGM@public.gmane.org
Quoted 10 lines · Alexander Graf
Quoted 8 lines
Quoted 6 lines
Basically I envision that this is the easiest way to do PR for KVM on PPC. Releasing this properly will instantly raise awareness and thus potentially increase our user base by a lot. IMHO it'd also help KVM in general, keeping it in the news.
To me it seems the intersection of gamers with ppc desktop linux owners would be rather small.
There are no ppc desktop linux owners left. Well - almost none. There's only servers and gamers.
... so what's the use case? server owners won't run games, and console owners don't need kvm to run games. Unless you propose to run Wii games on PlayStation, or something.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Alexander Graf
Quoted 6 lines
Quoted 6 lines
Basically I envision that this is the easiest way to do PR for KVM on PPC. Releasing this properly will instantly raise awareness and thus potentially increase our user base by a lot. IMHO it'd also help KVM in general, keeping it in the news.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
To me it seems the intersection of gamers with ppc desktop linux owners would be rather small.
There are no ppc desktop linux owners left. Well - almost none. There's only servers and gamers.
... so what's the use case? server owners won't run games, and console owners don't need kvm to run games. Unless you propose to run Wii games on PlayStation, or something.
It's an experiment to verify a theory I've had for quite a while, based on this talk: http://www.heise.de/fastbin/eventmanager/file/?media_idF6
The general idea is that the only difference between proprietary software and open software for a company developing open source is that they need less marketing. Simply because people outside will be aware of the products.
Take Windows Server as an example. The reason people use it is because they know how to use Windows at home. They take a brand they know, usability they know and expectations they have to the server world.
I think the same thing could apply to KVM. If there was a UI that's as easy to use and as user focused as virtual box, the awareness of kvm would rise because people use it on their workstations and would thus expect the same piece of software on their servers.
Since I'm no UI programmer and I figured creating a usable UI takes way too long for me to spend time on, I figured I'd go with something where I could make a difference myself. And that's PPC.
If you're using KVM on your game console and it works well, why not use it on your server?
Also, as mentioned earlier, I've had different levels of awareness of stuff I did so far. People were extremely fascinated by and eager to see osx running on kvm. For nesting, the audience was smaller, thus the news it generated was way less. When it comes to commodity hardware (game consoles), I think the audience is a lot bigger again, thus generating more traction.
The same thing happened with the PPC btw. Since Apple stopped shipping PPC based Macs, people are way less aware and way less interested in it. I'm pretty sure the number of server sales are still the same as they used to be, but because there's no commodity hardware that generates awareness, people believe it to be dead.
So far for the theory. We'll see if my point gets proven soon enough I guess :-).
Alex
Quoted 37 lines · Avi Kivity
Quoted 27 lines · Alexander Graf
Quoted 7 lines
It's not a good idea for the kernel either, if it happens all the time. If a typical Gekko application uses the fpu and the emulated instructions intensively, performance will suck badly (as in: qemu/tcg will be faster).
Yeah, I haven't really gotten far enough to run full-blown guests yet. So far I'm on demos and they look pretty good.
But as far as intercept speed goes - I just tried running this little piece of code in kvmctl:
.global
_start_start:
li r3, 42
mtsprg 0, r3
mfsprg r4, 0
b_startand measured the amount of exits I get on my test machine:
processor : 0
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)Quoted 8 lines
--->
exits 1811108
I have no idea how we manage to get that many exits, but apparently we are. So I'm less concerned about the speed of the FPU rerouting at the moment.
Quoted 4 lines
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Measuring a typical Gekko application, I get about 200k-250k of fpu (incl. paired singles) instructions per second.
Alex
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Alexander Graf
Quoted 6 lines
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Measuring a typical Gekko application, I get about 200k-250k of fpu (incl. paired singles) instructions per second.
Virtualized, yes? What's the rate on bare metal?
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Alexander Graf
Quoted 6 lines
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Quoted 5 lines
Quoted 3 lines
Measuring a typical Gekko application, I get about 200k-250k of fpu (incl. paired singles) instructions per second.
Virtualized, yes? What's the rate on bare metal?
Emulated. I can't measure anything on bare metal.
Alex
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Alexander Graf
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Alexander Graf
Quoted 7 lines
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines
Measuring a typical Gekko application, I get about 200k-250k of fpu (incl. paired singles) instructions per second.
Virtualized, yes? What's the rate on bare metal?
Emulated. I can't measure anything on bare metal.
Well, then, the rate may be low due to virtualization overhead. Any way to compare absolute performance?
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 16 lines · Avi Kivity
Quoted 15 lines · Alexander Graf
Quoted 13 lines · Avi Kivity
Quoted 11 lines · Alexander Graf
Quoted 7 lines
That's pretty impressive (never saw x86 with this exit rate) but it's more than 1000 times slower than the hardware, assuming 1 fpu IPC (and the processor can probably do more). An fpu intensive application will slow to a crawl.
Measuring a typical Gekko application, I get about 200k-250k of fpu (incl. paired singles) instructions per second.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Virtualized, yes? What's the rate on bare metal?
Emulated. I can't measure anything on bare metal.
Well, then, the rate may be low due to virtualization overhead. Any way to compare absolute performance?
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234
std r2, 0(r1)
lfd f3, 0(r1)
lfd f4, 0(r1)
do_mul:
fmul f0, f3, f4
b do_mulWith the following kvm_stat output:
dec 2236 53
exits 60797802 1171403
ext_intr 379 4
halt_wakeup 0 0
inst_emu 60795247 1171344
ld 60795132 1171348
So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): 80F0B53A-6F83-4166-8F85-5D9B07526158-l3A5Bk7waGM@public.gmane.org
Quoted 16 lines · Alexander Graf
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234 std r2, 0(r1) lfd f3, 0(r1) lfd f4, 0(r1) do_mul: fmul f0, f3, f4 b do_mulWith the following
kvm_statoutput:
Quoted 9 lines
dec 2236 53
exits 60797802 1171403
ext_intr379 4
halt_wakeup0 0
inst_emu60795247 1171344
ld 60795132 1171348So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
It's a large number, but won't real hardware be three orders of magnitude faster?
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · Avi Kivity
Quoted 23 lines · Alexander Graf
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234 std r2, 0(r1) lfd f3, 0(r1) lfd f4, 0(r1) do_mul: fmul f0, f3, f4 b do_mulWith the following
kvm_statoutput:dec 2236 53
exits 60797802 1171403
ext_intr379 4
halt_wakeup0 0
inst_emu60795247 1171344
ld 60795132 1171348
Quoted 4 lines
Quoted 2 lines
So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
It's a large number, but won't real hardware be three orders of magnitude faster?
Yes, it would. But we don't have to care. The only thing we need to worry about is being fast enough to emulate enough FPU instructions actually used in normal guests so the guest runs in full speed. And 1000k > 250k, so we can do that apparently, leaving some spare cycles for non-fpu instructions.
The kernel on my PS3 is still compiling. Let's see how fast I get there.
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 27 lines · Alexander Graf
Quoted 24 lines · Avi Kivity
Quoted 22 lines · Alexander Graf
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234 std r2, 0(r1) lfd f3, 0(r1) lfd f4, 0(r1) do_mul: fmul f0, f3, f4 b do_mulWith the following
kvm_statoutput:dec 2236 53
exits 60797802 1171403
ext_intr379 4
halt_wakeup0 0
inst_emu60795247 1171344
ld 60795132 1171348
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
It's a large number, but won't real hardware be three orders of magnitude faster?
Yes, it would. But we don't have to care. The only thing we need to worry about is being fast enough to emulate enough FPU instructions actually used in normal guests so the guest runs in full speed. And 1000k> 250k, so we can do that apparently, leaving some spare cycles for non-fpu instructions.
I'm sure 250k isn't representative of a floating point intensive program (but maybe there aren't fpu intensive applications on that cpu).
Signature · 1 line
error compiling committee.c: too many arguments to function
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 31 lines · Avi Kivity
Quoted 30 lines · Alexander Graf
Quoted 27 lines · Avi Kivity
Quoted 25 lines · Alexander Graf
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234 std r2, 0(r1) lfd f3, 0(r1) lfd f4, 0(r1) do_mul: fmul f0, f3, f4 b do_mulWith the following
kvm_statoutput:dec 2236 53
exits 60797802 1171403
ext_intr379 4
halt_wakeup0 0
inst_emu60795247 1171344
ld 60795132 1171348So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
It's a large number, but won't real hardware be three orders of magnitude faster?
Yes, it would. But we don't have to care. The only thing we need to worry about is being fast enough to emulate enough FPU instructions actually used in normal guests so the guest runs in full speed. And 1000k> 250k, so we can do that apparently, leaving some spare cycles for non-fpu instructions.
I'm sure 250k isn't representative of a floating point intensive program (but maybe there aren't fpu intensive applications on that cpu).
Now you made me check how fast the real hw is. I get about 65,000,000 fmul operations per second on it.
So we're 65x slower on a PowerStation. And that's for a tight FPU only loop. I'm still not convinced we're running into major problems.
Alex--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): D3C1F39A-E9C8-46B8-9C4A-12E56D34B5AC-l3A5Bk7waGM@public.gmane.org
Quoted 41 lines · Alexander Graf
Quoted 38 lines · Avi Kivity
Quoted 36 lines · Alexander Graf
Quoted 32 lines · Avi Kivity
Quoted 15 lines · Alexander Graf
So I changed to code according to your input by making all FPU calls explicit, getting rid of all binary patching.
On the PowerStation again I'm running this code (simplified to the important instructions) using kvmctl:
li r2, 0x1234 std r2, 0(r1) lfd f3, 0(r1) lfd f4, 0(r1) do_mul: fmul f0, f3, f4 b do_mulWith the following
kvm_statoutput:Quoted 11 lines
dec 2236 53
exits 60797802 1171403
ext_intr379 4
halt_wakeup0 0
inst_emu60795247 1171344
ld 60795132 1171348So I'm getting 1171403 fmul operations per second. And that's even with non-optimized instruction fetching. Not bad.
It's a large number, but won't real hardware be three orders of magnitude faster?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Yes, it would. But we don't have to care. The only thing we need to worry about is being fast enough to emulate enough FPU instructions actually used in normal guests so the guest runs in full speed. And 1000k> 250k, so we can do that apparently, leaving some spare cycles for non-fpu instructions.
I'm sure 250k isn't representative of a floating point intensive program (but maybe there aren't fpu intensive applications on that cpu).
Now you made me check how fast the real hw is. I get about 65,000,000 fmul operations per second on it.
That's surprisingly low.
Quoted 2 lines
So we're 65x slower on a PowerStation. And that's for a tight FPU only loop. I'm still not convinced we're running into major problems.
Well, it's up to you. I just hope we don't end up underperforming due to this.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 4 lines · Avi Kivity
Quoted 2 lines
Now you made me check how fast the real hw is. I get about 65,000,000 fmul operations per second on it.
That's surprisingly low.
I get 3.7 Gflops on my home machine (1G loops, 4 fmul and 4 fadds, all independent, in 2.15 seconds; otherwise I can't saturate the pipeline).
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
PATCH 7/18 KVM: PPC: Combine extension interrupt handlers
In reply to Alexander GrafWhen we for example get an Altivec interrupt, but our guest doesn't support altivec, we need to inject a program interrupt, not an altivec interrupt.
The same goes for paired singles. When an altivec interrupt arrives, we're pretty sure we need to emulate the instruction because it's a paired single operation.
So let's make all the ext handlers aware that they need to jump to the program interrupt handler when an extension interrupt arrives that was not supposed to arrive for the guest CPU.
--- arch/powerpc/kvm/book3s.c | 55 ++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 50 insertions(+), 5 deletions(-)
arch/powerpc/kvm/book3s.c +50 -5
@@ -36,6 +36,8 @@/* #define DEBUG_EXT */ static void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr); +static int kvmppc_handle_ext (struct kvm_vcpu *vcpu,unsigned int exit_nr, +ulong msr);struct kvm_stats_debugfs_item debugfs_entries[] = { {"exits" ,VCPU_STAT (sum_exits) },
@@ -628,6 +630,30 @@ static void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr)kvmppc_recalc_shadow_msr (vcpu); } +static int kvmppc_check_ext (struct kvm_vcpu *vcpu,unsigned int exit_nr) +{ +ulong srr0 = vcpu->arch.pc; +int ret; + +/* Need to do paired single emulation? */ +if (!(vcpu->arch.hflags &BOOK3S_HFLAG_PAIRED_SINGLE )) +return EMULATE_DONE ; + +/* Read out the instruction */ + ret =kvmppc_ld (vcpu, &srr0,sizeof (u32), &vcpu->arch.last_inst, false); +if (ret = -ENOENT ) { + vcpu->arch.msr =kvmppc_set_field (vcpu->arch.msr,33 ,33 ,1 ); + vcpu->arch.msr =kvmppc_set_field (vcpu->arch.msr,34 ,36 ,0 ); + vcpu->arch.msr =kvmppc_set_field (vcpu->arch.msr,42 ,47 ,0 ); +kvmppc_book3s_queue_irqprio (vcpu,BOOK3S_INTERRUPT_INST_STORAGE ); + }else if (ret =EMULATE_DONE ) { +/* Need to emulate */ +return EMULATE_FAIL ; + } + +return EMULATE_AGAIN ; +} +/* Handle external providers (FPU, Altivec, VSX) */ static int kvmppc_handle_ext (struct kvm_vcpu *vcpu,unsigned int exit_nr,ulong msr)
@@ -772,6 +798,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,enum emulation_result er;ulong flags; +program_interrupt: flags = vcpu->arch.shadow_srr1 &0x1f0000ull ;if (vcpu->arch.msr &MSR_PR ) {
@@ -815,14 +842,32 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu, r =RESUME_GUEST ;break ;case BOOK3S_INTERRUPT_FP_UNAVAIL : - r =kvmppc_handle_ext (vcpu, exit_nr,MSR_FP ); -break ;case BOOK3S_INTERRUPT_ALTIVEC : - r =kvmppc_handle_ext (vcpu, exit_nr,MSR_VEC ); -break ;case BOOK3S_INTERRUPT_VSX : - r =kvmppc_handle_ext (vcpu, exit_nr,MSR_VSX ); + { +int ext_msr =0 ; + +switch (exit_nr) { +case BOOK3S_INTERRUPT_FP_UNAVAIL : ext_msr =MSR_FP ;break ; +case BOOK3S_INTERRUPT_ALTIVEC : ext_msr =MSR_VEC ;break ; +case BOOK3S_INTERRUPT_VSX : ext_msr =MSR_VSX ;break ; + } + +switch (kvmppc_check_ext (vcpu, exit_nr)) { +case EMULATE_DONE : +/* everything ok - let's enable the ext */ + r =kvmppc_handle_ext (vcpu, exit_nr, ext_msr); +break ; +case EMULATE_FAIL : +/* we need to emulate this instruction */ + goto program_interrupt; +break ; +default : +/* nothing to worry about - go again */ +break ; + }break ; + }case BOOK3S_INTERRUPT_MACHINE_CHECK :case BOOK3S_INTERRUPT_TRACE :kvmppc_book3s_queue_irqprio (vcpu, exit_nr);
Signature · 1 line
1.6.0.2
PATCH 8/18 KVM: PPC: Preload FPU when possible
In reply to Alexander GrafThere are some situations when we're pretty sure the guest will use the FPU soon. So we can save the churn of going into the guest, finding out it does want to use the FPU and going out again.
This patch adds preloading of the FPU when it's reasonable.
--- arch/powerpc/kvm/book3s.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
arch/powerpc/kvm/book3s.c +8 -0
@@ -137,6 +137,10 @@ void kvmppc_set_msr(struct kvm_vcpu *vcpu, u64 msr)kvmppc_mmu_flush_segments (vcpu);kvmppc_mmu_map_segment (vcpu, vcpu->arch.pc); } + +/* Preload FPU if it's enabled */ +if (vcpu->arch.msr &MSR_FP ) +kvmppc_handle_ext (vcpu,BOOK3S_INTERRUPT_FP_UNAVAIL ,MSR_FP ); }void kvmppc_inject_interrupt (struct kvm_vcpu *vcpu,int vec,u64 flags)
@@ -1194,6 +1198,10 @@ int __kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)/* XXX we get called with irq disabled - change that! */ local_irq_enable (); +/* Preload FPU if it's enabled */ +if (vcpu->arch.msr &MSR_FP ) +kvmppc_handle_ext (vcpu,BOOK3S_INTERRUPT_FP_UNAVAIL ,MSR_FP ); + ret =__kvmppc_vcpu_entry (kvm_run, vcpu);local_irq_disable ();
Signature · 1 line
1.6.0.2
PATCH 9/18 KVM: PPC: Fix typo in book3s_32 debug code
In reply to Alexander GrafThere's a typo in the debug ifdef of the book3s_32 mmu emulation. While trying to debug something I stumbled across that and wanted to save anyone after me (or myself later) from having to debug that again.
So let's fix the ifdef.
--- arch/powerpc/kvm/book3s_32_mmu.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/powerpc/kvm/book3s_32_mmu.c +1 -1
@@ -37,7 +37,7 @@ #define dprintk(X...) do { } while(0)#endif -#ifdef DEBUG_PTE +#ifdef DEBUG_MMU_PTE #define dprintk_pte(X...) printk(KERN_INFO X)#else #define dprintk_pte(X...) do { } while(0)
Signature · 1 line
1.6.0.2
PATCH 10/18 KVM: PPC: Implement mtsr instruction emulation
In reply to Alexander GrafThe Book3S_32 specifications allows for two instructions to modify segment
registers: mtsrin and mtsr.
Most normal operating systems use mtsrin, because it allows to define which segment it wants to change using a register. But since I was trying to run an embedded guest, it turned out to be using mtsr with hardcoded values.
So let's also emulate mtsr. It's a valid instruction after all.
--- arch/powerpc/kvm/book3s_64_emulate.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/powerpc/kvm/book3s_64_emulate.c +6 -0
@@ -28,6 +28,7 @@#define OP_31_XOP_MFMSR 83#define OP_31_XOP_MTMSR 146#define OP_31_XOP_MTMSRD 178 +#define OP_31_XOP_MTSR 210#define OP_31_XOP_MTSRIN 242#define OP_31_XOP_TLBIEL 274#define OP_31_XOP_TLBIE 306
@@ -101,6 +102,11 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu, }break ; } +case OP_31_XOP_MTSR : + vcpu->arch.mmu.mtsrin (vcpu, + (inst >>16 ) &0xf , +kvmppc_get_gpr (vcpu,get_rs (inst))); +break ;case OP_31_XOP_MTSRIN : vcpu->arch.mmu.mtsrin(vcpu, (kvmppc_get_gpr(vcpu, get_rb(inst)) >> 28) & 0xf,
Signature · 1 line
1.6.0.2
PATCH 11/18 KVM: PPC: Make software load/store return eaddr
In reply to Alexander GrafThe Book3S KVM implementation contains some helper functions to load and store data from and to virtual addresses.
Unfortunately, this helper used to keep the physical address it so nicely found out for us to itself. So let's change that and make it return the physical address it resolved.
--- arch/powerpc/include/asm/kvm_book3s.h | 4 +- arch/powerpc/kvm/book3s.c | 41 ++++++++++++++++++++------------- arch/powerpc/kvm/book3s_64_emulate.c | 11 +++++---- 3 files changed, 33 insertions(+), 23 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +2 -2
@@ -115,8 +115,8 @@ extern int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte);extern int kvmppc_mmu_map_segment (struct kvm_vcpu *vcpu,ulong eaddr);extern void kvmppc_mmu_flush_segments (struct kvm_vcpu *vcpu);extern struct kvmppc_pte *kvmppc_mmu_find_pte (struct kvm_vcpu *vcpu,u64 ea,bool data); -extern int kvmppc_ld (struct kvm_vcpu *vcpu,ulong eaddr,int size,void *ptr,bool data); -extern int kvmppc_st (struct kvm_vcpu *vcpu,ulong eaddr,int size,void *ptr); +extern int kvmppc_ld (struct kvm_vcpu *vcpu,ulong *eaddr,int size,void *ptr,bool data); +extern int kvmppc_st (struct kvm_vcpu *vcpu,ulong *eaddr,int size,void *ptr,bool data);extern void kvmppc_book3s_queue_irqprio (struct kvm_vcpu *vcpu,unsigned int vec);extern void kvmppc_set_bat (struct kvm_vcpu *vcpu,struct kvmppc_bat *bat,bool upper,u32 val);
arch/powerpc/kvm/book3s.c +25 -16
80 lines not on this page · raw message
arch/powerpc/kvm/book3s_64_emulate.c +6 -5
@@ -169,7 +169,7 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu, {ulong rb =kvmppc_get_gpr (vcpu,get_rb (inst));ulong ra =0 ; -ulong addr; +ulong addr, vaddr;u32 zeros[8 ] = {0 ,0 ,0 ,0 ,0 ,0 ,0 ,0 };if (get_ra (inst))
@@ -178,15 +178,16 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu, addr = (ra + rb) & ~31ULL ;if (!(vcpu->arch.msr &MSR_SF )) addr &=0xffffffff ; + vaddr = addr; -if (kvmppc_st (vcpu, addr,32 , zeros)) { - vcpu->arch.dear = addr; - vcpu->arch.fault_dear = addr; +if (kvmppc_st (vcpu, &addr,32 , zeros, true)) { + vcpu->arch.dear = vaddr; + vcpu->arch.fault_dear = vaddr;to_book3s (vcpu)->dsisr =DSISR_PROTFAULT |DSISR_ISSTORE ;kvmppc_book3s_queue_irqprio (vcpu,BOOK3S_INTERRUPT_DATA_STORAGE ); -kvmppc_mmu_pte_flush (vcpu, addr, ~0xFFFULL ); +kvmppc_mmu_pte_flush (vcpu, vaddr, ~0xFFFULL ); }break ;
Signature · 1 line
1.6.0.2
PATCH 13/18 KVM: PPC: Add helpers to call FPU instructions
In reply to Alexander GrafTo emulate paired single instructions, we need to be able to call FPU operations from within the kernel. Since we don't want gcc to spill arbitrary FPU code everywhere, we tell it to use a soft fpu.
Since we know we can really call the FPU in safe areas, let's also add some calls that we can later use to actually execute real world FPU operations on the host's FPU.
--- arch/powerpc/include/asm/kvm_fpu.h | 45 +++++++++++++++++++++ arch/powerpc/kernel/ppc_ksyms.c | 2 + arch/powerpc/kvm/Makefile | 1 + arch/powerpc/kvm/fpu.S | 77 ++++++++++++++++++++++++++++++++++++ 4 files changed, 125 insertions(+), 0 deletions(-) create mode 100644 arch/powerpc/include/asm/kvm_fpu.h create mode 100644 arch/powerpc/kvm/fpu.S
arch/powerpc/include/asm/kvm_fpu.h new file +45 -0
@@ -0,0 +1,45 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License, version 2, as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +* +* Copyright Novell Inc. 2010 +* +* Authors: Alexander Graf <[email protected]> +*/ + +#ifndef __ASM_KVM_FPU_H__ +#define __ASM_KVM_FPU_H__ + +#include <linux/types.h> + +extern void fp_fres (struct thread_struct *t,u32 *dst,u32 *src1); +extern void fp_frsqrte (struct thread_struct *t,u32 *dst,u32 *src1); +extern void fp_fsqrts (struct thread_struct *t,u32 *dst,u32 *src1); + +extern void fp_fadds (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2); +extern void fp_fdivs (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2); +extern void fp_fmuls (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2); +extern void fp_fsubs (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2); + +extern void fp_fmadds (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2, +u32 *src3); +extern void fp_fmsubs (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2, +u32 *src3); +extern void fp_fnmadds (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2, +u32 *src3); +extern void fp_fnmsubs (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2, +u32 *src3); +extern void fp_fsel (struct thread_struct *t,u32 *dst,u32 *src1,u32 *src2, +u32 *src3); + +#endif
arch/powerpc/kernel/ppc_ksyms.c +2 -0
@@ -101,6 +101,8 @@ EXPORT_SYMBOL(pci_dram_offset);EXPORT_SYMBOL (start_thread);EXPORT_SYMBOL (kernel_thread); +EXPORT_SYMBOL_GPL (cvt_df); +EXPORT_SYMBOL_GPL (cvt_fd);EXPORT_SYMBOL (giveup_fpu);#ifdef CONFIG_ALTIVEC EXPORT_SYMBOL (giveup_altivec);
arch/powerpc/kvm/Makefile +1 -0
@@ -40,6 +40,7 @@ kvm-objs-$(CONFIG_KVM_E500) := $(kvm-e500-objs) kvm-book3s_64-objs := \ $(common-objs-y) \ + fpu.o \ book3s.o \ book3s_64_emulate.o \ book3s_64_interrupts.o \
arch/powerpc/kvm/fpu.S new file +77 -0
77 lines not on this page · raw message
Signature · 1 line
1.6.0.2
PATCH 17/18 KVM: PPC: Reserve a chunk of memory for opcodes
In reply to Alexander GrafWith paired singles we have a nifty instruction execution engine. That engine takes safe and properly cleared FPU opcodes and executes them directly on the hardware.
Since we can't run off the stack and modifying .bss isn't future-proof either, the best method seemed to be to vmalloc an executable chunk of memory.
This chunk will be used by the following patch.
--- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/include/asm/kvm_ppc.h | 4 ++++ arch/powerpc/kvm/book3s.c | 14 +++++++++++++- 3 files changed, 18 insertions(+), 1 deletions(-)
arch/powerpc/include/asm/kvm_book3s.h +1 -0
@@ -144,5 +144,6 @@ static inline ulong dsisr(void)extern void kvm_return_point (void );#define INS_DCBZ 0x7c0007ec +#define INS_BLR 0x4e800020#endif /* __ASM_KVM_BOOK3S_H__ */
arch/powerpc/include/asm/kvm_ppc.h +4 -0
@@ -103,6 +103,10 @@ extern void kvmppc_booke_exit(void);extern void kvmppc_core_destroy_mmu (struct kvm_vcpu *vcpu); +/* 16*NR_CPUS bytes filled with "blr" instructions. We use this to enable +code to execute arbitrary (checked!) opcodes. */ +extern u32 *kvmppc_call_stack; +/* * Cuts out inst bits with ordering according to spec. * That means the leftmost bit is zero. All given bits are included.
arch/powerpc/kvm/book3s.c +13 -1
@@ -35,6 +35,8 @@/* #define EXIT_DEBUG_SIMPLE */ /* #define DEBUG_EXT */ +u32 *kvmppc_call_stack; +static int kvmppc_handle_ext (struct kvm_vcpu *vcpu,unsigned int exit_nr,ulong msr);
@@ -1249,7 +1251,17 @@ int __kvmppc_vcpu_run(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)static int kvmppc_book3s_init (void ) { -return kvm_init (NULL ,sizeof (struct kvmppc_vcpu_book3s ),THIS_MODULE ); +int r, i; + + r =kvm_init (NULL ,sizeof (struct kvmppc_vcpu_book3s ),THIS_MODULE ); + +/* Prepare call blob we can use to execute single instructions */ + kvmppc_call_stack =__vmalloc (NR_CPUS *2 *sizeof (u32), +GFP_KERNEL | __GFP_HIGHMEM,PAGE_KERNEL_EXEC ); +for (i =0 ; i < (NR_CPUS *2 ); i++) + kvmppc_call_stack[i] =INS_BLR ; + +return r; }static void kvmppc_book3s_exit (void )
Signature · 1 line
1.6.0.2
PATCH 18/18 KVM: PPC: Implement Paired Single emulation
In reply to Alexander GrafThe one big thing about the Gekko is paired singles.
Paired singles are an extension to the instruction set, that adds 32 single precision floating point registers (qprs), some SPRs to modify the behavior of paired singled operations and instructions to deal with qprs to the instruction set.
Unfortunately, it also changes semantics of existing operations that affect single values in FPRs. In most cases they get mirrored to the coresponding QPR.
Thanks to that we need to emulate all FPU operations and all the new paired single operations too.
In order to achieve that, we take the guest's instruction, rip out the parameters, put in our own and execute the very same instruction, but also fix up the QPR values along the way.
That way we can execute paired single FPU operations without implementing a soft fpu.
--- arch/powerpc/include/asm/kvm_book3s.h | 1 + arch/powerpc/kvm/Makefile | 1 + arch/powerpc/kvm/book3s_64_emulate.c | 3 + arch/powerpc/kvm/book3s_paired_singles.c | 1356 ++++++++++++++++++++++++++++++ 4 files changed, 1361 insertions(+), 0 deletions(-) create mode 100644 arch/powerpc/kvm/book3s_paired_singles.c
arch/powerpc/include/asm/kvm_book3s.h +1 -0
@@ -121,6 +121,7 @@ extern void kvmppc_book3s_queue_irqprio(struct kvm_vcpu *vcpu, unsigned int vec)extern void kvmppc_set_bat (struct kvm_vcpu *vcpu,struct kvmppc_bat *bat,bool upper,u32 val);extern void kvmppc_giveup_ext (struct kvm_vcpu *vcpu,ulong msr); +extern int kvmppc_emulate_paired_single (struct kvm_run *run,struct kvm_vcpu *vcpu);extern u32 kvmppc_trampoline_lowmem;extern u32 kvmppc_trampoline_enter;
arch/powerpc/kvm/Makefile +1 -0
@@ -41,6 +41,7 @@ kvm-objs-$(CONFIG_KVM_E500) := $(kvm-e500-objs) kvm-book3s_64-objs := \ $(common-objs-y) \ fpu.o \ + book3s_paired_singles.o \ book3s.o \ book3s_64_emulate.o \ book3s_64_interrupts.o \
arch/powerpc/kvm/book3s_64_emulate.c +3 -0
@@ -200,6 +200,9 @@ int kvmppc_core_emulate_op(struct kvm_run *run, struct kvm_vcpu *vcpu, emulated =EMULATE_FAIL ; } +if (emulated =EMULATE_FAIL ) + emulated =kvmppc_emulate_paired_single (run, vcpu); +return emulated; }
arch/powerpc/kvm/book3s_paired_singles.c new file +1356 -0
1,356 lines not on this page · raw message
Signature · 1 line
1.6.0.2
A message that is not in the archive (sent off-list or not yet received): 1265298925-31954-19-git-send-email-agraf-l3A5Bk7waGM@public.gmane.org
Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation
Quoted 14 lines · Alexander Graf
The one big thing about the Gekko is paired singles.
Paired singles are an extension to the instruction set, that adds 32 single precision floating point registers (qprs), some SPRs to modify the behavior of paired singled operations and instructions to deal with qprs to the instruction set.
Unfortunately, it also changes semantics of existing operations that affect single values in FPRs. In most cases they get mirrored to the coresponding QPR.
Thanks to that we need to emulate all FPU operations and all the new paired single operations too.
Quoted 8 lines
In order to achieve that, we take the guest's instruction, rip out the parameters, put in our own and execute the very same instruction, but also fix up the QPR values along the way.
That way we can execute paired single FPU operations without implementing a soft fpu.
A little frightening. How many instructions are there? Maybe we can just have an array of all of them followed by a return instruction, so we don't jit code.
Quoted 16 lines
static void call_fpu_inst(u32 inst, u64 *out, u64 *in1, u64 *in2, u64 *in3,
arch/powerpc/kvm/book3s_paired_singles.c new file +14
-0+ u32 *cr, u32 *fpscr) +{ +u32 cr_val =0 ; +u32 *call_stack; +u64 inout[5 ] = {0 ,0 ,0 ,0 ,0 }; + +if (fpscr) + inout[0 ] = *fpscr; +if (in1) + inout[1 ] = *in1; +if (in2) + inout[2 ] = *in2; +if (in3) + inout[3 ] = *in3;
arch/powerpc/kvm/book3s_paired_singles.c line 201
Quoted 10 lines
arch/powerpc/kvm/book3s_paired_singles.c new file +10
-0+if (cr) + cr_val = *cr; + +dprintk (KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x%llx )" , inst, + inout[1 ], inout[2 ], inout[3 ]); + + call_stack =&kvmppc_call_stack[(smp_processor_id () *2 )]; + call_stack[0 ] = inst; +/* call_stack[1] is INS_BLR */ +
Would be easier on the cache to do this per-cpu?
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation
Am 07.02.2010 um 13:50 schrieb Avi Kivity <[email protected]>:
Quoted 19 lines · Alexander Graf
Quoted 18 lines
The one big thing about the Gekko is paired singles.
Paired singles are an extension to the instruction set, that adds 32 single precision floating point registers (qprs), some SPRs to modify the behavior of paired singled operations and instructions to deal with qprs to the instruction set.
Unfortunately, it also changes semantics of existing operations that affect single values in FPRs. In most cases they get mirrored to the coresponding QPR.
Thanks to that we need to emulate all FPU operations and all the new paired single operations too.
In order to achieve that, we take the guest's instruction, rip out the parameters, put in our own and execute the very same instruction, but also fix up the QPR values along the way.
Quoted 6 lines
Quoted 4 lines
That way we can execute paired single FPU operations without implementing a soft fpu.
A little frightening. How many instructions are there? Maybe we can just have an array of all of them followed by a return instruction, so we don't jit code.
There's all the instructions in the list, most can have the rc (compare) bit set to modify CC and iirc there were a couple ones with immediate values.
But maybe you're right. I probably could just always set rc and either ignore the result or use it. I could maybe find alternatives to immediate using instructions. Let me check this on the bus trip back from brussels.
Quoted 22 lines
Quoted 22 lines
static void call_fpu_inst(u32 inst, u64 *out, u64 *in1, u64 *in2, u64 *in3,
arch/powerpc/kvm/book3s_paired_singles.c new file +18
-0+ u32 *cr, u32 *fpscr) +{ + u32 cr_val = 0; +u32 *call_stack; +u64 inout[5 ] = {0 ,0 ,0 ,0 ,0 }; + +if (fpscr) + inout[0 ] = *fpscr; +if (in1) + inout[1 ] = *in1; +if (in2) + inout[2 ] = *in2; +if (in3) + inout[3 ] = *in3; +if (cr) + cr_val = *cr; + + dprintk(KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x%llx )", inst,
+ inout[1], inout[2], inout[3]);
Quoted 8 lines
arch/powerpc/kvm/book3s_paired_singles.c line 201
Quoted 6 lines
arch/powerpc/kvm/book3s_paired_singles.c new file +5
-0+ + call_stack =&kvmppc_call_stack[(smp_processor_id () *2 )]; + call_stack[0 ] = inst; +/* call_stack[1] is INS_BLR */ +Would be easier on the cache to do this per-cpu?
It is per-cpu. Or do you mean to actually use the PER_CPU definition? Is that guaranteed to be executable?
Alex
Re: [PATCH 18/18] KVM: PPC: Implement Paired Single emulation
On 02/07/2010 05:57 PM, Alexander Graf wrote:+
Quoted 12 lines
Quoted 11 lines
arch/powerpc/kvm/book3s_paired_singles.c line 201
Quoted 9 lines
+ dprintk(KERN_INFO "FPU Emulator 0x%x ( 0x%llx, 0x%llx, 0x%llx
)", inst,arch/powerpc/kvm/book3s_paired_singles.c new file +6
-0+ inout[1], inout[2], inout[3]); + + call_stack =&kvmppc_call_stack[(smp_processor_id () *2 )]; + call_stack[0 ] = inst; +/* call_stack[1] is INS_BLR */ +Would be easier on the cache to do this per-cpu?
Quoted 2 lines
It is per-cpu. Or do you mean to actually use the
PER_CPUdefinition? Is that guaranteed to be executable?
I meant, per-cpu vmalloc area, but it should be enough to have a per-cpu cache line.
Signature · 1 line
error compiling committee.c: too many arguments to function