HV KVM improvements in preparation for POWER8 support

Currently we are not saving and restoring the SIAR and SDAR registers in the PMU (performance monitor unit) on guest entry and exit. The result is that performance monitoring tools in the guest could get false information about where a program was executing and what data it was accessing at the time of a performance monitor interrupt. This fixes it by saving and restoring these registers along with the other PMU registers on guest entry/exit.

This also provides a way for userspace to access these values for a vcpu via the one_reg interface.

Signed-off-by Paul Mackerras <[email protected]>

This allows guests to have a different timebase origin from the host. This is needed for migration, where a guest can migrate from one host to another and the two hosts might have a different timebase origin. However, the timebase seen by the guest must not go backwards, and should go forwards only by a small amount corresponding to the time taken for the migration.

Therefore this provides a new per-vcpu value accessed via the one_reg interface using the new KVM_REG_PPC_TB_OFFSET identifier. This value defaults to 0 and is not modified by KVM. On entering the guest, this value is added onto the timebase, and on exiting the guest, it is subtracted from the timebase.

This is only supported for recent POWER hardware which has the TBU40 (timebase upper 40 bits) register. Writing to the TBU40 register only alters the upper 40 bits of the timebase, leaving the lower 24 bits unchanged. This provides a way to modify the timebase for guest migration without disturbing the synchronization of the timebase registers across CPU cores. The kernel rounds up the value given to a multiple of 2^24.

Timebase values stored in KVM structures (struct kvm_vcpu, struct kvmppc_vcore, etc.) are stored as host timebase values. The timebase values in the dispatch trace log need to be guest timebase values, however, since that is read directly by the guest. This moves the setting of vcpu->arch.dec_expires on guest exit to a point after we have restored the host timebase so that vcpu->arch.dec_expires is a host timebase value.

Signed-off-by Paul Mackerras <[email protected]>

arch/powerpc/kvm/book3s.c +10 -0

@@ -528,6 +528,9 @@ int kvm_vcpu_ioctl_get_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 			}
 			val = get_reg_val(reg->id, vcpu->arch.vscr.u[3]);
 			break;
+		case KVM_REG_PPC_VRSAVE:
+			val = get_reg_val(reg->id, vcpu->arch.vrsave);
+			break;
 #endif /* CONFIG_ALTIVEC */
 		case KVM_REG_PPC_DEBUG_INST: {
 			u32 opcode = INS_TW;

@@ -605,6 +608,13 @@ int kvm_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu, struct kvm_one_reg *reg)
 			}
 			vcpu->arch.vscr.u[3] = set_reg_val(reg->id, val);
 			break;
+		case KVM_REG_PPC_VRSAVE:
+			if (!cpu_has_feature(CPU_FTR_ALTIVEC)) {
+				r = -ENXIO;
+				break;
+			}
+			vcpu->arch.vrsave = set_reg_val(reg->id, val);
+			break;
 #endif /* CONFIG_ALTIVEC */
 #ifdef CONFIG_KVM_XICS
 		case KVM_REG_PPC_ICP_STATE:

Signature · 1 line

1.8.4.rc3

I don't like how this is available for book3s, but not for booke. In the long run it might make sense to create a generic one_reg handler for shared fields. But in the meantime, could you please just add handling for booke as well? I'll apply the patch in the meantime, but please send another one doing this for booke as well.

This adds the ability for userspace to read and write the LPCR (Logical Partitioning Control Register) value relating to a guest via the GET/SET_ONE_REG interface. There is only one LPCR value for the guest, which can be accessed through any vcpu. Userspace can only modify the following fields of the LPCR value:

DPFD Default prefetch depth
ILE Interrupt little-endian
TC Translation control (secondary HPT hash group search disable)

Signed-off-by Paul Mackerras <[email protected]>

Quoted 10 lines · Paul Mackerras

This adds the ability for userspace to read and write the LPCR (Logical Partitioning Control Register) value relating to a guest via the GET/SET_ONE_REG interface. There is only one LPCR value for the guest, which can be accessed through any vcpu. Userspace can only modify the following fields of the LPCR value:

DPFD Default prefetch depth
ILE Interrupt little-endian
TC Translation control (secondary HPT hash group search disable)

Quoted 1 line

Signed-off-by Paul Mackerras <[email protected]>

There are 3 things I dislike about this patch :)

1) A vcpu one_reg should only change the state of the vcpu it's targeting. You want a vm wide thing.

2) If anyone gets crazy enough to implement HV emulation in PR KVM this would overlap with the guest's guest LPCR, so we need to name it differently. It's really VM configuration, not a register. Maybe ENABLE_CAP is a better fit?

3) Checkpatch fails:

WARNING: please, no space before tabs
#59: FILE: arch/powerpc/include/asm/reg.h:295:
+#define LPCR_TC ^I0x00000200^I/* Translation control */$

Quoted 15 lines · Paul Mackerras

Quoted 12 lines

This adds the ability for userspace to read and write the LPCR (Logical Partitioning Control Register) value relating to a guest via the GET/SET_ONE_REG interface. There is only one LPCR value for the guest, which can be accessed through any vcpu. Userspace can only modify the following fields of the LPCR value:

DPFD Default prefetch depth
ILE Interrupt little-endian
TC Translation control (secondary HPT hash group search disable)

Signed-off-by Paul Mackerras <[email protected]>

Quoted 3 lines

There are 3 things I dislike about this patch :)

1) A vcpu one_reg should only change the state of the vcpu it's targeting. You want a vm wide thing.

In hardware there is in fact an LPCR per hardware CPU thread, though the architecture says that on threaded processors many of the fields have to be the same on each thread, including the three fields mentioned here.

Quoted 2 lines

2) If anyone gets crazy enough to implement HV emulation in PR KVM this would overlap with the guest's guest LPCR, so we need to name it differently. It's really VM configuration, not a register. Maybe ENABLE_CAP is a better fit?

If we were doing HV emulation in PR KVM then there would still only be one LPCR per guest vcpu. If there was a hypervisor running as the PR guest, it would be its job to context switch the LPCR between its guests. So I don't see why there would be any need for the top-level KVM to know about the guest's guest LPCR.

In that situation we would need a one_reg to get and set the guest's LPCR, which would be per vcpu, and there would be no restriction on which bits could be set by the host.

It sounds to me like the best option would be to keep an LPCR per vcpu and use the one_reg interface to let the host get and set it. The actual LPCR applied when the guest runs would use some bits from the per-vcpu value plus some bits set by the KVM host code (for the host's protection). How does that sound?

Alternatively I could define a new per-VM ioctl. ENABLE_CAP doesn't really help since it also is per-vcpu, not per-VM.

Quoted 5 lines

3) Checkpatch fails:

WARNING: please, no space before tabs
#59: FILE: arch/powerpc/include/asm/reg.h:295:
+#define LPCR_TC ^I0x00000200^I/* Translation control */$

Oops, will fix.

Paul.

Am 13.09.2013 um 21:21 schrieb Paul Mackerras <[email protected]>:

Quoted 4 lines

In hardware there is in fact an LPCR per hardware CPU thread, though the architecture says that on threaded processors many of the fields have to be the same on each thread, including the three fields mentioned here.

Ok. Is it mandatory to be identical per core? Or per vm?

Quoted 4 lines

Am 13.09.2013 um 21:21 schrieb Paul Mackerras <[email protected]>:

Quoted 6 lines

Quoted 4 lines

In hardware there is in fact an LPCR per hardware CPU thread, though the architecture says that on threaded processors many of the fields have to be the same on each thread, including the three fields mentioned here.

Ok. Is it mandatory to be identical per core? Or per vm?

Per core.

Paul.

POWER7 and later IBM server processors have a register called the Program Priority Register (PPR), which controls the priority of each hardware CPU SMT thread, and affects how fast it runs compared to other SMT threads. This priority can be controlled by writing to the PPR or by use of a set of instructions of the form or rN,rN,rN which are otherwise no-ops but have been defined to set the priority to particular levels.

This adds code to context switch the PPR when entering and exiting guests and to make the PPR value accessible through the SET/GET_ONE_REG interface. When entering the guest, we set the PPR as late as possible, because if we are setting a low thread priority it will make the code run slowly from that point on. Similarly, the first-level interrupt handlers save the PPR value in the PACA very early on, and set the thread priority to the medium level, so that the interrupt handling code runs at a reasonable speed.

Signed-off-by Paul Mackerras <[email protected]>

POWER7 and later IBM server processors have a register called the Program Priority Register (PPR), which controls the priority of each hardware CPU SMT thread, and affects how fast it runs compared to other SMT threads. This priority can be controlled by writing to the PPR or by use of a set of instructions of the form or rN,rN,rN which are otherwise no-ops but have been defined to set the priority to particular levels.

This adds code to context switch the PPR when entering and exiting guests and to make the PPR value accessible through the SET/GET_ONE_REG interface. When entering the guest, we set the PPR as late as possible, because if we are setting a low thread priority it will make the code run slowly from that point on. Similarly, the first-level interrupt handlers save the PPR value in the PACA very early on, and set the thread priority to the medium level, so that the interrupt handling code runs at a reasonable speed.

Signed-off-by Paul Mackerras <[email protected]>

---
 Documentation/virtual/kvm/api.txt         |  1 +
 arch/powerpc/include/asm/exception-64s.h  |  8 ++++++++
 arch/powerpc/include/asm/kvm_book3s_asm.h |  1 +
 arch/powerpc/include/asm/kvm_host.h       |  1 +
 arch/powerpc/include/uapi/asm/kvm.h       |  1 +
 arch/powerpc/kernel/asm-offsets.c         |  2 ++
 arch/powerpc/kvm/book3s_hv.c              |  6 ++++++
 arch/powerpc/kvm/book3s_hv_rmhandlers.S   | 12 +++++++++++-
 8 files changed, 31 insertions(+), 1 deletion(-)

Documentation/virtual/kvm/api.txt +1 -0

index 1030ac9..34a32b6 100644
@@ -1836,6 +1836,7 @@ registers, find a list below:
   PPC   | KVM_REG_PPC_ACOP	| 64
   PPC   | KVM_REG_PPC_VRSAVE	| 32
   PPC   | KVM_REG_PPC_LPCR	| 64
+  PPC   | KVM_REG_PPC_PPR	| 64
   PPC   | KVM_REG_PPC_TM_GPR0	| 64
...
   PPC   | KVM_REG_PPC_TM_GPR31	| 64

arch/powerpc/include/asm/exception-64s.h +8 -0

index 07ca627..b86c4db 100644
@@ -203,6 +203,10 @@ do_kvm_##n:								\
 	ld	r10,area+EX_CFAR(r13);					\
 	std	r10,HSTATE_CFAR(r13);					\
 	END_FTR_SECTION_NESTED(CPU_FTR_CFAR,CPU_FTR_CFAR,947);		\
+	BEGIN_FTR_SECTION_NESTED(948)					\
+	ld	r10,area+EX_PPR(r13);					\
+	std	r10,HSTATE_PPR(r13);					\
+	END_FTR_SECTION_NESTED(CPU_FTR_HAS_PPR,CPU_FTR_HAS_PPR,948);	\
 	ld	r10,area+EX_R10(r13);					\
 	stw	r9,HSTATE_SCRATCH1(r13);				\
 	ld	r9,area+EX_R9(r13);					\

@@ -216,6 +220,10 @@ do_kvm_##n:								\
 	ld	r10,area+EX_R10(r13);					\
 	beq	89f;							\
 	stw	r9,HSTATE_SCRATCH1(r13);			\
+	BEGIN_FTR_SECTION_NESTED(948)					\
+	ld	r9,area+EX_PPR(r13);					\
+	std	r9,HSTATE_PPR(r13);					\
+	END_FTR_SECTION_NESTED(CPU_FTR_HAS_PPR,CPU_FTR_HAS_PPR,948);	\
 	ld	r9,area+EX_R9(r13);					\
 	std	r12,HSTATE_SCRATCH0(r13);			\
 	li	r12,n;							\

arch/powerpc/include/asm/kvm_book3s_asm.h +1 -0

index 9039d3c..22f4606 100644
@@ -101,6 +101,7 @@ struct kvmppc_host_state {
 #endif
 #ifdef CONFIG_PPC_BOOK3S_64
 	u64 cfar;
+	u64 ppr;
 #endif
 };
 

arch/powerpc/include/asm/kvm_host.h +1 -0

index 9741bf0..b0dcd18 100644
@@ -464,6 +464,7 @@ struct kvm_vcpu_arch {
 	u32 ctrl;
 	ulong dabr;
 	ulong cfar;
+	ulong ppr;
 #endif
 	u32 vrsave; /* also USPRG0 */
 	u32 mmucr;

arch/powerpc/include/uapi/asm/kvm.h +1 -0

index e42127d..fab6bc1 100644
@@ -534,6 +534,7 @@ struct kvm_get_htab_header {
 
 #define KVM_REG_PPC_VRSAVE	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xb4)
 #define KVM_REG_PPC_LPCR	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xb5)
+#define KVM_REG_PPC_PPR		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xb6)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs

arch/powerpc/kernel/asm-offsets.c +2 -0

index ccb42cd..5c6ea96 100644
@@ -516,6 +516,7 @@ int main(void)
 	DEFINE(VCPU_TRAP, offsetof(struct kvm_vcpu, arch.trap));
 	DEFINE(VCPU_PTID, offsetof(struct kvm_vcpu, arch.ptid));
 	DEFINE(VCPU_CFAR, offsetof(struct kvm_vcpu, arch.cfar));
+	DEFINE(VCPU_PPR, offsetof(struct kvm_vcpu, arch.ppr));
 	DEFINE(VCORE_ENTRY_EXIT, offsetof(struct kvmppc_vcore, entry_exit_count));
 	DEFINE(VCORE_NAP_COUNT, offsetof(struct kvmppc_vcore, nap_count));
 	DEFINE(VCORE_IN_GUEST, offsetof(struct kvmppc_vcore, in_guest));

@@ -600,6 +601,7 @@ int main(void)
 
 #ifdef CONFIG_PPC_BOOK3S_64
 	HSTATE_FIELD(HSTATE_CFAR, cfar);
+	HSTATE_FIELD(HSTATE_PPR, ppr);
 #endif /* CONFIG_PPC_BOOK3S_64 */
 
 #else /* CONFIG_PPC_BOOK3S */

arch/powerpc/kvm/book3s_hv.c +6 -0

index 9c878d7..eceff7e 100644
@@ -814,6 +814,9 @@ int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	case KVM_REG_PPC_LPCR:
 		*val = get_reg_val(id, vcpu->kvm->arch.lpcr);
 		break;
+	case KVM_REG_PPC_PPR:
+		*val = get_reg_val(id, vcpu->arch.ppr);
+		break;
 	default:
 		r = -EINVAL;
 		break;

@@ -921,6 +924,9 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	case KVM_REG_PPC_LPCR:
 		kvmppc_set_lpcr(vcpu, set_reg_val(id, *val));
 		break;
+	case KVM_REG_PPC_PPR:
+		vcpu->arch.ppr = set_reg_val(id, *val);
+		break;
 	default:
 		r = -EINVAL;
 		break;

arch/powerpc/kvm/book3s_hv_rmhandlers.S +11 -1

index 85f8dd0..88e7068 100644
@@ -561,13 +561,15 @@ BEGIN_FTR_SECTION
 	ld	r5, VCPU_CFAR(r4)
 	mtspr	SPRN_CFAR, r5
 END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
+BEGIN_FTR_SECTION
+	ld	r0, VCPU_PPR(r4)
+END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
 
 	ld	r5, VCPU_LR(r4)
 	lwz	r6, VCPU_CR(r4)
 	mtlr	r5
 	mtcr	r6
 
-	ld	r0, VCPU_GPR(R0)(r4)
 	ld	r1, VCPU_GPR(R1)(r4)
 	ld	r2, VCPU_GPR(R2)(r4)
 	ld	r3, VCPU_GPR(R3)(r4)

@@ -581,6 +583,10 @@ END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
 	ld	r12, VCPU_GPR(R12)(r4)
 	ld	r13, VCPU_GPR(R13)(r4)
 
+BEGIN_FTR_SECTION
+	mtspr	SPRN_PPR, r0
+END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
+	ld	r0, VCPU_GPR(R0)(r4)
 	ld	r4, VCPU_GPR(R4)(r4)
 
 	hrfid

@@ -631,6 +637,10 @@ BEGIN_FTR_SECTION
 	ld	r3, HSTATE_CFAR(r13)
 	std	r3, VCPU_CFAR(r9)
 END_FTR_SECTION_IFSET(CPU_FTR_CFAR)
+BEGIN_FTR_SECTION
+	ld	r4, HSTATE_PPR(r13)
+	std	r4, VCPU_PPR(r9)
+END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR)
 
 	/* Restore R1/R2 so we can handle faults */
 	ld	r1, HSTATE_HOST_R1(r13)

Quoted 17 lines · Paul Mackerras

POWER7 and later IBM server processors have a register called the Program Priority Register (PPR), which controls the priority of each hardware CPU SMT thread, and affects how fast it runs compared to other SMT threads. This priority can be controlled by writing to the PPR or by use of a set of instructions of the form or rN,rN,rN which are otherwise no-ops but have been defined to set the priority to particular levels.

This adds code to context switch the PPR when entering and exiting guests and to make the PPR value accessible through the SET/GET_ONE_REG interface. When entering the guest, we set the PPR as late as possible, because if we are setting a low thread priority it will make the code run slowly from that point on. Similarly, the first-level interrupt handlers save the PPR value in the PACA very early on, and set the thread priority to the medium level, so that the interrupt handling code runs at a reasonable speed.

Quoted 1 line

Signed-off-by Paul Mackerras <[email protected]>

Acked-by Benjamin Herrenschmidt <[email protected]>

Alex, can you take this via your tree ?

This enables us to use the Processor Compatibility Register (PCR) on POWER7 to put the processor into architecture 2.05 compatibility mode when running a guest. In this mode the new instructions and registers that were introduced on POWER7 are disabled in user mode. This includes all the VSX facilities plus several other instructions such as ldbrx, stdbrx, popcntw, popcntd, etc.

To select this mode, we have a new register accessible through the set/get_one_reg interface, called KVM_REG_PPC_ARCH_COMPAT. Setting this to zero gives the full set of capabilities of the processor. Setting it to one of the "logical" PVR values defined in PAPR puts the vcpu into the compatibility mode for the corresponding architecture level. The supported values are:

0x0f000002 Architecture 2.05 (POWER6)
0x0f000003 Architecture 2.06 (POWER7)
0x0f100003 Architecture 2.06+ (POWER7+)

Since the PCR is per-core, the architecture compatibility level and the corresponding PCR value are stored in the struct kvmppc_vcore, and are therefore shared between all vcpus in a virtual core.

Quoted 18 lines · Paul Mackerras

This enables us to use the Processor Compatibility Register (PCR) on POWER7 to put the processor into architecture 2.05 compatibility mode when running a guest. In this mode the new instructions and registers that were introduced on POWER7 are disabled in user mode. This includes all the VSX facilities plus several other instructions such as ldbrx, stdbrx, popcntw, popcntd, etc.

To select this mode, we have a new register accessible through the set/get_one_reg interface, called KVM_REG_PPC_ARCH_COMPAT. Setting this to zero gives the full set of capabilities of the processor. Setting it to one of the "logical" PVR values defined in PAPR puts the vcpu into the compatibility mode for the corresponding architecture level. The supported values are:

0x0f000002 Architecture 2.05 (POWER6)
0x0f000003 Architecture 2.06 (POWER7)
0x0f100003 Architecture 2.06+ (POWER7+)

Quoted 3 lines

Since the PCR is per-core, the architecture compatibility level and the corresponding PCR value are stored in the struct kvmppc_vcore, and are therefore shared between all vcpus in a virtual core.

We already have KVM_SET_SREGS taking pvr as argument. Can't we do this kvmppc_set_pvr ?. Can you also share the qemu changes ? There I guess we need to do update the "cpu-version" in the device tree so that /proc/cpuinfo shows the right information in the guest

This enables us to use the Processor Compatibility Register (PCR) on POWER7 to put the processor into architecture 2.05 compatibility mode when running a guest. In this mode the new instructions and registers that were introduced on POWER7 are disabled in user mode. This includes all the VSX facilities plus several other instructions such as ldbrx, stdbrx, popcntw, popcntd, etc.

To select this mode, we have a new register accessible through the set/get_one_reg interface, called KVM_REG_PPC_ARCH_COMPAT. Setting this to zero gives the full set of capabilities of the processor. Setting it to one of the "logical" PVR values defined in PAPR puts the vcpu into the compatibility mode for the corresponding architecture level. The supported values are:

0x0f000002 Architecture 2.05 (POWER6)
0x0f000003 Architecture 2.06 (POWER7)
0x0f100003 Architecture 2.06+ (POWER7+)

Since the PCR is per-core, the architecture compatibility level and the corresponding PCR value are stored in the struct kvmppc_vcore, and are therefore shared between all vcpus in a virtual core.

Signed-off-by Paul Mackerras <[email protected]>

---

Documentation/virtual/kvm/api.txt       |  1 +
arch/powerpc/include/asm/kvm_host.h     |  2 ++
arch/powerpc/include/asm/reg.h          | 11 +++++++++++
arch/powerpc/include/uapi/asm/kvm.h     |  3 +++
arch/powerpc/kernel/asm-offsets.c       |  1 +
arch/powerpc/kvm/book3s_hv.c            | 35 +++++++++++++++++++++++++++++++++
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 11 +++++++++--
7 files changed, 62 insertions(+), 2 deletions(-)

Documentation/virtual/kvm/api.txt +1 -0

index 34a32b6..f1f300f 100644
@@ -1837,6 +1837,7 @@ registers, find a list below:
  PPC   | KVM_REG_PPC_VRSAVE	| 32
  PPC   | KVM_REG_PPC_LPCR	| 64
  PPC   | KVM_REG_PPC_PPR	| 64
+  PPC   | KVM_REG_PPC_ARCH_COMPAT | 32
  PPC   | KVM_REG_PPC_TM_GPR0	| 64
...
  PPC   | KVM_REG_PPC_TM_GPR31	| 64

arch/powerpc/include/asm/kvm_host.h +2 -0

index b0dcd18..5a40270 100644
@@ -295,6 +295,8 @@ struct kvmppc_vcore {
 	u64 preempt_tb;
 	struct kvm_vcpu *runner;
 	u64 tb_offset;		/* guest timebase - host timebase */
+	u32 arch_compat;
+	ulong pcr;
 };
 
 #define VCORE_ENTRY_COUNT(vc)	((vc)->entry_exit_count & 0xff)

arch/powerpc/include/asm/reg.h +11 -0

index 3fc0d06..52ff962 100644
@@ -305,6 +305,10 @@
 #define   LPID_RSVD	0x3ff		/* Reserved LPID for partn switching */
 #define	SPRN_HMER	0x150	/* Hardware m? error recovery */
 #define	SPRN_HMEER	0x151	/* Hardware m? enable error recovery */
+#define SPRN_PCR	0x152	/* Processor compatibility register */
+#define   PCR_VEC_DIS	(1ul << (63-0))	/* Vec. disable (pre POWER8) */
+#define   PCR_VSX_DIS	(1ul << (63-1))	/* VSX disable (pre POWER8) */
+#define   PCR_ARCH_205	0x2		/* Architecture 2.05 */
 #define	SPRN_HEIR	0x153	/* Hypervisor Emulated Instruction Register */
 #define SPRN_TLBINDEXR	0x154	/* P7 TLB control register */
 #define SPRN_TLBVPNR	0x155	/* P7 TLB control register */

@@ -1095,6 +1099,13 @@
 #define PVR_BE		0x0070
 #define PVR_PA6T	0x0090
 
+/* "Logical" PVR values defined in PAPR, representing architecture levels */
+#define PVR_ARCH_204	0x0f000001
+#define PVR_ARCH_205	0x0f000002
+#define PVR_ARCH_206	0x0f000003
+#define PVR_ARCH_206p	0x0f100003
+#define PVR_ARCH_207	0x0f000004
+
 /* Macros for setting and retrieving special purpose registers */
 #ifndef __ASSEMBLY__
 #define mfmsr()		({unsigned long rval; \

arch/powerpc/include/uapi/asm/kvm.h +3 -0

index fab6bc1..e420d46 100644
@@ -536,6 +536,9 @@ struct kvm_get_htab_header {
 #define KVM_REG_PPC_LPCR	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xb5)
 #define KVM_REG_PPC_PPR		(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xb6)
 
+/* Architecture compatibility level */
+#define KVM_REG_PPC_ARCH_COMPAT	(KVM_REG_PPC | KVM_REG_SIZE_U32 | 0xb7)
+
 /* Transactional Memory checkpointed state:
 * This is all GPRs, all VSX regs and a subset of SPRs
 */

arch/powerpc/kernel/asm-offsets.c +1 -0

index 5c6ea96..115dd64 100644
@@ -522,6 +522,7 @@ int main(void)
 	DEFINE(VCORE_IN_GUEST, offsetof(struct kvmppc_vcore, in_guest));
 	DEFINE(VCORE_NAPPING_THREADS, offsetof(struct kvmppc_vcore, napping_threads));
 	DEFINE(VCORE_TB_OFFSET, offsetof(struct kvmppc_vcore, tb_offset));
+	DEFINE(VCORE_PCR, offsetof(struct kvmppc_vcore, pcr));
 	DEFINE(VCPU_SVCPU, offsetof(struct kvmppc_vcpu_book3s, shadow_vcpu) -
 			   offsetof(struct kvmppc_vcpu_book3s, vcpu));
 	DEFINE(VCPU_SLB_E, offsetof(struct kvmppc_slb, orige));

arch/powerpc/kvm/book3s_hv.c +35 -0

index eceff7e..1a10afa 100644
@@ -166,6 +166,35 @@ void kvmppc_set_pvr(struct kvm_vcpu *vcpu, u32 pvr)
 	vcpu->arch.pvr = pvr;
 }
 
+int kvmppc_set_arch_compat(struct kvm_vcpu *vcpu, u32 arch_compat)
+{
+	unsigned long pcr = 0;
+	struct kvmppc_vcore *vc = vcpu->arch.vcore;
+
+	if (arch_compat) {
+		if (!cpu_has_feature(CPU_FTR_ARCH_206))
+			return -EINVAL;	/* 970 has no compat mode support */
+
+		switch (arch_compat) {
+		case PVR_ARCH_205:
+			pcr = PCR_ARCH_205;
+			break;
+		case PVR_ARCH_206:
+		case PVR_ARCH_206p:
+			break;
+		default:
+			return -EINVAL;
+		}
+	}
+
+	spin_lock(&vc->lock);
+	vc->arch_compat = arch_compat;
+	vc->pcr = pcr;
+	spin_unlock(&vc->lock);
+
+	return 0;
+}
+
 void kvmppc_dump_regs(struct kvm_vcpu *vcpu)
 {
 	int r;

@@ -817,6 +846,9 @@ int kvmppc_get_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	case KVM_REG_PPC_PPR:
 		*val = get_reg_val(id, vcpu->arch.ppr);
 		break;
+	case KVM_REG_PPC_ARCH_COMPAT:
+		*val = get_reg_val(id, vcpu->arch.vcore->arch_compat);
+		break;
 	default:
 		r = -EINVAL;
 		break;

@@ -927,6 +959,9 @@ int kvmppc_set_one_reg(struct kvm_vcpu *vcpu, u64 id, union kvmppc_one_reg *val)
 	case KVM_REG_PPC_PPR:
 		vcpu->arch.ppr = set_reg_val(id, *val);
 		break;
+	case KVM_REG_PPC_ARCH_COMPAT:
+		r = kvmppc_set_arch_compat(vcpu, set_reg_val(id, *val));
+		break;
 	default:
 		r = -EINVAL;
 		break;

arch/powerpc/kvm/book3s_hv_rmhandlers.S +6 -2

index 88e7068..023d8600 100644
@@ -358,7 +358,11 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_201)
 	addis	r8,r8,0x100	/* if so, increment upper 40 bits */
 	mtspr	SPRN_TBU40,r8
 
-37:	li	r0,1
+	/* Load guest PCR value to select appropriate compat mode */
+37:	ld	r7, VCORE_PCR(r5)
+	mtspr	SPRN_PCR, r7
+
+	li	r0,1
 	stb	r0,VCORE_IN_GUEST(r5)	/* signal secondaries to continue */
 	b	10f
 

@@ -984,8 +988,11 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_201)
 	addis	r8,r8,0x100		/* if so, increment upper 40 bits */
 	mtspr	SPRN_TBU40,r8
 
-	/* Signal secondary CPUs to continue */
+	/* Reset PCR */

17: li r0,0

How long does writing to PCR take? Is it faster than a load+branch to see whether we actually need it? I would assume the normal fast path is going to be guest cpu == host.

The H_CONFER hypercall is used when a guest vcpu is spinning on a lock held by another vcpu which has been preempted, and the spinning vcpu wishes to give its timeslice to the lock holder. We implement this in the straightforward way using kvm_vcpu_yield_to().

Signed-off-by Paul Mackerras <[email protected]>

We have two paths into and out of the low-level guest entry and exit
code: from a vcpu task via kvmppc_hv_entry_trampoline, and from the system reset vector for an offline secondary thread on POWER7 via kvm_start_guest. Currently both just branch to kvmppc_hv_entry to enter the guest, and on guest exit, we test the vcpu physical thread ID to detect which way we came in and thus whether we should return to the vcpu task or go back to nap mode.

In order to make the code flow clearer, and to keep the code relating to each flow together, this turns kvmppc_hv_entry into a subroutine that follows the normal conventions for call and return. This means that kvmppc_hv_entry_trampoline() and kvmppc_hv_entry() now establish normal stack frames, and we use the normal stack slots for saving return addresses rather than local_paca->kvm_hstate.vmhandler. Apart from that this is mostly moving code around unchanged.

Signed-off-by Paul Mackerras <[email protected]>

This moves the code in book3s_hv_rmhandlers.S that reads any pending interrupt from the XICS interrupt controller, and works out whether it is an IPI for the guest, an IPI for the host, or a device interrupt, into a new function called kvmppc_read_intr. Later patches will need this.

Signed-off-by Paul Mackerras <[email protected]>

The yield count in the VPA is supposed to be incremented every time we enter the guest, and every time we exit the guest, so that its value is even when the vcpu is running in the guest and odd when it isn't. However, it's currently possible that we increment the yield count on the way into the guest but then find that other CPU threads are already exiting the guest, so we go back to nap mode via the secondary_too_late label. In this situation we don't increment the yield count again, breaking the relationship between the LSB of the count and whether the vcpu is in the guest.

To fix this, we move the increment of the yield count to a point after we have checked whether other CPU threads are exiting.

Signed-off-by Paul Mackerras <[email protected]>

From: Michael Ellerman <[email protected]>

This means that if we do happen to get a trap that we don't know about, we abort the guest rather than crashing the host kernel.

Signed-off-by Michael Ellerman <[email protected]>

Signed-off-by Paul Mackerras <[email protected]>

---
arch/powerpc/kvm/book3s_hv.c | 3 +--

1 file changed, 1 insertion(+), 2 deletions(-)

arch/powerpc/kvm/book3s_hv.c +0 -0

index 0bb23a9..731e46e 100644

This series of patches is based on Alex Graf's kvm-ppc-queue branch. It fixes some bugs, makes some more registers accessible through the one_reg interface, and implements some missing features such as support for the compatibility modes in recent POWER cpus and support for the guest having a different timebase origin from the host. These patches are all useful on POWER7 and will be needed for good POWER8 support.

Please apply.