Skip to contentSkip to conversation

KVM: PPC: Book3S HV: Nested guest migration fixes

PATCH In review Shivaprasad G Bhat 3 participants 16 messages Jun 3, 2024 – Jun 5, 2024

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover KVM: PPC: Book3S HV: Nested guest migration fixes

The series fixes the issues exposed by the kvm-unit-tests[1] sprs-migration test.

The SDAR, MMCR3 were seen to have some typo/refactoring bugs. The first two patches fix them.

Though the nestedv2 APIs defined the guest state elements for Power ISA 3.1B SPRs to save-restore with PHYP during entry-exit, the DEXCR and HASHKEYR were ignored in code. The KVM_PPC_REG too for them are missing without which the Qemu is not setting them to their 'previous' value during guest migration at destination. The remaining patches take care of this.

References:
[1]: https://github.com/kvm-unit-tests/kvm-unit-tests

---

Shivaprasad G Bhat (6):
KVM: PPC: Book3S HV: Fix the set_one_reg for MMCR3
KVM: PPC: Book3S HV: Fix the get_one_reg of SDAR
KVM: PPC: Book3S HV nestedv2: Keep nested guest DEXCR in sync
KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register
KVM: PPC: Book3S HV nestedv2: Keep nested guest HASHKEYR in sync
KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

 Documentation/virt/kvm/api.rst            |  2 ++
 arch/powerpc/include/asm/kvm_host.h       |  2 ++
 arch/powerpc/include/uapi/asm/kvm.h       |  2 ++
 arch/powerpc/kvm/book3s_hv.c              | 16 ++++++++++++++--
 arch/powerpc/kvm/book3s_hv.h              |  2 ++
 arch/powerpc/kvm/book3s_hv_nestedv2.c     | 12 ++++++++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |  2 ++
 7 files changed, 36 insertions(+), 2 deletions(-)

--
Signature

#1 lore raw

PATCH 1/6 KVM: PPC: Book3S HV: Fix the set_one_reg for MMCR3

The kvmppc_set_one_reg_hv() wrongly get() the value instead of set() for MMCR3. Fix the same.

Fixes 5752fe0b811b ("KVM: PPC: Book3S HV: Save/restore new PMU registers")
---
 arch/powerpc/kvm/book3s_hv.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +1 -1
index daaf7faf21a5..a4f34f94c86f 100644
@@ -2540,7 +2540,7 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 		vcpu->arch.mmcrs = set_reg_val(id, *val);
 		break;
 	case KVM_REG_PPC_MMCR3:
-		*val = get_reg_val(id, vcpu->arch.mmcr[3]);
+		kvmppc_set_mmcr_hv(vcpu, 3, set_reg_val(id, *val));
 		break;
 	case KVM_REG_PPC_PMC1 ... KVM_REG_PPC_PMC8:
 		i = id - KVM_REG_PPC_PMC1;
#2 lore raw

PATCH 2/6 KVM: PPC: Book3S HV: Fix the get_one_reg of SDAR

In reply to Shivaprasad G Bhat

The kvmppc_get_one_reg_hv() for SDAR is wrongly getting the SIAR instead of SDAR, possibly a paste error emanating from the previous refactoring.

Patch fixes the wrong get_one_reg() for the same.

Fixes ebc88ea7a6ad ("KVM: PPC: Book3S HV: Use accessors for VCPU registers")
---
 arch/powerpc/kvm/book3s_hv.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/kvm/book3s_hv.c +1 -1
index a4f34f94c86f..b576781d58d5 100644
@@ -2305,7 +2305,7 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 		*val = get_reg_val(id, kvmppc_get_siar_hv(vcpu));
 		break;
 	case KVM_REG_PPC_SDAR:
-		*val = get_reg_val(id, kvmppc_get_siar_hv(vcpu));
+		*val = get_reg_val(id, kvmppc_get_sdar_hv(vcpu));
 		break;
 	case KVM_REG_PPC_SIER:
 		*val = get_reg_val(id, kvmppc_get_sier_hv(vcpu, 0));
#3 lore raw

PATCH 3/6 KVM: PPC: Book3S HV nestedv2: Keep nested guest DEXCR in sync

In reply to Shivaprasad G Bhat

The nestedv2 APIs has the guest state element defined for DEXCR for the save-restore with L0. However, its ignored in the code.

The patch takes care of this for the DEXCR GSID.

---
 arch/powerpc/include/asm/kvm_host.h   |    1 +
 arch/powerpc/kvm/book3s_hv.h          |    1 +
 arch/powerpc/kvm/book3s_hv_nestedv2.c |    6 ++++++
 3 files changed, 8 insertions(+)
arch/powerpc/include/asm/kvm_host.h +1 -0
index 8abac532146e..1e2fdcbecffd 100644
@@ -599,6 +599,7 @@ struct kvm_vcpu_arch {
 	ulong dawrx0;
 	ulong dawr1;
 	ulong dawrx1;
+	ulong dexcr;
 	ulong ciabr;
 	ulong cfar;
 	ulong ppr;
arch/powerpc/kvm/book3s_hv.h +1 -0
index 47b2c815641e..7b0fd282fe95 100644
@@ -116,6 +116,7 @@ KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr0, 64, KVMPPC_GSID_DAWR0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr1, 64, KVMPPC_GSID_DAWR1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx0, 64, KVMPPC_GSID_DAWRX0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx1, 64, KVMPPC_GSID_DAWRX1)
+KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dexcr, 64, KVMPPC_GSID_DEXCR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ciabr, 64, KVMPPC_GSID_CIABR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(wort, 64, KVMPPC_GSID_WORT)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ppr, 64, KVMPPC_GSID_PPR)
arch/powerpc/kvm/book3s_hv_nestedv2.c +6 -0
index 1091f7a83b25..d207a6d936ff 100644
@@ -193,6 +193,9 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb,
 		case KVMPPC_GSID_DAWRX1:
 			rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.dawrx1);
 			break;
+		case KVMPPC_GSID_DEXCR:
+			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dexcr);
+			break;
 		case KVMPPC_GSID_CIABR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ciabr);
 			break;
@@ -441,6 +444,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,
 		case KVMPPC_GSID_DAWRX1:
 			vcpu->arch.dawrx1 = kvmppc_gse_get_u32(gse);
 			break;
+		case KVMPPC_GSID_DEXCR:
+			vcpu->arch.dexcr = kvmppc_gse_get_u64(gse);
+			break;
 		case KVMPPC_GSID_CIABR:
 			vcpu->arch.ciabr = kvmppc_gse_get_u64(gse);
 			break;
reviewer Reviewed-by on v1 3/6
#4 lore raw

Re: [PATCH 3/6] KVM: PPC: Book3S HV nestedv2: Keep nested guest DEXCR in sync

Quoted 17 lines · Shivaprasad G Bhat

The nestedv2 APIs has the guest state element defined for DEXCR for the save-restore with L0. However, its ignored in the code.

The patch takes care of this for the DEXCR GSID.

---
 arch/powerpc/include/asm/kvm_host.h   |    1 +
 arch/powerpc/kvm/book3s_hv.h          |    1 +
 arch/powerpc/kvm/book3s_hv_nestedv2.c |    6 ++++++
 3 files changed, 8 insertions(+)
arch/powerpc/include/asm/kvm_host.h +0 -0
index 8abac532146e..1e2fdcbecffd 100644

arch/powerpc/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 605

Quoted 7 lines
arch/powerpc/include/asm/kvm_host.h +1 -0
@@ -599,6 +599,7 @@ struct kvm_vcpu_arch {
 	ulong dawrx0;
 	ulong dawr1;
 	ulong dawrx1;
+	ulong dexcr;
 	ulong ciabr;
 	ulong cfar;
 	ulong ppr;

Actually I would reorder the patches so you introduce the KVM reg first, and put this hunk there.

The nested v2 bits look okay. For them,

Quoted 36 lines
arch/powerpc/kvm/book3s_hv.h +1 -0
index 47b2c815641e..7b0fd282fe95 100644
@@ -116,6 +116,7 @@ KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr0, 64, KVMPPC_GSID_DAWR0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr1, 64, KVMPPC_GSID_DAWR1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx0, 64, KVMPPC_GSID_DAWRX0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx1, 64, KVMPPC_GSID_DAWRX1)
+KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dexcr, 64, KVMPPC_GSID_DEXCR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ciabr, 64, KVMPPC_GSID_CIABR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(wort, 64, KVMPPC_GSID_WORT)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ppr, 64, KVMPPC_GSID_PPR)
arch/powerpc/kvm/book3s_hv_nestedv2.c +6 -0
index 1091f7a83b25..d207a6d936ff 100644
@@ -193,6 +193,9 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb,
 		case KVMPPC_GSID_DAWRX1:
 			rc = kvmppc_gse_put_u32(gsb, iden, vcpu->arch.dawrx1);
 			break;
+		case KVMPPC_GSID_DEXCR:
+			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dexcr);
+			break;
 		case KVMPPC_GSID_CIABR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ciabr);
 			break;
@@ -441,6 +444,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,
 		case KVMPPC_GSID_DAWRX1:
 			vcpu->arch.dawrx1 = kvmppc_gse_get_u32(gse);
 			break;
+		case KVMPPC_GSID_DEXCR:
+			vcpu->arch.dexcr = kvmppc_gse_get_u64(gse);
+			break;
 		case KVMPPC_GSID_CIABR:
 			vcpu->arch.ciabr = kvmppc_gse_get_u64(gse);
 			break;
#5 lore raw

PATCH 4/6 KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register

In reply to Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the DEXCR for the guest during enter/exit with KVM_REG_PPC_DEXCR. The specific SPR KVM API documentation too updated.

---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
 4 files changed, 9 insertions(+)
Documentation/virt/kvm/api.rst +1 -0
index a71d91978d9e..81077c654281 100644
@@ -2441,6 +2441,7 @@ registers, find a list below:
   PPC     KVM_REG_PPC_PTCR                64
   PPC     KVM_REG_PPC_DAWR1               64
   PPC     KVM_REG_PPC_DAWRX1              64
+  PPC     KVM_REG_PPC_DEXCR               64
   PPC     KVM_REG_PPC_TM_GPR0             64
   ...
   PPC     KVM_REG_PPC_TM_GPR31            64
arch/powerpc/include/uapi/asm/kvm.h +1 -0
index 1691297a766a..fcb947f65667 100644
@@ -645,6 +645,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_SIER3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
+#define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
arch/powerpc/kvm/book3s_hv.c +6 -0
index b576781d58d5..1294c6839d37 100644
@@ -2349,6 +2349,9 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DAWRX1:
 		*val = get_reg_val(id, kvmppc_get_dawrx1_hv(vcpu));
 		break;
+	case KVM_REG_PPC_DEXCR:
+		*val = get_reg_val(id, kvmppc_get_dexcr_hv(vcpu));
+		break;
 	case KVM_REG_PPC_CIABR:
 		*val = get_reg_val(id, kvmppc_get_ciabr_hv(vcpu));
 		break;
@@ -2592,6 +2595,9 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DAWRX1:
 		kvmppc_set_dawrx1_hv(vcpu, set_reg_val(id, *val) & ~DAWRX_HYP);
 		break;
+	case KVM_REG_PPC_DEXCR:
+		kvmppc_set_dexcr_hv(vcpu, set_reg_val(id, *val));
+		break;
 	case KVM_REG_PPC_CIABR:
 		kvmppc_set_ciabr_hv(vcpu, set_reg_val(id, *val));
 		/* Don't allow setting breakpoints in hypervisor code */
tools/arch/powerpc/include/uapi/asm/kvm.h +1 -0
index 1691297a766a..fcb947f65667 100644
@@ -645,6 +645,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_SIER3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
+#define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
reviewer Reviewed-by on v1 4/6
#6 lore raw

Re: [PATCH 4/6] KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register

Quoted 4 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the DEXCR for the guest during enter/exit with KVM_REG_PPC_DEXCR. The specific SPR KVM API documentation too updated.

I wonder if the uapi and documentation parts should go in their own patch in a ppc kvm uapi topic branch? Otherwise looks okay.

Quoted 69 lines
---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
 4 files changed, 9 insertions(+)
Documentation/virt/kvm/api.rst +1 -0
index a71d91978d9e..81077c654281 100644
@@ -2441,6 +2441,7 @@ registers, find a list below:
   PPC     KVM_REG_PPC_PTCR                64
   PPC     KVM_REG_PPC_DAWR1               64
   PPC     KVM_REG_PPC_DAWRX1              64
+  PPC     KVM_REG_PPC_DEXCR               64
   PPC     KVM_REG_PPC_TM_GPR0             64
...
   PPC     KVM_REG_PPC_TM_GPR31            64
arch/powerpc/include/uapi/asm/kvm.h +1 -0
index 1691297a766a..fcb947f65667 100644
@@ -645,6 +645,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_SIER3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
+#define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
arch/powerpc/kvm/book3s_hv.c +6 -0
index b576781d58d5..1294c6839d37 100644
@@ -2349,6 +2349,9 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DAWRX1:
 		*val = get_reg_val(id, kvmppc_get_dawrx1_hv(vcpu));
 		break;
+	case KVM_REG_PPC_DEXCR:
+		*val = get_reg_val(id, kvmppc_get_dexcr_hv(vcpu));
+		break;
 	case KVM_REG_PPC_CIABR:
 		*val = get_reg_val(id, kvmppc_get_ciabr_hv(vcpu));
 		break;
@@ -2592,6 +2595,9 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DAWRX1:
 		kvmppc_set_dawrx1_hv(vcpu, set_reg_val(id, *val) & ~DAWRX_HYP);
 		break;
+	case KVM_REG_PPC_DEXCR:
+		kvmppc_set_dexcr_hv(vcpu, set_reg_val(id, *val));
+		break;
 	case KVM_REG_PPC_CIABR:
 		kvmppc_set_ciabr_hv(vcpu, set_reg_val(id, *val));
 		/* Don't allow setting breakpoints in hypervisor code */
tools/arch/powerpc/include/uapi/asm/kvm.h +1 -0
index 1691297a766a..fcb947f65667 100644
@@ -645,6 +645,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_SIER3	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc3)
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
+#define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
#7 lore raw

Re: [PATCH 4/6] KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register

Quoted 8 lines · Nicholas Piggin
Quoted 4 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the DEXCR for the guest during enter/exit with KVM_REG_PPC_DEXCR. The specific SPR KVM API documentation too updated.

I wonder if the uapi and documentation parts should go in their own patch in a ppc kvm uapi topic branch?

I'll put the whole series in the topic/ppc-kvm branch, which I think is probably sufficient.

cheers

#8 lore raw

Re: [PATCH 4/6] KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register

In reply to Shivaprasad G Bhat
Quoted 11 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the DEXCR for the guest during enter/exit with KVM_REG_PPC_DEXCR. The specific SPR KVM API documentation too updated.

---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +

Headers under tools/ are not supposed to be updated directly, they're synced later by the perf developers.

See: https://lore.kernel.org/all/ZlYxAdHjyAkvGtMW@x1/

cheers

#9 lore raw

PATCH 5/6 KVM: PPC: Book3S HV nestedv2: Keep nested guest HASHKEYR in sync

In reply to Shivaprasad G Bhat

The nestedv2 APIs has the guest state element defined for HASHKEYR for the save-restore with L0. However, its ignored in the code.

The patch takes care of this for the HASHKEYR GSID.

---
 arch/powerpc/include/asm/kvm_host.h   |    1 +
 arch/powerpc/kvm/book3s_hv.h          |    1 +
 arch/powerpc/kvm/book3s_hv_nestedv2.c |    6 ++++++
 3 files changed, 8 insertions(+)
arch/powerpc/include/asm/kvm_host.h +1 -0
index 1e2fdcbecffd..a0cd9dbf534f 100644
@@ -600,6 +600,7 @@ struct kvm_vcpu_arch {
 	ulong dawr1;
 	ulong dawrx1;
 	ulong dexcr;
+	ulong hashkeyr;
 	ulong ciabr;
 	ulong cfar;
 	ulong ppr;
arch/powerpc/kvm/book3s_hv.h +1 -0
index 7b0fd282fe95..c073fdfa7dc4 100644
@@ -117,6 +117,7 @@ KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr1, 64, KVMPPC_GSID_DAWR1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx0, 64, KVMPPC_GSID_DAWRX0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx1, 64, KVMPPC_GSID_DAWRX1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dexcr, 64, KVMPPC_GSID_DEXCR)
+KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(hashkeyr, 64, KVMPPC_GSID_HASHKEYR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ciabr, 64, KVMPPC_GSID_CIABR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(wort, 64, KVMPPC_GSID_WORT)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ppr, 64, KVMPPC_GSID_PPR)
arch/powerpc/kvm/book3s_hv_nestedv2.c +6 -0
index d207a6d936ff..bbff933f2ccc 100644
@@ -196,6 +196,9 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb,
 		case KVMPPC_GSID_DEXCR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dexcr);
 			break;
+		case KVMPPC_GSID_HASHKEYR:
+			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.hashkeyr);
+			break;
 		case KVMPPC_GSID_CIABR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ciabr);
 			break;
@@ -447,6 +450,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,
 		case KVMPPC_GSID_DEXCR:
 			vcpu->arch.dexcr = kvmppc_gse_get_u64(gse);
 			break;
+		case KVMPPC_GSID_HASHKEYR:
+			vcpu->arch.hashkeyr = kvmppc_gse_get_u64(gse);
+			break;
 		case KVMPPC_GSID_CIABR:
 			vcpu->arch.ciabr = kvmppc_gse_get_u64(gse);
 			break;
reviewer Reviewed-by on v1 5/6
#10 lore raw

Re: [PATCH 5/6] KVM: PPC: Book3S HV nestedv2: Keep nested guest HASHKEYR in sync

Quoted 17 lines · Shivaprasad G Bhat

The nestedv2 APIs has the guest state element defined for HASHKEYR for the save-restore with L0. However, its ignored in the code.

The patch takes care of this for the HASHKEYR GSID.

---
 arch/powerpc/include/asm/kvm_host.h   |    1 +
 arch/powerpc/kvm/book3s_hv.h          |    1 +
 arch/powerpc/kvm/book3s_hv_nestedv2.c |    6 ++++++
 3 files changed, 8 insertions(+)
arch/powerpc/include/asm/kvm_host.h +0 -0
index 1e2fdcbecffd..a0cd9dbf534f 100644

arch/powerpc/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 606

Quoted 7 lines
arch/powerpc/include/asm/kvm_host.h +1 -0
@@ -600,6 +600,7 @@ struct kvm_vcpu_arch {
 	ulong dawr1;
 	ulong dawrx1;
 	ulong dexcr;
+	ulong hashkeyr;
 	ulong ciabr;
 	ulong cfar;
 	ulong ppr;

Same comment applies

Thanks,
Nick

Quoted 36 lines
arch/powerpc/kvm/book3s_hv.h +1 -0
index 7b0fd282fe95..c073fdfa7dc4 100644
@@ -117,6 +117,7 @@ KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawr1, 64, KVMPPC_GSID_DAWR1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx0, 64, KVMPPC_GSID_DAWRX0)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dawrx1, 64, KVMPPC_GSID_DAWRX1)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(dexcr, 64, KVMPPC_GSID_DEXCR)
+KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(hashkeyr, 64, KVMPPC_GSID_HASHKEYR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ciabr, 64, KVMPPC_GSID_CIABR)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(wort, 64, KVMPPC_GSID_WORT)
 KVMPPC_BOOK3S_HV_VCPU_ACCESSOR(ppr, 64, KVMPPC_GSID_PPR)
arch/powerpc/kvm/book3s_hv_nestedv2.c +6 -0
index d207a6d936ff..bbff933f2ccc 100644
@@ -196,6 +196,9 @@ static int gs_msg_ops_vcpu_fill_info(struct kvmppc_gs_buff *gsb,
 		case KVMPPC_GSID_DEXCR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.dexcr);
 			break;
+		case KVMPPC_GSID_HASHKEYR:
+			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.hashkeyr);
+			break;
 		case KVMPPC_GSID_CIABR:
 			rc = kvmppc_gse_put_u64(gsb, iden, vcpu->arch.ciabr);
 			break;
@@ -447,6 +450,9 @@ static int gs_msg_ops_vcpu_refresh_info(struct kvmppc_gs_msg *gsm,
 		case KVMPPC_GSID_DEXCR:
 			vcpu->arch.dexcr = kvmppc_gse_get_u64(gse);
 			break;
+		case KVMPPC_GSID_HASHKEYR:
+			vcpu->arch.hashkeyr = kvmppc_gse_get_u64(gse);
+			break;
 		case KVMPPC_GSID_CIABR:
 			vcpu->arch.ciabr = kvmppc_gse_get_u64(gse);
 			break;
#11 lore raw

PATCH 6/6 KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

In reply to Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the virtual HASHKEYR for the guest during enter/exit with KVM_REG_PPC_HASHKEYR. The specific SPR KVM API documentation too updated.

---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
 4 files changed, 9 insertions(+)
Documentation/virt/kvm/api.rst +1 -0
index 81077c654281..0c22cb4196d8 100644
@@ -2439,6 +2439,7 @@ registers, find a list below:
   PPC     KVM_REG_PPC_PSSCR               64
   PPC     KVM_REG_PPC_DEC_EXPIRY          64
   PPC     KVM_REG_PPC_PTCR                64
+  PPC     KVM_REG_PPC_HASHKEYR            64
   PPC     KVM_REG_PPC_DAWR1               64
   PPC     KVM_REG_PPC_DAWRX1              64
   PPC     KVM_REG_PPC_DEXCR               64
arch/powerpc/include/uapi/asm/kvm.h +1 -0
index fcb947f65667..23a0af739c78 100644
@@ -646,6 +646,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
 #define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
+#define KVM_REG_PPC_HASHKEYR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc7)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
arch/powerpc/kvm/book3s_hv.c +6 -0
index 1294c6839d37..ccc9564c5a31 100644
@@ -2352,6 +2352,9 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DEXCR:
 		*val = get_reg_val(id, kvmppc_get_dexcr_hv(vcpu));
 		break;
+	case KVM_REG_PPC_HASHKEYR:
+		*val = get_reg_val(id, kvmppc_get_hashkeyr_hv(vcpu));
+		break;
 	case KVM_REG_PPC_CIABR:
 		*val = get_reg_val(id, kvmppc_get_ciabr_hv(vcpu));
 		break;
@@ -2598,6 +2601,9 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DEXCR:
 		kvmppc_set_dexcr_hv(vcpu, set_reg_val(id, *val));
 		break;
+	case KVM_REG_PPC_HASHKEYR:
+		kvmppc_set_hashkeyr_hv(vcpu, set_reg_val(id, *val));
+		break;
 	case KVM_REG_PPC_CIABR:
 		kvmppc_set_ciabr_hv(vcpu, set_reg_val(id, *val));
 		/* Don't allow setting breakpoints in hypervisor code */
tools/arch/powerpc/include/uapi/asm/kvm.h +1 -0
index fcb947f65667..23a0af739c78 100644
@@ -646,6 +646,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
 #define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
+#define KVM_REG_PPC_HASHKEYR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc7)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
reviewer Reviewed-by on v1 6/6
#12 lore raw

Re: [PATCH 6/6] KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

Quoted 4 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the virtual HASHKEYR for the guest during enter/exit with KVM_REG_PPC_HASHKEYR. The specific SPR KVM API documentation too updated.

Quoted 69 lines
---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
 4 files changed, 9 insertions(+)
Documentation/virt/kvm/api.rst +1 -0
index 81077c654281..0c22cb4196d8 100644
@@ -2439,6 +2439,7 @@ registers, find a list below:
   PPC     KVM_REG_PPC_PSSCR               64
   PPC     KVM_REG_PPC_DEC_EXPIRY          64
   PPC     KVM_REG_PPC_PTCR                64
+  PPC     KVM_REG_PPC_HASHKEYR            64
   PPC     KVM_REG_PPC_DAWR1               64
   PPC     KVM_REG_PPC_DAWRX1              64
   PPC     KVM_REG_PPC_DEXCR               64
arch/powerpc/include/uapi/asm/kvm.h +1 -0
index fcb947f65667..23a0af739c78 100644
@@ -646,6 +646,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
 #define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
+#define KVM_REG_PPC_HASHKEYR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc7)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
arch/powerpc/kvm/book3s_hv.c +6 -0
index 1294c6839d37..ccc9564c5a31 100644
@@ -2352,6 +2352,9 @@ static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DEXCR:
 		*val = get_reg_val(id, kvmppc_get_dexcr_hv(vcpu));
 		break;
+	case KVM_REG_PPC_HASHKEYR:
+		*val = get_reg_val(id, kvmppc_get_hashkeyr_hv(vcpu));
+		break;
 	case KVM_REG_PPC_CIABR:
 		*val = get_reg_val(id, kvmppc_get_ciabr_hv(vcpu));
 		break;
@@ -2598,6 +2601,9 @@ static int kvmppc_set_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
 	case KVM_REG_PPC_DEXCR:
 		kvmppc_set_dexcr_hv(vcpu, set_reg_val(id, *val));
 		break;
+	case KVM_REG_PPC_HASHKEYR:
+		kvmppc_set_hashkeyr_hv(vcpu, set_reg_val(id, *val));
+		break;
 	case KVM_REG_PPC_CIABR:
 		kvmppc_set_ciabr_hv(vcpu, set_reg_val(id, *val));
 		/* Don't allow setting breakpoints in hypervisor code */
tools/arch/powerpc/include/uapi/asm/kvm.h +1 -0
index fcb947f65667..23a0af739c78 100644
@@ -646,6 +646,7 @@ struct kvm_ppc_cpu_char {
 #define KVM_REG_PPC_DAWR1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc4)
 #define KVM_REG_PPC_DAWRX1	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc5)
 #define KVM_REG_PPC_DEXCR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc6)
+#define KVM_REG_PPC_HASHKEYR	(KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc7)
 
 /* Transactional Memory checkpointed state:
  * This is all GPRs, all VSX regs and a subset of SPRs
#13 lore raw

Re: [PATCH 6/6] KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

In reply to Shivaprasad G Bhat
Quoted 18 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the virtual HASHKEYR for the guest during enter/exit with KVM_REG_PPC_HASHKEYR. The specific SPR KVM API documentation too updated.

---
 Documentation/virt/kvm/api.rst            |    1 +
 arch/powerpc/include/uapi/asm/kvm.h       |    1 +
 arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
 4 files changed, 9 insertions(+)
Documentation/virt/kvm/api.rst +0 -0
index 81077c654281..0c22cb4196d8 100644

Documentation/virt/kvm/api.rst registers, find a list below: · line 2442

Quoted 4 lines
Documentation/virt/kvm/api.rst +1 -0
@@ -2439,6 +2439,7 @@ registers, find a list below:
   PPC     KVM_REG_PPC_PSSCR               64
   PPC     KVM_REG_PPC_DEC_EXPIRY          64
   PPC     KVM_REG_PPC_PTCR                64
+  PPC     KVM_REG_PPC_HASHKEYR            64

Just looking at the QEMU side of this change made me think... AFAIKS we need to also set and get and migrate the HASHPKEY SPR.

The hashst/hashchk test cases might be "working" by chance if the SPR is always zero :/

Thanks,
Nick

#14 lore raw

Re: [PATCH 6/6] KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

Quoted 14 lines · Nicholas Piggin
Quoted 13 lines · Shivaprasad G Bhat

The patch adds a one-reg register identifier which can be used to read and set the virtual HASHKEYR for the guest during enter/exit with KVM_REG_PPC_HASHKEYR. The specific SPR KVM API documentation too updated.

---
  Documentation/virt/kvm/api.rst            |    1 +
  arch/powerpc/include/uapi/asm/kvm.h       |    1 +
  arch/powerpc/kvm/book3s_hv.c              |    6 ++++++
  tools/arch/powerpc/include/uapi/asm/kvm.h |    1 +
  4 files changed, 9 insertions(+)
Quoted 11 lines

Documentation/virt/kvm/api.rst registers, find a list below: · line 2442

Quoted 9 lines
Documentation/virt/kvm/api.rst +1 -0
index 81077c654281..0c22cb4196d8 100644
@@ -2439,6 +2439,7 @@ registers, find a list below:
    PPC     KVM_REG_PPC_PSSCR               64
    PPC     KVM_REG_PPC_DEC_EXPIRY          64
    PPC     KVM_REG_PPC_PTCR                64
+  PPC     KVM_REG_PPC_HASHKEYR            64

Just looking at the QEMU side of this change made me think... AFAIKS we need to also set and get and migrate the HASHPKEY SPR.

Thanks Nick. I have posted the v2 with changes for HASHPKEYR

and your other suggestions at

[email protected]

Regards,

Shivaprasad

Quoted 6 lines

The hashst/hashchk test cases might be "working" by chance if the SPR is always zero :/

Thanks,
Nick

#15 lore raw
In reply to Shivaprasad G Bhat
Quoted 6 lines · Shivaprasad G Bhat

The series fixes the issues exposed by the kvm-unit-tests[1] sprs-migration test.

The SDAR, MMCR3 were seen to have some typo/refactoring bugs. The first two patches fix them.

Quoted 6 lines

Though the nestedv2 APIs defined the guest state elements for Power ISA 3.1B SPRs to save-restore with PHYP during entry-exit, the DEXCR and HASHKEYR were ignored in code. The KVM_PPC_REG too for them are missing without which the Qemu is not setting them to their 'previous' value during guest migration at destination. The remaining patches take care of this.

These aren't just fixes for nested v2 or even just migration, by the way. Good fixes.

Thanks,
Nick

Quoted 26 lines

References:
[1]: https://github.com/kvm-unit-tests/kvm-unit-tests

---

Shivaprasad G Bhat (6):
KVM: PPC: Book3S HV: Fix the set_one_reg for MMCR3
KVM: PPC: Book3S HV: Fix the get_one_reg of SDAR
KVM: PPC: Book3S HV nestedv2: Keep nested guest DEXCR in sync
KVM: PPC: Book3S HV: Add one-reg interface for DEXCR register
KVM: PPC: Book3S HV nestedv2: Keep nested guest HASHKEYR in sync
KVM: PPC: Book3S HV: Add one-reg interface for HASHKEYR register

 Documentation/virt/kvm/api.rst            |  2 ++
 arch/powerpc/include/asm/kvm_host.h       |  2 ++
 arch/powerpc/include/uapi/asm/kvm.h       |  2 ++
 arch/powerpc/kvm/book3s_hv.c              | 16 ++++++++++++++--
 arch/powerpc/kvm/book3s_hv.h              |  2 ++
 arch/powerpc/kvm/book3s_hv_nestedv2.c     | 12 ++++++++++++
 tools/arch/powerpc/include/uapi/asm/kvm.h |  2 ++
 7 files changed, 36 insertions(+), 2 deletions(-)

--
Signature