Skip to contentSkip to conversation

KVM: PPC: MMIO fixes

PATCH Applied Fabiano Rosas 3 participants 8 messages Jan 25, 2022 – Feb 16, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v5 cover KVM: PPC: MMIO fixes

Changes from v4:

-patch 4: switched to kvm_debug_ratelimited.

-patch 5: kept the Program interrupt for BookE.

v4:
https://lore.kernel.org/r/[email protected]

v3:
https://lore.kernel.org/r/[email protected]

v2:
https://lore.kernel.org/r/[email protected]

v1:
https://lore.kernel.org/r/[email protected]

Fabiano Rosas (5):
KVM: PPC: Book3S HV: Stop returning internal values to userspace
KVM: PPC: Fix vmx/vsx mixup in mmio emulation
KVM: PPC: mmio: Reject instructions that access more than mmio.data
size
KVM: PPC: mmio: Return to guest after emulation failure
KVM: PPC: Book3s: mmio: Deliver DSI after emulation failure

 arch/powerpc/kvm/emulate_loadstore.c | 10 ++---
 arch/powerpc/kvm/powerpc.c           | 56 ++++++++++++++++++++--------
 2 files changed, 43 insertions(+), 23 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v5 1/5 KVM: PPC: Book3S HV: Stop returning internal values to userspace

Our kvm_arch_vcpu_ioctl_run currently returns the RESUME_HOST values to userspace, against the API of the KVM_RUN ioctl which returns 0 on success.

---
This was noticed while enabling the kvm selftests for powerpc. There's an assert at the _vcpu_run function when we return a value different from the expected.

---
 arch/powerpc/kvm/powerpc.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/kvm/powerpc.c +8 -0
index 2ad0ccd202d5..50414fb2a5ea 100644
@@ -1841,6 +1841,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
 #ifdef CONFIG_ALTIVEC
 out:
 #endif
+
+	/*
+	 * We're already returning to userspace, don't pass the
+	 * RESUME_HOST flags along.
+	 */
+	if (r > 0)
+		r = 0;
+
 	vcpu_put(vcpu);
 	return r;
 }
Signature · 1 line
2.34.1
#2 lore raw

PATCH v5 2/5 KVM: PPC: Fix vmx/vsx mixup in mmio emulation

In reply to Fabiano Rosas

The MMIO emulation code for vector instructions is duplicated between VSX and VMX. When emulating VMX we should check the VMX copy size instead of the VSX one.

Fixes acc9eb9305fe ("KVM: PPC: Reimplement LOAD_VMX/STORE_VMX instruction ...")
---
 arch/powerpc/kvm/powerpc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/kvm/powerpc.c +2 -2
index 50414fb2a5ea..c2bd29e90314 100644
@@ -1499,7 +1499,7 @@ int kvmppc_handle_vmx_load(struct kvm_vcpu *vcpu,
 {
 	enum emulation_result emulated = EMULATE_DONE;
 
-	if (vcpu->arch.mmio_vsx_copy_nums > 2)
+	if (vcpu->arch.mmio_vmx_copy_nums > 2)
 		return EMULATE_FAIL;
 
 	while (vcpu->arch.mmio_vmx_copy_nums) {
@@ -1596,7 +1596,7 @@ int kvmppc_handle_vmx_store(struct kvm_vcpu *vcpu,
 	unsigned int index = rs & KVM_MMIO_REG_MASK;
 	enum emulation_result emulated = EMULATE_DONE;
 
-	if (vcpu->arch.mmio_vsx_copy_nums > 2)
+	if (vcpu->arch.mmio_vmx_copy_nums > 2)
 		return EMULATE_FAIL;
 
 	vcpu->arch.io_gpr = rs;
Signature · 1 line
2.34.1
#3 lore raw

PATCH v5 3/5 KVM: PPC: mmio: Reject instructions that access more than mmio.data size

In reply to Fabiano Rosas

The MMIO interface between the kernel and userspace uses a structure that supports a maximum of 8-bytes of data. Instructions that access more than that need to be emulated in parts.

We currently don't have generic support for splitting the emulation in parts and each set of instructions needs to be explicitly included.

There's already an error message being printed when a load or store exceeds the mmio.data buffer but we don't fail the emulation until later at kvmppc_complete_mmio_load and even then we allow userspace to make a partial copy of the data, which ends up overwriting some fields of the mmio structure.

This patch makes the emulation fail earlier at kvmppc_handle_load|store, which will send a Program interrupt to the guest. This is better than allowing the guest to proceed with partial data.

Note that this was caught in a somewhat artificial scenario using quadword instructions (lq/stq), there's no account of an actual guest in the wild running instructions that are not properly emulated.

(While here, remove the "bad MMIO" messages. The caller already has an error message.)

---
 arch/powerpc/kvm/powerpc.c | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)
arch/powerpc/kvm/powerpc.c +5 -11
index c2bd29e90314..27fb2b70f631 100644
@@ -1114,10 +1114,8 @@ static void kvmppc_complete_mmio_load(struct kvm_vcpu *vcpu)
 	struct kvm_run *run = vcpu->run;
 	u64 gpr;
 
-	if (run->mmio.len > sizeof(gpr)) {
-		printk(KERN_ERR "bad MMIO length: %d\n", run->mmio.len);
+	if (run->mmio.len > sizeof(gpr))
 		return;
-	}
 
 	if (!vcpu->arch.mmio_host_swabbed) {
 		switch (run->mmio.len) {
@@ -1236,10 +1234,8 @@ static int __kvmppc_handle_load(struct kvm_vcpu *vcpu,
 		host_swabbed = !is_default_endian;
 	}
 
-	if (bytes > sizeof(run->mmio.data)) {
-		printk(KERN_ERR "%s: bad MMIO length: %d\n", __func__,
-		       run->mmio.len);
-	}
+	if (bytes > sizeof(run->mmio.data))
+		return EMULATE_FAIL;
 
 	run->mmio.phys_addr = vcpu->arch.paddr_accessed;
 	run->mmio.len = bytes;
@@ -1325,10 +1321,8 @@ int kvmppc_handle_store(struct kvm_vcpu *vcpu,
 		host_swabbed = !is_default_endian;
 	}
 
-	if (bytes > sizeof(run->mmio.data)) {
-		printk(KERN_ERR "%s: bad MMIO length: %d\n", __func__,
-		       run->mmio.len);
-	}
+	if (bytes > sizeof(run->mmio.data))
+		return EMULATE_FAIL;
 
 	run->mmio.phys_addr = vcpu->arch.paddr_accessed;
 	run->mmio.len = bytes;
Signature · 1 line
2.34.1
#4 lore raw

PATCH v5 4/5 KVM: PPC: mmio: Return to guest after emulation failure

In reply to Fabiano Rosas

If MMIO emulation fails we don't want to crash the whole guest by returning to userspace.

The original commit bbf45ba57eae ("KVM: ppc: PowerPC 440 KVM implementation") added a todo:

/* XXX Deliver Program interrupt to guest. */

and later the commit d69614a295ae ("KVM: PPC: Separate loadstore emulation from priv emulation") added the Program interrupt injection but in another file, so I'm assuming it was missed that this block needed to be altered.

Also change the message to a ratelimited one since we're letting the guest run and it could flood the host logs.

---
 arch/powerpc/kvm/powerpc.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/powerpc/kvm/powerpc.c +3 -3
index 27fb2b70f631..acb0d2a4bdb9 100644
@@ -307,9 +307,9 @@ int kvmppc_emulate_mmio(struct kvm_vcpu *vcpu)
 		u32 last_inst;
 
 		kvmppc_get_last_inst(vcpu, INST_GENERIC, &last_inst);
-		/* XXX Deliver Program interrupt to guest. */
-		pr_emerg("%s: emulation failed (%08x)\n", __func__, last_inst);
-		r = RESUME_HOST;
+		kvm_debug_ratelimited("Guest access to device memory using unsupported instruction (opcode: %#08x)\n",
+				      last_inst);
+		r = RESUME_GUEST;
 		break;
 	}
 	default:
Signature · 1 line
2.34.1
#5 lore raw

PATCH v5 5/5 KVM: PPC: Book3s: mmio: Deliver DSI after emulation failure

In reply to Fabiano Rosas

MMIO emulation can fail if the guest uses an instruction that we are not prepared to emulate. Since these instructions can be and most likely are valid ones, this is (slightly) closer to an access fault than to an illegal instruction, so deliver a Data Storage interrupt instead of a Program interrupt.

BookE ignores bad faults, so it will keep using a Program interrupt because a DSI would cause a fault loop in the guest.

---
 arch/powerpc/kvm/emulate_loadstore.c | 10 +++-------
 arch/powerpc/kvm/powerpc.c           | 22 ++++++++++++++++++++++
 2 files changed, 25 insertions(+), 7 deletions(-)
arch/powerpc/kvm/emulate_loadstore.c +3 -7
index 48272a9b9c30..cfc9114b87d0 100644
@@ -73,7 +73,6 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 {
 	u32 inst;
 	enum emulation_result emulated = EMULATE_FAIL;
-	int advance = 1;
 	struct instruction_op op;
 
 	/* this default type might be overwritten by subcategories */
@@ -98,6 +97,8 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 		int type = op.type & INSTR_TYPE_MASK;
 		int size = GETSIZE(op.type);
 
+		vcpu->mmio_is_write = OP_IS_STORE(type);
+
 		switch (type) {
 		case LOAD:  {
 			int instr_byte_swap = op.type & BYTEREV;
@@ -355,15 +356,10 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 		}
 	}
 
-	if (emulated = EMULATE_FAIL) {
-		advance = 0;
-		kvmppc_core_queue_program(vcpu, 0);
-	}
-
 	trace_kvm_ppc_instr(inst, kvmppc_get_pc(vcpu), emulated);
 
 	/* Advance past emulated instruction. */
-	if (advance)
+	if (emulated != EMULATE_FAIL)
 		kvmppc_set_pc(vcpu, kvmppc_get_pc(vcpu) + 4);
 
 	return emulated;
arch/powerpc/kvm/powerpc.c +22 -0
index acb0d2a4bdb9..82d889db2b6b 100644
@@ -309,6 +309,28 @@ int kvmppc_emulate_mmio(struct kvm_vcpu *vcpu)
 		kvmppc_get_last_inst(vcpu, INST_GENERIC, &last_inst);
 		kvm_debug_ratelimited("Guest access to device memory using unsupported instruction (opcode: %#08x)\n",
 				      last_inst);
+
+		/*
+		 * Injecting a Data Storage here is a bit more
+		 * accurate since the instruction that caused the
+		 * access could still be a valid one.
+		 */
+		if (!IS_ENABLED(CONFIG_BOOKE)) {
+			ulong dsisr = DSISR_BADACCESS;
+
+			if (vcpu->mmio_is_write)
+				dsisr |= DSISR_ISSTORE;
+
+			kvmppc_core_queue_data_storage(vcpu, vcpu->arch.vaddr_accessed, dsisr);
+		} else {
+			/*
+			 * BookE does not send a SIGBUS on a bad
+			 * fault, so use a Program interrupt instead
+			 * to avoid a fault loop.
+			 */
+			kvmppc_core_queue_program(vcpu, 0);
+		}
+
 		r = RESUME_GUEST;
 		break;
 	}
Signature · 1 line
2.34.1
reviewer Reviewed-by on v5 5/5
#6 lore raw

Re: [PATCH v5 5/5] KVM: PPC: Book3s: mmio: Deliver DSI after emulation failure

Excerpts from Fabiano Rosas's message of January 26, 2022 7:56 am:

Quoted 11 lines

MMIO emulation can fail if the guest uses an instruction that we are not prepared to emulate. Since these instructions can be and most likely are valid ones, this is (slightly) closer to an access fault than to an illegal instruction, so deliver a Data Storage interrupt instead of a Program interrupt.

BookE ignores bad faults, so it will keep using a Program interrupt because a DSI would cause a fault loop in the guest.

Thanks this looks good to me. (And thanks for updating patch 4/5 with the kvm debug print helper.)

Quoted 80 lines
---
 arch/powerpc/kvm/emulate_loadstore.c | 10 +++-------
 arch/powerpc/kvm/powerpc.c           | 22 ++++++++++++++++++++++
 2 files changed, 25 insertions(+), 7 deletions(-)
arch/powerpc/kvm/emulate_loadstore.c +3 -7
index 48272a9b9c30..cfc9114b87d0 100644
@@ -73,7 +73,6 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 {
 	u32 inst;
 	enum emulation_result emulated = EMULATE_FAIL;
-	int advance = 1;
 	struct instruction_op op;
 
 	/* this default type might be overwritten by subcategories */
@@ -98,6 +97,8 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 		int type = op.type & INSTR_TYPE_MASK;
 		int size = GETSIZE(op.type);
 
+		vcpu->mmio_is_write = OP_IS_STORE(type);
+
 		switch (type) {
 		case LOAD:  {
 			int instr_byte_swap = op.type & BYTEREV;
@@ -355,15 +356,10 @@ int kvmppc_emulate_loadstore(struct kvm_vcpu *vcpu)
 		}
 	}
 
-	if (emulated == EMULATE_FAIL) {
-		advance = 0;
-		kvmppc_core_queue_program(vcpu, 0);
-	}
-
 	trace_kvm_ppc_instr(inst, kvmppc_get_pc(vcpu), emulated);
 
 	/* Advance past emulated instruction. */
-	if (advance)
+	if (emulated != EMULATE_FAIL)
 		kvmppc_set_pc(vcpu, kvmppc_get_pc(vcpu) + 4);
 
 	return emulated;
arch/powerpc/kvm/powerpc.c +22 -0
index acb0d2a4bdb9..82d889db2b6b 100644
@@ -309,6 +309,28 @@ int kvmppc_emulate_mmio(struct kvm_vcpu *vcpu)
 		kvmppc_get_last_inst(vcpu, INST_GENERIC, &last_inst);
 		kvm_debug_ratelimited("Guest access to device memory using unsupported instruction (opcode: %#08x)\n",
 				      last_inst);
+
+		/*
+		 * Injecting a Data Storage here is a bit more
+		 * accurate since the instruction that caused the
+		 * access could still be a valid one.
+		 */
+		if (!IS_ENABLED(CONFIG_BOOKE)) {
+			ulong dsisr = DSISR_BADACCESS;
+
+			if (vcpu->mmio_is_write)
+				dsisr |= DSISR_ISSTORE;
+
+			kvmppc_core_queue_data_storage(vcpu, vcpu->arch.vaddr_accessed, dsisr);
+		} else {
+			/*
+			 * BookE does not send a SIGBUS on a bad
+			 * fault, so use a Program interrupt instead
+			 * to avoid a fault loop.
+			 */
+			kvmppc_core_queue_program(vcpu, 0);
+		}
+
 		r = RESUME_GUEST;
 		break;
 	}
Signature · 1 line
2.34.1
#7 lore raw
In reply to Fabiano Rosas
Quoted 10 lines · Fabiano Rosas

Changes from v4:

(quoted diff) +0 -2
-patch 4: switched to kvm_debug_ratelimited.
 
-patch 5: kept the Program interrupt for BookE.

v4:
https://lore.kernel.org/r/[email protected]

[...]

Applied to powerpc/topic/ppc-kvm.

[1/5] KVM: PPC: Book3S HV: Stop returning internal values to userspace

      https://git.kernel.org/powerpc/c/36d014d37d59065087e51b8381e37993f1ca99bc
[2/5] KVM: PPC: Fix vmx/vsx mixup in mmio emulation
      https://git.kernel.org/powerpc/c/b99234b918c6e36b9aa0a5b2981e86b6bd11f8e2
[3/5] KVM: PPC: mmio: Reject instructions that access more than mmio.data size
      https://git.kernel.org/powerpc/c/3f831504482ab0d0d53d1966987959d1485345cc
[4/5] KVM: PPC: mmio: Return to guest after emulation failure
      https://git.kernel.org/powerpc/c/349fbfe9b918e6dea00734f07c0fbaf4c2e2df5e
[5/5] KVM: PPC: Book3s: mmio: Deliver DSI after emulation failure
      https://git.kernel.org/powerpc/c/c1c8a66367a35aabbad9bd629b105b9fb49f2c1f

cheers