Skip to contentSkip to conversation

KVM: s390: Fix AR parameter in MEM_OP ioctl

PATCH Superseded Eric Farman 2 participants 6 messages Feb 9, 2024 – Feb 13, 2024

Subsystems (4)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover KVM: s390: Fix AR parameter in MEM_OP ioctl

Hi Christian, Janosch, Heiko,

Here is a pair of patches to replace the RFC I recently sent [1].

Patch 1 performs the host/guest access register swap that Christian suggested (instead of a full sync_reg/store_reg process).

Patch 2 provides a selftest patch that exercises this scenario. Applying patch 2 without patch 1 fails in the following way:

[eric@host linux]# ./tools/testing/selftests/kvm/s390x/memop TAP version 13
1..16
ok 1 simple copy
ok 2 generic error checks
ok 3 copy with storage keys
ok 4 cmpxchg with storage keys
ok 5 concurrently cmpxchg with storage keys
ok 6 copy with key storage protection override
ok 7 copy with key fetch protection
ok 8 copy with key fetch protection override
==== Test Assertion Failure ====
s390x/memop.c:186: !r
pid=5720 tid=5720 errno=4 - Interrupted system call

     1	0x00000000010042af: memop_ioctl at memop.c:186 (discriminator 3)
     2	0x0000000001006697: test_copy_access_register at memop.c:400 (discriminator 2)
     3	0x0000000001002aaf: main at memop.c:1181
     4	0x000003ffaec33a5b: ?? ??:0
     5	0x000003ffaec33b5d: ?? ??:0
     6	0x0000000001002ba9: _start at ??:?

KVM_S390_MEM_OP failed, rc: 40 errno: 4 (Interrupted system call)

Thoughts on this approach?

Thanks,
Eric

[1] https://lore.kernel.org/r/[email protected]/

Eric Farman (2):
KVM: s390: load guest access registers in MEM_OP ioctl
KVM: s390: selftests: memop: add a simple AR test

 arch/s390/kvm/kvm-s390.c                  |  6 +++++
 tools/testing/selftests/kvm/s390x/memop.c | 28 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
Signature · 1 line
2.40.1
#1 lore raw

PATCH 1/2 KVM: s390: load guest access registers in MEM_OP ioctl

The routine ar_translation() is called by get_vcpu_asce(), which is called by both the instruction intercept path (where the access registers had been loaded with the guest's values), and the MEM_OP ioctl (which hadn't). This means that any ALET the guest expects to be used would be ignored.

Furthermore, the logic in ar_translation() will store the contents of the access registers back to the KVM_RUN struct. This unexpected change of AR values can lead to problems after invoking the MEM_OP, for example an ALET Specification Exception.

Fix this by swapping the host/guest access registers around the MEM_OP ioctl, in the same way that the KVM_RUN ioctl does with sync_regs()/store_regs(). The full register swap isn't needed here, since only the access registers are used in this interface.

---
 arch/s390/kvm/kvm-s390.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/s390/kvm/kvm-s390.c +6 -0
index ea63ac769889..c2dfeea55dcf 100644
@@ -5391,6 +5391,10 @@ static long kvm_s390_vcpu_mem_op(struct kvm_vcpu *vcpu,
 			return -ENOMEM;
 	}
 
+	/* Swap host/guest access registers in the event of a MEM_OP with AR specified */
+	save_access_regs(vcpu->arch.host_acrs);
+	restore_access_regs(vcpu->run->s.regs.acrs);
+
 	acc_mode = mop->op == KVM_S390_MEMOP_LOGICAL_READ ? GACC_FETCH : GACC_STORE;
 	if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
 		r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size,
@@ -5420,6 +5424,8 @@ static long kvm_s390_vcpu_mem_op(struct kvm_vcpu *vcpu,
 		kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
 
 out_free:
+	save_access_regs(vcpu->run->s.regs.acrs);
+	restore_access_regs(vcpu->arch.host_acrs);
 	vfree(tmpbuf);
 	return r;
 }
Signature · 1 line
2.40.1
maintainer
#2 lore raw

Re: [PATCH v1 1/2] KVM: s390: load guest access registers in MEM_OP ioctl

Quoted 38 lines · Eric Farman

The routine ar_translation() is called by get_vcpu_asce(), which is called by both the instruction intercept path (where the access registers had been loaded with the guest's values), and the MEM_OP ioctl (which hadn't). This means that any ALET the guest expects to be used would be ignored.

Furthermore, the logic in ar_translation() will store the contents of the access registers back to the KVM_RUN struct. This unexpected change of AR values can lead to problems after invoking the MEM_OP, for example an ALET Specification Exception.

Fix this by swapping the host/guest access registers around the MEM_OP ioctl, in the same way that the KVM_RUN ioctl does with sync_regs()/store_regs(). The full register swap isn't needed here, since only the access registers are used in this interface.

---
 arch/s390/kvm/kvm-s390.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/s390/kvm/kvm-s390.c +4 -0
index ea63ac769889..c2dfeea55dcf 100644
@@ -5391,6 +5391,10 @@ static long kvm_s390_vcpu_mem_op(struct kvm_vcpu *vcpu,
 			return -ENOMEM;
 	}
 
+	/* Swap host/guest access registers in the event of a MEM_OP with AR specified */
+	save_access_regs(vcpu->arch.host_acrs);
+	restore_access_regs(vcpu->run->s.regs.acrs);
+
 	acc_mode = mop->op == KVM_S390_MEMOP_LOGICAL_READ ? GACC_FETCH : GACC_STORE;
 	if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
 		r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size,

arch/s390/kvm/kvm-s390.c static long kvm_s390_vcpu_mem_op(struct kvm_vcpu *vcpu, · line 5428

Quoted 5 lines
arch/s390/kvm/kvm-s390.c +2 -0
@@ -5420,6 +5424,8 @@ static long kvm_s390_vcpu_mem_op(struct kvm_vcpu *vcpu,
 		kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
 
 out_free:
+	save_access_regs(vcpu->run->s.regs.acrs);
+	restore_access_regs(vcpu->arch.host_acrs);

I guess we will end up with more and more of such constructs until nobody knows when which register contents are loaded. I would higly prefer a TIF flag which indicates if the access registers contain the host or guest register contents, and actual users grab the required content from the correct location
- or better: always take it from guest save area, and write to the guest save area if the to be invented TIF flag indicates that access registers contain guest registers...

Or maybe a TIF flag with different semantics: "guest save area does not reflect current state - which is within registers".

maintainer
#3 lore raw

Re: [PATCH v1 1/2] KVM: s390: load guest access registers in MEM_OP ioctl

Quoted 2 lines · Heiko Carstens

Or maybe a TIF flag with different semantics: "guest save area does not reflect current state - which is within registers".

Something like the below; untested of course. But I guess there must be some arch specific vcpu flags, which can be used to achieve the same?

arch/s390/include/asm/thread_info.h +1 -0
index a674c7d25da5..b9ff8b125fb8 100644
@@ -69,6 +69,7 @@ void arch_setup_new_exec(void);
 #define TIF_PATCH_PENDING	5	/* pending live patching update */
 #define TIF_PGSTE		6	/* New mm's will use 4K page tables */
 #define TIF_NOTIFY_SIGNAL	7	/* signal notifications exist */
+#define TIF_KVM_ACRS		8	/* access registers contain guest content */
 #define TIF_ISOLATE_BP_GUEST	9	/* Run KVM guests with isolated BP */
 #define TIF_PER_TRAP		10	/* Need to handle PER trap on exit to usermode */
 
arch/s390/kvm/gaccess.c +2 -1
index 5bfcc50c1a68..b0ef242d2371 100644
@@ -391,7 +391,8 @@ static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, u8 ar,
 	if (ar >= NUM_ACRS)
 		return -EINVAL;
 
-	save_access_regs(vcpu->run->s.regs.acrs);
+	if (test_thread_flag(TIF_KVM_ACRS))
+		save_access_regs(vcpu->run->s.regs.acrs);
 	alet.val = vcpu->run->s.regs.acrs[ar];
 
 	if (ar == 0 || alet.val == 0) {
arch/s390/kvm/kvm-s390.c +2 -0
index ea63ac769889..3ee0913639d5 100644
@@ -4951,6 +4951,7 @@ static void sync_regs(struct kvm_vcpu *vcpu)
 	}
 	save_access_regs(vcpu->arch.host_acrs);
 	restore_access_regs(vcpu->run->s.regs.acrs);
+	set_thread_flag(TIF_KVM_ACRS);
 	/* save host (userspace) fprs/vrs */
 	save_fpu_regs();
 	vcpu->arch.host_fpregs.fpc = current->thread.fpu.fpc;
@@ -5020,6 +5021,7 @@ static void store_regs(struct kvm_vcpu *vcpu)
 	kvm_run->s.regs.pfs = vcpu->arch.pfault_select;
 	kvm_run->s.regs.pfc = vcpu->arch.pfault_compare;
 	save_access_regs(vcpu->run->s.regs.acrs);
+	clear_thread_flag(TIF_KVM_ACRS);
 	restore_access_regs(vcpu->arch.host_acrs);
 	/* Save guest register state */
 	save_fpu_regs();
#4 lore raw

Re: [PATCH v1 1/2] KVM: s390: load guest access registers in MEM_OP ioctl

Quoted 7 lines · Heiko Carstens
Quoted 7 lines · Heiko Carstens
Quoted 4 lines
Quoted 4 lines

Or maybe a TIF flag with different semantics: "guest save area does
not
reflect current state - which is within registers".

Something like the below; untested of course.

Ooops, yeah. Christian suggested something similar in his first response to the RFC which I'd overlooked.

Quoted 3 lines
Quoted 3 lines

 But I guess there must be
some arch specific vcpu flags, which can be used to achieve the same?

Agreed. Putting something there probably makes sense to keep it in the KVM sphere

Quoted 22 lines
Quoted 22 lines
arch/s390/include/asm/thread_info.h +1 -0
index a674c7d25da5..b9ff8b125fb8 100644
@@ -69,6 +69,7 @@ void arch_setup_new_exec(void);
  #define TIF_PATCH_PENDING 5 /* pending live patching update */
  #define TIF_PGSTE 6 /* New mm's will use 4K page tables */
  #define TIF_NOTIFY_SIGNAL 7 /* signal notifications exist */
+#define TIF_KVM_ACRS 8 /* access registers contain guest content
 */
  #define TIF_ISOLATE_BP_GUEST 9 /* Run KVM guests with isolated BP
 */

 #define TIF_PER_TRAP 10 /* Need to handle PER trap on exit to usermode */

arch/s390/kvm/gaccess.c +0 -0
index 5bfcc50c1a68..b0ef242d2371 100644
@@ -391,7 +391,8 @@ static int ar_translation(struct kvm_vcpu
Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/gaccess.c +0 -0
 *vcpu,
 union asce *asce, u8 ar,
   if (ar >= NUM_ACRS)
   return -EINVAL;


- save_access_regs(vcpu->run->s.regs.acrs);
+ if (test_thread_flag(TIF_KVM_ACRS))
+ save_access_regs(vcpu->run->s.regs.acrs);

...or WARN if not set, so that we know of the missing path. Will send this all as a v2. Thanks.

Quoted 23 lines
Quoted 23 lines

  alet.val = vcpu->run->s.regs.acrs[ar];

  if (ar == 0 || alet.val == 0) {

arch/s390/kvm/kvm-s390.c +2 -0
index ea63ac769889..3ee0913639d5 100644
@@ -4951,6 +4951,7 @@ static void sync_regs(struct kvm_vcpu *vcpu)
   }
   save_access_regs(vcpu->arch.host_acrs);
   restore_access_regs(vcpu->run->s.regs.acrs);
+ set_thread_flag(TIF_KVM_ACRS);
   /* save host (userspace) fprs/vrs */
   save_fpu_regs();
   vcpu->arch.host_fpregs.fpc = current->thread.fpu.fpc;
@@ -5020,6 +5021,7 @@ static void store_regs(struct kvm_vcpu *vcpu)
   kvm_run->s.regs.pfs = vcpu->arch.pfault_select;
   kvm_run->s.regs.pfc = vcpu->arch.pfault_compare;
   save_access_regs(vcpu->run->s.regs.acrs);
+ clear_thread_flag(TIF_KVM_ACRS);
   restore_access_regs(vcpu->arch.host_acrs);
   /* Save guest register state */
   save_fpu_regs();
#5 lore raw

PATCH 2/2 KVM: s390: selftests: memop: add a simple AR test

In reply to Eric Farman

There is a selftest that checks for an (expected) error when an invalid AR is specified, but not one that exercises the AR path.

Add a simple test that mirrors the vanilla write/read test while providing an AR. An AR that contains zero will direct the CPU to use the primary address space normally used anyway. AR[1] is selected for this test because the host AR[1] is usually non-zero, and KVM needs to correctly swap those values.

---
 tools/testing/selftests/kvm/s390x/memop.c | 28 +++++++++++++++++++++++
 1 file changed, 28 insertions(+)
tools/testing/selftests/kvm/s390x/memop.c +28 -0
index bb3ca9a5d731..be20c26ee545 100644
@@ -375,6 +375,29 @@ static void test_copy(void)
 	kvm_vm_free(t.kvm_vm);
 }
 
+static void test_copy_access_register(void)
+{
+	struct test_default t = test_default_init(guest_copy);
+
+	HOST_SYNC(t.vcpu, STAGE_INITED);
+
+	prepare_mem12();
+	t.run->psw_mask &= ~(3UL << (63 - 17));
+	t.run->psw_mask |= 1UL << (63 - 17);  /* Enable AR mode */
+
+	CHECK_N_DO(MOP, t.vcpu, LOGICAL, WRITE, mem1, t.size,
+		   GADDR_V(mem1), AR(1));
+	HOST_SYNC(t.vcpu, STAGE_COPIED);
+
+	CHECK_N_DO(MOP, t.vcpu, LOGICAL, READ, mem2, t.size,
+		   GADDR_V(mem2), AR(1));
+	ASSERT_MEM_EQ(mem1, mem2, t.size);
+
+	t.run->psw_mask &= ~(3UL << (63 - 17));   /* Disable AR mode */
+
+	kvm_vm_free(t.kvm_vm);
+}
+
 static void set_storage_key_range(void *addr, size_t len, uint8_t key)
 {
 	uintptr_t _addr, abs, i;
@@ -1101,6 +1124,11 @@ int main(int argc, char *argv[])
 			.test = test_copy_key_fetch_prot_override,
 			.requirements_met = extension_cap > 0,
 		},
+		{
+			.name = "copy with access register mode",
+			.test = test_copy_access_register,
+			.requirements_met = true,
+		},
 		{
 			.name = "error checks with key",
 			.test = test_errors_key,
Signature · 1 line
2.40.1