KVM: s390: Add new reset vcpu API

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

What am I missing?

(we could get rid of the kvm_arch_vcpu_ioctl_initial_reset() wrapper)

Quoted 34 lines · On Thu, 30 Jan 2020 10:49:35 +0100 David Hildenbrand <[email protected]> wrote:

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

Quoted 1 line

What am I missing?

I have been staring at this patch for some time now, and I fear I'm missing something as well. Can we please get more explanation?

Quoted 37 lines · Cornelia Huck

Quoted 34 lines · On Thu, 30 Jan 2020 10:49:35 +0100 David Hildenbrand <[email protected]> wrote:

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

Quoted 4 lines

Quoted 1 line

What am I missing?

I have been staring at this patch for some time now, and I fear I'm missing something as well. Can we please get more explanation?

Could we please get a test for this issue in the kvm selftests, too? I.e. host sets a value in its FPC, then calls the INITIAL_RESET ioctl and then checks that the value in its FPC is still there?

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

What am I missing?

Quoted 34 lines · David Hildenbrand

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

Quoted 1 line

What am I missing?

vcpu_load/put does no longer reload the registers lazily. We moved that out into the vcpu_run ioctl itself. (this avoids register reloading during schedule).

Quoted 39 lines · Christian Borntraeger

Quoted 35 lines · David Hildenbrand

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

What am I missing?

Quoted 2 lines

vcpu_load/put does no longer reload the registers lazily. We moved that out into the vcpu_run ioctl itself. (this avoids register reloading during schedule).

see
e1788bb KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load 31d8b8d KVM: s390: handle access registers in the run ioctl not in vcpu_put/load

so maybe we want to change the Fixes tag to this patch.

Quoted 42 lines · Christian Borntraeger

Quoted 39 lines · Christian Borntraeger

Quoted 35 lines · David Hildenbrand

Quoted 20 lines · Christian Borntraeger

The initial CPU reset currently clobbers the userspace fpc. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes 9abc2a08a7d6 ("KVM: s390: fix memory overwrites when vx is disabled")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

arch/s390/kvm/kvm-s390.c +0 -0

index c059b86..eb789cd 100644

arch/s390/kvm/kvm-s390.c static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu) · line 2830

Quoted 9 lines

arch/s390/kvm/kvm-s390.c +1 -2

@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

kvm_arch_vcpu_ioctl() does a vcpu_load(vcpu), followed by the call to kvm_arch_vcpu_ioctl_initial_reset(), followed by a vcpu_put().

What am I missing?

Quoted 9 lines

Quoted 2 lines

vcpu_load/put does no longer reload the registers lazily. We moved that out into the vcpu_run ioctl itself. (this avoids register reloading during schedule).

see
e1788bb KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load 31d8b8d KVM: s390: handle access registers in the run ioctl not in vcpu_put/load

so maybe we want to change the Fixes tag to this patch.

Yes, because

e1788bb KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load

broke it.

We should audit all users of save_fpu_regs().

The two ioctls for initial CPU reset and store status currently clobber the userspace fpc and potentially access registers. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes e1788bb995be ("KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load")

Fixes 31d8b8d41a7e ("KVM: s390: handle access registers in the run ioctl not in vcpu_put/load")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

arch/s390/kvm/kvm-s390.c +1 -2

index c059b86..936415b 100644
@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

arch/s390/kvm/kvm-s390.c +0 -0

 		srcu_read_unlock(&vcpu->kvm->srcu, idx);
 		break;
 	case KVM_S390_SET_INITIAL_PSW: {

The two ioctls for initial CPU reset and store status currently clobber the userspace fpc and potentially access registers. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

Cc [email protected]

Fixes e1788bb995be ("KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load")

Fixes 31d8b8d41a7e ("KVM: s390: handle access registers in the run ioctl not in vcpu_put/load")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

arch/s390/kvm/kvm-s390.c +2 -3

index c059b86..936415b 100644
@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

@@ -4343,7 +4342,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 	switch (ioctl) {
 	case KVM_S390_STORE_STATUS:
 		idx = srcu_read_lock(&vcpu->kvm->srcu);
-		r = kvm_s390_vcpu_store_status(vcpu, arg);
+		r = kvm_s390_vcpu_store_status_unloaded(vcpu, arg);
 		srcu_read_unlock(&vcpu->kvm->srcu, idx);
 		break;
 	case KVM_S390_SET_INITIAL_PSW: {

Quoted 7 lines · Christian Borntraeger

The two ioctls for initial CPU reset and store status currently clobber the userspace fpc and potentially access registers. This was an oversight during a fixup for the lazy fpu reloading rework. The reset calls are only done from userspace ioctls. No CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context. Otherwise the fpu restore call will restore the zeroes fpc to userspace.

New patch description:

KVM: s390: do not clobber registers during guest reset/store status

The initial CPU reset clobbers the userspace fpc and the store status ioctl clobbers the guest acrs + fpr. As these calls are only done via ioctl (and not via vcpu_run), no CPU context is loaded, so we can (and must) act directly on the sync regs, not on the thread context.

Cc [email protected]

Fixes e1788bb995be ("KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load")

Fixes 31d8b8d41a7e ("KVM: s390: handle access registers in the run ioctl not in vcpu_put/load")

Signed-off-by Christian Borntraeger <[email protected]>

Quoted 25 lines

Cc [email protected]

Fixes e1788bb995be ("KVM: s390: handle floating point registers in the run ioctl not in vcpu_put/load")

Fixes 31d8b8d41a7e ("KVM: s390: handle access registers in the run ioctl not in vcpu_put/load")

Signed-off-by Christian Borntraeger <[email protected]>

---
 arch/s390/kvm/kvm-s390.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

arch/s390/kvm/kvm-s390.c +1 -2

index c059b86..936415b 100644
@@ -2824,8 +2824,7 @@ static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
 	vcpu->arch.sie_block->gcr[14] = CR14_UNUSED_32 |
 					CR14_UNUSED_33 |
 					CR14_EXTERNAL_DAMAGE_SUBMASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
+	vcpu->run->s.regs.fpc = 0;
 	current->thread.fpu.fpc = 0;
 	vcpu->arch.sie_block->gbea = 1;
 	vcpu->arch.sie_block->pp = 0;

@@ -4343,7 +4342,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
 	switch (ioctl) {

arch/s390/kvm/kvm-s390.c long kvm_arch_vcpu_ioctl(struct file *filp, · line 4348

Quoted 8 lines

arch/s390/kvm/kvm-s390.c +1 -1

 	case KVM_S390_STORE_STATUS:
 		idx = srcu_read_lock(&vcpu->kvm->srcu);
-		r = kvm_s390_vcpu_store_status(vcpu, arg);
+		r = kvm_s390_vcpu_store_status_unloaded(vcpu, arg);
 		srcu_read_unlock(&vcpu->kvm->srcu, idx);
 		break;
 	case KVM_S390_SET_INITIAL_PSW: {

The architecture states that we need to reset local IRQs for all CPU resets. Because the old reset interface did not support the normal CPU reset we never did that on a normal reset.

Let's implement an interface for the missing normal and clear resets and reset all local IRQs, registers and control structures as stated in the architecture.

Userspace might already reset the registers via the vcpu run struct, but as we need the interface for the interrupt clearing part anyway, we implement the resets fully and don't rely on userspace to reset the rest.

Signed-off-by Janosch Frank <[email protected]>

Reviewed-by Cornelia Huck <[email protected]>

---
 Documentation/virt/kvm/api.txt |  43 ++++++++++++++
 arch/s390/kvm/kvm-s390.c       | 103 +++++++++++++++++++++++----------
 include/uapi/linux/kvm.h       |   5 ++
 3 files changed, 122 insertions(+), 29 deletions(-)

Documentation/virt/kvm/api.txt +43 -0

index ebb37b34dcfc..73448764f544 100644
@@ -4168,6 +4168,42 @@ This ioctl issues an ultravisor call to terminate the secure guest,
 unpins the VPA pages and releases all the device pages that are used to
 track the secure pages by hypervisor.
 
+4.122 KVM_S390_NORMAL_RESET
+
+Capability: KVM_CAP_S390_VCPU_RESETS
+Architectures: s390
+Type: vcpu ioctl
+Parameters: none
+Returns: 0
+
+This ioctl resets VCPU registers and control structures according to
+the cpu reset definition in the POP (Principles Of Operation).
+
+4.123 KVM_S390_INITIAL_RESET
+
+Capability: none
+Architectures: s390
+Type: vcpu ioctl
+Parameters: none
+Returns: 0
+
+This ioctl resets VCPU registers and control structures according to
+the initial cpu reset definition in the POP. However, the cpu is not
+put into ESA mode. This reset is a superset of the normal reset.
+
+4.124 KVM_S390_CLEAR_RESET
+
+Capability: KVM_CAP_S390_VCPU_RESETS
+Architectures: s390
+Type: vcpu ioctl
+Parameters: none
+Returns: 0
+
+This ioctl resets VCPU registers and control structures according to
+the clear cpu reset definition in the POP. However, the cpu is not put
+into ESA mode. This reset is a superset of the initial reset.
+
+
 5. The kvm_run structure
 ------------------------
 

@@ -5396,3 +5432,10 @@ handling by KVM (as some KVM hypercall may be mistakenly treated as TLB
 flush hypercalls by Hyper-V) so userspace should disable KVM identification
 in CPUID and only exposes Hyper-V identification. In this case, guest
 thinks it's running on Hyper-V and only use Hyper-V hypercalls.
+
+8.22 KVM_CAP_S390_VCPU_RESETS
+
+Architectures: s390
+
+This capability indicates that the KVM_S390_NORMAL_RESET and
+KVM_S390_CLEAR_RESET ioctls are available.

arch/s390/kvm/kvm-s390.c +17 -28

index c5f520de39a6..6aebaf08db64 100644
@@ -529,6 +529,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_S390_CMMA_MIGRATION:
 	case KVM_CAP_S390_AIS:
 	case KVM_CAP_S390_AIS_MIGRATION:
+	case KVM_CAP_S390_VCPU_RESETS:
 		r = 1;
 		break;
 	case KVM_CAP_S390_HPAGE_1M:

@@ -2844,31 +2845,6 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
 
 }
 
-static void kvm_s390_vcpu_initial_reset(struct kvm_vcpu *vcpu)
-{
-	/* this equals initial cpu reset in pop, but we don't switch to ESA */
-	vcpu->arch.sie_block->gpsw.mask = 0;
-	vcpu->arch.sie_block->gpsw.addr = 0;
-	kvm_s390_set_prefix(vcpu, 0);
-	kvm_s390_set_cpu_timer(vcpu, 0);
-	vcpu->arch.sie_block->ckc = 0;
-	vcpu->arch.sie_block->todpr = 0;
-	memset(vcpu->arch.sie_block->gcr, 0, sizeof(vcpu->arch.sie_block->gcr));
-	vcpu->arch.sie_block->gcr[0] = CR0_INITIAL_MASK;
-	vcpu->arch.sie_block->gcr[14] = CR14_INITIAL_MASK;
-	/* make sure the new fpc will be lazily loaded */
-	save_fpu_regs();
-	current->thread.fpu.fpc = 0;
-	vcpu->arch.sie_block->gbea = 1;
-	vcpu->arch.sie_block->pp = 0;
-	vcpu->arch.sie_block->fpf &= ~FPF_BPBC;
-	vcpu->arch.pfault_token = KVM_S390_PFAULT_TOKEN_INVALID;
-	kvm_clear_async_pf_completion_queue(vcpu);
-	if (!kvm_s390_user_cpu_state_ctrl(vcpu->kvm))
-		kvm_s390_vcpu_stop(vcpu);
-	kvm_s390_clear_local_irqs(vcpu);
-}
-
 void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 {
 	mutex_lock(&vcpu->kvm->lock);

@@ -3283,10 +3259,70 @@ static int kvm_arch_vcpu_ioctl_set_one_reg(struct kvm_vcpu *vcpu,
 	return r;
 }
 
-static int kvm_arch_vcpu_ioctl_initial_reset(struct kvm_vcpu *vcpu)
+static void kvm_arch_vcpu_ioctl_normal_reset(struct kvm_vcpu *vcpu)
 {
-	kvm_s390_vcpu_initial_reset(vcpu);
-	return 0;
+	vcpu->arch.sie_block->gpsw.mask &= ~PSW_MASK_RI;
+	vcpu->arch.pfault_token = KVM_S390_PFAULT_TOKEN_INVALID;
+	memset(vcpu->run->s.regs.riccb, 0, sizeof(vcpu->run->s.regs.riccb));
+
+	kvm_clear_async_pf_completion_queue(vcpu);
+	if (!kvm_s390_user_cpu_state_ctrl(vcpu->kvm))
+		kvm_s390_vcpu_stop(vcpu);
+	kvm_s390_clear_local_irqs(vcpu);
+}
+
+static void kvm_arch_vcpu_ioctl_initial_reset(struct kvm_vcpu *vcpu)
+{
+	/* Initial reset is a superset of the normal reset */
+	kvm_arch_vcpu_ioctl_normal_reset(vcpu);
+

arch/s390/kvm/kvm-s390.c +6 -0

+{
+	/* Initial reset is a superset of the normal reset */
+	kvm_arch_vcpu_ioctl_normal_reset(vcpu);
+
+	/* this equals initial cpu reset in pop, but we don't switch to ESA */
+	vcpu->arch.sie_block->gpsw.mask = 0;

arch/s390/kvm/kvm-s390.c +25 -0

+	current->thread.fpu.fpc = 0;
+	vcpu->arch.sie_block->gbea = 1;
+	vcpu->arch.sie_block->pp = 0;
+	vcpu->arch.sie_block->fpf &= ~FPF_BPBC;
+}
+
+static void kvm_arch_vcpu_ioctl_clear_reset(struct kvm_vcpu *vcpu)
+{
+	struct kvm_sync_regs *regs = &vcpu->run->s.regs;
+
+	/* Clear reset is a superset of the initial reset */
+	kvm_arch_vcpu_ioctl_initial_reset(vcpu);
+
+	memset(&regs->gprs, 0, sizeof(regs->gprs));
+	memset(&regs->vrs, 0, sizeof(regs->vrs));
+	memset(&regs->acrs, 0, sizeof(regs->acrs));
+
+	regs->etoken = 0;
+	regs->etoken_extension = 0;
+
+	memset(&regs->gscb, 0, sizeof(regs->gscb));
 
 
+	if (MACHINE_HAS_GS) {
+		preempt_disable();
+		__ctl_set_bit(2, 4);
+		if (current->thread.gs_cb) {

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

tools/testing/selftests/kvm/include/kvm_util.h +6 -0

index 29cccaf96baf..ae0d14c2540a 100644
@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +41 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+

Quoted 79 lines · Janosch Frank

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

tools/testing/selftests/kvm/include/kvm_util.h +6 -0

index 29cccaf96baf..ae0d14c2540a 100644
@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +46 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_SET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);

tools/testing/selftests/kvm/lib/kvm_util.c int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs) · line 1426

Quoted 6 lines

tools/testing/selftests/kvm/lib/kvm_util.c +2 -0

+}
+
 /*
  * VCPU Ioctl
  *

Reviewed-by Thomas Huth <[email protected]>

Quoted 80 lines · Thomas Huth

Quoted 79 lines · Janosch Frank

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

tools/testing/selftests/kvm/include/kvm_util.h +6 -0

index 29cccaf96baf..ae0d14c2540a 100644
@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +46 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_SET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);

Quoted 9 lines

tools/testing/selftests/kvm/lib/kvm_util.c int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs) · line 1426

Quoted 6 lines

tools/testing/selftests/kvm/lib/kvm_util.c +2 -0

+}
+
 /*
  * VCPU Ioctl
  *

Reviewed-by Thomas Huth <[email protected]>

How about what's below instead. It should be equivalent.

Thanks,
drew

tools/testing/selftests/kvm/include/kvm_util.h +25 -0

index 29cccaf96baf..d96a072e69bf 100644
@@ -125,6 +125,31 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+
+static inline void
+vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_FPU, fpu);
+}
+
+static inline void
+vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_FPU, fpu);
+}
+
+static inline void
+vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, reg);
+}
+
+static inline void
+vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, reg);
+}
+
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

Quoted 10 lines · Andrew Jones

Quoted 9 lines · Thomas Huth

Quoted 8 lines · Janosch Frank

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

tools/testing/selftests/kvm/include/kvm_util.h +0 -0

index 29cccaf96baf..ae0d14c2540a 100644

Quoted 8 lines

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, · line 131

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h +4 -0

@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);

Quoted 69 lines

Quoted 68 lines

Quoted 55 lines

tools/testing/selftests/kvm/include/kvm_util.h +2 -0

+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +41 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+

tools/testing/selftests/kvm/lib/kvm_util.c int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs) · line 1426

Quoted 10 lines

tools/testing/selftests/kvm/lib/kvm_util.c +7 -0

+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_SET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
 /*
  * VCPU Ioctl
  *

Reviewed-by Thomas Huth <[email protected]>

Quoted 1 line

How about what's below instead. It should be equivalent.

With your proposed changes we loose a bit verbosity in the error messages. I need to think about which I like more.

Quoted 42 lines

Thanks,
drew

tools/testing/selftests/kvm/include/kvm_util.h +25 -0

index 29cccaf96baf..d96a072e69bf 100644
@@ -125,6 +125,31 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+
+static inline void
+vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_FPU, fpu);
+}
+
+static inline void
+vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_FPU, fpu);
+}
+
+static inline void
+vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, reg);
+}
+
+static inline void
+vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, reg);
+}
+
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

Quoted 11 lines · Janosch Frank

Quoted 10 lines · Andrew Jones

Quoted 9 lines · Thomas Huth

Quoted 8 lines · Janosch Frank

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

Quoted 12 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

tools/testing/selftests/kvm/include/kvm_util.h +0 -0

index 29cccaf96baf..ae0d14c2540a 100644

Quoted 8 lines

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, · line 131

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h +4 -0

@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);

nit: no need for the above line breaks. We don't even get to 80 char.

Quoted 71 lines

Quoted 70 lines

Quoted 59 lines

Quoted 59 lines

tools/testing/selftests/kvm/include/kvm_util.h +2 -0

+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +46 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_SET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);

Quoted 9 lines

tools/testing/selftests/kvm/lib/kvm_util.c int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs) · line 1426

Quoted 6 lines

tools/testing/selftests/kvm/lib/kvm_util.c +2 -0

+}
+
 /*
  * VCPU Ioctl
  *

Reviewed-by Thomas Huth <[email protected]>

How about what's below instead. It should be equivalent.

Quoted 2 lines

With your proposed changes we loose a bit verbosity in the error messages. I need to think about which I like more.

Looks like both error messages are missing something. The ones above are missing the string version of errno. The ones below are missing the string version of cmd. It's easy to add the string version of errno, which is an argument for keeping the functions above (but we could at least use _vcpu_ioctl to avoid duplicating the vcpu_find and vcpu!=NULL assert).

Or, we could consider adding a kvm_ioctl_cmd_to_string() function, which might be nice for other ioctl wrappers now and in the future. It shouldn't be too bad to generate a string table from kvm.h, but of course we'd have to keep it maintained.

Thanks,
drew

Quoted 44 lines

Quoted 42 lines

Thanks,
drew

tools/testing/selftests/kvm/include/kvm_util.h +25 -0

index 29cccaf96baf..d96a072e69bf 100644
@@ -125,6 +125,31 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+
+static inline void
+vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_FPU, fpu);
+}
+
+static inline void
+vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_FPU, fpu);
+}
+
+static inline void
+vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_GET_ONE_REG, reg);
+}
+
+static inline void
+vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, reg);
+}
+
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

Quoted 98 lines · Andrew Jones

Quoted 11 lines · Janosch Frank

Quoted 10 lines · Andrew Jones

Quoted 9 lines · Thomas Huth

Quoted 8 lines · Janosch Frank

Add library access to more registers.

Signed-off-by Janosch Frank <[email protected]>

---
 .../testing/selftests/kvm/include/kvm_util.h  |  6 +++
 tools/testing/selftests/kvm/lib/kvm_util.c    | 48 +++++++++++++++++++
 2 files changed, 54 insertions(+)

Quoted 12 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

tools/testing/selftests/kvm/include/kvm_util.h +0 -0

index 29cccaf96baf..ae0d14c2540a 100644

Quoted 8 lines

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, · line 131

Quoted 8 lines

tools/testing/selftests/kvm/include/kvm_util.h +4 -0

@@ -125,6 +125,12 @@ void vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
 int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid,
 		    struct kvm_sregs *sregs);
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid,
+		  struct kvm_fpu *fpu);

nit: no need for the above line breaks. We don't even get to 80 char.

Quoted 71 lines

Quoted 70 lines

Quoted 59 lines

Quoted 59 lines

tools/testing/selftests/kvm/include/kvm_util.h +2 -0

+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg);
 #ifdef __KVM_HAVE_VCPU_EVENTS
 void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid,
 		     struct kvm_vcpu_events *events);

tools/testing/selftests/kvm/lib/kvm_util.c +46 -0

index 41cf45416060..dae117728ec6 100644
@@ -1373,6 +1373,54 @@ int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs)
 	return ioctl(vcpu->fd, KVM_SET_SREGS, sregs);
 }
 
+void vcpu_fpu_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_GET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_fpu_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_fpu *fpu)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_FPU, fpu);
+	TEST_ASSERT(ret == 0, "KVM_SET_FPU failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_get_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_GET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_GET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);
+}
+
+void vcpu_set_reg(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_one_reg *reg)
+{
+	struct vcpu *vcpu = vcpu_find(vm, vcpuid);
+	int ret;
+
+	TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid);
+
+	ret = ioctl(vcpu->fd, KVM_SET_ONE_REG, reg);
+	TEST_ASSERT(ret == 0, "KVM_SET_ONE_REG failed, rc: %i errno: %i",
+		    ret, errno);

Quoted 9 lines

tools/testing/selftests/kvm/lib/kvm_util.c int _vcpu_sregs_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_sregs *sregs) · line 1426

Quoted 6 lines

tools/testing/selftests/kvm/lib/kvm_util.c +2 -0

+}
+
 /*
  * VCPU Ioctl
  *

Reviewed-by Thomas Huth <[email protected]>

How about what's below instead. It should be equivalent.

Quoted 8 lines

Quoted 2 lines

With your proposed changes we loose a bit verbosity in the error messages. I need to think about which I like more.

Looks like both error messages are missing something. The ones above are missing the string version of errno. The ones below are missing the string version of cmd. It's easy to add the string version of errno, which is an argument for keeping the functions above (but we could at least use _vcpu_ioctl to avoid duplicating the vcpu_find and vcpu!=NULL assert).

Will do

Quoted 4 lines

Or, we could consider adding a kvm_ioctl_cmd_to_string() function, which might be nice for other ioctl wrappers now and in the future. It shouldn't be too bad to generate a string table from kvm.h, but of course we'd have to keep it maintained.

I'm currently occupied with managing a lot of patches, so something like that is not very high on my todo list.

Test if the registers end up having the correct values after a normal, initial and clear reset.

Signed-off-by Janosch Frank <[email protected]>

---
 tools/testing/selftests/kvm/Makefile       |   1 +
 tools/testing/selftests/kvm/s390x/resets.c | 165 +++++++++++++++++++++
 2 files changed, 166 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/s390x/resets.c

tools/testing/selftests/kvm/Makefile +1 -0

index 3138a916574a..fe1ea294730c 100644
@@ -36,6 +36,7 @@ TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
 
 TEST_GEN_PROGS_s390x = s390x/memop
 TEST_GEN_PROGS_s390x += s390x/sync_regs_test
+TEST_GEN_PROGS_s390x += s390x/resets
 TEST_GEN_PROGS_s390x += dirty_log_test
 TEST_GEN_PROGS_s390x += kvm_create_max_vcpus
 

tools/testing/selftests/kvm/s390x/resets.c new file +31 -0

new file mode 100644
index 000000000000..2b2378cc9e80
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Test for s390x CPU resets
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+
+#define VCPU_ID 3
+
+struct kvm_vm *vm;
+struct kvm_run *run;
+struct kvm_sync_regs *regs;
+static uint64_t regs_null[16];
+
+static uint64_t crs[16] = { 0x40000ULL,
+			    0x42000ULL,
+			    0, 0, 0, 0, 0,
+			    0x43000ULL,
+			    0, 0, 0, 0, 0,
+			    0x44000ULL,
+			    0, 0
+};
+

tools/testing/selftests/kvm/s390x/resets.c new file +55 -0

+}
+
+static void test_one_reg(uint64_t id, uint64_t value)
+{
+	struct kvm_one_reg reg;
+	uint64_t eval_reg;
+
+	reg.addr = (uintptr_t)&eval_reg;
+	reg.id = id;
+	vcpu_get_reg(vm, VCPU_ID, &reg);
+	TEST_ASSERT(eval_reg == value, "value == %s", value);
+}
+
+static void assert_clear(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_regs regs;
+	struct kvm_fpu fpu;
+
+	vcpu_regs_get(vm, VCPU_ID, &regs);
+	TEST_ASSERT(!memcmp(&regs.gprs, regs_null, sizeof(regs.gprs)), "grs == 0");
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(!memcmp(&sregs.acrs, regs_null, sizeof(sregs.acrs)), "acrs == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!memcmp(&fpu.fprs, regs_null, sizeof(fpu.fprs)), "fprs == 0");
+}
+
+static void assert_initial(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_fpu fpu;
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(sregs.crs[0] == 0xE0UL, "cr0 == 0xE0");
+	TEST_ASSERT(sregs.crs[14] == 0xC2000000UL, "cr14 == 0xC2000000");
+	TEST_ASSERT(!memcmp(&sregs.crs[1], regs_null, sizeof(sregs.crs[1]) * 12),
+		    "cr1-13 == 0");
+	TEST_ASSERT(sregs.crs[15] == 0, "cr15 == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!fpu.fpc, "fpc == 0");
+
+	test_one_reg(KVM_REG_S390_GBEA, 1);
+	test_one_reg(KVM_REG_S390_PP, 0);
+	test_one_reg(KVM_REG_S390_TODPR, 0);
+	test_one_reg(KVM_REG_S390_CPU_TIMER, 0);
+	test_one_reg(KVM_REG_S390_CLOCK_COMP, 0);
+}
+
+static void assert_normal(void)
+{
+	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+}

tools/testing/selftests/kvm/s390x/resets.c new file +6 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_NORMAL_RESET, 0);
+	assert_normal();
+	kvm_vm_free(vm);
+}
+
+static int test_initial(void)

tools/testing/selftests/kvm/s390x/resets.c new file +26 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_INITIAL_RESET, 0);
+	assert_normal();
+	assert_initial();
+	kvm_vm_free(vm);
+	return rv;
+}
+
+static int test_clear(void)
+{
+	int rv;
+
+	printf("Testing clear reset\n");
+	/* Create VM */
+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	rv = _vcpu_run(vm, VCPU_ID);
+
+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_CLEAR_RESET, 0);
+	assert_normal();
+	assert_initial();
+	assert_clear();
+	kvm_vm_free(vm);
+	return rv;
+}

Quoted 65 lines · Janosch Frank

Test if the registers end up having the correct values after a normal, initial and clear reset.

Signed-off-by Janosch Frank <[email protected]>

---
 tools/testing/selftests/kvm/Makefile       |   1 +
 tools/testing/selftests/kvm/s390x/resets.c | 165 +++++++++++++++++++++
 2 files changed, 166 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/s390x/resets.c

tools/testing/selftests/kvm/Makefile +1 -0

index 3138a916574a..fe1ea294730c 100644
@@ -36,6 +36,7 @@ TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
 
 TEST_GEN_PROGS_s390x = s390x/memop
 TEST_GEN_PROGS_s390x += s390x/sync_regs_test
+TEST_GEN_PROGS_s390x += s390x/resets
 TEST_GEN_PROGS_s390x += dirty_log_test
 TEST_GEN_PROGS_s390x += kvm_create_max_vcpus
 

tools/testing/selftests/kvm/s390x/resets.c new file +37 -0

new file mode 100644
index 000000000000..2b2378cc9e80
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Test for s390x CPU resets
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+
+#define VCPU_ID 3
+
+struct kvm_vm *vm;
+struct kvm_run *run;
+struct kvm_sync_regs *regs;
+static uint64_t regs_null[16];
+
+static uint64_t crs[16] = { 0x40000ULL,
+			    0x42000ULL,
+			    0, 0, 0, 0, 0,
+			    0x43000ULL,
+			    0, 0, 0, 0, 0,
+			    0x44000ULL,
+			    0, 0
+};
+
+static void guest_code_initial(void)
+{
+	/* Round toward 0 */
+	uint32_t fpc = 0x11;
+
+	/* Dirty registers */

tools/testing/selftests/kvm/s390x/resets.c line 41

Quoted 4 lines

tools/testing/selftests/kvm/s390x/resets.c new file +4 -0

+	asm volatile (
+		"	lctlg	0,15,%0\n"
+		"	sfpc	%1\n"
+		: : "Q" (crs), "d" (fpc));

I'd recommend to add a GUEST_SYNC(0) here ... otherwise the guest code tries to return from this function and will cause a crash - which will also finish execution of the guest, but might have unexpected side effects.

Quoted 60 lines

tools/testing/selftests/kvm/s390x/resets.c new file +59 -0

+}
+
+static void test_one_reg(uint64_t id, uint64_t value)
+{
+	struct kvm_one_reg reg;
+	uint64_t eval_reg;
+
+	reg.addr = (uintptr_t)&eval_reg;
+	reg.id = id;
+	vcpu_get_reg(vm, VCPU_ID, &reg);
+	TEST_ASSERT(eval_reg == value, "value == %s", value);
+}
+
+static void assert_clear(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_regs regs;
+	struct kvm_fpu fpu;
+
+	vcpu_regs_get(vm, VCPU_ID, &regs);
+	TEST_ASSERT(!memcmp(&regs.gprs, regs_null, sizeof(regs.gprs)), "grs == 0");
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(!memcmp(&sregs.acrs, regs_null, sizeof(sregs.acrs)), "acrs == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!memcmp(&fpu.fprs, regs_null, sizeof(fpu.fprs)), "fprs == 0");
+}
+
+static void assert_initial(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_fpu fpu;
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(sregs.crs[0] == 0xE0UL, "cr0 == 0xE0");
+	TEST_ASSERT(sregs.crs[14] == 0xC2000000UL, "cr14 == 0xC2000000");
+	TEST_ASSERT(!memcmp(&sregs.crs[1], regs_null, sizeof(sregs.crs[1]) * 12),
+		    "cr1-13 == 0");
+	TEST_ASSERT(sregs.crs[15] == 0, "cr15 == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!fpu.fpc, "fpc == 0");
+
+	test_one_reg(KVM_REG_S390_GBEA, 1);
+	test_one_reg(KVM_REG_S390_PP, 0);
+	test_one_reg(KVM_REG_S390_TODPR, 0);
+	test_one_reg(KVM_REG_S390_CPU_TIMER, 0);
+	test_one_reg(KVM_REG_S390_CLOCK_COMP, 0);
+}
+
+static void assert_normal(void)
+{
+	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+}
+
+static void test_normal(void)
+{
+	printf("Testing notmal reset\n");

tools/testing/selftests/kvm/s390x/resets.c line 106

Quoted 6 lines

tools/testing/selftests/kvm/s390x/resets.c new file +6 -0

+	/* Create VM */
+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	_vcpu_run(vm, VCPU_ID);

Could you use vcpu_run() instead of _vcpu_run() ?

Quoted 12 lines

tools/testing/selftests/kvm/s390x/resets.c new file +11 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_NORMAL_RESET, 0);
+	assert_normal();
+	kvm_vm_free(vm);
+}
+
+static int test_initial(void)
+{
+	int rv;
+
+	printf("Testing initial reset\n");
+	/* Create VM */

tools/testing/selftests/kvm/s390x/resets.c line 123

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c new file +5 -0

+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	rv = _vcpu_run(vm, VCPU_ID);

Extra bonus points if you check here that the registers contain the values that have been set by the guest ;-)

Quoted 32 lines

tools/testing/selftests/kvm/s390x/resets.c new file +31 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_INITIAL_RESET, 0);
+	assert_normal();
+	assert_initial();
+	kvm_vm_free(vm);
+	return rv;
+}
+
+static int test_clear(void)
+{
+	int rv;
+
+	printf("Testing clear reset\n");
+	/* Create VM */
+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	rv = _vcpu_run(vm, VCPU_ID);
+
+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_CLEAR_RESET, 0);
+	assert_normal();
+	assert_initial();
+	assert_clear();
+	kvm_vm_free(vm);
+	return rv;
+}
+
+int main(int argc, char *argv[])
+{
+	int addl_resets;
+

tools/testing/selftests/kvm/s390x/resets.c line 160

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c new file +5 -0

+	setbuf(stdout, NULL);	/* Tell stdout not to buffer its content */
+	addl_resets = kvm_check_cap(KVM_CAP_S390_VCPU_RESETS);
+
+	test_initial();
+	if (addl_resets) {

I think you could still fit this into one line, without the need to declare the addl_resets variable:

	if (kvm_check_cap(KVM_CAP_S390_VCPU_RESETS)) {

tools/testing/selftests/kvm/s390x/resets.c line 165

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c new file +5 -0

+		test_normal();
+		test_clear();
+	}
+	return 0;
+}

Apart from the nits, this looks pretty good already, thanks for putting it together!

Thomas

Quoted 66 lines · Thomas Huth

Quoted 65 lines · Janosch Frank

Test if the registers end up having the correct values after a normal, initial and clear reset.

Signed-off-by Janosch Frank <[email protected]>

---
 tools/testing/selftests/kvm/Makefile       |   1 +
 tools/testing/selftests/kvm/s390x/resets.c | 165 +++++++++++++++++++++
 2 files changed, 166 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/s390x/resets.c

tools/testing/selftests/kvm/Makefile +1 -0

index 3138a916574a..fe1ea294730c 100644
@@ -36,6 +36,7 @@ TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
 
 TEST_GEN_PROGS_s390x = s390x/memop
 TEST_GEN_PROGS_s390x += s390x/sync_regs_test
+TEST_GEN_PROGS_s390x += s390x/resets
 TEST_GEN_PROGS_s390x += dirty_log_test
 TEST_GEN_PROGS_s390x += kvm_create_max_vcpus
 

tools/testing/selftests/kvm/s390x/resets.c new file +37 -0

new file mode 100644
index 000000000000..2b2378cc9e80
@@ -0,0 +1,165 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Test for s390x CPU resets
+ *
+ * Copyright (C) 2020, IBM
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+
+#include "test_util.h"
+#include "kvm_util.h"
+
+#define VCPU_ID 3
+
+struct kvm_vm *vm;
+struct kvm_run *run;
+struct kvm_sync_regs *regs;
+static uint64_t regs_null[16];
+
+static uint64_t crs[16] = { 0x40000ULL,
+			    0x42000ULL,
+			    0, 0, 0, 0, 0,
+			    0x43000ULL,
+			    0, 0, 0, 0, 0,
+			    0x44000ULL,
+			    0, 0
+};
+
+static void guest_code_initial(void)
+{
+	/* Round toward 0 */
+	uint32_t fpc = 0x11;
+
+	/* Dirty registers */

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c line 41

Quoted 4 lines

tools/testing/selftests/kvm/s390x/resets.c new file +4 -0

+	asm volatile (
+		"	lctlg	0,15,%0\n"
+		"	sfpc	%1\n"
+		: : "Q" (crs), "d" (fpc));

I'd recommend to add a GUEST_SYNC(0) here ... otherwise the guest code tries to return from this function and will cause a crash - which will also finish execution of the guest, but might have unexpected side effects.

Ok

Quoted 60 lines

Quoted 60 lines

tools/testing/selftests/kvm/s390x/resets.c new file +59 -0

+}
+
+static void test_one_reg(uint64_t id, uint64_t value)
+{
+	struct kvm_one_reg reg;
+	uint64_t eval_reg;
+
+	reg.addr = (uintptr_t)&eval_reg;
+	reg.id = id;
+	vcpu_get_reg(vm, VCPU_ID, &reg);
+	TEST_ASSERT(eval_reg == value, "value == %s", value);
+}
+
+static void assert_clear(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_regs regs;
+	struct kvm_fpu fpu;
+
+	vcpu_regs_get(vm, VCPU_ID, &regs);
+	TEST_ASSERT(!memcmp(&regs.gprs, regs_null, sizeof(regs.gprs)), "grs == 0");
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(!memcmp(&sregs.acrs, regs_null, sizeof(sregs.acrs)), "acrs == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!memcmp(&fpu.fprs, regs_null, sizeof(fpu.fprs)), "fprs == 0");
+}
+
+static void assert_initial(void)
+{
+	struct kvm_sregs sregs;
+	struct kvm_fpu fpu;
+
+	vcpu_sregs_get(vm, VCPU_ID, &sregs);
+	TEST_ASSERT(sregs.crs[0] == 0xE0UL, "cr0 == 0xE0");
+	TEST_ASSERT(sregs.crs[14] == 0xC2000000UL, "cr14 == 0xC2000000");
+	TEST_ASSERT(!memcmp(&sregs.crs[1], regs_null, sizeof(sregs.crs[1]) * 12),
+		    "cr1-13 == 0");
+	TEST_ASSERT(sregs.crs[15] == 0, "cr15 == 0");
+
+	vcpu_fpu_get(vm, VCPU_ID, &fpu);
+	TEST_ASSERT(!fpu.fpc, "fpc == 0");
+
+	test_one_reg(KVM_REG_S390_GBEA, 1);
+	test_one_reg(KVM_REG_S390_PP, 0);
+	test_one_reg(KVM_REG_S390_TODPR, 0);
+	test_one_reg(KVM_REG_S390_CPU_TIMER, 0);
+	test_one_reg(KVM_REG_S390_CLOCK_COMP, 0);
+}
+
+static void assert_normal(void)
+{
+	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+}
+
+static void test_normal(void)
+{
+	printf("Testing notmal reset\n");

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c line 106

Quoted 6 lines

tools/testing/selftests/kvm/s390x/resets.c new file +6 -0

+	/* Create VM */
+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	_vcpu_run(vm, VCPU_ID);

Could you use vcpu_run() instead of _vcpu_run() ?

Done.

Quoted 12 lines

Quoted 12 lines

tools/testing/selftests/kvm/s390x/resets.c new file +11 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_NORMAL_RESET, 0);
+	assert_normal();
+	kvm_vm_free(vm);
+}
+
+static int test_initial(void)
+{
+	int rv;
+
+	printf("Testing initial reset\n");
+	/* Create VM */

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c line 123

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c new file +5 -0

+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	rv = _vcpu_run(vm, VCPU_ID);

Extra bonus points if you check here that the registers contain the values that have been set by the guest ;-)

I started working on that yesterday

Quoted 32 lines

Quoted 32 lines

tools/testing/selftests/kvm/s390x/resets.c new file +31 -0

+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_INITIAL_RESET, 0);
+	assert_normal();
+	assert_initial();
+	kvm_vm_free(vm);
+	return rv;
+}
+
+static int test_clear(void)
+{
+	int rv;
+
+	printf("Testing clear reset\n");
+	/* Create VM */
+	vm = vm_create_default(VCPU_ID, 0, guest_code_initial);
+	run = vcpu_state(vm, VCPU_ID);
+	regs = &run->s.regs;
+
+	rv = _vcpu_run(vm, VCPU_ID);
+
+	vcpu_ioctl(vm, VCPU_ID, KVM_S390_CLEAR_RESET, 0);
+	assert_normal();
+	assert_initial();
+	assert_clear();
+	kvm_vm_free(vm);
+	return rv;
+}
+
+int main(int argc, char *argv[])
+{
+	int addl_resets;
+

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c line 160

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c new file +5 -0

+	setbuf(stdout, NULL);	/* Tell stdout not to buffer its content */
+	addl_resets = kvm_check_cap(KVM_CAP_S390_VCPU_RESETS);
+
+	test_initial();
+	if (addl_resets) {

I think you could still fit this into one line, without the need to declare the addl_resets variable:

The other question is if we still need to check that if the test is bundled with the kernel anyway?

From: Pierre Morel <[email protected]>

Local IRQs are reset by a normal cpu reset. The initial cpu reset and the clear cpu reset, as superset of the normal reset, both clear the IRQs too.

Let's inject an interrupt to a vCPU before calling a reset and see if it is gone after the reset.

We choose to inject only an emergency interrupt at this point and can extend the test to other types of IRQs later.

Signed-off-by Pierre Morel <[email protected]>

tools/testing/selftests/kvm/s390x/resets.c +2 -0

+
+struct kvm_s390_irq buf[VCPU_ID + LOCAL_IRQS];
 
 struct kvm_vm *vm;
 struct kvm_run *run;

@@ -52,6 +55,29 @@ static void test_one_reg(uint64_t id, uint64_t value)
 	TEST_ASSERT(eval_reg == value, "value == %s", value);

tools/testing/selftests/kvm/s390x/resets.c +4 -0

+
+	TEST_ASSERT(!irqs, "IRQ pending");
+}
+
 static void assert_clear(void)
 {
 	struct kvm_sregs sregs;

tools/testing/selftests/kvm/s390x/resets.c +7 -0

+{
+	struct kvm_s390_irq_state irq_state;
+	struct kvm_s390_irq *irq = &buf[0];
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))

tools/testing/selftests/kvm/s390x/resets.c +8 -0

+		return errno;
+	}
+
+	return 0;
 }
 
 static void test_normal(void)

@@ -105,6 +156,8 @@ static void test_normal(void)
 
 	_vcpu_run(vm, VCPU_ID);
 
+	inject_irq(VCPU_ID);
+
 	vcpu_ioctl(vm, VCPU_ID, KVM_S390_NORMAL_RESET, 0);
 	assert_normal();
 	kvm_vm_free(vm);

@@ -122,6 +175,8 @@ static int test_initial(void)
 
 	rv = _vcpu_run(vm, VCPU_ID);
 
+	inject_irq(VCPU_ID);
+
 	vcpu_ioctl(vm, VCPU_ID, KVM_S390_INITIAL_RESET, 0);
 	assert_normal();
 	assert_initial();

Quoted 6 lines · On Wed, 29 Jan 2020 15:03:12 -0500 Janosch Frank <[email protected]> wrote:

From: Pierre Morel <[email protected]>

Local IRQs are reset by a normal cpu reset. The initial cpu reset and the clear cpu reset, as superset of the normal reset, both clear the IRQs too.

Quoted 7 lines

Let's inject an interrupt to a vCPU before calling a reset and see if it is gone after the reset.

We choose to inject only an emergency interrupt at this point and can extend the test to other types of IRQs later.

Signed-off-by Pierre Morel <[email protected]>

You probably should add your s-o-b here as well.

tools/testing/selftests/kvm/s390x/resets.c line 17

Quoted 13 lines

---
 tools/testing/selftests/kvm/s390x/resets.c | 57 ++++++++++++++++++++++
 1 file changed, 57 insertions(+)

tools/testing/selftests/kvm/s390x/resets.c +1 -0

index 2b2378cc9e80..299c1686f98c 100644
@@ -14,6 +14,9 @@
 #include "kvm_util.h"
 
 #define VCPU_ID 3
+#define LOCAL_IRQS 32

Why 32?

Quoted 7 lines

tools/testing/selftests/kvm/s390x/resets.c +2 -0

+
+struct kvm_s390_irq buf[VCPU_ID + LOCAL_IRQS];
 
 struct kvm_vm *vm;
 struct kvm_run *run;

@@ -52,6 +55,29 @@ static void test_one_reg(uint64_t id, uint64_t value)
 	TEST_ASSERT(eval_reg == value, "value == %s", value);

tools/testing/selftests/kvm/s390x/resets.c static void test_one_reg(uint64_t id, uint64_t value) · line 65

Quoted 10 lines

tools/testing/selftests/kvm/s390x/resets.c +8 -0

 }
 
+static void assert_noirq(void)
+{
+	struct kvm_s390_irq_state irq_state;
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return;

Might want to do a

irq_introspection_supported = (check stuff);

once for this test? Works fine as is, of course.

tools/testing/selftests/kvm/s390x/resets.c static void test_one_reg(uint64_t id, uint64_t value) · line 73

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c +8 -0

+
+	irq_state.len = sizeof(buf);
+	irq_state.buf = (unsigned long)buf;
+	irqs = _vcpu_ioctl(vm, VCPU_ID, KVM_S390_GET_IRQ_STATE, &irq_state);
+	/*
+	 * irqs contains the number of retrieved interrupts, apart from the
+	 * emergency call that should be cleared by the resets, there should be
+	 * none.

Even if there were any, they should have been cleared by the reset, right?

tools/testing/selftests/kvm/s390x/resets.c static void test_one_reg(uint64_t id, uint64_t value) · line 76

Quoted 4 lines

tools/testing/selftests/kvm/s390x/resets.c +3 -0

+	 */
+	if (irqs < 0)
+		printf("Error by getting IRQ: errno %d\n", errno);

"Error getting pending IRQs" ?

Quoted 9 lines

tools/testing/selftests/kvm/s390x/resets.c +4 -0

+
+	TEST_ASSERT(!irqs, "IRQ pending");
+}
+
 static void assert_clear(void)
 {
 	struct kvm_sregs sregs;

tools/testing/selftests/kvm/s390x/resets.c static void assert_initial(void) · line 125

Quoted 7 lines

tools/testing/selftests/kvm/s390x/resets.c +4 -0

@@ -93,6 +119,31 @@ static void assert_initial(void)
 static void assert_normal(void)
 {
 	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+	assert_noirq();
+}
+
+static int inject_irq(int cpu_id)

You never seem to check the return code.

Quoted 11 lines

tools/testing/selftests/kvm/s390x/resets.c +11 -0

+{
+	struct kvm_s390_irq_state irq_state;
+	struct kvm_s390_irq *irq = &buf[0];
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return 0;
+
+	/* Inject IRQ */
+	irq_state.len = sizeof(struct kvm_s390_irq);

tools/testing/selftests/kvm/s390x/resets.c static void assert_initial(void) · line 142

Quoted 6 lines

tools/testing/selftests/kvm/s390x/resets.c +6 -0

+	irq_state.buf = (unsigned long)buf;
+	irq->type = KVM_S390_INT_EMERGENCY;
+	irq->u.emerg.code = cpu_id;
+	irqs = _vcpu_ioctl(vm, cpu_id, KVM_S390_SET_IRQ_STATE, &irq_state);
+	if (irqs < 0) {
+		printf("Error by injecting INT_EMERGENCY: errno %d\n", errno);

"Error injecting EMERGENCY IRQ" ?

Quoted 27 lines

tools/testing/selftests/kvm/s390x/resets.c +8 -0

+		return errno;
+	}
+
+	return 0;
 }
 
 static void test_normal(void)

@@ -105,6 +156,8 @@ static void test_normal(void)
 
 	_vcpu_run(vm, VCPU_ID);
 
+	inject_irq(VCPU_ID);
+
 	vcpu_ioctl(vm, VCPU_ID, KVM_S390_NORMAL_RESET, 0);
 	assert_normal();
 	kvm_vm_free(vm);

@@ -122,6 +175,8 @@ static int test_initial(void)
 
 	rv = _vcpu_run(vm, VCPU_ID);
 
+	inject_irq(VCPU_ID);
+
 	vcpu_ioctl(vm, VCPU_ID, KVM_S390_INITIAL_RESET, 0);
 	assert_normal();
 	assert_initial();

tools/testing/selftests/kvm/s390x/resets.c static int test_clear(void) · line 203

Quoted 8 lines

tools/testing/selftests/kvm/s390x/resets.c +2 -0

@@ -141,6 +196,8 @@ static int test_clear(void)
 
 	rv = _vcpu_run(vm, VCPU_ID);
 
+	inject_irq(VCPU_ID);
+
 	vcpu_ioctl(vm, VCPU_ID, KVM_S390_CLEAR_RESET, 0);
 	assert_normal();
 	assert_initial();

On the whole, looks good to me.

Quoted 13 lines · Cornelia Huck

tools/testing/selftests/kvm/s390x/resets.c static void test_one_reg(uint64_t id, uint64_t value) · line 73

Quoted 7 lines · On Wed, 29 Jan 2020 15:03:12 -0500 Janosch Frank <[email protected]> wrote:

tools/testing/selftests/kvm/s390x/resets.c +7 -0

+	irq_state.len = sizeof(buf);
+	irq_state.buf = (unsigned long)buf;
+	irqs = _vcpu_ioctl(vm, VCPU_ID, KVM_S390_GET_IRQ_STATE, &irq_state);
+	/*
+	 * irqs contains the number of retrieved interrupts, apart from the
+	 * emergency call that should be cleared by the resets, there should be
+	 * none.  

Even if there were any, they should have been cleared by the reset, right?

Yes, that's what "there should be none" should actually express. I added the comment before sending out.

Quoted 7 lines

tools/testing/selftests/kvm/s390x/resets.c static void test_one_reg(uint64_t id, uint64_t value) · line 76

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c +3 -0

+	 */
+	if (irqs < 0)
+		printf("Error by getting IRQ: errno %d\n", errno);  

"Error getting pending IRQs" ?

"Could not fetch IRQs: errno %d\n" ?

Quoted 5 lines

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c +3 -0

+
+	TEST_ASSERT(!irqs, "IRQ pending");
+}

/*
* irqs contains the number of retrieved interrupts. Any interrupt
* (notably, the emergency call interrupt we have injected) should
* be cleared by the resets, so this should be 0.
*/

From: Pierre Morel <[email protected]>

Local IRQs are reset by a normal cpu reset. The initial cpu reset and the clear cpu reset, as superset of the normal reset, both clear the IRQs too.

Let's inject an interrupt to a vCPU before calling a reset and see if it is gone after the reset.

We choose to inject only an emergency interrupt at this point and can extend the test to other types of IRQs later.

Signed-off-by Pierre Morel <[email protected]>

---
 tools/testing/selftests/kvm/s390x/resets.c | 57 ++++++++++++++++++++++
 1 file changed, 57 insertions(+)

tools/testing/selftests/kvm/s390x/resets.c +39 -0

index 2b2378cc9e80..299c1686f98c 100644
@@ -14,6 +14,9 @@
 #include "kvm_util.h"
 
 #define VCPU_ID 3
+#define LOCAL_IRQS 32
+
+struct kvm_s390_irq buf[VCPU_ID + LOCAL_IRQS];
 
 struct kvm_vm *vm;
 struct kvm_run *run;

@@ -52,6 +55,29 @@ static void test_one_reg(uint64_t id, uint64_t value)
 	TEST_ASSERT(eval_reg == value, "value == %s", value);
 }
 
+static void assert_noirq(void)
+{
+	struct kvm_s390_irq_state irq_state;
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return;
+
+	irq_state.len = sizeof(buf);
+	irq_state.buf = (unsigned long)buf;
+	irqs = _vcpu_ioctl(vm, VCPU_ID, KVM_S390_GET_IRQ_STATE, &irq_state);
+	/*
+	 * irqs contains the number of retrieved interrupts, apart from the
+	 * emergency call that should be cleared by the resets, there should be
+	 * none.
+	 */
+	if (irqs < 0)
+		printf("Error by getting IRQ: errno %d\n", errno);
+
+	TEST_ASSERT(!irqs, "IRQ pending");
+}
+
 static void assert_clear(void)
 {
 	struct kvm_sregs sregs;

@@ -93,6 +119,31 @@ static void assert_initial(void)
 static void assert_normal(void)
 {
 	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+	assert_noirq();
+}
+
+static int inject_irq(int cpu_id)
+{
+	struct kvm_s390_irq_state irq_state;
+	struct kvm_s390_irq *irq = &buf[0];
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return 0;
+

Quoted 83 lines · Janosch Frank

From: Pierre Morel <[email protected]>

Local IRQs are reset by a normal cpu reset. The initial cpu reset and the clear cpu reset, as superset of the normal reset, both clear the IRQs too.

Let's inject an interrupt to a vCPU before calling a reset and see if it is gone after the reset.

We choose to inject only an emergency interrupt at this point and can extend the test to other types of IRQs later.

Signed-off-by Pierre Morel <[email protected]>

---
 tools/testing/selftests/kvm/s390x/resets.c | 57 ++++++++++++++++++++++
 1 file changed, 57 insertions(+)

tools/testing/selftests/kvm/s390x/resets.c +44 -0

index 2b2378cc9e80..299c1686f98c 100644
@@ -14,6 +14,9 @@
 #include "kvm_util.h"
 
 #define VCPU_ID 3
+#define LOCAL_IRQS 32
+
+struct kvm_s390_irq buf[VCPU_ID + LOCAL_IRQS];
 
 struct kvm_vm *vm;
 struct kvm_run *run;

@@ -52,6 +55,29 @@ static void test_one_reg(uint64_t id, uint64_t value)
 	TEST_ASSERT(eval_reg == value, "value == %s", value);
 }
 
+static void assert_noirq(void)
+{
+	struct kvm_s390_irq_state irq_state;
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return;
+
+	irq_state.len = sizeof(buf);
+	irq_state.buf = (unsigned long)buf;
+	irqs = _vcpu_ioctl(vm, VCPU_ID, KVM_S390_GET_IRQ_STATE, &irq_state);
+	/*
+	 * irqs contains the number of retrieved interrupts, apart from the
+	 * emergency call that should be cleared by the resets, there should be
+	 * none.
+	 */
+	if (irqs < 0)
+		printf("Error by getting IRQ: errno %d\n", errno);
+
+	TEST_ASSERT(!irqs, "IRQ pending");
+}
+
 static void assert_clear(void)
 {
 	struct kvm_sregs sregs;

@@ -93,6 +119,31 @@ static void assert_initial(void)
 static void assert_normal(void)
 {
 	test_one_reg(KVM_REG_S390_PFTOKEN, KVM_S390_PFAULT_TOKEN_INVALID);
+	assert_noirq();
+}
+
+static int inject_irq(int cpu_id)
+{
+	struct kvm_s390_irq_state irq_state;
+	struct kvm_s390_irq *irq = &buf[0];
+	int irqs;
+
+	if (!(kvm_check_cap(KVM_CAP_S390_INJECT_IRQ) &&
+	    kvm_check_cap(KVM_CAP_S390_IRQ_STATE)))
+		return 0;
+
+	/* Inject IRQ */
+	irq_state.len = sizeof(struct kvm_s390_irq);
+	irq_state.buf = (unsigned long)buf;
+	irq->type = KVM_S390_INT_EMERGENCY;
+	irq->u.emerg.code = cpu_id;

tools/testing/selftests/kvm/s390x/resets.c static void assert_initial(void) · line 144

Quoted 5 lines

tools/testing/selftests/kvm/s390x/resets.c +5 -0

+	irqs = _vcpu_ioctl(vm, cpu_id, KVM_S390_SET_IRQ_STATE, &irq_state);
+	if (irqs < 0) {
+		printf("Error by injecting INT_EMERGENCY: errno %d\n", errno);
+		return errno;
+	}

Can you turn this into a TEST_ASSERT() instead? Otherwise the printf() error might go unnoticed.

Apart from that (and the nits that Cornelia already mentioned), the patch looks fine to me.

Thomas