Skip to contentSkip to conversation

s390x: KVM: CPU Topology

PATCH Superseded Pierre Morel 3 participants 9 messages Jul 22, 2021 – Jul 23, 2021

Subsystems (3)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover s390x: KVM: CPU Topology

To provide Topology information to the guest through the STSI instruction, we need to forward STSI with Function Code 15 to QEMU which will take care to provide the right information to the guest.

To let the guest use the PTF instruction and ask if a topology change occured we add a new KVM capability to enable the topology facility.

Pierre Morel (2):
s390x: KVM: accept STSI for CPU topology information s390:kvm: Topology expose TOPOLOGY facility

 arch/s390/kvm/kvm-s390.c | 1 +
 arch/s390/kvm/priv.c     | 7 ++++++-
 include/uapi/linux/kvm.h | 1 +
 3 files changed, 8 insertions(+), 1 deletion(-)
Signature · 15 lines
2.25.1

Changelog:

- Add a KVM capability to let QEMU know we support PTF and STSI 15
  (David)

- check KVM facility 11 before accepting STSI fc 15
  (David)

- handle all we can in userland
  (David)

- add tracing to STSI fc 15
  (Connie)
#1 lore raw

PATCH v2 1/2 s390x: KVM: accept STSI for CPU topology information

STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.

---
 arch/s390/kvm/priv.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
arch/s390/kvm/priv.c +6 -1
index 9928f785c677..8581b6881212 100644
@@ -856,7 +856,8 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	if (fc > 3) {
+	if ((fc > 3 && fc != 15) ||
+	    (fc == 15 && !test_kvm_facility(vcpu->kvm, 11))) {
 		kvm_s390_set_psw_cc(vcpu, 3);
 		return 0;
 	}
@@ -893,6 +894,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
 			goto out_no_data;
 		handle_stsi_3_2_2(vcpu, (void *) mem);
 		break;
+	case 15:
+		trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
+		insert_stsi_usr_data(vcpu, operand2, ar, fc, sel1, sel2);
+		return -EREMOTE;
 	}
 	if (kvm_s390_pv_cpu_is_protected(vcpu)) {
 		memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH v2 1/2] s390x: KVM: accept STSI for CPU topology information

Quoted 15 lines · Pierre Morel

STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.

---
  arch/s390/kvm/priv.c | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
arch/s390/kvm/priv.c +0 -0
index 9928f785c677..8581b6881212 100644

arch/s390/kvm/priv.c static int handle_stsi(struct kvm_vcpu *vcpu) · line 860

Quoted 6 lines
arch/s390/kvm/priv.c +2 -1
@@ -856,7 +856,8 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
  	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
  		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
 
-	if (fc > 3) {
+	if ((fc > 3 && fc != 15) ||
+	    (fc == 15 && !test_kvm_facility(vcpu->kvm, 11))) {

Especially as you use test_kvm_facility here you really want to have the cpu model in patch 2 (and not CAP).

Quoted 15 lines
arch/s390/kvm/priv.c +4 -0
  		kvm_s390_set_psw_cc(vcpu, 3);
  		return 0;
  	}
@@ -893,6 +894,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
  			goto out_no_data;
  		handle_stsi_3_2_2(vcpu, (void *) mem);
  		break;
+	case 15:
+		trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
+		insert_stsi_usr_data(vcpu, operand2, ar, fc, sel1, sel2);
+		return -EREMOTE;
  	}
  	if (kvm_s390_pv_cpu_is_protected(vcpu)) {
  		memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,
#3 lore raw

PATCH v2 2/2 s390:kvm: Topology expose TOPOLOGY facility

In reply to Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

---
 arch/s390/kvm/kvm-s390.c | 1 +
 include/uapi/linux/kvm.h | 1 +
 2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_S390_VCPU_RESETS:
 	case KVM_CAP_SET_GUEST_DEBUG:
 	case KVM_CAP_S390_DIAG318:
+	case KVM_CAP_S390_CPU_TOPOLOGY:
 		r = 1;
 		break;
 	case KVM_CAP_SET_GUEST_DEBUG2:
include/uapi/linux/kvm.h +1 -0
index d9e4aabcb31a..081ce0cd44b9 100644
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_BINARY_STATS_FD 203
 #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
 #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
Signature · 1 line
2.25.1
#4 lore raw

Re: [PATCH v2 2/2] s390:kvm: Topology expose TOPOLOGY facility

Quoted 2 lines · Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

I think the old variant of using the CPU model was actually better. It was just the patch description that was wrong.

Quoted 31 lines
---
  arch/s390/kvm/kvm-s390.c | 1 +
  include/uapi/linux/kvm.h | 1 +
  2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
  	case KVM_CAP_S390_VCPU_RESETS:
  	case KVM_CAP_SET_GUEST_DEBUG:
  	case KVM_CAP_S390_DIAG318:
+	case KVM_CAP_S390_CPU_TOPOLOGY:
  		r = 1;
  		break;
  	case KVM_CAP_SET_GUEST_DEBUG2:
include/uapi/linux/kvm.h +1 -0
index d9e4aabcb31a..081ce0cd44b9 100644
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
  #define KVM_CAP_BINARY_STATS_FD 203
  #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
  #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
 
  #ifdef KVM_CAP_IRQ_ROUTING
 
#5 lore raw

Re: [PATCH v2 2/2] s390:kvm: Topology expose TOPOLOGY facility

Quoted 6 lines · Christian Borntraeger
Quoted 2 lines · Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

I think the old variant of using the CPU model was actually better. It was just the patch description that was wrong.

I thought we wanted a cap that userspace can enable to get ptf intercepts? I'm confused.

Quoted 29 lines
Quoted 29 lines
---
  arch/s390/kvm/kvm-s390.c | 1 +
  include/uapi/linux/kvm.h | 1 +
  2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
  	case KVM_CAP_S390_VCPU_RESETS:
  	case KVM_CAP_SET_GUEST_DEBUG:
  	case KVM_CAP_S390_DIAG318:
+	case KVM_CAP_S390_CPU_TOPOLOGY:
  		r = 1;
  		break;
  	case KVM_CAP_SET_GUEST_DEBUG2:
include/uapi/linux/kvm.h +1 -0
index d9e4aabcb31a..081ce0cd44b9 100644
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
  #define KVM_CAP_BINARY_STATS_FD 203
  #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
  #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
 
Quoted 3 lines

include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1118

Quoted 3 lines
include/uapi/linux/kvm.h +0 -0
  #ifdef KVM_CAP_IRQ_ROUTING
 

Regardless of what we end up with: we need documentation for any new cap :)

#6 lore raw

Re: [PATCH v2 2/2] s390:kvm: Topology expose TOPOLOGY facility

Quoted 12 lines · Cornelia Huck
Quoted 6 lines · Christian Borntraeger
Quoted 2 lines · Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

I think the old variant of using the CPU model was actually better. It was just the patch description that was wrong.

I thought we wanted a cap that userspace can enable to get ptf intercepts? I'm confused.

PTF goes to userspace in any case as every instruction that is not handled by kvm and where interpretion is not enabled. Now, having said that, we actually want PTF interpretion to be enabled for "Check topology-change status" as this is supposed to be a fast operation. Some OSes do query that in their interrupt handlers.

Quoted 36 lines
Quoted 29 lines
Quoted 29 lines
---
   arch/s390/kvm/kvm-s390.c | 1 +
   include/uapi/linux/kvm.h | 1 +
   2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
   	case KVM_CAP_S390_VCPU_RESETS:
   	case KVM_CAP_SET_GUEST_DEBUG:
   	case KVM_CAP_S390_DIAG318:
+	case KVM_CAP_S390_CPU_TOPOLOGY:
   		r = 1;
   		break;
   	case KVM_CAP_SET_GUEST_DEBUG2:
include/uapi/linux/kvm.h +1 -0
index d9e4aabcb31a..081ce0cd44b9 100644
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
   #define KVM_CAP_BINARY_STATS_FD 203
   #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
   #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
 
Quoted 3 lines

include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1118

Quoted 3 lines
include/uapi/linux/kvm.h +0 -0
   #ifdef KVM_CAP_IRQ_ROUTING
 

Regardless of what we end up with: we need documentation for any new cap :)

#7 lore raw

Re: [PATCH v2 2/2] s390:kvm: Topology expose TOPOLOGY facility

Quoted 16 lines · Christian Borntraeger
Quoted 12 lines · Cornelia Huck
Quoted 6 lines · Christian Borntraeger
Quoted 2 lines · Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

I think the old variant of using the CPU model was actually better. It was just the patch description that was wrong.

I thought we wanted a cap that userspace can enable to get ptf intercepts? I'm confused.

Quoted 6 lines

PTF goes to userspace in any case as every instruction that is not handled by kvm and where interpretion is not enabled. Now, having said that, we actually want PTF interpretion to be enabled for "Check topology-change status" as this is supposed to be a fast operation. Some OSes do query that in their interrupt handlers.

An old QEMU getting the PTF instruction will send a OPERATION exception to the guest if the facility 11 is actzivated. Facility 11 is in QEMU since GAEN10_GA1, if I enable the facility in the CPU model all cpu model starting with GEN10_GA1 will panic on PTF.

So I think we need the capability so that new QEMU enable the facility once it has the right handling for PTF.

Quoted 35 lines
Quoted 35 lines
Quoted 23 lines
Quoted 23 lines

---
   arch/s390/kvm/kvm-s390.c | 1 +
   include/uapi/linux/kvm.h | 1 +

   2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +0 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm 
 *kvm, long ext)

       case KVM_CAP_S390_VCPU_RESETS:
       case KVM_CAP_SET_GUEST_DEBUG:
       case KVM_CAP_S390_DIAG318:
+    case KVM_CAP_S390_CPU_TOPOLOGY:
           r = 1;
           break;
       case KVM_CAP_SET_GUEST_DEBUG2:

include/uapi/linux/kvm.h +0 -1
index d9e4aabcb31a..081ce0cd44b9 100644
--- a/include/uapi/linux/kvm.h
Quoted 8 lines

include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1117

Quoted 8 lines

+++ b/include/uapi/linux/kvm.h

include/uapi/linux/kvm.h +1 -0
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
    #define KVM_CAP_BINARY_STATS_FD 203
    #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
    #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
    #ifdef KVM_CAP_IRQ_ROUTING
 

Regardless of what we end up with: we need documentation for any new cap :)

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#8 lore raw

Re: [PATCH v2 2/2] s390:kvm: Topology expose TOPOLOGY facility

Quoted 26 lines · Pierre Morel
Quoted 16 lines · Christian Borntraeger
Quoted 12 lines · Cornelia Huck
Quoted 6 lines · Christian Borntraeger
Quoted 2 lines · Pierre Morel

We add a KVM extension KVM_CAP_S390_CPU_TOPOLOGY to tell the userland hypervisor it is safe to activate the CPU Topology facility.

I think the old variant of using the CPU model was actually better. It was just the patch description that was wrong.

I thought we wanted a cap that userspace can enable to get ptf intercepts? I'm confused.

Quoted 6 lines

PTF goes to userspace in any case as every instruction that is not handled by kvm and where interpretion is not enabled. Now, having said that, we actually want PTF interpretion to be enabled for "Check topology-change status" as this is supposed to be a fast operation. Some OSes do query that in their interrupt handlers.

Quoted 2 lines

An old QEMU getting the PTF instruction will send a OPERATION exception to the guest if the facility 11 is actzivated. Facility 11 is in QEMU since GAEN10_GA1, if I enable the facility in the CPU model all cpu model starting with GEN10_GA1 will panic on PTF.

Hmm, if qemu enables facility 11 this would be a bug in QEMU. And indeed that seems to be the case here. So yes, the proper solution (cpu model) does not work unfortunately.

As I said, we want PTF interpretion enabled in KVM anyway as PTF FC==2 Check topology-change status is performance critical.
So this needs some bigger rework still. For example QEMU must then be able to tell KVM to tell SIE that the interpreted PTF will indicate a topology change.

Quoted 38 lines

So I think we need the capability so that new QEMU enable the facility once it has the right handling for PTF.

Quoted 34 lines
Quoted 34 lines
Quoted 22 lines
Quoted 22 lines

---
   arch/s390/kvm/kvm-s390.c | 1 +
   include/uapi/linux/kvm.h | 1 +

   2 files changed, 2 insertions(+)
arch/s390/kvm/kvm-s390.c +0 -0
index b655a7d82bf0..8c695ee79612 100644
@@ -568,6 +568,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)

       case KVM_CAP_S390_VCPU_RESETS:
       case KVM_CAP_SET_GUEST_DEBUG:
       case KVM_CAP_S390_DIAG318:
+    case KVM_CAP_S390_CPU_TOPOLOGY:
           r = 1;
           break;
       case KVM_CAP_SET_GUEST_DEBUG2:

include/uapi/linux/kvm.h +0 -1
index d9e4aabcb31a..081ce0cd44b9 100644
--- a/include/uapi/linux/kvm.h
Quoted 8 lines

include/uapi/linux/kvm.h struct kvm_ppc_resize_hpt { · line 1117

Quoted 8 lines

+++ b/include/uapi/linux/kvm.h

include/uapi/linux/kvm.h +1 -0
@@ -1112,6 +1112,7 @@ struct kvm_ppc_resize_hpt {
    #define KVM_CAP_BINARY_STATS_FD 203
    #define KVM_CAP_EXIT_ON_EMULATION_FAILURE 204
    #define KVM_CAP_ARM_MTE 205
+#define KVM_CAP_S390_CPU_TOPOLOGY 206
    #ifdef KVM_CAP_IRQ_ROUTING
 

Regardless of what we end up with: we need documentation for any new cap :)