s390x: KVM: CPU Topology
Subsystems (4)
v10 cover s390x: KVM: CPU Topology
Hi all,
This new spin suppress the check for real cpu migration and modify the checking of valid function code inside the interception of the STSI instruction.
The series provides:
0- Modification of the ipte lock handling to use KVM instead of the
vcpu as an argument because ipte lock work on SCA which is uniq per KVM structure and common to all vCPUs.
1- interception of the STSI instruction forwarding the CPU topology 2- interpretation of the PTF instruction
3- a KVM capability for the userland hypervisor to ask KVM to
setup PTF interpretation.
4- KVM ioctl to get and set the MTCR bit of the SCA in order to
migrate this bit during a migration.
0- Foreword
The S390 CPU topology is reported using two instructions:
- PTF, to get information if the CPU topology did change since last PTF instruction or a subsystem reset.
- STSI, to get the topology information, consisting of the topology of the CPU inside the sockets, of the sockets inside the books etc.
The PTF(2) instruction report a change if the STSI(15.1.2) instruction will report a difference with the last STSI(15.1.2) instruction*. With the SIE interpretation, the PTF(2) instruction will report a change to the guest if the host sets the SCA.MTCR bit.
*The STSI(15.1.2) instruction reports:
- The cores address within a socket
- The polarization of the cores
- The CPU type of the cores
- If the cores are dedicated or not
We decided to implement the CPU topology for S390 in several steps:
- first we report CPU hotplug
In future development we will provide:
- modification of the CPU mask inside sockets
- handling of shared CPUs
- reporting of the CPU Type
- reporting of the polarization
1- Interception of STSI
To provide Topology information to the guest through the STSI instruction, we forward STSI with Function Code 15 to the userland hypervisor which will take care to provide the right information to the guest.
To let the guest use both the PTF instruction to check if a topology change occurred and sthe STSI_15.x.x instruction we add a new KVM capability to enable the topology facility.
2- Interpretation of PTF with FC(2)
The PTF instruction reports a topology change if there is any change with a previous STSI(15.1.2) SYSIB.
Changes inside a STSI(15.1.2) SYSIB occur if CPU bits are set or clear inside the CPU Topology List Entry CPU mask field, which happens with changes in CPU polarization, dedication, CPU types and adding or removing CPUs in a socket.
Considering that the KVM guests currently only supports:
- horizontal polarization
- type 3 (Linux) CPU
And that we decide to support only:
- dedicated CPUs on the host
- pinned vCPUs on the guest
the creation of vCPU will is the only trigger to set the MTCR bit for a guest.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry of the guest's SCA which will be cleared during the interpretation of PTF.
Regards,
Pierre
Pierre Morel (3):
KVM: s390: ipte lock for SCA access should be contained in KVM
KVM: s390: guest support for topology function
KVM: s390: resetting the Topology-Change-Report
Documentation/virt/kvm/api.rst | 31 ++++++++ arch/s390/include/asm/kvm_host.h | 11 ++- arch/s390/include/uapi/asm/kvm.h | 10 +++ arch/s390/kvm/gaccess.c | 96 ++++++++++++------------ arch/s390/kvm/gaccess.h | 6 +- arch/s390/kvm/kvm-s390.c | 123 ++++++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 21 ++++-- arch/s390/kvm/vsie.c | 3 + include/uapi/linux/kvm.h | 1 + 9 files changed, 240 insertions(+), 62 deletions(-)
Signature · 96 lines
2.31.1 Changelog: from v9 to v10 - Suppression of the check on real CPU migration (Christian) - Changed the check on fc in handle_stsi (David) from v8 to v9 - bug correction in kvm_s390_topology_changed (Heiko) - simplification for ipte_lock/unlock to use kvm as arg instead of vcpu and test on sclp.has_siif instead of the SIE ECA_SII. (David) - use of a single value for reporting if the topology changed instead of a structure (David) from v7 to v8 - implement reset handling (Janosch) - change the way to check if the topology changed (Nico, Heiko) from v6 to v7 - rebase from v5 to v6 - make the subject more accurate (Claudio) - Change the kvm_s390_set_mtcr() function to have vcpu in the name (Janosch) - Replace the checks on ECB_PTF wit the check of facility 11 (Janosch) - modify kvm_arch_vcpu_load, move the check in a function in the header file (Janosh) - No magical number replace the "new cpu value" of -1 with a define (Janosch) - Make the checks for STSI validity clearer (Janosch) from v4 tp v5 - modify the way KVM_CAP is tested to be OK with vsie (David) from v3 to v4 - squatch both patches (David) - Added Documentation (David) - Modified the detection for new vCPUs (Pierre) from v2 to v3 - use PTF interpretation (Christian) - optimize arch_update_cpu_topology using PTF (Pierre) from v1 to v2: - Add a KVM capability to let QEMU know we support PTF and STSI 15 (David) - check KVM facility 11 before accepting STSI fc 15 (David) - handle all we can in userland (David) - add tracing to STSI fc 15 (Connie)
PATCH v10 1/3 KVM: s390: ipte lock for SCA access should be contained in KVM
We can check if SIIF is enabled by testing the sclp_info struct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to set ECA_SII anyway so we can go straight to the source for that.
--- arch/s390/kvm/gaccess.c | 96 ++++++++++++++++++++--------------------- arch/s390/kvm/gaccess.h | 6 +-- arch/s390/kvm/priv.c | 6 +-- 3 files changed, 54 insertions(+), 54 deletions(-)
arch/s390/kvm/gaccess.c +48 -48
@@ -262,77 +262,77 @@ struct aste {/* .. more fields there */ }; -int ipte_lock_held (struct kvm_vcpu *vcpu) +int ipte_lock_held (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) { +if (sclp.has_siif) {int rc; -read_lock (&vcpu->kvm->arch.sca_lock); - rc =kvm_s390_get_ipte_control (vcpu->kvm)->kh !=0 ; -read_unlock (&vcpu->kvm->arch.sca_lock); +read_lock (&kvm->arch.sca_lock); + rc =kvm_s390_get_ipte_control (kvm)->kh !=0 ; +read_unlock (&kvm->arch.sca_lock);return rc; } -return vcpu->kvm->arch.ipte_lock_count !=0 ; +return kvm->arch.ipte_lock_count !=0 ; } -static void ipte_lock_simple (struct kvm_vcpu *vcpu) +static void ipte_lock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count++; -if (vcpu->kvm->arch.ipte_lock_count >1 ) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count++; +if (kvm->arch.ipte_lock_count >1 ) goto out; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic);if (old.k) { -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);cond_resched (); goto retry; } new = old; new.k =1 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_unlock_simple (struct kvm_vcpu *vcpu) +static void ipte_unlock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count--; -if (vcpu->kvm->arch.ipte_lock_count) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count--; +if (kvm->arch.ipte_lock_count) goto out; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic); new = old; new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); -wake_up (&vcpu->kvm->arch.ipte_wq); +read_unlock (&kvm->arch.sca_lock); +wake_up (&kvm->arch.ipte_wq); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_lock_siif (struct kvm_vcpu *vcpu) +static void ipte_lock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); if (old.kg) { - read_unlock(&vcpu->kvm->arch.sca_lock); + read_unlock(&kvm->arch.sca_lock); cond_resched(); goto retry; }
@@ -340,15 +340,15 @@ static void ipte_lock_siif(struct kvm_vcpu *vcpu) new.k =1 ; new.kh++; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); } -static void ipte_unlock_siif (struct kvm_vcpu *vcpu) +static void ipte_unlock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); new = old;
@@ -356,25 +356,25 @@ static void ipte_unlock_siif(struct kvm_vcpu *vcpu)if (!new.kh) new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);if (!new.kh) -wake_up (&vcpu->kvm->arch.ipte_wq); +wake_up (&kvm->arch.ipte_wq); } -void ipte_lock (struct kvm_vcpu *vcpu) +void ipte_lock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_lock_siif (vcpu); +if (sclp.has_siif) +ipte_lock_siif (kvm);else -ipte_lock_simple (vcpu); +ipte_lock_simple (kvm); } -void ipte_unlock (struct kvm_vcpu *vcpu) +void ipte_unlock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_unlock_siif (vcpu); +if (sclp.has_siif) +ipte_unlock_siif (kvm);else -ipte_unlock_simple (vcpu); +ipte_unlock_simple (kvm); } static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, u8 ar,
@@ -1086,7 +1086,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, try_storage_prot_override =storage_prot_override_applicable (vcpu); need_ipte_lock =psw_bits (*psw).dat && !asce.r;if (need_ipte_lock) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm);/* * Since we do the access further down ultimately via a move instruction * that does key checking and returns an error in case of a protection
@@ -1127,7 +1127,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, } out_unlock:if (need_ipte_lock) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);if (nr_pages >ARRAY_SIZE (gpa_array))vfree (gpas);return rc;
@@ -1199,10 +1199,10 @@ int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, u8 ar, rc =get_vcpu_asce (vcpu, &asce, gva, ar, mode);if (rc)return rc; -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =guest_range_to_gpas (vcpu, gva, ar,NULL , length, asce, mode, access_key); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return rc; }
@@ -1465,7 +1465,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg,* tables/pointers we read stay valid - unshadowing is however * always possible - only guest_table_lock protects us. */ -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =gmap_shadow_pgt_lookup (sg, saddr, &pgt, &dat_protection, &fake);if (rc)
@@ -1499,7 +1499,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg, pte.p |= dat_protection;if (!rc) rc =gmap_shadow_page (sg, saddr,__pte (pte.val)); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);mmap_read_unlock (sg->mm);return rc; }
arch/s390/kvm/gaccess.h +3 -3
@@ -440,9 +440,9 @@ int read_guest_real(struct kvm_vcpu *vcpu, unsigned long gra, void *data,return access_guest_real (vcpu, gra, data, len,0 ); } -void ipte_lock (struct kvm_vcpu *vcpu); -void ipte_unlock (struct kvm_vcpu *vcpu); -int ipte_lock_held (struct kvm_vcpu *vcpu); +void ipte_lock (struct kvm *kvm); +void ipte_unlock (struct kvm *kvm); +int ipte_lock_held (struct kvm *kvm);int kvm_s390_check_low_addr_prot_real (struct kvm_vcpu *vcpu,unsigned long gra);/* MVPG PEI indication bits */
arch/s390/kvm/priv.c +3 -3
@@ -442,7 +442,7 @@ static int handle_ipte_interlock(struct kvm_vcpu *vcpu) vcpu->stat.instruction_ipte_interlock++;if (psw_bits (vcpu->arch.sie_block->gpsw).pstate)return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu)); +wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu->kvm));kvm_s390_retry_instr (vcpu);VCPU_EVENT (vcpu,4 ,"%s" ,"retrying ipte interlock operation" );return 0 ;
@@ -1471,7 +1471,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) access_key = (operand2 &0xf0 ) >>4 ;if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); ret =guest_translate_address_with_key (vcpu, address, ar, &gpa,GACC_STORE , access_key);
@@ -1508,7 +1508,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) }if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return ret; }
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v10 1/3] KVM: s390: ipte lock for SCA access should be contained in KVM
Quoted 4 lines · Pierre Morel
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to setECA_SIIanyway so we can go straight to the source for that.
The subject and commit description don't fit together. You're doing two things in this patch and only describe one of them.
I'd suggest something like this:
KVM: s390: Cleanup ipte lock access and SIIF facility checks
We can check if SIIF is enabled by testing the sclp_info struct instead of testing the sie control block eca variable as that facility is always enabled if available.
Also let's cleanup all the ipte related struct member accesses which currently happen by referencing the KVM struct via the VCPU struct. Making the KVM struct the parameter to the ipte_* functions removes one level of indirection which makes the code more readable.
Other than that I'm happy with this patch.
Quoted 273 lines
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by David Hildenbrand <[email protected]>--- arch/s390/kvm/gaccess.c | 96 ++++++++++++++++++++--------------------- arch/s390/kvm/gaccess.h | 6 +-- arch/s390/kvm/priv.c | 6 +-- 3 files changed, 54 insertions(+), 54 deletions(-)arch/s390/kvm/gaccess.c +48
-48@@ -262,77 +262,77 @@ struct aste {/* .. more fields there */ }; -int ipte_lock_held (struct kvm_vcpu *vcpu) +int ipte_lock_held (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) { +if (sclp.has_siif) {int rc; -read_lock (&vcpu->kvm->arch.sca_lock); - rc =kvm_s390_get_ipte_control (vcpu->kvm)->kh !=0 ; -read_unlock (&vcpu->kvm->arch.sca_lock); +read_lock (&kvm->arch.sca_lock); + rc =kvm_s390_get_ipte_control (kvm)->kh !=0 ; +read_unlock (&kvm->arch.sca_lock);return rc; } -return vcpu->kvm->arch.ipte_lock_count !=0 ; +return kvm->arch.ipte_lock_count !=0 ; } -static void ipte_lock_simple (struct kvm_vcpu *vcpu) +static void ipte_lock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count++; -if (vcpu->kvm->arch.ipte_lock_count >1 ) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count++; +if (kvm->arch.ipte_lock_count >1 ) goto out; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic);if (old.k) { -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);cond_resched (); goto retry; } new = old; new.k =1 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_unlock_simple (struct kvm_vcpu *vcpu) +static void ipte_unlock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count--; -if (vcpu->kvm->arch.ipte_lock_count) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count--; +if (kvm->arch.ipte_lock_count) goto out; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic); new = old; new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); -wake_up (&vcpu->kvm->arch.ipte_wq); +read_unlock (&kvm->arch.sca_lock); +wake_up (&kvm->arch.ipte_wq); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_lock_siif (struct kvm_vcpu *vcpu) +static void ipte_lock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); if (old.kg) { - read_unlock(&vcpu->kvm->arch.sca_lock); + read_unlock(&kvm->arch.sca_lock); cond_resched(); goto retry; }@@ -340,15 +340,15 @@ static void ipte_lock_siif(struct kvm_vcpu *vcpu) new.k =1 ; new.kh++; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); } -static void ipte_unlock_siif (struct kvm_vcpu *vcpu) +static void ipte_unlock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); new = old;@@ -356,25 +356,25 @@ static void ipte_unlock_siif(struct kvm_vcpu *vcpu)if (!new.kh) new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);if (!new.kh) -wake_up (&vcpu->kvm->arch.ipte_wq); +wake_up (&kvm->arch.ipte_wq); } -void ipte_lock (struct kvm_vcpu *vcpu) +void ipte_lock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_lock_siif (vcpu); +if (sclp.has_siif) +ipte_lock_siif (kvm);else -ipte_lock_simple (vcpu); +ipte_lock_simple (kvm); } -void ipte_unlock (struct kvm_vcpu *vcpu) +void ipte_unlock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_unlock_siif (vcpu); +if (sclp.has_siif) +ipte_unlock_siif (kvm);else -ipte_unlock_simple (vcpu); +ipte_unlock_simple (kvm); } static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, u8 ar,@@ -1086,7 +1086,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, try_storage_prot_override =storage_prot_override_applicable (vcpu); need_ipte_lock =psw_bits (*psw).dat && !asce.r;if (need_ipte_lock) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm);/* * Since we do the access further down ultimately via a move instruction * that does key checking and returns an error in case of a protection @@ -1127,7 +1127,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, } out_unlock:if (need_ipte_lock) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);if (nr_pages >ARRAY_SIZE (gpa_array))vfree (gpas);return rc;@@ -1199,10 +1199,10 @@ int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, u8 ar, rc =get_vcpu_asce (vcpu, &asce, gva, ar, mode);if (rc)return rc; -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =guest_range_to_gpas (vcpu, gva, ar,NULL , length, asce, mode, access_key); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return rc; }@@ -1465,7 +1465,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg,* tables/pointers we read stay valid - unshadowing is however * always possible - only guest_table_lock protects us. */ -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =gmap_shadow_pgt_lookup (sg, saddr, &pgt, &dat_protection, &fake);if (rc)@@ -1499,7 +1499,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg, pte.p |= dat_protection;if (!rc) rc =gmap_shadow_page (sg, saddr,__pte (pte.val)); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);mmap_read_unlock (sg->mm);return rc; }arch/s390/kvm/gaccess.h +3
-3@@ -440,9 +440,9 @@ int read_guest_real(struct kvm_vcpu *vcpu, unsigned long gra, void *data,return access_guest_real (vcpu, gra, data, len,0 ); } -void ipte_lock (struct kvm_vcpu *vcpu); -void ipte_unlock (struct kvm_vcpu *vcpu); -int ipte_lock_held (struct kvm_vcpu *vcpu); +void ipte_lock (struct kvm *kvm); +void ipte_unlock (struct kvm *kvm); +int ipte_lock_held (struct kvm *kvm);int kvm_s390_check_low_addr_prot_real (struct kvm_vcpu *vcpu,unsigned long gra);/* MVPG PEI indication bits */ arch/s390/kvm/priv.c +3
-3@@ -442,7 +442,7 @@ static int handle_ipte_interlock(struct kvm_vcpu *vcpu) vcpu->stat.instruction_ipte_interlock++;if (psw_bits (vcpu->arch.sie_block->gpsw).pstate)return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu)); +wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu->kvm));kvm_s390_retry_instr (vcpu);VCPU_EVENT (vcpu,4 ,"%s" ,"retrying ipte interlock operation" );return 0 ;@@ -1471,7 +1471,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) access_key = (operand2 &0xf0 ) >>4 ;if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); ret =guest_translate_address_with_key (vcpu, address, ar, &gpa,GACC_STORE , access_key);@@ -1508,7 +1508,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) }if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return ret; }
Re: [PATCH v10 1/3] KVM: s390: ipte lock for SCA access should be contained in KVM
Quoted 10 lines · Janosch Frank
Quoted 4 lines · Pierre Morel
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to setECA_SIIanyway so we can go straight to the source for that.The subject and commit description don't fit together. You're doing two things in this patch and only describe one of them.
Quoted 8 lines
I'd suggest something like this:
KVM: s390: Cleanup ipte lock access and SIIF facility checks
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable as that facility is always enabled if available.Also let's cleanup all the ipte related struct member accesses which currently happen by referencing the KVM struct via the VCPU struct. Making the KVM struct the parameter to the ipte_* functions removes one level of indirection which makes the code more readable.
OK done.
Quoted 2 lines
Other than that I'm happy with this patch.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 1/3] KVM: s390: ipte lock for SCA access should be contained in KVM
In reply to Pierre MorelQuoting Pierre Morel (2022-06-20 14:54:35)
Quoted 8 lines
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to setECA_SIIanyway so we can go straight to the source for that.Signed-off-by Pierre Morel <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by David Hildenbrand <[email protected]>
Re: [PATCH v10 1/3] KVM: s390: ipte lock for SCA access should be contained in KVM
Quoted 6 lines · Nico Boehr
Quoting Pierre Morel (2022-06-20 14:54:35)
Quoted 5 lines
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to setECA_SIIanyway so we can go straight to the source for that.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by David Hildenbrand <[email protected]>Reviewed-by Nico Boehr <[email protected]>
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 1/3] KVM: s390: ipte lock for SCA access should be contained in KVM
In reply to Pierre MorelQuoted 8 lines · On Mon, 20 Jun 2022 14:54:35 +0200 Pierre Morel <[email protected]> wrote:
We can check if SIIF is enabled by testing the
sclp_infostruct instead of testing the sie control block eca variable. sclp.has_ssif is the only requirement to setECA_SIIanyway so we can go straight to the source for that.Signed-off-by Pierre Morel <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by David Hildenbrand <[email protected]>
Quoted 269 lines
--- arch/s390/kvm/gaccess.c | 96 ++++++++++++++++++++--------------------- arch/s390/kvm/gaccess.h | 6 +-- arch/s390/kvm/priv.c | 6 +-- 3 files changed, 54 insertions(+), 54 deletions(-)arch/s390/kvm/gaccess.c +48
-48@@ -262,77 +262,77 @@ struct aste {/* .. more fields there */ }; -int ipte_lock_held (struct kvm_vcpu *vcpu) +int ipte_lock_held (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) { +if (sclp.has_siif) {int rc; -read_lock (&vcpu->kvm->arch.sca_lock); - rc =kvm_s390_get_ipte_control (vcpu->kvm)->kh !=0 ; -read_unlock (&vcpu->kvm->arch.sca_lock); +read_lock (&kvm->arch.sca_lock); + rc =kvm_s390_get_ipte_control (kvm)->kh !=0 ; +read_unlock (&kvm->arch.sca_lock);return rc; } -return vcpu->kvm->arch.ipte_lock_count !=0 ; +return kvm->arch.ipte_lock_count !=0 ; } -static void ipte_lock_simple (struct kvm_vcpu *vcpu) +static void ipte_lock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count++; -if (vcpu->kvm->arch.ipte_lock_count >1 ) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count++; +if (kvm->arch.ipte_lock_count >1 ) goto out; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic);if (old.k) { -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);cond_resched (); goto retry; } new = old; new.k =1 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_unlock_simple (struct kvm_vcpu *vcpu) +static void ipte_unlock_simple (struct kvm *kvm) {union ipte_control old, new, *ic; -mutex_lock (&vcpu->kvm->arch.ipte_mutex); - vcpu->kvm->arch.ipte_lock_count--; -if (vcpu->kvm->arch.ipte_lock_count) +mutex_lock (&kvm->arch.ipte_mutex); + kvm->arch.ipte_lock_count--; +if (kvm->arch.ipte_lock_count) goto out; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm);do { old =READ_ONCE (*ic); new = old; new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); -wake_up (&vcpu->kvm->arch.ipte_wq); +read_unlock (&kvm->arch.sca_lock); +wake_up (&kvm->arch.ipte_wq); out: -mutex_unlock (&vcpu->kvm->arch.ipte_mutex); +mutex_unlock (&kvm->arch.ipte_mutex); } -static void ipte_lock_siif (struct kvm_vcpu *vcpu) +static void ipte_lock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; retry: -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); if (old.kg) { - read_unlock(&vcpu->kvm->arch.sca_lock); + read_unlock(&kvm->arch.sca_lock); cond_resched(); goto retry; }@@ -340,15 +340,15 @@ static void ipte_lock_siif(struct kvm_vcpu *vcpu) new.k =1 ; new.kh++; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock); } -static void ipte_unlock_siif (struct kvm_vcpu *vcpu) +static void ipte_unlock_siif (struct kvm *kvm) {union ipte_control old, new, *ic; -read_lock (&vcpu->kvm->arch.sca_lock); - ic =kvm_s390_get_ipte_control (vcpu->kvm); +read_lock (&kvm->arch.sca_lock); + ic =kvm_s390_get_ipte_control (kvm); do { old = READ_ONCE(*ic); new = old;@@ -356,25 +356,25 @@ static void ipte_unlock_siif(struct kvm_vcpu *vcpu)if (!new.kh) new.k =0 ; }while (cmpxchg (&ic->val, old.val, new.val) != old.val); -read_unlock (&vcpu->kvm->arch.sca_lock); +read_unlock (&kvm->arch.sca_lock);if (!new.kh) -wake_up (&vcpu->kvm->arch.ipte_wq); +wake_up (&kvm->arch.ipte_wq); } -void ipte_lock (struct kvm_vcpu *vcpu) +void ipte_lock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_lock_siif (vcpu); +if (sclp.has_siif) +ipte_lock_siif (kvm);else -ipte_lock_simple (vcpu); +ipte_lock_simple (kvm); } -void ipte_unlock (struct kvm_vcpu *vcpu) +void ipte_unlock (struct kvm *kvm) { -if (vcpu->arch.sie_block->eca &ECA_SII ) -ipte_unlock_siif (vcpu); +if (sclp.has_siif) +ipte_unlock_siif (kvm);else -ipte_unlock_simple (vcpu); +ipte_unlock_simple (kvm); } static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, u8 ar,@@ -1086,7 +1086,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, try_storage_prot_override =storage_prot_override_applicable (vcpu); need_ipte_lock =psw_bits (*psw).dat && !asce.r;if (need_ipte_lock) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm);/* * Since we do the access further down ultimately via a move instruction * that does key checking and returns an error in case of a protection @@ -1127,7 +1127,7 @@ int access_guest_with_key(struct kvm_vcpu *vcpu, unsigned long ga, u8 ar, } out_unlock:if (need_ipte_lock) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);if (nr_pages >ARRAY_SIZE (gpa_array))vfree (gpas);return rc;@@ -1199,10 +1199,10 @@ int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, u8 ar, rc =get_vcpu_asce (vcpu, &asce, gva, ar, mode);if (rc)return rc; -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =guest_range_to_gpas (vcpu, gva, ar,NULL , length, asce, mode, access_key); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return rc; }@@ -1465,7 +1465,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg,* tables/pointers we read stay valid - unshadowing is however * always possible - only guest_table_lock protects us. */ -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); rc =gmap_shadow_pgt_lookup (sg, saddr, &pgt, &dat_protection, &fake);if (rc)@@ -1499,7 +1499,7 @@ int kvm_s390_shadow_fault(struct kvm_vcpu *vcpu, struct gmap *sg, pte.p |= dat_protection;if (!rc) rc =gmap_shadow_page (sg, saddr,__pte (pte.val)); -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);mmap_read_unlock (sg->mm);return rc; }arch/s390/kvm/gaccess.h +3
-3@@ -440,9 +440,9 @@ int read_guest_real(struct kvm_vcpu *vcpu, unsigned long gra, void *data,return access_guest_real (vcpu, gra, data, len,0 ); } -void ipte_lock (struct kvm_vcpu *vcpu); -void ipte_unlock (struct kvm_vcpu *vcpu); -int ipte_lock_held (struct kvm_vcpu *vcpu); +void ipte_lock (struct kvm *kvm); +void ipte_unlock (struct kvm *kvm); +int ipte_lock_held (struct kvm *kvm);int kvm_s390_check_low_addr_prot_real (struct kvm_vcpu *vcpu,unsigned long gra);/* MVPG PEI indication bits */ arch/s390/kvm/priv.c +3
-3@@ -442,7 +442,7 @@ static int handle_ipte_interlock(struct kvm_vcpu *vcpu) vcpu->stat.instruction_ipte_interlock++;if (psw_bits (vcpu->arch.sie_block->gpsw).pstate)return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu)); +wait_event (vcpu->kvm->arch.ipte_wq, !ipte_lock_held (vcpu->kvm));kvm_s390_retry_instr (vcpu);VCPU_EVENT (vcpu,4 ,"%s" ,"retrying ipte interlock operation" );return 0 ;@@ -1471,7 +1471,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) access_key = (operand2 &0xf0 ) >>4 ;if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_lock (vcpu); +ipte_lock (vcpu->kvm); ret =guest_translate_address_with_key (vcpu, address, ar, &gpa,GACC_STORE , access_key);@@ -1508,7 +1508,7 @@ static int handle_tprot(struct kvm_vcpu *vcpu) }if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_DAT ) -ipte_unlock (vcpu); +ipte_unlock (vcpu->kvm);return ret; }
PATCH v10 2/3 KVM: s390: guest support for topology function
In reply to Pierre MorelWe report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)
arch/s390/include/asm/kvm_host.h +8 -3
@@ -97,15 +97,19 @@ struct bsca_block {union ipte_control ipte_control;__u64 reserved[5 ];__u64 mcn; -__u64 reserved2; +#define SCA_UTILITY_MTCR 0x8000 +__u16 utility; +__u8 reserved2[6 ];struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS ]; };struct esca_block {union ipte_control ipte_control; -__u64 reserved1[7 ]; +__u64 reserved1[6 ]; +__u16 utility; +__u8 reserved2[6 ];__u64 mcn[4 ]; -__u64 reserved2[20 ]; +__u64 reserved3[20 ];struct esca_entry cpu[KVM_S390_ESCA_CPU_SLOTS ]; };
@@ -249,6 +253,7 @@ struct kvm_s390_sie_block {#define ECB_SPECI 0x08#define ECB_SRSI 0x04#define ECB_HOSTPROTINT 0x02 +#define ECB_PTF 0x01__u8 ecb;/* 0x0061 */ #define ECB2_CMMA 0x80#define ECB2_IEP 0x20
arch/s390/kvm/kvm-s390.c +26 -1
@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr +* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ +struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm); + sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +} +static int kvm_s390_vm_set_attr (struct kvm *kvm,struct kvm_device_attr *attr) {int ret;
@@ -3143,7 +3162,6 @@ __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -gmap_enable (vcpu->arch.enabled_gmap);kvm_s390_set_cpuflags (vcpu,CPUSTAT_RUNNING );if (vcpu->arch.cputm_enabled && !is_vcpu_idle (vcpu))
@@ -3272,6 +3290,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu) vcpu->arch.sie_block->ecb |=ECB_HOSTPROTINT ;if (test_kvm_facility (vcpu->kvm,9 )) vcpu->arch.sie_block->ecb |=ECB_SRSI ; + +/* PTF needs guest facilities to enable interpretation */ +if (test_kvm_facility (vcpu->kvm,11 )) + vcpu->arch.sie_block->ecb |=ECB_PTF ; +if (test_kvm_facility (vcpu->kvm,73 )) vcpu->arch.sie_block->ecb |=ECB_TE ;if (!kvm_is_ucontrol (vcpu->kvm))
@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);return 0 ; out_ucontrol_uninit:
arch/s390/kvm/priv.c +11 -4
@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_PSTATE )return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -if (fc >3 ) { -kvm_s390_set_psw_cc (vcpu,3 ); -return 0 ; - } +/* Bailout forbidden function codes */ +if (fc >3 && fc !=15 ) + goto out_no_data; + +/* fc 15 is provided with PTF/CPU topology support */ +if (fc ==15 && !test_kvm_facility (vcpu->kvm,11 )) + goto out_no_data;if (vcpu->run->s.regs.gprs[0 ] &0x0fffff00 || vcpu->run->s.regs.gprs[1 ] &0xffff0000 )
@@ -910,6 +913,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu) goto out_no_data;handle_stsi_3_2_2 (vcpu, (void *) mem);break ; +case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ; }if (kvm_s390_pv_cpu_is_protected (vcpu)) { memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,
arch/s390/kvm/vsie.c +3 -0
@@ -503,6 +503,9 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)/* Host-protection-interruption introduced with ESOP */ if (test_kvm_cpu_feat (vcpu->kvm,KVM_S390_VM_CPU_FEAT_ESOP )) scb_s->ecb |= scb_o->ecb &ECB_HOSTPROTINT ; +/* CPU Topology */ +if (test_kvm_facility (vcpu->kvm,11 )) + scb_s->ecb |= scb_o->ecb &ECB_PTF ;/* transactional execution */ if (test_kvm_facility (vcpu->kvm,73 ) && wants_tx) {/* remap the prefix is tx is toggled on */
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 29 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)arch/s390/include/asm/kvm_host.h +0
-0
arch/s390/include/asm/kvm_host.h struct bsca_block { · line 100
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -97,15 +97,19 @@ struct bsca_block {union ipte_control ipte_control;__u64 reserved[5 ];__u64 mcn; -__u64 reserved2; +#define SCA_UTILITY_MTCR 0x8000
I'm not too happy having this in the bsca but not in the esca. I'd suggest putting it outside the structs or to go with my next suggestion:
Just make it a bit field struct and make that a member in bsca/esca. No messing about with ANDing, ORing etc.
It's unfortunate that we only use one bit in that field but I'd still find it easier to read.
Quoted 30 lines
arch/s390/include/asm/kvm_host.h +7
-2+__u16 utility; +__u8 reserved2[6 ];struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS ]; };struct esca_block {union ipte_control ipte_control; -__u64 reserved1[7 ]; +__u64 reserved1[6 ]; +__u16 utility; +__u8 reserved2[6 ];__u64 mcn[4 ]; -__u64 reserved2[20 ]; +__u64 reserved3[20 ];struct esca_entry cpu[KVM_S390_ESCA_CPU_SLOTS ]; };@@ -249,6 +253,7 @@ struct kvm_s390_sie_block {#define ECB_SPECI 0x08#define ECB_SRSI 0x04#define ECB_HOSTPROTINT 0x02 +#define ECB_PTF 0x01__u8 ecb;/* 0x0061 */ #define ECB2_CMMA 0x80#define ECB2_IEP 0x20arch/s390/kvm/kvm-s390.c +0
-0
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1699
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +6
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr +* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11
I'm not sure that this statement make sense since you set the mctr in kvm_s390_vcpu_setup() unconditionally and don't check stfle 11.
I think we can remove the second line from this.
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1702
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3
-0+* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change.
Please swap those two comments
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1706
Quoted 4 lines
arch/s390/kvm/kvm-s390.c +4
-0+*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ +struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */
Please put the comment above the statement and maybe extend it a bit:
SCA version doesn't matter, the utility field always has the same offset.
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +6
-0+ +ipte_lock (kvm); + sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +} +static int kvm_s390_vm_set_attr (struct kvm *kvm,struct kvm_device_attr *attr) {int ret;
arch/s390/kvm/kvm-s390.c __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu) · line 3146
Quoted 4 lines
arch/s390/kvm/kvm-s390.c +0
-1@@ -3143,7 +3162,6 @@ __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -
Please remove that change
Quoted 62 lines
arch/s390/kvm/kvm-s390.c +7
-0gmap_enable (vcpu->arch.enabled_gmap);kvm_s390_set_cpuflags (vcpu,CPUSTAT_RUNNING );if (vcpu->arch.cputm_enabled && !is_vcpu_idle (vcpu))@@ -3272,6 +3290,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu) vcpu->arch.sie_block->ecb |=ECB_HOSTPROTINT ;if (test_kvm_facility (vcpu->kvm,9 )) vcpu->arch.sie_block->ecb |=ECB_SRSI ; + +/* PTF needs guest facilities to enable interpretation */ +if (test_kvm_facility (vcpu->kvm,11 )) + vcpu->arch.sie_block->ecb |=ECB_PTF ; +if (test_kvm_facility (vcpu->kvm,73 )) vcpu->arch.sie_block->ecb |=ECB_TE ;if (!kvm_is_ucontrol (vcpu->kvm))@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);return 0 ; out_ucontrol_uninit:arch/s390/kvm/priv.c +11
-4@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_PSTATE )return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -if (fc >3 ) { -kvm_s390_set_psw_cc (vcpu,3 ); -return 0 ; - } +/* Bailout forbidden function codes */ +if (fc >3 && fc !=15 ) + goto out_no_data; + +/* fc 15 is provided with PTF/CPU topology support */ +if (fc ==15 && !test_kvm_facility (vcpu->kvm,11 )) + goto out_no_data;if (vcpu->run->s.regs.gprs[0 ] &0x0fffff00 || vcpu->run->s.regs.gprs[1 ] &0xffff0000 )@@ -910,6 +913,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu) goto out_no_data;handle_stsi_3_2_2 (vcpu, (void *) mem);break ; +case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ; }if (kvm_s390_pv_cpu_is_protected (vcpu)) { memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,arch/s390/kvm/vsie.c +0
-0
arch/s390/kvm/vsie.c static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) · line 506
Quoted 4 lines
arch/s390/kvm/vsie.c +1
-0@@ -503,6 +503,9 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)/* Host-protection-interruption introduced with ESOP */ if (test_kvm_cpu_feat (vcpu->kvm,KVM_S390_VM_CPU_FEAT_ESOP )) scb_s->ecb |= scb_o->ecb &ECB_HOSTPROTINT ; +/* CPU Topology */
Maybe also add:
This facility only uses the utility field of the SCA and none of the cpu entries that are problematic with the other interpretation facilities so we can pass it through.
Quoted 5 lines
arch/s390/kvm/vsie.c +2
-0+if (test_kvm_facility (vcpu->kvm,11 )) + scb_s->ecb |= scb_o->ecb &ECB_PTF ;/* transactional execution */ if (test_kvm_facility (vcpu->kvm,73 ) && wants_tx) {/* remap the prefix is tx is toggled on */
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 38 lines · Janosch Frank
Quoted 29 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>---
arch/s390/include/asm/kvm_host.h| 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++4 files changed, 48 insertions(+), 8 deletions(-)arch/s390/include/asm/kvm_host.h +0
-0arch/s390/include/asm/kvm_host.h struct bsca_block { · line 100
Quoted 5 lines
arch/s390/include/asm/kvm_host.h +1
-1@@ -97,15 +97,19 @@ struct bsca_block { union ipte_control ipte_control; __u64 reserved[5]; __u64 mcn; - __u64 reserved2; +#define SCA_UTILITY_MTCR 0x8000I'm not too happy having this in the bsca but not in the esca. I'd suggest putting it outside the structs or to go with my next suggestion:
Quoted 4 lines
Just make it a bit field struct and make that a member in bsca/esca. No messing about with ANDing, ORing etc.
It's unfortunate that we only use one bit in that field but I'd still find it easier to read.
OK
Quoted 134 lines
Quoted 29 lines
arch/s390/include/asm/kvm_host.h +7
-2+ __u16 utility; + __u8 reserved2[6]; struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS]; }; struct esca_block { union ipte_control ipte_control; - __u64 reserved1[7]; + __u64 reserved1[6]; + __u16 utility; + __u8 reserved2[6]; __u64 mcn[4]; - __u64 reserved2[20]; + __u64 reserved3[20]; struct esca_entry cpu[KVM_S390_ESCA_CPU_SLOTS]; };@@ -249,6 +253,7 @@ struct kvm_s390_sie_block { #define ECB_SPECI 0x08 #define ECB_SRSI 0x04 #define ECB_HOSTPROTINT 0x02 +#define ECB_PTF 0x01__u8 ecb; /*
0x0061*/
#define ECB2_CMMA0x80
#define ECB2_IEP0x20arch/s390/kvm/kvm-s390.c +0
-0arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm · line 1699
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +6
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) return ret; } +/** +* kvm_s390_sca_set_mtcr +* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 I'm not sure that this statement make sense since you set the mctr in
kvm_s390_vcpu_setup()unconditionally and don't check stfle 11.I think we can remove the second line from this.
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1702
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3
-0+* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. Please swap those two comments
Quoted 5 lines
arch/s390/kvm/kvm-s390.c +4
-0+*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ + struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn'tmatter */
Please put the comment above the statement and maybe extend it a bit:
SCA version doesn't matter, the utility field always has the same offset.arch/s390/kvm/kvm-s390.c __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu) · line 3145
Quoted 13 lines
arch/s390/kvm/kvm-s390.c +6
-0+ + ipte_lock(kvm); + sca->utility |= SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} +static int
kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr){
int ret;arch/s390/kvm/kvm-s390.c +0
-1@@ -3143,7 +3162,6 @@ __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu) void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { -Please remove that change
Quoted 60 lines
arch/s390/kvm/kvm-s390.c +5
-0gmap_enable(vcpu->arch.enabled_gmap); kvm_s390_set_cpuflags(vcpu, CPUSTAT_RUNNING); if (vcpu->arch.cputm_enabled && !is_vcpu_idle(vcpu))@@ -3272,6 +3290,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu) vcpu->arch.sie_block->ecb |= ECB_HOSTPROTINT; if (test_kvm_facility(vcpu->kvm, 9)) vcpu->arch.sie_block->ecb |= ECB_SRSI; + + /* PTF needs guest facilities to enable interpretation */ + if (test_kvm_facility(vcpu->kvm, 11)) + vcpu->arch.sie_block->ecb |= ECB_PTF; + if (test_kvm_facility(vcpu->kvm, 73)) vcpu->arch.sie_block->ecb |= ECB_TE;if (!kvm_is_ucontrol(vcpu->kvm))
arch/s390/kvm/kvm-s390.c +2
-0@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc = kvm_s390_vcpu_setup(vcpu); if (rc) goto out_ucontrol_uninit; + + kvm_s390_sca_set_mtcr(vcpu->kvm); return 0;
out_ucontrol_uninit:arch/s390/kvm/priv.c +7
-4@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu) if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE) return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP); - if (fc > 3) { - kvm_s390_set_psw_cc(vcpu, 3); - return 0; - } + /* Bailout forbidden function codes */ + if (fc > 3 && fc != 15) + goto out_no_data; + + /* fc 15 is provided with PTF/CPU topology support */ + if (fc == 15 && !test_kvm_facility(vcpu->kvm, 11)) + goto out_no_data;if (vcpu->run->s.regs.gprs[0] &
0x0fffff00|| vcpu->run->s.regs.gprs[1] &0xffff0000)arch/s390/kvm/priv.c +4
-0@@ -910,6 +913,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu) goto out_no_data; handle_stsi_3_2_2(vcpu, (void *) mem); break; + case 15: + trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2); + insert_stsi_usr_data(vcpu, operand2, ar, fc, sel1, sel2); + return -EREMOTE; } if (kvm_s390_pv_cpu_is_protected(vcpu)) { memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,arch/s390/kvm/vsie.c +0
-0@@ -503,6 +503,9 @@ static int shadow_scb(struct kvm_vcpu *vcpu,
Quoted 15 lines
arch/s390/kvm/vsie.c static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) · line 506
Quoted 5 lines
arch/s390/kvm/vsie.c +1
-0struct vsie_page *vsie_page) /* Host-protection-interruption introduced with ESOP */ if (test_kvm_cpu_feat(vcpu->kvm, KVM_S390_VM_CPU_FEAT_ESOP)) scb_s->ecb |= scb_o->ecb & ECB_HOSTPROTINT; + /* CPU Topology */Maybe also add:
This facility only uses the utility field of the SCA and none of the cpu entries that are problematic with the other interpretation facilities so we can pass it through.Quoted 5 lines
arch/s390/kvm/vsie.c +2
-0+ if (test_kvm_facility(vcpu->kvm, 11)) + scb_s->ecb |= scb_o->ecb & ECB_PTF; /* transactional execution */ if (test_kvm_facility(vcpu->kvm, 73) && wants_tx) {/* remap the prefix is tx is toggled on */
OK with all comments,
I make the changes.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Pierre MorelQuoting Pierre Morel (2022-06-20 14:54:36)
[...]
Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0
[...]
arch/s390/include/asm/kvm_host.h int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · line 3430
Quoted 6 lines
arch/s390/include/asm/kvm_host.h +2
-0@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);
We set the MTCR in the vcpu create. Does it also make sense to set it in kvm_arch_vcpu_destroy?
[...]
Quoted 5 lines
arch/s390/kvm/priv.c +0
-0@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)
[...]
arch/s390/kvm/priv.c static int handle_stsi(struct kvm_vcpu *vcpu) · line 919
Quoted 4 lines
arch/s390/kvm/priv.c +4
-0+case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ;
Maybe the API documentation should clearly note that once you turn on KVM_CAP_S390_CPU_TOPOLOGY, you will get exits to userspace for STSI 15.x.y, regardless of whether KVM_CAP_S390_USER_STSI is on or off.
Other than that, looks good, hence:
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 7 lines · Nico Boehr
Quoting Pierre Morel (2022-06-20 14:54:36)
[...]Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0[...]
Quoted 8 lines
arch/s390/include/asm/kvm_host.h int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · line 3430
Quoted 6 lines
arch/s390/include/asm/kvm_host.h +2
-0@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);We set the MTCR in the vcpu create. Does it also make sense to set it in
kvm_arch_vcpu_destroy?
I think you are right.
Even we only destroy vCPU when we destroy the VM and I think that it is not currently needed, it would be more logical to do so, and we will be ready for the day we can unplug vCPUs.
Unless somebody has another opinion I add the entry.
Quoted 19 lines
[...]
Quoted 5 lines
arch/s390/kvm/priv.c +0
-0@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)[...]
arch/s390/kvm/priv.c static int handle_stsi(struct kvm_vcpu *vcpu) · line 919
Quoted 4 lines
arch/s390/kvm/priv.c +4
-0+case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ;Maybe the API documentation should clearly note that once you turn on
KVM_CAP_S390_CPU_TOPOLOGY, you will get exits to userspace for STSI 15.x.y, regardless of whetherKVM_CAP_S390_USER_STSIis on or off.Other than that, looks good, hence:
Reviewed-by Nico Boehr <[email protected]>
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Nico BoehrQuoted 23 lines · Nico Boehr
Quoting Pierre Morel (2022-06-20 14:54:36)
[...]Quoted 4 lines
arch/s390/include/asm/kvm_host.h +0
-0[...]
arch/s390/include/asm/kvm_host.h int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · line 3430
Quoted 6 lines
arch/s390/include/asm/kvm_host.h +2
-0@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);We set the MTCR in the vcpu create. Does it also make sense to set it in
kvm_arch_vcpu_destroy?[...]
Quoted 5 lines
arch/s390/kvm/priv.c +0
-0@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)[...]
Quoted 6 lines
arch/s390/kvm/priv.c static int handle_stsi(struct kvm_vcpu *vcpu) · line 919
Quoted 4 lines
arch/s390/kvm/priv.c +4
-0+case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ;Maybe the API documentation should clearly note that once you turn on
KVM_CAP_S390_CPU_TOPOLOGY, you will get exits to userspace for STSI 15.x.y, regardless of whetherKVM_CAP_S390_USER_STSIis on or off.
Humm, right, I must change this, it is not right to have STSI(15) but not STSI([123])
I will also have two other changes in this patch, the locking and add a check for PV on STSI(15)
So I will respin in a short time.
Quoted 5 lines
Other than that, looks good, hence:
Reviewed-by Nico Boehr <[email protected]>
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Pierre MorelQuoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)
[...]
Quoted 13 lines
arch/s390/kvm/kvm-s390.c +5
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr +* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present,
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1708
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ +struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm);
Why do we need to take the ipte lock here and in patch 3?
Quoted 3 lines
arch/s390/kvm/kvm-s390.c +3
-0+ sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +}
[...]
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 44 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)[...]
Quoted 17 lines
arch/s390/kvm/kvm-s390.c +9
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr +* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change.
Quoted 8 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1708
Quoted 6 lines
arch/s390/kvm/kvm-s390.c +6
-0+*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ +struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm);Why do we need to take the ipte lock here and in patch 3?
That is a good question.
I fear I was tired as I understood that from the documentation, after re-reading, we need an interlocked-update not an ipte lock update. ... I have to change that
Quoted 7 lines
Quoted 4 lines
arch/s390/kvm/kvm-s390.c +3
-0+ sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +}[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Claudio Imbrenda
maintainerRe: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Pierre MorelQuoted 64 lines · On Mon, 20 Jun 2022 14:54:36 +0200 Pierre Morel <[email protected]> wrote:
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)arch/s390/include/asm/kvm_host.h +8
-3@@ -97,15 +97,19 @@ struct bsca_block {union ipte_control ipte_control;__u64 reserved[5 ];__u64 mcn; -__u64 reserved2; +#define SCA_UTILITY_MTCR 0x8000 +__u16 utility; +__u8 reserved2[6 ];struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS ]; };struct esca_block {union ipte_control ipte_control; -__u64 reserved1[7 ]; +__u64 reserved1[6 ]; +__u16 utility; +__u8 reserved2[6 ];__u64 mcn[4 ]; -__u64 reserved2[20 ]; +__u64 reserved3[20 ];struct esca_entry cpu[KVM_S390_ESCA_CPU_SLOTS ]; };@@ -249,6 +253,7 @@ struct kvm_s390_sie_block {#define ECB_SPECI 0x08#define ECB_SRSI 0x04#define ECB_HOSTPROTINT 0x02 +#define ECB_PTF 0x01__u8 ecb;/* 0x0061 */ #define ECB2_CMMA 0x80#define ECB2_IEP 0x20arch/s390/kvm/kvm-s390.c +0
-0
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 5 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr
the format for kdoc is:
function_name - very short description
please add a very short description. something like:
kvm_s390_sca_set_mtcr - update mtcr to signal topology change
Quoted 96 lines
arch/s390/kvm/kvm-s390.c +24
-1+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{ +struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm); + sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +} +static int kvm_s390_vm_set_attr (struct kvm *kvm,struct kvm_device_attr *attr) {int ret;@@ -3143,7 +3162,6 @@ __u64 kvm_s390_get_cpu_timer(struct kvm_vcpu *vcpu)void kvm_arch_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -gmap_enable (vcpu->arch.enabled_gmap);kvm_s390_set_cpuflags (vcpu,CPUSTAT_RUNNING );if (vcpu->arch.cputm_enabled && !is_vcpu_idle (vcpu))@@ -3272,6 +3290,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu) vcpu->arch.sie_block->ecb |=ECB_HOSTPROTINT ;if (test_kvm_facility (vcpu->kvm,9 )) vcpu->arch.sie_block->ecb |=ECB_SRSI ; + +/* PTF needs guest facilities to enable interpretation */ +if (test_kvm_facility (vcpu->kvm,11 )) + vcpu->arch.sie_block->ecb |=ECB_PTF ; +if (test_kvm_facility (vcpu->kvm,73 )) vcpu->arch.sie_block->ecb |=ECB_TE ;if (!kvm_is_ucontrol (vcpu->kvm))@@ -3403,6 +3426,8 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) rc =kvm_s390_vcpu_setup (vcpu);if (rc) goto out_ucontrol_uninit; + +kvm_s390_sca_set_mtcr (vcpu->kvm);return 0 ; out_ucontrol_uninit:arch/s390/kvm/priv.c +11
-4@@ -873,10 +873,13 @@ static int handle_stsi(struct kvm_vcpu *vcpu)if (vcpu->arch.sie_block->gpsw.mask &PSW_MASK_PSTATE )return kvm_s390_inject_program_int (vcpu,PGM_PRIVILEGED_OP ); -if (fc >3 ) { -kvm_s390_set_psw_cc (vcpu,3 ); -return 0 ; - } +/* Bailout forbidden function codes */ +if (fc >3 && fc !=15 ) + goto out_no_data; + +/* fc 15 is provided with PTF/CPU topology support */ +if (fc ==15 && !test_kvm_facility (vcpu->kvm,11 )) + goto out_no_data;if (vcpu->run->s.regs.gprs[0 ] &0x0fffff00 || vcpu->run->s.regs.gprs[1 ] &0xffff0000 )@@ -910,6 +913,10 @@ static int handle_stsi(struct kvm_vcpu *vcpu) goto out_no_data;handle_stsi_3_2_2 (vcpu, (void *) mem);break ; +case 15 : +trace_kvm_s390_handle_stsi (vcpu, fc, sel1, sel2, operand2); +insert_stsi_usr_data (vcpu, operand2, ar, fc, sel1, sel2); +return -EREMOTE ; }if (kvm_s390_pv_cpu_is_protected (vcpu)) { memcpy((void *)sida_origin(vcpu->arch.sie_block), (void *)mem,arch/s390/kvm/vsie.c +3
-0@@ -503,6 +503,9 @@ static int shadow_scb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page)/* Host-protection-interruption introduced with ESOP */ if (test_kvm_cpu_feat (vcpu->kvm,KVM_S390_VM_CPU_FEAT_ESOP )) scb_s->ecb |= scb_o->ecb &ECB_HOSTPROTINT ; +/* CPU Topology */ +if (test_kvm_facility (vcpu->kvm,11 )) + scb_s->ecb |= scb_o->ecb &ECB_PTF ;/* transactional execution */ if (test_kvm_facility (vcpu->kvm,73 ) && wants_tx) {/* remap the prefix is tx is toggled on */
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 75 lines · Claudio Imbrenda
Quoted 64 lines · On Mon, 20 Jun 2022 14:54:36 +0200 Pierre Morel <[email protected]> wrote:
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)arch/s390/include/asm/kvm_host.h +8
-3@@ -97,15 +97,19 @@ struct bsca_block {union ipte_control ipte_control;__u64 reserved[5 ];__u64 mcn; -__u64 reserved2; +#define SCA_UTILITY_MTCR 0x8000 +__u16 utility; +__u8 reserved2[6 ];struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS ]; };struct esca_block {union ipte_control ipte_control; -__u64 reserved1[7 ]; +__u64 reserved1[6 ]; +__u16 utility; +__u8 reserved2[6 ];__u64 mcn[4 ]; -__u64 reserved2[20 ]; +__u64 reserved3[20 ];struct esca_entry cpu[KVM_S390_ESCA_CPU_SLOTS ]; };@@ -249,6 +253,7 @@ struct kvm_s390_sie_block {#define ECB_SPECI 0x08#define ECB_SRSI 0x04#define ECB_HOSTPROTINT 0x02 +#define ECB_PTF 0x01__u8 ecb;/* 0x0061 */ #define ECB2_CMMA 0x80#define ECB2_IEP 0x20arch/s390/kvm/kvm-s390.c +0
-0arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 5 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr the format for kdoc is:
Quoted 5 lines
function_name- very short descriptionplease add a very short description. something like:
kvm_s390_sca_set_mtcr- update mtcr to signal topology change
OK, thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Pierre MorelQuoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)
[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr
I wonder if there is a better name, kvm_s390_report_topology_change maybe?
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{
Do we need a sca_lock read_section here? If we don't why not? Did not see one up the stack, but I might have overlooked something.
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +7
-0+struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm); + sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +} +
[...]
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 40 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; } +/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?
Quoted 13 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{Do we need a
sca_lockread_sectionhere? If we don't why not? Did not see one up the stack, but I might have overlooked something.
Yes we do.
As I said about your well justified comment in a previous mail, ipte_lock is not the right thing to use here and I will replace with an inter locked update.
Quoted 11 lines
Quoted 8 lines
arch/s390/kvm/kvm-s390.c +7
-0+struct bsca_block *sca = kvm->arch.sca;/* SCA version doesn't matter */ + +ipte_lock (kvm); + sca->utility |=SCA_UTILITY_MTCR ; +ipte_unlock (kvm); +} +[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 57 lines · Pierre Morel
Quoted 40 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>---
arch/s390/include/asm/kvm_host.h| 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++4 files changed, 48 insertions(+), 8 deletions(-)[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) return ret; } +/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?Quoted 13 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{Do we need a
sca_lockread_sectionhere? If we don't why not? Did not see one up the stack, but I might have overlooked something.
Quoted 2 lines
Yes we do.
As I said about your well justified comment in a previous mail,ipte_lockis not the right thing to use here and I will replace with an inter locked update.
Not sure I'm understanding you right, you're saying we need both? i.e.:
struct bsca_block *sca;
read_lock(&vcpu->kvm->arch.sca_lock);
sca = kvm->arch.sca;atomic_or(SCA_UTILITY_MTCR, &sca->utility);read_unlock(&vcpu->kvm->arch.sca_lock);
Obviously you would need to change the definition of the utility field and could not use a bit field like Janosch suggested, unless you want to use a cmpxchg loop.
It's a bit ugly that utility is a two byte value.
Maybe there is a nicer way to set that bit, OR (OI, OIY) seem appropriate, but I don't know if they have a nice abstraction in Linux or if you'd need inline asm.
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +7
-0+ struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */ + + ipte_lock(kvm); + sca->utility |= SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} +[...]
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 70 lines · Janis Schoetterl-Glausch
Quoted 57 lines · Pierre Morel
Quoted 40 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) return ret; } +/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?Quoted 13 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{Do we need a
sca_lockread_sectionhere? If we don't why not? Did not see one up the stack, but I might have overlooked something.Quoted 2 lines
Yes we do.
As I said about your well justified comment in a previous mail,ipte_lockis not the right thing to use here and I will replace with an inter locked update.Not sure I'm understanding you right, you're saying we need both? i.e.:
struct
bsca_block*sca;
read_lock(&vcpu->kvm->arch.sca_lock);
sca =kvm->arch.sca;atomic_or(SCA_UTILITY_MTCR, &sca->utility);read_unlock(&vcpu->kvm->arch.sca_lock);
Quoted 5 lines
Obviously you would need to change the definition of the utility field and could not use a bit field like Janosch suggested, unless you want to use a cmpxchg loop.
It's a bit ugly that utility is a two byte value.
Maybe there is a nicer way to set that bit, OR (OI, OIY) seem appropriate, but I don't know if they have a nice abstraction in Linux or if you'd need inline asm.
I was think to something like this because it is what is used most of the time when a bit is to be change concurrently with firmware.
+union sca_utility {
+ __u16 val;
+ struct {
+ __u16 mtcr : 1;
+ __u16 reserved : 15;
+ };
+};
+
struct bsca_block {
union ipte_control ipte_control;
__u64 reserved[5];
__u64 mcn;
- __u64 reserved2;
+ union sca_utility utility;
+ __u8 reserved2[6];
struct bsca_entry cpu[KVM_S390_BSCA_CPU_SLOTS];}
....
static void kvm_s390_sca_set_mtcr(struct kvm *kvm, int val)
{
struct bsca_block *sca = kvm->arch.sca;
union sca_utility new, old;
read_lock(&kvm->arch.sca_lock);
do {
old = READ_ONCE(sca->utility);
new = old;
new.mtcr = val ? 1 : 0;
} while (cmpxchg(&sca->utility.val, old.val, new.val) != old.val);
read_lock(&kvm->arch.sca_lock);
}Quoted 14 lines
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +7
-0+ struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */ + + ipte_lock(kvm); + sca->utility |= SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} +[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 73 lines · Pierre Morel
Quoted 70 lines · Janis Schoetterl-Glausch
Quoted 57 lines · Pierre Morel
Quoted 40 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>---
arch/s390/include/asm/kvm_host.h| 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++4 files changed, 48 insertions(+), 8 deletions(-)[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) return ret; } +/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?Quoted 13 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{Do we need a
sca_lockread_sectionhere? If we don't why not? Did not see one up the stack, but I might have overlooked something.Quoted 2 lines
Yes we do.
As I said about your well justified comment in a previous mail,ipte_lockis not the right thing to use here and I will replace with an inter locked update.Not sure I'm understanding you right, you're saying we need both? i.e.:
struct
bsca_block*sca;
read_lock(&vcpu->kvm->arch.sca_lock);
sca =kvm->arch.sca;atomic_or(SCA_UTILITY_MTCR, &sca->utility);read_unlock(&vcpu->kvm->arch.sca_lock);
Quoted 7 lines
Quoted 5 lines
Obviously you would need to change the definition of the utility field and could not use a bit field like Janosch suggested, unless you want to use a cmpxchg loop.
It's a bit ugly that utility is a two byte value.
Maybe there is a nicer way to set that bit, OR (OI, OIY) seem appropriate, but I don't know if they have a nice abstraction in Linux or if you'd need inline asm.I was think to something like this because it is what is used most of the time when a bit is to be change concurrently with firmware.
Ah, ok you want to keep the bitfield.
[...]
Quoted 2 lines
static void
kvm_s390_sca_set_mtcr(struct kvm *kvm, int val)
If you use a bool val you can simply do new.mtcr = val; below.
Quoted 5 lines
{
structbsca_block*sca =kvm->arch.sca;
unionsca_utilitynew, old;
read_lock(&kvm->arch.sca_lock);
Don't forget to move the sca = kvm->arch.sca; under the lock here.
Quoted 23 lines
do {
old =READ_ONCE(sca->utility);
new = old;
new.mtcr = val ? 1 : 0;
} while (cmpxchg(&sca->utility.val, old.val, new.val)!= old.val);read_lock(&kvm->arch.sca_lock);
}Quoted 14 lines
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +7
-0+ struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */ + + ipte_lock(kvm); + sca->utility |= SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} +[...]
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
Quoted 88 lines · Janis Schoetterl-Glausch
Quoted 73 lines · Pierre Morel
Quoted 70 lines · Janis Schoetterl-Glausch
Quoted 57 lines · Pierre Morel
Quoted 40 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)[...]
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) return ret; } +/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?Quoted 13 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1705
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+* @kvm: guest KVM description +* +* Is only relevant if the topology facility is present, +* the caller should check KVM facility 11 +* +* Updates the Multiprocessor Topology-Change-Report to signal +* the guest with a topology change. +*/ +static void kvm_s390_sca_set_mtcr (struct kvm *kvm) +{Do we need a
sca_lockread_sectionhere? If we don't why not? Did not see one up the stack, but I might have overlooked something.Quoted 2 lines
Yes we do.
As I said about your well justified comment in a previous mail,ipte_lockis not the right thing to use here and I will replace with an inter locked update.Not sure I'm understanding you right, you're saying we need both? i.e.:
struct
bsca_block*sca;
read_lock(&vcpu->kvm->arch.sca_lock);
sca =kvm->arch.sca;atomic_or(SCA_UTILITY_MTCR, &sca->utility);read_unlock(&vcpu->kvm->arch.sca_lock);Quoted 7 lines
Quoted 5 lines
Obviously you would need to change the definition of the utility field and could not use a bit field like Janosch suggested, unless you want to use a cmpxchg loop.
It's a bit ugly that utility is a two byte value.
Maybe there is a nicer way to set that bit, OR (OI, OIY) seem appropriate, but I don't know if they have a nice abstraction in Linux or if you'd need inline asm.I was think to something like this because it is what is used most of the time when a bit is to be change concurrently with firmware.
Ah, ok you want to keep the bitfield.
[...]
Quoted 2 lines
static void
kvm_s390_sca_set_mtcr(struct kvm *kvm, int val)
Quoted 15 lines
If you use a bool val you can simply do new.mtcr = val; below.
Quoted 5 lines
{
structbsca_block*sca =kvm->arch.sca;
unionsca_utilitynew, old;
read_lock(&kvm->arch.sca_lock);Don't forget to move the sca =
kvm->arch.sca; under the lock here.Quoted 7 lines
do {
old =READ_ONCE(sca->utility);
new = old;
new.mtcr = val ? 1 : 0;
} while (cmpxchg(&sca->utility.val, old.val, new.val)!= old.val);read_lock(&kvm->arch.sca_lock);
}
right, thanks, and to unlock at the end :)
Quoted 17 lines
Quoted 16 lines
Quoted 15 lines
Quoted 14 lines
Quoted 13 lines
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +7
-0+ struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */ + + ipte_lock(kvm); + sca->utility |= SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} +[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 2/3] KVM: s390: guest support for topology function
In reply to Janis Schoetterl-GlauschQuoted 35 lines · Janis Schoetterl-Glausch
Quoted 24 lines · Pierre Morel
We report a topology change to the guest for any CPU hotplug.
The reporting to the guest is done using the Multiprocessor Topology-Change-Report (MTCR) bit of the utility entry in the guest's SCA which will be cleared during the interpretation of PTF.
On every vCPU creation we set the MCTR bit to let the guest know the next time he uses the PTF with command 2 instruction that the topology changed and that he should use the STSI(15.1.x) instruction to get the topology details.
STSI(15.1.x) gives information on the CPU configuration topology. Let's accept the interception of STSI with the function code 15 and let the userland part of the hypervisor handle it when userland support the CPU Topology facility.
Signed-off-by Pierre Morel <[email protected]>--- arch/s390/include/asm/kvm_host.h | 11 ++++++++--- arch/s390/kvm/kvm-s390.c | 27 ++++++++++++++++++++++++++- arch/s390/kvm/priv.c | 15 +++++++++++---- arch/s390/kvm/vsie.c | 3 +++ 4 files changed, 48 insertions(+), 8 deletions(-)[...]
Quoted 8 lines
arch/s390/kvm/kvm-s390.c +0
-0@@ -1691,6 +1691,25 @@ static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr)return ret; }
Quoted 4 lines
arch/s390/kvm/kvm-s390.c static int kvm_s390_get_cpu_model(struct kvm *kvm, struct kvm_device_attr *attr) · line 1695
Quoted 2 lines
arch/s390/kvm/kvm-s390.c +2
-0+/** +* kvm_s390_sca_set_mtcr I wonder if there is a better name,
kvm_s390_report_topology_changemaybe?
OK, for me, with a good name I can suppress a comment.
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v10 3/3 KVM: s390: resetting the Topology-Change-Report
In reply to Pierre MorelDuring a subsystem reset the Topology-Change-Report is cleared. Let's give userland the possibility to clear the MTCR in the case of a subsystem reset.
To migrate the MTCR, we give userland the possibility to query the MTCR state.
We indicate KVM support for the CPU topology facility with a new KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
--- Documentation/virt/kvm/api.rst | 31 +++++++++++ arch/s390/include/uapi/asm/kvm.h | 10 ++++ arch/s390/kvm/kvm-s390.c | 96 ++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 138 insertions(+)
Documentation/virt/kvm/api.rst +31 -0
@@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled. When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request. +8.37 KVM_CAP_S390_CPU_TOPOLOGY +------------------------------ + +:Capability: KVM_CAP_S390_CPU_TOPOLOGY +:Architectures: s390 +:Type: vm + +This capability indicates that KVM will provide the S390 CPU Topology +facility which consist of the interpretation of the PTF instruction for +the Function Code 2 along with interception and forwarding of both the +PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x) +instruction to the userland hypervisor. + +The stfle facility 11, CPU Topology facility, should not be provided +to the guest without this capability. + +When this capability is present, KVM provides a new attribute group +on vm fd, KVM_S390_VM_CPU_TOPOLOGY. +This new attribute allows to get, set or clear the Modified Change +Topology Report (MTCR) bit of the SCA through the kvm_device_attr +structure. + +Getting the MTCR bit is realized by using a kvm_device_attr attr +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr +entry pointing to the address of a struct kvm_cpu_topology. +The value of the MTCR is return by the bit mtcr of the structure. + +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR. + 9. Known KVM API problems =========================
arch/s390/include/uapi/asm/kvm.h +10 -0
@@ -74,6 +74,7 @@ struct kvm_s390_io_adapter_req {#define KVM_S390_VM_CRYPTO 2#define KVM_S390_VM_CPU_MODEL 3#define KVM_S390_VM_MIGRATION 4 +#define KVM_S390_VM_CPU_TOPOLOGY 5/* kvm attributes for mem_ctrl */ #define KVM_S390_VM_MEM_ENABLE_CMMA 0
@@ -171,6 +172,15 @@ struct kvm_s390_vm_cpu_subfunc {#define KVM_S390_VM_MIGRATION_START 1#define KVM_S390_VM_MIGRATION_STATUS 2 +/* kvm attributes for cpu topology */ +#define KVM_S390_VM_CPU_TOPO_MTCR_CLEAR 0 +#define KVM_S390_VM_CPU_TOPO_MTCR_SET 1 + +struct kvm_cpu_topology { +__u16 mtcr :1 ; +__u16 reserved :15 ; +}; +/* for KVM_GET_REGS and KVM_SET_REGS */ struct kvm_regs {/* general purpose regs for s390 */
arch/s390/kvm/kvm-s390.c +96 -0
132 lines not on this page · raw message
include/uapi/linux/kvm.h +1 -0
@@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_TSC_CONTROL 214#define KVM_CAP_SYSTEM_EVENT_DATA 215#define KVM_CAP_ARM_SYSTEM_SUSPEND 216 +#define KVM_CAP_S390_CPU_TOPOLOGY 217#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
Quoted 26 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared. Let's give userland the possibility to clear the MTCR in the case of a subsystem reset.
To migrate the MTCR, we give userland the possibility to query the MTCR state.
We indicate KVM support for the CPU topology facility with a new KVM capability:
KVM_CAP_S390_CPU_TOPOLOGY.Signed-off-by Pierre Morel <[email protected]>--- Documentation/virt/kvm/api.rst | 31 +++++++++++ arch/s390/include/uapi/asm/kvm.h | 10 ++++ arch/s390/kvm/kvm-s390.c | 96 ++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 138 insertions(+)Documentation/virt/kvm/api.rst +0
-0@@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled. When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request.
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7968
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+8.37 KVM_CAP_S390_CPU_TOPOLOGY +------------------------------ + +:Capability: KVM_CAP_S390_CPU_TOPOLOGY +:Architectures: s390 +:Type: vm + +This capability indicates that KVM will provide the S390 CPU Topology +facility which consist of the interpretation of the PTF instruction for +the Function Code 2 along with interception and forwarding of both the
Making function code capital surprises me when reading.
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7972
Quoted 4 lines
Documentation/virt/kvm/api.rst +4
-0+PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x) +instruction to the userland hypervisor. + +The stfle facility 11, CPU Topology facility, should not be provided
s/provided/indicated
Quoted 5 lines
Documentation/virt/kvm/api.rst +5
-0+to the guest without this capability. + +When this capability is present, KVM provides a new attribute group +on vm fd, KVM_S390_VM_CPU_TOPOLOGY. +This new attribute allows to get, set or clear the Modified Change
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7988
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+Topology Report (MTCR) bit of the SCA through the kvm_device_attr +structure. + +Getting the MTCR bit is realized by using a kvm_device_attr attr +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr +entry pointing to the address of a struct kvm_cpu_topology. +The value of the MTCR is return by the bit mtcr of the structure. > + +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR.
I have the feeling that we can drop the two blocks above and we won't loose information.
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled.
Quoted 11 lines
Documentation/virt/kvm/api.rst +11
-0+/** + * kvm_s390_sca_clear_mtcr + * @kvm: guest KVM description + * + * Is only relevant if the topology facility is present, + * the caller should check KVM facility 11 + * + * Updates the Multiprocessor Topology-Change-Report to signal + * the guest with a topology change. + */ +static void kvm_s390_sca_clear_mtcr(struct kvm *kvm)
This is a set operation with the value 0 and that's clearly visible by the copied code. If you make the utility entry a bitfield you can easily set 0/1 via one function without doing the bit manipulation by hand.
I.e. please only use one set function.
Quoted 11 lines
Documentation/virt/kvm/api.rst +11
-0+{ + struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */ + + ipte_lock(kvm); + sca->utility &= ~SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} + +static int kvm_s390_set_topology(struct kvm *kvm, struct kvm_device_attr *attr) +{ + if (!test_kvm_facility(kvm, 11))
Documentation/virt/kvm/api.rst
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ return -ENXIO; + + switch (attr->attr) { + case KVM_S390_VM_CPU_TOPO_MTCR_SET: + kvm_s390_sca_set_mtcr(kvm); + break; + case KVM_S390_VM_CPU_TOPO_MTCR_CLEAR: + kvm_s390_sca_clear_mtcr(kvm); + break; + }
By having two endpoints here we trade an easy check with having to access process memory to grab the value we want to set.
I'm still torn about this.
Quoted 5 lines
Documentation/virt/kvm/api.rst +5
-0+ return 0; +} + +/** + * kvm_s390_sca_get_mtcr
Documentation/virt/kvm/api.rst
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ * @kvm: guest KVM description + * + * Is only relevant if the topology facility is present, + * the caller should check KVM facility 11 + * + * reports to QEMU the Multiprocessor Topology-Change-Report. + */ +static int kvm_s390_sca_get_mtcr(struct kvm *kvm) +{ + struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */
Same comments as with the set_mtcr()
Quoted 68 lines
Documentation/virt/kvm/api.rst +32
-0+ int val; + + ipte_lock(kvm); + val = sca->utility & SCA_UTILITY_MTCR; + ipte_unlock(kvm); + + return val; +} + +static int kvm_s390_get_topology(struct kvm *kvm, struct kvm_device_attr *attr) +{ + struct kvm_cpu_topology topo = {}; + + if (!test_kvm_facility(kvm, 11)) + return -ENXIO; + + topo.mtcr = kvm_s390_sca_get_mtcr(kvm) ? 1 : 0; + if (copy_to_user((void __user *)attr->addr, &topo, sizeof(topo))) + return -EFAULT; + + return 0; +} + static int kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr) { int ret;@@ -1730,6 +1817,9 @@ static int kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr) case KVM_S390_VM_MIGRATION: ret = kvm_s390_vm_set_migration(kvm, attr); break; + case KVM_S390_VM_CPU_TOPOLOGY: + ret = kvm_s390_set_topology(kvm, attr); + break; default: ret = -ENXIO; break;@@ -1755,6 +1845,9 @@ static int kvm_s390_vm_get_attr(struct kvm *kvm, struct kvm_device_attr *attr) case KVM_S390_VM_MIGRATION: ret = kvm_s390_vm_get_migration(kvm, attr); break; + case KVM_S390_VM_CPU_TOPOLOGY: + ret = kvm_s390_get_topology(kvm, attr); + break; default: ret = -ENXIO; break;@@ -1828,6 +1921,9 @@ static int kvm_s390_vm_has_attr(struct kvm *kvm, struct kvm_device_attr *attr) case KVM_S390_VM_MIGRATION: ret = 0; break; + case KVM_S390_VM_CPU_TOPOLOGY: + ret = test_kvm_facility(kvm, 11) ? 0 : -ENXIO; + break; default: ret = -ENXIO; break;include/uapi/linux/kvm.h +1
-0@@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_TSC_CONTROL 214#define KVM_CAP_SYSTEM_EVENT_DATA 215#define KVM_CAP_ARM_SYSTEM_SUSPEND 216 +#define KVM_CAP_S390_CPU_TOPOLOGY 217#ifdef KVM_CAP_IRQ_ROUTING
Re: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
Quoted 31 lines · Janosch Frank
Quoted 30 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared. Let's give userland the possibility to clear the MTCR in the case of a subsystem reset.
To migrate the MTCR, we give userland the possibility to query the MTCR state.
We indicate KVM support for the CPU topology facility with a new KVM capability:
KVM_CAP_S390_CPU_TOPOLOGY.Signed-off-by Pierre Morel <[email protected]>---
Documentation/virt/kvm/api.rst | 31 +++++++++++arch/s390/include/uapi/asm/kvm.h| 10 ++++ arch/s390/kvm/kvm-s390.c | 96 ++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 +4 files changed, 138 insertions(+)Documentation/virt/kvm/api.rst +0
-0@@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect thatthe PMU is disabled.
When enabled, KVM will exit to userspace withKVM_EXIT_SYSTEM_EVENTof typeKVM_SYSTEM_EVENT_SUSPENDto process the guest suspend request. +8.37KVM_CAP_S390_CPU_TOPOLOGY
+------------------------------
+
+:Capability:KVM_CAP_S390_CPU_TOPOLOGY
Quoted 8 lines
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7968
Quoted 6 lines
+:Architectures: s390
+:Type: vmDocumentation/virt/kvm/api.rst +4
-0+ +This capability indicates that KVM will provide the S390 CPU Topology +facility which consist of the interpretation of the PTF instruction for +the Function Code 2 along with interception and forwarding of both theMaking function code capital surprises me when reading.
wanted to highlight FC.
I remove it.
Quoted 8 lines
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7972
Quoted 5 lines
Documentation/virt/kvm/api.rst +4
-0+PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x) +instruction to the userland hypervisor. + +The stfle facility 11, CPU Topology facility, should not be provideds/provided/indicated
OK
Quoted 18 lines
Quoted 5 lines
Documentation/virt/kvm/api.rst +5
-0+to the guest without this capability. + +When this capability is present, KVM provides a new attribute group +on vm fd, KVM_S390_VM_CPU_TOPOLOGY. +This new attribute allows to get, set or clear the Modified ChangeDocumentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7988
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+Topology Report (MTCR) bit of the SCA through the kvm_device_attr +structure. + +Getting the MTCR bit is realized by using a kvm_device_attr attr +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr +entry pointing to the address of a struct kvm_cpu_topology. +The value of the MTCR is return by the bit mtcr of the structure. > + +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR.I have the feeling that we can drop the two blocks above and we won't loose information.
Quoted 13 lines
Documentation/virt/kvm/api.rst
Quoted 11 lines
Documentation/virt/kvm/api.rst +11
-0+/** + * kvm_s390_sca_clear_mtcr + * @kvm: guest KVM description + * + * Is only relevant if the topology facility is present, + * the caller should check KVM facility 11 + * + * Updates the Multiprocessor Topology-Change-Report to signal + * the guest with a topology change. + */ +static void kvm_s390_sca_clear_mtcr(struct kvm *kvm)This is a set operation with the value 0 and that's clearly visible by the copied code. If you make the utility entry a bitfield you can easily set 0/1 via one function without doing the bit manipulation by hand.
OK
Quoted 41 lines
I.e. please only use one set function.
Quoted 13 lines
+{
+ structbsca_block*sca =kvm->arch.sca; /* SCA version doesn't matter */Documentation/virt/kvm/api.rst +7
-0+ + ipte_lock(kvm); + sca->utility &= ~SCA_UTILITY_MTCR; + ipte_unlock(kvm); +} + +static int kvm_s390_set_topology(struct kvm *kvm, struct
kvm_device_attr*attr)Documentation/virt/kvm/api.rst +2
-0+{ + if (!test_kvm_facility(kvm, 11))Documentation/virt/kvm/api.rst
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ return -ENXIO; + + switch (attr->attr) { + case KVM_S390_VM_CPU_TOPO_MTCR_SET: + kvm_s390_sca_set_mtcr(kvm); + break; + case KVM_S390_VM_CPU_TOPO_MTCR_CLEAR: + kvm_s390_sca_clear_mtcr(kvm); + break; + }By having two endpoints here we trade an easy check with having to access process memory to grab the value we want to set.
I'm still torn about this.
Quoted 10 lines
+ return 0;
Documentation/virt/kvm/api.rst +9
-0+} + +/** + * kvm_s390_sca_get_mtcr + * @kvm: guest KVM description + * + * Is only relevant if the topology facility is present, + * the caller should check KVM facility 11 + *
Quoted 8 lines
Quoted 6 lines
+ * reports to QEMU the Multiprocessor Topology-Change-Report.
+ */Documentation/virt/kvm/api.rst +3
-0+static int kvm_s390_sca_get_mtcr(struct kvm *kvm) +{ + struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn'tmatter */
Same comments as with the
set_mtcr()
OK
Quoted 72 lines
Quoted 71 lines
+ int val;
Documentation/virt/kvm/api.rst +9
-0+ + ipte_lock(kvm); + val = sca->utility & SCA_UTILITY_MTCR; + ipte_unlock(kvm); + + return val; +} + +static int kvm_s390_get_topology(struct kvm *kvm, struct
kvm_device_attr*attr)Documentation/virt/kvm/api.rst +13
-0+{ + struct kvm_cpu_topology topo = {}; + + if (!test_kvm_facility(kvm, 11)) + return -ENXIO; + + topo.mtcr = kvm_s390_sca_get_mtcr(kvm) ? 1 : 0; + if (copy_to_user((void __user *)attr->addr, &topo, sizeof(topo))) + return -EFAULT; + + return 0; +} +static int
kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr){
int ret;Documentation/virt/kvm/api.rst +0
-0@@ -1730,6 +1817,9 @@ static int kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr)case
KVM_S390_VM_MIGRATION:
ret =kvm_s390_vm_set_migration(kvm, attr); break;
+ caseKVM_S390_VM_CPU_TOPOLOGY:
+ ret =kvm_s390_set_topology(kvm, attr); + break;
default:
ret =-ENXIO;
break;Documentation/virt/kvm/api.rst +0
-0@@ -1755,6 +1845,9 @@ static int kvm_s390_vm_get_attr(struct kvm *kvm, struct kvm_device_attr *attr)case
KVM_S390_VM_MIGRATION:
ret =kvm_s390_vm_get_migration(kvm, attr); break;
+ caseKVM_S390_VM_CPU_TOPOLOGY:
+ ret =kvm_s390_get_topology(kvm, attr); + break;
default:
ret =-ENXIO;
break;Documentation/virt/kvm/api.rst +0
-0@@ -1828,6 +1921,9 @@ static int kvm_s390_vm_has_attr(struct kvm *kvm, struct kvm_device_attr *attr)case
KVM_S390_VM_MIGRATION:
ret = 0;
break;
+ caseKVM_S390_VM_CPU_TOPOLOGY:
+ ret =test_kvm_facility(kvm, 11)? 0 :-ENXIO; + break;
default:
ret =-ENXIO;
break;include/uapi/linux/kvm.h +1
-0@@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_TSC_CONTROL 214 #define KVM_CAP_SYSTEM_EVENT_DATA 215 #define KVM_CAP_ARM_SYSTEM_SUSPEND 216 +#define KVM_CAP_S390_CPU_TOPOLOGY 217 #ifdef KVM_CAP_IRQ_ROUTING
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
In reply to Pierre MorelQuoted 28 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared. Let's give userland the possibility to clear the MTCR in the case of a subsystem reset.
To migrate the MTCR, we give userland the possibility to query the MTCR state.
We indicate KVM support for the CPU topology facility with a new KVM capability:
KVM_CAP_S390_CPU_TOPOLOGY.Signed-off-by Pierre Morel <[email protected]>--- Documentation/virt/kvm/api.rst | 31 +++++++++++ arch/s390/include/uapi/asm/kvm.h | 10 ++++ arch/s390/kvm/kvm-s390.c | 96 ++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 138 insertions(+)Documentation/virt/kvm/api.rst +2
-0@@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled. When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request. +8.37 KVM_CAP_S390_CPU_TOPOLOGY +------------------------------
Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7970
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ +:Capability: KVM_CAP_S390_CPU_TOPOLOGY +:Architectures: s390 +:Type: vm + +This capability indicates that KVM will provide the S390 CPU Topology +facility which consist of the interpretation of the PTF instruction for +the Function Code 2 along with interception and forwarding of both the +PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x) +instruction to the userland hypervisor.
The way the code is written, STSI 15.x.x is forwarded to user space, might actually make sense to future proof the code by restricting that to 15.1.2-6 in priv.c.
Quoted 35 lines
Documentation/virt/kvm/api.rst +19
-0+ +The stfle facility 11, CPU Topology facility, should not be provided +to the guest without this capability. + +When this capability is present, KVM provides a new attribute group +on vm fd, KVM_S390_VM_CPU_TOPOLOGY. +This new attribute allows to get, set or clear the Modified Change +Topology Report (MTCR) bit of the SCA through the kvm_device_attr +structure. + +Getting the MTCR bit is realized by using a kvm_device_attr attr +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr +entry pointing to the address of a struct kvm_cpu_topology. +The value of the MTCR is return by the bit mtcr of the structure. + +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR. + 9. Known KVM API problems =========================arch/s390/include/uapi/asm/kvm.h +1
-0@@ -74,6 +74,7 @@ struct kvm_s390_io_adapter_req {#define KVM_S390_VM_CRYPTO 2#define KVM_S390_VM_CPU_MODEL 3#define KVM_S390_VM_MIGRATION 4 +#define KVM_S390_VM_CPU_TOPOLOGY 5/* kvm attributes for mem_ctrl */ #define KVM_S390_VM_MEM_ENABLE_CMMA 0
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 177
Quoted 6 lines
arch/s390/include/uapi/asm/kvm.h +3
-0@@ -171,6 +172,15 @@ struct kvm_s390_vm_cpu_subfunc {#define KVM_S390_VM_MIGRATION_START 1#define KVM_S390_VM_MIGRATION_STATUS 2 +/* kvm attributes for cpu topology */ +#define KVM_S390_VM_CPU_TOPO_MTCR_CLEAR 0 +#define KVM_S390_VM_CPU_TOPO_MTCR_SET 1
Are you going to transition to a set-value-provided-by-user API with the next series? I don't particularly like that MTCR is user visible, it's kind of an implementation detail.
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 180
Quoted 3 lines
arch/s390/include/uapi/asm/kvm.h +3
-0+ +struct kvm_cpu_topology { +__u16 mtcr :1 ;
So I'd give this a more descriptive name, report_topology_change/topo_change_report_pending ?
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 181
Quoted 1 line
arch/s390/include/uapi/asm/kvm.h +1
-0+ __u16 reserved : 15;
Are these bits for future proofing? If so a few more would do no harm IMO.
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 182
Quoted 1 line
arch/s390/include/uapi/asm/kvm.h +1
-0+};
The use of a bit field in uapi surprised my, but I guess it's fine and kvm_sync_regs has them too.
Quoted 24 lines
arch/s390/include/uapi/asm/kvm.h +1
-0+/* for KVM_GET_REGS and KVM_SET_REGS */ struct kvm_regs {/* general purpose regs for s390 */ arch/s390/kvm/kvm-s390.c +5
-0@@ -606,6 +606,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_S390_PROTECTED : r =is_prot_virt_host ();break ; +case KVM_CAP_S390_CPU_TOPOLOGY : + r =test_facility (11 ); +break ;default : r =0 ; }@@ -817,6 +820,20 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)icpt_operexc_on_all_vcpus (kvm); r =0 ;break ; +case KVM_CAP_S390_CPU_TOPOLOGY : + r = -EINVAL ;
arch/s390/kvm/kvm-s390.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 834
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10
-0+mutex_lock (&kvm->lock); +if (kvm->created_vcpus) { + r = -EBUSY ; + }else if (test_facility (11 )) { +set_kvm_facility (kvm->arch.model.fac_mask,11 ); +set_kvm_facility (kvm->arch.model.fac_list,11 ); + r =0 ; + } +mutex_unlock (&kvm->lock); + VM_EVENT(kvm, 3, "ENABLE: CPU TOPOLOGY %s",
Most of the other cases spell out the cap, so it'd be "ENABLE: CAP_S390_CPU_TOPOLOGY %s".
arch/s390/kvm/kvm-s390.c static void kvm_s390_sca_set_mtcr(struct kvm *kvm) · line 1729
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +2
-0+ r ? "(not available)" : "(success)"); +break ;default : r = -EINVAL ;break ;@@ -1710,6 +1727,76 @@ static void kvm_s390_sca_set_mtcr(struct kvm *kvm)ipte_unlock (kvm); }
Some brainstorming function names:
kvm_s390_get_topo_change_report kvm_s390_(un|re)set_topo_change_report kvm_s390_(publish|revoke|unpublish)_topo_change_report kvm_s390_(report|signal|revoke)_topology_change
[...]
Re: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
Quoted 81 lines · Janis Schoetterl-Glausch
Quoted 28 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared. Let's give userland the possibility to clear the MTCR in the case of a subsystem reset.
To migrate the MTCR, we give userland the possibility to query the MTCR state.
We indicate KVM support for the CPU topology facility with a new KVM capability:
KVM_CAP_S390_CPU_TOPOLOGY.Signed-off-by Pierre Morel <[email protected]>--- Documentation/virt/kvm/api.rst | 31 +++++++++++ arch/s390/include/uapi/asm/kvm.h | 10 ++++ arch/s390/kvm/kvm-s390.c | 96 ++++++++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 138 insertions(+)Documentation/virt/kvm/api.rst +2
-0@@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled. When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request. +8.37 KVM_CAP_S390_CPU_TOPOLOGY +------------------------------Documentation/virt/kvm/api.rst should adjust CPUID leaf 0xA to reflect that the PMU is disabled. · line 7970
Quoted 10 lines
Documentation/virt/kvm/api.rst +10
-0+ +:Capability: KVM_CAP_S390_CPU_TOPOLOGY +:Architectures: s390 +:Type: vm + +This capability indicates that KVM will provide the S390 CPU Topology +facility which consist of the interpretation of the PTF instruction for +the Function Code 2 along with interception and forwarding of both the +PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x) +instruction to the userland hypervisor.The way the code is written, STSI 15.x.x is forwarded to user space, might actually make sense to future proof the code by restricting that to 15.1.2-6 in
priv.c.Quoted 38 lines
Documentation/virt/kvm/api.rst +19
-0+ +The stfle facility 11, CPU Topology facility, should not be provided +to the guest without this capability. + +When this capability is present, KVM provides a new attribute group +on vm fd, KVM_S390_VM_CPU_TOPOLOGY. +This new attribute allows to get, set or clear the Modified Change +Topology Report (MTCR) bit of the SCA through the kvm_device_attr +structure. + +Getting the MTCR bit is realized by using a kvm_device_attr attr +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr +entry pointing to the address of a struct kvm_cpu_topology. +The value of the MTCR is return by the bit mtcr of the structure. + +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR. + 9. Known KVM API problems =========================arch/s390/include/uapi/asm/kvm.h +1
-0@@ -74,6 +74,7 @@ struct kvm_s390_io_adapter_req {#define KVM_S390_VM_CRYPTO 2#define KVM_S390_VM_CPU_MODEL 3#define KVM_S390_VM_MIGRATION 4 +#define KVM_S390_VM_CPU_TOPOLOGY 5/* kvm attributes for mem_ctrl */ #define KVM_S390_VM_MEM_ENABLE_CMMA 0@@ -171,6 +172,15 @@ struct kvm_s390_vm_cpu_subfunc {#define KVM_S390_VM_MIGRATION_START 1#define KVM_S390_VM_MIGRATION_STATUS 2
Quoted 6 lines
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 177
Quoted 3 lines
arch/s390/include/uapi/asm/kvm.h +3
-0+/* kvm attributes for cpu topology */ +#define KVM_S390_VM_CPU_TOPO_MTCR_CLEAR 0 +#define KVM_S390_VM_CPU_TOPO_MTCR_SET 1Are you going to transition to a set-value-provided-by-user API with the next series? I don't particularly like that MTCR is user visible, it's kind of an implementation detail.
It is not the same structure as the hardware structure. Even it looks like it.
I am OK to use something else, like a u8
in that case I need to say userland that the size of the data returned by get KVM_S390_VM_CPU_TOPOLOGY is u8.
I find this is a lack in the definition of the kvm_device_attr, it should have a size entry.
All other user of kvm_device_attr have structures and it is easy to the userland to get the size using the sizeof(struct...) one can say that userland knows that the parameter for topology is a u8 but that hurt me somehow. May be it is stupid, for the other calls the user has to know the name of the structure anyway.
Then we can say the value of u8 bit 1 is the value of the mtcr. OK for me.
What do you think?
Quoted 41 lines
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 180
Quoted 4 lines
arch/s390/include/uapi/asm/kvm.h +3
-0+ +struct kvm_cpu_topology { +__u16 mtcr :1 ;So I'd give this a more descriptive name, report_topology_change/topo_change_report_pending ?
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 181
Quoted 1 line
arch/s390/include/uapi/asm/kvm.h +1
-0+ __u16 reserved : 15;Are these bits for future proofing? If so a few more would do no harm IMO.
arch/s390/include/uapi/asm/kvm.h struct kvm_s390_vm_cpu_subfunc { · line 182
Quoted 1 line
arch/s390/include/uapi/asm/kvm.h +1
-0+};The use of a bit field in uapi surprised my, but I guess it's fine and
kvm_sync_regshas them too.Quoted 28 lines
arch/s390/include/uapi/asm/kvm.h +1
-0+/* for KVM_GET_REGS and KVM_SET_REGS */ struct kvm_regs {/* general purpose regs for s390 */ arch/s390/kvm/kvm-s390.c +9
-0@@ -606,6 +606,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_S390_PROTECTED : r =is_prot_virt_host ();break ; +case KVM_CAP_S390_CPU_TOPOLOGY : + r =test_facility (11 ); +break ;default : r =0 ; }@@ -817,6 +820,20 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)icpt_operexc_on_all_vcpus (kvm); r =0 ;break ; +case KVM_CAP_S390_CPU_TOPOLOGY : + r = -EINVAL ; +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) { + r = -EBUSY ; + }else if (test_facility (11 )) {
Quoted 8 lines
arch/s390/kvm/kvm-s390.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 834
Quoted 6 lines
arch/s390/kvm/kvm-s390.c +6
-0+set_kvm_facility (kvm->arch.model.fac_mask,11 ); +set_kvm_facility (kvm->arch.model.fac_list,11 ); + r =0 ; + } +mutex_unlock (&kvm->lock); + VM_EVENT(kvm, 3, "ENABLE: CPU TOPOLOGY %s",Most of the other cases spell out the cap, so it'd be "ENABLE:
CAP_S390_CPU_TOPOLOGY%s".
OK
Quoted 13 lines
arch/s390/kvm/kvm-s390.c static void kvm_s390_sca_set_mtcr(struct kvm *kvm) · line 1729
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +2
-0+ r ? "(not available)" : "(success)"); +break ;default : r = -EINVAL ;break ;@@ -1710,6 +1727,76 @@ static void kvm_s390_sca_set_mtcr(struct kvm *kvm)ipte_unlock (kvm); }Some brainstorming function names:
Quoted 4 lines
kvm_s390_get_topo_change_reportkvm_s390_(un|re)set_topo_change_report kvm_s390_(publish|revoke|unpublish)_topo_change_report kvm_s390_(report|signal|revoke)_topology_change
kvm_s390_update_topology_change_report ?
Quoted 3 lines
[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen