s390x: kvm: Honor storage keys during emulation
Subsystems (2)
cover s390x: kvm: Honor storage keys during emulation
Make use of the storage key support of the MEMOP ioctl, if available, in order to support storage key checking during emulation.
I did not update all the headers, since that broke the build, not sure what the best way of dealing with that is.
Janis Schoetterl-Glausch (2):
Pull in MEMOP changes in linux-headers target/s390x: kvm: Honor storage keys during emulation
linux-headers/linux/kvm.h | 11 +++++++++-- target/s390x/kvm/kvm.c | 9 +++++++++ 2 files changed, 18 insertions(+), 2 deletions(-)
base-commit: 31abf61c4929a91275fe32f1fafe6e6b3e840b2a
Signature · 1 line
2.32.0
PATCH 1/2 Pull in MEMOP changes in linux-headers
Since a full update of the linux headers pulls in changes in vfio.h that break compilation, pull in only the required changes for storage key support.
--- linux-headers/linux/kvm.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
linux-headers/linux/kvm.h +9 -2
@@ -562,9 +562,12 @@ struct kvm_s390_mem_op {__u32 op;/* type of operation */ __u64 buf;/* buffer in userspace */ union { -__u8 ar;/* the access register number */ +struct { +__u8 ar;/* the access register number */ +__u8 key;/* access key, ignored if flag unset */ + };__u32 sida_offset;/* offset into the sida */ -__u8 reserved[32 ];/* should be set to 0 */ +__u8 reserved[32 ];/* ignored */ }; };/* types for kvm_s390_mem_op->op */
@@ -572,9 +575,12 @@ struct kvm_s390_mem_op {#define KVM_S390_MEMOP_LOGICAL_WRITE 1#define KVM_S390_MEMOP_SIDA_READ 2#define KVM_S390_MEMOP_SIDA_WRITE 3 +#define KVM_S390_MEMOP_ABSOLUTE_READ 4 +#define KVM_S390_MEMOP_ABSOLUTE_WRITE 5/* flags for kvm_s390_mem_op->flags */ #define KVM_S390_MEMOP_F_CHECK_ONLY (1ULL << 0)#define KVM_S390_MEMOP_F_INJECT_EXCEPTION (1ULL << 1) +#define KVM_S390_MEMOP_F_SKEY_PROTECTION (1ULL << 2)/* for KVM_INTERRUPT */ struct kvm_interrupt {
@@ -1134,6 +1140,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_GPA_BITS 207#define KVM_CAP_XSAVE2 208#define KVM_CAP_SYS_ATTRIBUTES 209 +#define KVM_CAP_S390_MEM_OP_EXTENSION 211#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.32.0
PATCH 2/2 target/s390x: kvm: Honor storage keys during emulation
In reply to Janis Schoetterl-GlauschStorage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for the MEM_OP ioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:
* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFC
--- target/s390x/kvm/kvm.c | 9 +++++++++ 1 file changed, 9 insertions(+)
target/s390x/kvm/kvm.c +9 -0
@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = {static int cap_sync_regs;static int cap_async_pf;static int cap_mem_op; +static int cap_mem_op_extension;static int cap_s390_irq;static int cap_ri;static int cap_hpage_1m;static int cap_vcpu_resets;static int cap_protected; +static bool mem_op_storage_key_support; +static int active_cmma;static int kvm_s390_query_mem_limit (uint64_t *memory_limit)
@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs =kvm_check_extension (s,KVM_CAP_SYNC_REGS ); cap_async_pf =kvm_check_extension (s,KVM_CAP_ASYNC_PF ); cap_mem_op =kvm_check_extension (s,KVM_CAP_S390_MEM_OP ); + cap_mem_op_extension =kvm_check_extension (s,KVM_CAP_S390_MEM_OP_EXTENSION ); + mem_op_storage_key_support = cap_mem_op_extension >0 ; cap_s390_irq =kvm_check_extension (s,KVM_CAP_S390_INJECT_IRQ ); cap_vcpu_resets =kvm_check_extension (s,KVM_CAP_S390_VCPU_RESETS ); cap_protected =kvm_check_extension (s,KVM_CAP_S390_PROTECTED );
@@ -842,6 +847,7 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf, : KVM_S390_MEMOP_LOGICAL_READ, .buf = (uint64_t)hostbuf, .ar = ar, + .key = (cpu->env.psw.mask & PSW_MASK_KEY) >> PSW_SHIFT_KEY, };int ret;
@@ -851,6 +857,9 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf,if (!hostbuf) { mem_op.flags |=KVM_S390_MEMOP_F_CHECK_ONLY ; } +if (mem_op_storage_key_support) { + mem_op.flags |=KVM_S390_MEMOP_F_SKEY_PROTECTION ; + } ret =kvm_vcpu_ioctl (CPU (cpu),KVM_S390_MEM_OP , &mem_op);if (ret <0 ) {
Signature · 1 line
2.32.0
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 55 lines · Janis Schoetterl-Glausch
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>--- target/s390x/kvm/kvm.c | 9 +++++++++ 1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +3
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = {static int cap_sync_regs;static int cap_async_pf;static int cap_mem_op; +static int cap_mem_op_extension;static int cap_s390_irq;static int cap_ri;static int cap_hpage_1m;static int cap_vcpu_resets;static int cap_protected; +static bool mem_op_storage_key_support; +static int active_cmma;static int kvm_s390_query_mem_limit (uint64_t *memory_limit)
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 5 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs =kvm_check_extension (s,KVM_CAP_SYNC_REGS ); cap_async_pf =kvm_check_extension (s,KVM_CAP_ASYNC_PF ); cap_mem_op =kvm_check_extension (s,KVM_CAP_S390_MEM_OP ); + cap_mem_op_extension =kvm_check_extension (s,KVM_CAP_S390_MEM_OP_EXTENSION ); + mem_op_storage_key_support = cap_mem_op_extension >0 ;
Ah, so KVM_CAP_S390_MEM_OP_EXTENSION is a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)
(would it be better to treat it as a flag field, so that certain extensions could go away again in the future? In that case, it would be better to check with "& 1" instead of "> 0" here)
Thomas
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 53 lines · Thomas Huth
Quoted 52 lines · Janis Schoetterl-Glausch
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>---
target/s390x/kvm/kvm.c| 9 +++++++++1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)
Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written in api.rst that it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.
Quoted 5 lines
(would it be better to treat it as a flag field, so that certain extensions could go away again in the future? In that case, it would be better to check with "& 1" instead of "> 0" here)
Thomas
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 63 lines · Janis Schoetterl-Glausch
Quoted 53 lines · Thomas Huth
Quoted 52 lines · Janis Schoetterl-Glausch
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>--- target/s390x/kvm/kvm.c | 9 +++++++++ 1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)
Quoted 5 lines
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.
Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
I'm in strong favor of treating the KVM_CAP_S390_MEM_OP_EXTENSION as a flag field instead of a version number. What do others think? Christian? Halil?
Thomas
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Am 24.05.22 um 12:43 schrieb Thomas Huth:
Quoted 70 lines · Janis Schoetterl-Glausch
Quoted 63 lines · Thomas Huth
Quoted 53 lines · Janis Schoetterl-Glausch
Quoted 52 lines
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>---
target/s390x/kvm/kvm.c| 9 +++++++++1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)Quoted 5 lines
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.
Quoted 5 lines
Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
I'm in strong favor of treating the
KVM_CAP_S390_MEM_OP_EXTENSIONas a flag field instead of a version number. What do others think? Christian? Halil?
Its too late for that. This is part of 5.18.
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 75 lines · Christian Borntraeger
Am 24.05.22 um 12:43 schrieb Thomas Huth:
Quoted 72 lines · Janis Schoetterl-Glausch
Quoted 70 lines · Thomas Huth
Quoted 55 lines · Janis Schoetterl-Glausch
Quoted 54 lines
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>---
target/s390x/kvm/kvm.c| 9 +++++++++1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 362
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s,KVM_CAP_S390_MEM_OP_EXTENSION ); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.
Quoted 7 lines
Quoted 5 lines
Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
I'm in strong favor of treating the
KVM_CAP_S390_MEM_OP_EXTENSIONas a flag field instead of a version number. What do others think? Christian? Halil?Its too late for that. This is part of 5.18.
Is it? We don't have to change the source code of the kernel, it's just about rewording what we have in api.rst documentation (which should be OK as long as there is no userspace program using this yet), e.g.:
Documentation/virt/kvm/api.rst +2 -2
@@ -3759,7 +3759,7 @@ If the KVM_S390_MEMOP_F_SKEY_PROTECTION flag is set, storage key protection is also in effect and may cause exceptions if accesses are prohibited given the access key designated by "key"; the valid range is 0..15. KVM_S390_MEMOP_F_SKEY_PROTECTION is available if KVM_CAP_S390_MEM_OP_EXTENSION -is > 0. +has the lowest bit set. Absolute read/write: ^^^^^^^^^^^^^^^^^^^^
@@ -3770,7 +3770,7 @@ the checks required for storage key protection as one operation (as opposed to user space getting the storage keys, performing the checks, and accessing memory thereafter, which could lead to a delay between check and access). Absolute accesses are permitted for the VM ioctl if KVM_CAP_S390_MEM_OP_EXTENSION -is > 0. +has the lowest bit set. Currently absolute accesses are not permitted for VCPU ioctls. Absolute accesses are permitted for non-protected guests only.
What do you think?
Thomas
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 78 lines · Thomas Huth
Quoted 77 lines · Christian Borntraeger
Am 24.05.22 um 12:43 schrieb Thomas Huth:
Quoted 74 lines · Janis Schoetterl-Glausch
Quoted 63 lines · Thomas Huth
Quoted 53 lines · Janis Schoetterl-Glausch
Quoted 52 lines
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>---
target/s390x/kvm/kvm.c| 9 +++++++++1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)Quoted 5 lines
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
I'm in strong favor of treating the
KVM_CAP_S390_MEM_OP_EXTENSIONas a flag field instead of a version number. What do others think? Christian? Halil?Its too late for that. This is part of 5.18.
Is it? We don't have to change the source code of the kernel, it's just about rewording what we have in
api.rstdocumentation (which should be OK as long as there is no userspace program using this yet), e.g.:
api.rst says about KVM_CHECK_EXTENSION:
:Returns: 0 if unsupported; 1 (or some other positive integer) if supported
but if we can return a negative value, we can define flags for possible future extensions and flip the sign bit if we want to get rid of the storage key extension.
A bit ugly, but doesn't require any changes now.
Thomas Huth
AppliedRe: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
Quoted 88 lines · Janis Schoetterl-Glausch
Quoted 78 lines · Thomas Huth
Quoted 77 lines · Christian Borntraeger
Am 24.05.22 um 12:43 schrieb Thomas Huth:
Quoted 74 lines · Janis Schoetterl-Glausch
Quoted 63 lines · Thomas Huth
Quoted 53 lines · Janis Schoetterl-Glausch
Quoted 52 lines
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>--- target/s390x/kvm/kvm.c | 9 +++++++++ 1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)Quoted 5 lines
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
I'm in strong favor of treating the
KVM_CAP_S390_MEM_OP_EXTENSIONas a flag field instead of a version number. What do others think? Christian? Halil?Its too late for that. This is part of 5.18.
Is it? We don't have to change the source code of the kernel, it's just about rewording what we have in
api.rstdocumentation (which should be OK as long as there is no userspace program using this yet), e.g.:
Quoted 7 lines
api.rstsays aboutKVM_CHECK_EXTENSION:
:Returns: 0 if unsupported; 1 (or some other positive integer) if supportedbut if we can return a negative value, we can define flags for possible future extensions and flip the sign bit if we want to get rid of the storage key extension.
A bit ugly, but doesn't require any changes now.
Oh well, I hope we'll never end up in that situation ...
I guess it will likely be better to drop the MEM_OP_EXTENSION capability in that case and come up with something new instead.
Anyway, since I'm apparently the only one with my opinion, and since it's very unlikely that we want to get rid of these extensions in the future again, and we still have the big hammer of removing MEM_OP_EXTENSION completely, I won't insist on a rework here.
Queued to s390x-next now:
https://gitlab.com/thuth/qemu/-/commits/s390x-next/
Thomas
Re: [PATCH 2/2] target/s390x: kvm: Honor storage keys during emulation
In reply to Thomas HuthQuoted 74 lines · On Tue, 24 May 2022 12:43:29 +0200 Thomas Huth <[email protected]> wrote:
Quoted 63 lines · Janis Schoetterl-Glausch
Quoted 53 lines · Thomas Huth
Quoted 52 lines · Janis Schoetterl-Glausch
Storage key controlled protection is currently not honored when emulating instructions.
If available, enable key protection for theMEM_OPioctl, thereby enabling it for the s390_cpu_virt_mem_* functions, when using kvm. As a result, the emulation of the following instructions honors storage keys:* CLP
The Synch I/O CLP command would need special handling in order to support storage keys, but is currently not supported.
* CHSC
Performing commands asynchronously would require special handling, but commands are currently always synchronous.
* STSI
* TSCH
Must (and does) not change channel if terminated due to protection.
* MSCH
Suppressed on protection, works because fetching instruction.
* SSCH
Suppressed on protection, works because fetching instruction.
* STSCH
* STCRW
Suppressed on protection, this works because no partial store is possible, because the operand cannot span multiple pages.
* PCISTB
* MPCIFC
* STPCIFCSigned-off-by Janis Schoetterl-Glausch <[email protected]>--- target/s390x/kvm/kvm.c | 9 +++++++++ 1 file changed, 9 insertions(+)target/s390x/kvm/kvm.c +2
-0@@ -151,12 +151,15 @@ const KVMCapabilityInfo kvm_arch_required_capabilities[] = { static int cap_sync_regs; static int cap_async_pf; static int cap_mem_op; +static int cap_mem_op_extension; static int cap_s390_irq; static int cap_ri; static int cap_hpage_1m; static int cap_vcpu_resets; static int cap_protected; +static bool mem_op_storage_key_support; + static int active_cmma; static int kvm_s390_query_mem_limit(uint64_t *memory_limit)Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 361
Quoted 6 lines
target/s390x/kvm/kvm.c +2
-0@@ -354,6 +357,8 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP); + cap_mem_op_extension = kvm_check_extension(s, KVM_CAP_S390_MEM_OP_EXTENSION); + mem_op_storage_key_support = cap_mem_op_extension > 0;Ah, so
KVM_CAP_S390_MEM_OP_EXTENSIONis a "version number", not a boolean flag? ... ok, now I've finally understood that ... ;-)Quoted 5 lines
Yeah, potentially having a bunch of memop capabilities didn't seem nice to me.
We can remove extensions if, when introducing an extension, we define that version x supports functionality y, z..., but for the storage keys I've written inapi.rstthat it's supported if the cap > 0.
So we'd need a new cap if we want to get rid of the skey extension and still support some other extension, but that doesn't seem particularly likely.Oh well, never say that ... we've seen it in the past, that sometimes we want to get rid of features again, and if they don't have a separate feature flag bit somewhere, it's getting very ugly to disable them again.
Quoted 6 lines
So since we don't have merged this patch yet, and thus we don't have a public userspace program using this interface yet, this is our last chance to redefine this interface before we might regret it later.
I'm in strong favor of treating the
KVM_CAP_S390_MEM_OP_EXTENSIONas a flag field instead of a version number. What do others think? Christian? Halil?
I don't fully understand the problem, and I don't have a strong opinion. What I understand is KVM_CAP_S390_MEM_OP_EXTENSION tells me if some mem op extensions may be available if non-zero or that none are available. Which mem-op extensions are available is not yet actually defined.
I can think some more, but feel free to proceed without me.
Regards,
Halil
Quoted 5 lines · Janis Schoetterl-Glausch
Make use of the storage key support of the MEMOP ioctl, if available, in order to support storage key checking during emulation.
I did not update all the headers, since that broke the build, not sure what the best way of dealing with that is.
Yeah, the vfio change is expected to break the build; the fix should be easy (simple rename), and the code affected is deprecated anyway (there hasn't been any upstream implementation that actually exposed the interfaces). I think we should do that in a single commit to preserve bisectability; I have not seen any patches posted yet to actually use the new vfio migration interface, so a simple compile fixup should be all that is needed.
Quoted 13 lines
Janis Schoetterl-Glausch (2):
Pull in MEMOP changes in linux-headers target/s390x: kvm: Honor storage keys during emulationlinux-headers/linux/kvm.h | 11 +++++++++-- target/s390x/kvm/kvm.c | 9 +++++++++ 2 files changed, 18 insertions(+), 2 deletions(-)base-commit:
31abf61c4929a91275fe32f1fafe6e6b3e840b2aSignature · 1 line
2.32.0
Quoted 8 lines · Cornelia Huck
Quoted 5 lines · Janis Schoetterl-Glausch
Make use of the storage key support of the MEMOP ioctl, if available, in order to support storage key checking during emulation.
I did not update all the headers, since that broke the build, not sure what the best way of dealing with that is.
Quoted 7 lines
Yeah, the vfio change is expected to break the build; the fix should be easy (simple rename), and the code affected is deprecated anyway (there hasn't been any upstream implementation that actually exposed the interfaces). I think we should do that in a single commit to preserve bisectability; I have not seen any patches posted yet to actually use the new vfio migration interface, so a simple compile fixup should be all that is needed.
So basically this patch (pasted below)
https://lore.kernel.org/qemu-devel/[email protected]/
squashed with the updated headers.
Subject: [PATCH v5 2/9] vfio: tolerate migration protocol v1 uapi renames
Date: Mon, 4 Apr 2022 14:17:19 -0400 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
The v1 uapi is deprecated and will be replaced by v2 at some point; this patch just tolerates the renaming of uapi fields to reflect v1 / deprecated status.
--- hw/vfio/common.c | 2 +- hw/vfio/migration.c | 19 +++++++++++-------- 2 files changed, 12 insertions(+), 9 deletions(-)
hw/vfio/common.c +1 -1
@@ -380,7 +380,7 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container)return false; } -if ((migration->device_state &VFIO_DEVICE_STATE_SAVING ) && +if ((migration->device_state &VFIO_DEVICE_STATE_V1_SAVING ) && (migration->device_state &VFIO_DEVICE_STATE_RUNNING )) {continue ; }else {
hw/vfio/migration.c +11 -8
@@ -432,7 +432,7 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) } ret =vfio_migration_set_state (vbasedev,VFIO_DEVICE_STATE_MASK , -VFIO_DEVICE_STATE_SAVING ); +VFIO_DEVICE_STATE_V1_SAVING );if (ret) {error_report ("%s: Failed to set state SAVING" , vbasedev->name);return ret;
@@ -532,7 +532,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)int ret; ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_RUNNING , -VFIO_DEVICE_STATE_SAVING ); +VFIO_DEVICE_STATE_V1_SAVING );if (ret) {error_report ("%s: Failed to set state STOP and SAVING" , vbasedev->name);
@@ -569,7 +569,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)return ret; } - ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_SAVING ,0 ); + ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING ,0 );if (ret) {error_report ("%s: Failed to set state STOPPED" , vbasedev->name);return ret;
@@ -730,7 +730,7 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state)* start saving data. */ if (state ==RUN_STATE_SAVE_VM ) { - value =VFIO_DEVICE_STATE_SAVING ; + value =VFIO_DEVICE_STATE_V1_SAVING ; }else { value =0 ; }
@@ -768,8 +768,9 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data)case MIGRATION_STATUS_FAILED : bytes_transferred =0 ; ret =vfio_migration_set_state (vbasedev, - ~(VFIO_DEVICE_STATE_SAVING |VFIO_DEVICE_STATE_RESUMING ), -VFIO_DEVICE_STATE_RUNNING ); + ~(VFIO_DEVICE_STATE_V1_SAVING | +VFIO_DEVICE_STATE_RESUMING ), +VFIO_DEVICE_STATE_RUNNING );if (ret) {error_report ("%s: Failed to set state RUNNING" , vbasedev->name); }
@@ -864,8 +865,10 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) goto add_blocker; } - ret =vfio_get_dev_region_info (vbasedev,VFIO_REGION_TYPE_MIGRATION , -VFIO_REGION_SUBTYPE_MIGRATION , &info); + ret =vfio_get_dev_region_info (vbasedev, +VFIO_REGION_TYPE_MIGRATION_DEPRECATED , +VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED , + &info);if (ret) { goto add_blocker; }
Quoted 15 lines
Quoted 14 lines
Janis Schoetterl-Glausch (2):
Pull in MEMOP changes in linux-headers target/s390x: kvm: Honor storage keys during emulationlinux-headers/linux/kvm.h | 11 +++++++++-- target/s390x/kvm/kvm.c | 9 +++++++++ 2 files changed, 18 insertions(+), 2 deletions(-)base-commit:
31abf61c4929a91275fe32f1fafe6e6b3e840b2aSignature · 1 line
2.32.0
Quoted 17 lines · Janis Schoetterl-Glausch
Quoted 8 lines · Cornelia Huck
Quoted 5 lines · Janis Schoetterl-Glausch
Make use of the storage key support of the MEMOP ioctl, if available, in order to support storage key checking during emulation.
I did not update all the headers, since that broke the build, not sure what the best way of dealing with that is.
Quoted 7 lines
Yeah, the vfio change is expected to break the build; the fix should be easy (simple rename), and the code affected is deprecated anyway (there hasn't been any upstream implementation that actually exposed the interfaces). I think we should do that in a single commit to preserve bisectability; I have not seen any patches posted yet to actually use the new vfio migration interface, so a simple compile fixup should be all that is needed.
Quoted 3 lines
So basically this patch (pasted below)
https://lore.kernel.org/qemu-devel/[email protected]/
squashed with the updated headers.
Yes. We should probably queue that seperately, just to disarm that trap for everyone; unless there's already a vfio update in flight? (Sorry, I've lost track a bit.)
Quoted 109 lines
Subject: [PATCH v5 2/9] vfio: tolerate migration protocol v1 uapi renames
Date: Mon, 4 Apr 2022 14:17:19 -0400 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>The v1 uapi is deprecated and will be replaced by v2 at some point; this patch just tolerates the renaming of uapi fields to reflect v1 / deprecated status.
Signed-off-by Matthew Rosato <[email protected]>--- hw/vfio/common.c | 2 +- hw/vfio/migration.c | 19 +++++++++++-------- 2 files changed, 12 insertions(+), 9 deletions(-)hw/vfio/common.c +1
-1@@ -380,7 +380,7 @@ static bool vfio_devices_all_running_and_saving(VFIOContainer *container)return false; } -if ((migration->device_state &VFIO_DEVICE_STATE_SAVING ) && +if ((migration->device_state &VFIO_DEVICE_STATE_V1_SAVING ) && (migration->device_state &VFIO_DEVICE_STATE_RUNNING )) {continue ; }else {hw/vfio/migration.c +11
-8@@ -432,7 +432,7 @@ static int vfio_save_setup(QEMUFile *f, void *opaque) } ret =vfio_migration_set_state (vbasedev,VFIO_DEVICE_STATE_MASK , -VFIO_DEVICE_STATE_SAVING ); +VFIO_DEVICE_STATE_V1_SAVING );if (ret) {error_report ("%s: Failed to set state SAVING" , vbasedev->name);return ret;@@ -532,7 +532,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)int ret; ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_RUNNING , -VFIO_DEVICE_STATE_SAVING ); +VFIO_DEVICE_STATE_V1_SAVING );if (ret) {error_report ("%s: Failed to set state STOP and SAVING" , vbasedev->name);@@ -569,7 +569,7 @@ static int vfio_save_complete_precopy(QEMUFile *f, void *opaque)return ret; } - ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_SAVING ,0 ); + ret =vfio_migration_set_state (vbasedev, ~VFIO_DEVICE_STATE_V1_SAVING ,0 );if (ret) {error_report ("%s: Failed to set state STOPPED" , vbasedev->name);return ret;@@ -730,7 +730,7 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state)* start saving data. */ if (state ==RUN_STATE_SAVE_VM ) { - value =VFIO_DEVICE_STATE_SAVING ; + value =VFIO_DEVICE_STATE_V1_SAVING ; }else { value =0 ; }@@ -768,8 +768,9 @@ static void vfio_migration_state_notifier(Notifier *notifier, void *data)case MIGRATION_STATUS_FAILED : bytes_transferred =0 ; ret =vfio_migration_set_state (vbasedev, - ~(VFIO_DEVICE_STATE_SAVING |VFIO_DEVICE_STATE_RESUMING ), -VFIO_DEVICE_STATE_RUNNING ); + ~(VFIO_DEVICE_STATE_V1_SAVING | +VFIO_DEVICE_STATE_RESUMING ), +VFIO_DEVICE_STATE_RUNNING );if (ret) {error_report ("%s: Failed to set state RUNNING" , vbasedev->name); }@@ -864,8 +865,10 @@ int vfio_migration_probe(VFIODevice *vbasedev, Error **errp) goto add_blocker; } - ret =vfio_get_dev_region_info (vbasedev,VFIO_REGION_TYPE_MIGRATION , -VFIO_REGION_SUBTYPE_MIGRATION , &info); + ret =vfio_get_dev_region_info (vbasedev, +VFIO_REGION_TYPE_MIGRATION_DEPRECATED , +VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED , + &info);if (ret) { goto add_blocker; }Quoted 15 lines
Quoted 14 lines
Janis Schoetterl-Glausch (2):
Pull in MEMOP changes in linux-headers target/s390x: kvm: Honor storage keys during emulationlinux-headers/linux/kvm.h | 11 +++++++++-- target/s390x/kvm/kvm.c | 9 +++++++++ 2 files changed, 18 insertions(+), 2 deletions(-)base-commit:
31abf61c4929a91275fe32f1fafe6e6b3e840b2aSignature · 1 line
2.32.0
Quoted 19 lines · Cornelia Huck
Quoted 17 lines · Janis Schoetterl-Glausch
Quoted 15 lines · Cornelia Huck
Quoted 5 lines · Janis Schoetterl-Glausch
Make use of the storage key support of the MEMOP ioctl, if available, in order to support storage key checking during emulation.
I did not update all the headers, since that broke the build, not sure what the best way of dealing with that is.
Yeah, the vfio change is expected to break the build; the fix should be easy (simple rename), and the code affected is deprecated anyway (there hasn't been any upstream implementation that actually exposed the interfaces). I think we should do that in a single commit to preserve bisectability; I have not seen any patches posted yet to actually use the new vfio migration interface, so a simple compile fixup should be all that is needed.
Quoted 7 lines
Quoted 3 lines
So basically this patch (pasted below)
https://lore.kernel.org/qemu-devel/[email protected]/
squashed with the updated headers.Yes. We should probably queue that seperately, just to disarm that trap for everyone; unless there's already a vfio update in flight? (Sorry, I've lost track a bit.)
Unless somebody else has queued this already, I can try to come up with a separate pull request for the header update today or tomorrow.
Thomas