s390x: Protected Virtualization support
v3 cover s390x: Protected Virtualization support
Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.
The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.
V3:
* Use dedicated functions to access SIDA
* Smaller cleanups and segfault fixes
* Error reporting for Ultravisor calls
* Inject of RC of diag308 subcode 10 fails
V2:
* Split out cleanups
* Internal PV state tracking
* Review feedback
Janosch Frank (17):
Header sync
s390x: Add missing vcpu reset functions
Sync pv
s390x: protvirt: Add diag308 subcodes 8 - 10
s390x: protvirt: Support unpack facility
s390x: protvirt: Add migration blocker
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390: protvirt: Move STSI data over SIDAD
s390x: Add SIDA memory ops
s390x: protvirt: SCLP interpretation
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Move IO control structures over SIDA
s390x: protvirt: Handle SIGP store status correctly
s390x: For now add unpack feature to GA1
hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 80 +++++++++++++- hw/s390x/ipl.h | 33 ++++++ hw/s390x/pv.c | 160 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 40 +++++++ hw/s390x/s390-virtio-ccw.c | 136 ++++++++++++++++++++++- hw/s390x/sclp.c | 17 +++ include/hw/s390x/s390-virtio-ccw.h | 1 + include/hw/s390x/sclp.h | 2 + linux-headers/linux/kvm.h | 48 ++++++++- target/s390x/cpu.c | 41 +++++-- target/s390x/cpu.h | 8 +- target/s390x/cpu_features_def.inc.h | 1 + target/s390x/diag.c | 63 +++++++++-- target/s390x/gen-features.c | 1 + target/s390x/helper.c | 4 + target/s390x/ioinst.c | 113 ++++++++++++++------ target/s390x/kvm-stub.c | 10 +- target/s390x/kvm.c | 89 ++++++++++++++-- target/s390x/kvm_s390x.h | 6 +- target/s390x/mmu_helper.c | 9 ++ target/s390x/sigp.c | 1 + 22 files changed, 789 insertions(+), 75 deletions(-) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h
Signature · 1 line
2.20.1
PATCH v3 8/17 s390x: protvirt: KVM intercept changes
Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.
The 104 mirrors the 4 interception.
The 108 is a notification interception to let KVM and QEMU know that something changed and we need to update tracking information or perform specific tasks. It's currently taken for the following instructions:
* stpx (To inform about the changed prefix location)
* sclp (On incorrect SCCB values, so we can inject a IRQ)
* sigp (All but "stop and store status")
* diag308 (Subcodes 0/1)
--- target/s390x/kvm.c | 4 ++++ 1 file changed, 4 insertions(+)
target/s390x/kvm.c +4 -0
@@ -115,6 +115,8 @@#define ICPT_CPU_STOP 0x28#define ICPT_OPEREXC 0x2c#define ICPT_IO 0x40 +#define ICPT_PV_INSTR 0x68 +#define ICPT_PV_INSTR_NOTIFICATION 0x6c#define NR_LOCAL_IRQS 32/*
@@ -1693,6 +1695,8 @@ static int handle_intercept(S390CPU *cpu) (long)cs->kvm_run->psw_addr);switch (icpt_code) {case ICPT_INSTRUCTION : +case ICPT_PV_INSTR : +case ICPT_PV_INSTR_NOTIFICATION : r =handle_instruction (cpu, run);break ;case ICPT_PROGRAM :
Signature · 1 line
2.20.1
PATCH v3 7/17 s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
In reply to Janosch FrankAs we now have access to the protection state of the cpus, we can implement special handling of diag 308 subcodes for cpus in the protected state.
For subcodes 0 and 1 we need to unshare all pages before continuing, so the guest doesn't accidentally expose data when dumping.
For subcode 3/4 we tear down the protected VM and reboot into unprotected mode. We do not provide a secure reboot.
Before we can do the unshare calls, we need to mark all cpus as stopped.
--- hw/s390x/s390-virtio-ccw.c | 41 +++++++++++++++++++++++++++++++++++--- target/s390x/diag.c | 4 ++++ 2 files changed, 42 insertions(+), 3 deletions(-)
hw/s390x/s390-virtio-ccw.c +38 -3
@@ -370,6 +370,20 @@ static void s390_machine_inject_pv_error(CPUState *cs) env->regs[r1 +1 ] =0xa02 ; } +static void s390_pv_prepare_reset (CPUS390XState *env) +{ +CPUState *cs; + +if (!env->pv) { +return ; + } +CPU_FOREACH (cs) { +s390_cpu_set_state (S390_CPU_STATE_STOPPED ,S390_CPU (cs)); + } +s390_pv_unshare (); +s390_pv_perf_clear_reset (); +} +static void s390_machine_reset (MachineState *machine) {enum s390_reset reset_type;
@@ -377,6 +391,7 @@ static void s390_machine_reset(MachineState *machine)S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine);static Error *local_err; +CPUS390XState *env;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);
@@ -385,10 +400,20 @@ static void s390_machine_reset(MachineState *machine)s390_cmma_reset (); cpu =S390_CPU (cs); + env = &cpu->env;switch (reset_type) {case S390_RESET_EXTERNAL :case S390_RESET_REIPL : +if (ms->pv) { +CPU_FOREACH (t) { +s390_pv_vcpu_destroy (t); + } +s390_pv_vm_destroy (); + ms->pv = false; +migrate_del_blocker (pv_mig_blocker); + } +qemu_devices_reset ();s390_crypto_reset ();
@@ -396,21 +421,31 @@ static void s390_machine_reset(MachineState *machine)run_on_cpu (cs, s390_do_cpu_ipl,RUN_ON_CPU_NULL );break ;case S390_RESET_MODIFIED_CLEAR : +/* +* Susbsystem reset needs to be done before we unshare memory +* and loose access to VIRTIO structures in guest memory. +*/ +subsystem_reset (); +s390_crypto_reset (); +s390_pv_prepare_reset (env); CPU_FOREACH(t) {run_on_cpu (t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL); } -subsystem_reset (); -s390_crypto_reset ();run_on_cpu (cs, s390_do_cpu_load_normal,RUN_ON_CPU_NULL );break ;case S390_RESET_LOAD_NORMAL : +/* +* Susbsystem reset needs to be done before we unshare memory +* and loose access to VIRTIO structures in guest memory. +*/ +subsystem_reset (); +s390_pv_prepare_reset (env); CPU_FOREACH(t) {if (t == cs) {continue ; }run_on_cpu (t, s390_do_cpu_reset,RUN_ON_CPU_NULL ); } -subsystem_reset ();run_on_cpu (cs, s390_do_cpu_initial_reset,RUN_ON_CPU_NULL );run_on_cpu (cs, s390_do_cpu_load_normal,RUN_ON_CPU_NULL );break ;
target/s390x/diag.c +4 -0
@@ -68,6 +68,10 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)static int diag308_parm_check (CPUS390XState *env,uint64_t r1,uint64_t addr,uintptr_t ra,bool write) { +/* Handled by the Ultravisor */ +if (env->pv) { +return 0 ; + }if ((r1 &1 ) || (addr & ~TARGET_PAGE_MASK )) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return -1 ;
Signature · 1 line
2.20.1
PATCH v3 2/17 s390x: Add missing vcpu reset functions
In reply to Janosch FrankUp to now we only had an ioctl to reset vcpu data QEMU couldn't reach for the initial reset, which was also called for the clear reset. To be architecture compliant, we also need to clear local interrupts on a normal reset.
Because of this and the upcoming protvirt support we need to add ioctls for the missing clear and normal resets.
--- target/s390x/cpu.c | 14 ++++++++++++-- target/s390x/kvm-stub.c | 10 +++++++++- target/s390x/kvm.c | 42 ++++++++++++++++++++++++++++++++-------- target/s390x/kvm_s390x.h | 4 +++- 4 files changed, 58 insertions(+), 12 deletions(-)
target/s390x/cpu.c +12 -2
@@ -144,8 +144,18 @@ static void s390_cpu_reset(CPUState *s, cpu_reset_type type) }/* Reset state inside the kernel that we cannot access yet from QEMU. */ -if (kvm_enabled () && type !=S390_CPU_RESET_NORMAL ) { -kvm_s390_reset_vcpu (cpu); +if (kvm_enabled ()) { +switch (type) { +case S390_CPU_RESET_CLEAR : +kvm_s390_reset_vcpu_clear (cpu); +break ; +case S390_CPU_RESET_INITIAL : +kvm_s390_reset_vcpu_initial (cpu); +break ; +case S390_CPU_RESET_NORMAL : +kvm_s390_reset_vcpu_normal (cpu); +break ; + } } }
target/s390x/kvm-stub.c +9 -1
@@ -83,7 +83,15 @@ void kvm_s390_cmma_reset(void) { } -void kvm_s390_reset_vcpu (S390CPU *cpu) +void kvm_s390_reset_vcpu_initial (S390CPU *cpu) +{ +} + +void kvm_s390_reset_vcpu_clear (S390CPU *cpu) +{ +} + +void kvm_s390_reset_vcpu_normal (S390CPU *cpu) { }
target/s390x/kvm.c +34 -8
63 lines not on this page · raw message
target/s390x/kvm_s390x.h +3 -1
@@ -34,7 +34,9 @@ int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch, int vq, bool assign);int kvm_s390_cmma_active (void );void kvm_s390_cmma_reset (void ); -void kvm_s390_reset_vcpu (S390CPU *cpu); +void kvm_s390_reset_vcpu_clear (S390CPU *cpu); +void kvm_s390_reset_vcpu_normal (S390CPU *cpu); +void kvm_s390_reset_vcpu_initial (S390CPU *cpu);int kvm_s390_set_mem_limit (uint64_t new_limit,uint64_t *hw_limit);void kvm_s390_set_max_pagesize (uint64_t pagesize,Error **errp);void kvm_s390_crypto_reset (void );
Signature · 1 line
2.20.1
PATCH v3 1/17 Header sync
In reply to Janosch FrankSync in the new vcpu resets.
--- linux-headers/linux/kvm.h | 4 ++++ 1 file changed, 4 insertions(+)
linux-headers/linux/kvm.h +4 -0
@@ -1009,6 +1009,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_PPC_GUEST_DEBUG_SSTEP 176#define KVM_CAP_ARM_NISV_TO_USER 177#define KVM_CAP_ARM_INJECT_EXT_DABT 178 +#define KVM_CAP_S390_VCPU_RESETS 179#ifdef KVM_CAP_IRQ_ROUTING
@@ -1473,6 +1474,9 @@ struct kvm_enc_region {/* Available with KVM_CAP_ARM_SVE */ #define KVM_ARM_VCPU_FINALIZE _IOW(KVMIO, 0xc2, int) +#define KVM_S390_NORMAL_RESET _IO(KVMIO, 0xc3) +#define KVM_S390_CLEAR_RESET _IO(KVMIO, 0xc4) +/* Secure Encrypted Virtualization command */ enum sev_cmd_id {/* Guest initialization commands */
Signature · 1 line
2.20.1
PATCH v3 3/17 Sync pv
In reply to Janosch Frank--- linux-headers/linux/kvm.h | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
linux-headers/linux/kvm.h +37 -0
@@ -1010,6 +1010,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_ARM_NISV_TO_USER 177#define KVM_CAP_ARM_INJECT_EXT_DABT 178#define KVM_CAP_S390_VCPU_RESETS 179 +#define KVM_CAP_S390_PROTECTED 181#ifdef KVM_CAP_IRQ_ROUTING
@@ -1477,6 +1478,42 @@ struct kvm_enc_region {#define KVM_S390_NORMAL_RESET _IO(KVMIO, 0xc3)#define KVM_S390_CLEAR_RESET _IO(KVMIO, 0xc4) +struct kvm_s390_pv_sec_parm { +__u64 origin; +__u64 length; +}; + +struct kvm_s390_pv_unp { +__u64 addr; +__u64 size; +__u64 tweak; +}; + +enum pv_cmd_id { +KVM_PV_VM_CREATE , +KVM_PV_VM_DESTROY , +KVM_PV_VM_SET_SEC_PARMS , +KVM_PV_VM_UNPACK , +KVM_PV_VM_VERIFY , +KVM_PV_VM_PREP_RESET , +KVM_PV_VM_UNSHARE_ALL , +KVM_PV_VCPU_CREATE , +KVM_PV_VCPU_DESTROY , +}; + +struct kvm_pv_cmd { +__u32 cmd;/* Command to be executed */ +__u16 rc;/* Ultravisor return code */ +__u16 rrc;/* Ultravisor return reason code */ +__u64 data;/* Data or address */ +__u32 flags;/* flags for future extensions. Must be 0 for now */ +__u32 reserved[3 ]; +}; + +/* Available with KVM_CAP_S390_PROTECTED */ +#define KVM_S390_PV_COMMAND _IOWR(KVMIO, 0xc5, struct kvm_pv_cmd) +#define KVM_S390_PV_COMMAND_VCPU _IOWR(KVMIO, 0xc6, struct kvm_pv_cmd) +/* Secure Encrypted Virtualization command */ enum sev_cmd_id {/* Guest initialization commands */
Signature · 1 line
2.20.1
PATCH v3 5/17 s390x: protvirt: Support unpack facility
In reply to Janosch FrankWhen a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.
The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity
Only after step 5 a protected VM is allowed to run.
to machine]
--- hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 32 ++++++ hw/s390x/ipl.h | 2 + hw/s390x/pv.c | 154 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 38 +++++++ hw/s390x/s390-virtio-ccw.c | 79 ++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 1 + target/s390x/cpu.c | 4 + target/s390x/cpu.h | 1 + target/s390x/cpu_features_def.inc.h | 1 + 10 files changed, 313 insertions(+) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h
hw/s390x/Makefile.objs +1 -0
@@ -31,6 +31,7 @@ obj-y += tod-qemu.o obj-$(CONFIG_KVM) += tod-kvm.o obj-$(CONFIG_KVM) += s390-skeys-kvm.o obj-$(CONFIG_KVM) += s390-stattrib-kvm.o +obj-$(CONFIG_KVM) += pv.o obj-y += s390-ccw.o obj-y += ap-device.o obj-y += ap-bridge.o
hw/s390x/ipl.c +32 -0
@@ -33,6 +33,7 @@#include "qemu/cutils.h" #include "qemu/option.h" #include "exec/exec-all.h" +#include "pv.h" #define KERN_IMAGE_START 0x010000UL#define LINUX_MAGIC_ADDR 0x010008UL
@@ -677,6 +678,37 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)cpu_physical_memory_unmap (addr, len,1 , len); } +int s390_ipl_prepare_pv_header (void ) +{ +S390IPLState *ipl =get_ipl_device (); +IPLBlockPV *ipib_pv = &ipl->iplb_pbt5.pv; +void *hdr =g_malloc (ipib_pv->pv_header_len); +int rc; + +cpu_physical_memory_read (ipib_pv->pv_header_addr, hdr, + ipib_pv->pv_header_len); + rc =s390_pv_set_sec_parms ((uint64_t )hdr, + ipib_pv->pv_header_len); +g_free (hdr); +return rc; +} + +int s390_ipl_pv_unpack (void ) +{ +int i, rc; +S390IPLState *ipl =get_ipl_device (); +IPLBlockPV *ipib_pv = &ipl->iplb_pbt5.pv; + +for (i =0 ; i < ipib_pv->num_comp; i++) { + rc =s390_pv_unpack (ipib_pv->components[i].addr, +TARGET_PAGE_ALIGN (ipib_pv->components[i].size), + ipib_pv->components[i].tweak_pref); +if (rc) +return rc; + } +return 0 ; +} +void s390_ipl_prepare_cpu (S390CPU *cpu) {S390IPLState *ipl =get_ipl_device ();
hw/s390x/ipl.h +2 -0
@@ -104,6 +104,8 @@ typedef union IplParameterBlock IplParameterBlock;int s390_ipl_set_loadparm (uint8_t *loadparm);int s390_ipl_pv_check_components (IplParameterBlock *iplb);void s390_ipl_update_diag308 (IplParameterBlock *iplb); +int s390_ipl_prepare_pv_header (void ); +int s390_ipl_pv_unpack (void );void s390_ipl_prepare_cpu (S390CPU *cpu);IplParameterBlock *s390_ipl_get_iplb (void );IplParameterBlock *s390_ipl_get_iplb_secure (void );
hw/s390x/pv.c new file +154 -0
154 lines not on this page · raw message
hw/s390x/pv.h new file +38 -0
@@ -0,0 +1,38 @@ +/* +* Protected Virtualization header +* +* Copyright IBM Corp. 2019 +* Author(s): +* Janosch Frank <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ + +#ifndef HW_S390_PV_H +#define HW_S390_PV_H + +#ifdef CONFIG_KVM +int s390_pv_vm_create (void ); +void s390_pv_vm_destroy (void ); +void s390_pv_vcpu_destroy (CPUState *cs); +int s390_pv_vcpu_create (CPUState *cs); +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length); +int s390_pv_unpack (uint64_t addr,uint64_t size,uint64_t tweak); +void s390_pv_perf_clear_reset (void ); +int s390_pv_verify (void ); +void s390_pv_unshare (void ); +#else +int s390_pv_vm_create (void ) {return 0 ; } +void s390_pv_vm_destroy (void ) {} +void s390_pv_vcpu_destroy (CPUState *cs) {} +int s390_pv_vcpu_create (CPUState *cs) {return 0 ; } +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length) {return 0 ; } +int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: } +void s390_pv_perf_clear_reset (void ) {} +int s390_pv_verify (void ) {return 0 ; } +void s390_pv_unshare (void ) {} +#endif + +#endif /* HW_S390_PV_H */
hw/s390x/s390-virtio-ccw.c +79 -0
116 lines not on this page · raw message
include/hw/s390x/s390-virtio-ccw.h +1 -0
@@ -28,6 +28,7 @@ typedef struct S390CcwMachineState {/*< public >*/ bool aes_key_wrap;bool dea_key_wrap; +bool pv;uint8_t loadparm[8 ]; } S390CcwMachineState;
target/s390x/cpu.c +4 -0
@@ -37,6 +37,8 @@#include "sysemu/hw_accel.h" #include "hw/qdev-properties.h" #ifndef CONFIG_USER_ONLY +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/pv.h" #include "hw/boards.h" #include "sysemu/arch_init.h" #include "sysemu/sysemu.h"
@@ -191,6 +193,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)#if !defined(CONFIG_USER_ONLY )MachineState *ms =MACHINE (qdev_get_machine ()); +S390CcwMachineState *ccw =S390_CCW_MACHINE (ms);unsigned int max_cpus = ms->smp.max_cpus;if (cpu->env.core_id >= max_cpus) { error_setg(&err, "Unable to add CPU with core-id: %" PRIu32
@@ -205,6 +208,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp) goto out; } + cpu->env.pv = ccw->pv;/* sync cs->cpu_index and env->core_id. The latter is needed for TCG. */ cs->cpu_index = cpu->env.core_id;#endif
target/s390x/cpu.h +1 -0
@@ -114,6 +114,7 @@ struct CPUS390XState {/* Fields up to this point are cleared by a CPU reset */ struct {} end_reset_fields; +bool pv;/* protected virtualization */ #if !defined(CONFIG_USER_ONLY )uint32_t core_id;/* PoP "CPU address", same as cpu_index */
target/s390x/cpu_features_def.inc.h +1 -0
@@ -107,6 +107,7 @@ DEF_FEAT(DEFLATE_BASE, "deflate-base", STFL, 151, "Deflate-conversion facility ( DEF_FEAT(VECTOR_PACKED_DECIMAL_ENH, "vxpdeh", STFL, 152, "Vector-Packed-Decimal-Enhancement Facility") DEF_FEAT(MSA_EXT_9, "msa9-base", STFL, 155, "Message-security-assist-extension-9 facility (excluding subfunctions)") DEF_FEAT(ETOKEN, "etoken", STFL, 156, "Etoken facility") +DEF_FEAT (UNPACK ,"unpack" ,STFL ,161 ,"Unpack facility" ) /* Features exposed via SCLP SCCB Byte 80 - 98 (bit numbers relative to byte-80) */DEF_FEAT (SIE_GSLS ,"gsls" ,SCLP_CONF_CHAR ,40 ,"SIE: Guest-storage-limit-suppression facility" )
Signature · 1 line
2.20.1
PATCH v3 12/17 s390x: protvirt: Set guest IPL PSW
In reply to Janosch FrankHandling of CPU reset and setting of the IPL psw from guest storage at offset 0 is done by a Ultravisor call. Let's only fetch it if necessary.
--- hw/s390x/pv.c | 6 ++++++ hw/s390x/pv.h | 2 ++ linux-headers/linux/kvm.h | 1 + target/s390x/cpu.c | 23 ++++++++++++++--------- 4 files changed, 23 insertions(+), 9 deletions(-)
hw/s390x/pv.c +6 -0
@@ -28,6 +28,7 @@ const char* cmd_names[] = {"VM_UNSHARE_ALL" ,"VCPU_CREATE" ,"VCPU_DESTROY" , +"VCPU_SET_IPL_PSW" , NULL };
@@ -124,6 +125,11 @@ int s390_pv_set_sec_parms(uint64_t origin, uint64_t length)return s390_pv_cmd (KVM_PV_VM_SET_SEC_PARMS , &args); } +void s390_pv_set_ipl_psw (CPUState *cs) +{ +s390_pv_cmd_vcpu_exit (cs,KVM_PV_VCPU_SET_IPL_PSW ,NULL ); +} +/* * Called for each component in the SE type IPL parameter block 0. */
hw/s390x/pv.h +2 -0
@@ -19,6 +19,7 @@ void s390_pv_vm_destroy(void);void s390_pv_vcpu_destroy (CPUState *cs);int s390_pv_vcpu_create (CPUState *cs);int s390_pv_set_sec_parms (uint64_t origin,uint64_t length); +void s390_pv_set_ipl_psw (CPUState *cs);int s390_pv_unpack (uint64_t addr,uint64_t size,uint64_t tweak);void s390_pv_perf_clear_reset (void );int s390_pv_verify (void );
@@ -29,6 +30,7 @@ void s390_pv_vm_destroy(void) {}void s390_pv_vcpu_destroy (CPUState *cs) {}int s390_pv_vcpu_create (CPUState *cs) {return 0 ; }int s390_pv_set_sec_parms (uint64_t origin,uint64_t length) {return 0 ; } +void s390_pv_set_ipl_psw (CPUState *cs) {} int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: }void s390_pv_perf_clear_reset (void ) {}int s390_pv_verify (void ) {return 0 ; }
linux-headers/linux/kvm.h +1 -0
@@ -1503,6 +1503,7 @@ enum pv_cmd_id {KVM_PV_VM_UNSHARE_ALL ,KVM_PV_VCPU_CREATE ,KVM_PV_VCPU_DESTROY , + KVM_PV_VCPU_SET_IPL_PSW, }; struct kvm_pv_cmd {
target/s390x/cpu.c +14 -9
@@ -78,16 +78,21 @@ static bool s390_cpu_has_work(CPUState *cs)static void s390_cpu_load_normal (CPUState *s) {S390CPU *cpu =S390_CPU (s); -uint64_t spsw =ldq_phys (s->as,0 ); - - cpu->env.psw.mask = spsw &0xffffffff80000000ULL ; -/* -* Invert short psw indication, so SIE will report a specification -* exception if it was not set. -*/ - cpu->env.psw.mask ^=PSW_MASK_SHORTPSW ; - cpu->env.psw.addr = spsw &0x7fffffffULL ; +CPUS390XState *env = &cpu->env; +uint64_t spsw; +if (!env->pv) { + spsw =ldq_phys (s->as,0 ); + cpu->env.psw.mask = spsw &0xffffffff80000000ULL ; +/* +* Invert short psw indication, so SIE will report a specification +* exception if it was not set. +*/ + cpu->env.psw.mask ^=PSW_MASK_SHORTPSW ; + cpu->env.psw.addr = spsw &0x7fffffffULL ; + }else { +s390_pv_set_ipl_psw (s); + }s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu); }#endif
Signature · 1 line
2.20.1
PATCH v3 10/17 s390x: Add SIDA memory ops
In reply to Janosch FrankProtected guests save the instruction control blocks in the SIDA instead of QEMU/KVM directly accessing the guest's memory.
Let's introduce new functions to access the SIDA.
--- linux-headers/linux/kvm.h | 6 +++++- target/s390x/cpu.h | 7 ++++++- target/s390x/kvm.c | 23 +++++++++++++++++++++++ target/s390x/kvm_s390x.h | 2 ++ target/s390x/mmu_helper.c | 9 +++++++++ 5 files changed, 45 insertions(+), 2 deletions(-)
linux-headers/linux/kvm.h +5 -1
@@ -475,11 +475,15 @@ struct kvm_s390_mem_op {__u32 op;/* type of operation */ __u64 buf;/* buffer in userspace */ __u8 ar;/* the access register number */ -__u8 reserved[31 ];/* should be set to 0 */ +__u8 reserved21[3 ];/* should be set to 0 */ +__u32 offset;/* offset into the sida */ +__u8 reserved28[24 ];/* should be set to 0 */ };/* types for kvm_s390_mem_op->op */ #define KVM_S390_MEMOP_LOGICAL_READ 0#define KVM_S390_MEMOP_LOGICAL_WRITE 1 +#define KVM_S390_MEMOP_SIDA_READ 2 +#define KVM_S390_MEMOP_SIDA_WRITE 3/* flags for kvm_s390_mem_op->flags */ #define KVM_S390_MEMOP_F_CHECK_ONLY (1ULL << 0)#define KVM_S390_MEMOP_F_INJECT_EXCEPTION (1ULL << 1)
target/s390x/cpu.h +6 -1
@@ -823,7 +823,12 @@ int s390_cpu_virt_mem_rw(S390CPU *cpu, vaddr laddr, uint8_t ar, void *hostbuf,#define s390_cpu_virt_mem_check_write (cpu, laddr, ar, len) \ s390_cpu_virt_mem_rw(cpu, laddr, ar, NULL, len, true)void s390_cpu_virt_mem_handle_exc (S390CPU *cpu,uintptr_t ra); - +int s390_cpu_pv_mem_rw (S390CPU *cpu,unsigned int offset,void *hostbuf, +int len,bool is_write); +#define s390_cpu_pv_mem_read (cpu, offset, dest, len) \ + s390_cpu_pv_mem_rw(cpu, offset, dest, len, false) +#define s390_cpu_pv_mem_write (cpu, offset, dest, len) \ + s390_cpu_pv_mem_rw(cpu, offset, dest, len, true)/* sigp.c */ int s390_cpu_restart (S390CPU *cpu);
target/s390x/kvm.c +23 -0
@@ -846,6 +846,29 @@ int kvm_s390_mem_op(S390CPU *cpu, vaddr addr, uint8_t ar, void *hostbuf,return ret; } +int kvm_s390_mem_op_pv (S390CPU *cpu,uint64_t offset,void *hostbuf, +int len,bool is_write) +{ +int ret =0 ; +struct kvm_s390_mem_op mem_op = { + .offset = offset, + .size = len, + .op = is_write ?KVM_S390_MEMOP_SIDA_WRITE + :KVM_S390_MEMOP_SIDA_READ , + .buf = (uint64_t )hostbuf, + }; + +if (!cap_mem_op) { +return -ENOSYS ; + } + + ret =kvm_vcpu_ioctl (CPU (cpu),KVM_S390_MEM_OP , &mem_op); +if (ret <0 ) { +warn_report ("KVM_S390_MEM_OP failed: %s" ,strerror (-ret)); + } +return ret; +} +/* * Legacy layout for s390: * Older S390 KVM requires the topmost vma of the RAM to be
target/s390x/kvm_s390x.h +2 -0
@@ -19,6 +19,8 @@ void kvm_s390_vcpu_interrupt(S390CPU *cpu, struct kvm_s390_irq *irq);void kvm_s390_access_exception (S390CPU *cpu,uint16_t code,uint64_t te_code);int kvm_s390_mem_op (S390CPU *cpu,vaddr addr,uint8_t ar,void *hostbuf,int len,bool is_write); +int kvm_s390_mem_op_pv (S390CPU *cpu,vaddr addr,void *hostbuf,int len, +bool is_write);void kvm_s390_program_interrupt (S390CPU *cpu,uint16_t code);int kvm_s390_set_cpu_state (S390CPU *cpu,uint8_t cpu_state);void kvm_s390_vcpu_interrupt_pre_save (S390CPU *cpu);
target/s390x/mmu_helper.c +9 -0
@@ -474,6 +474,15 @@ static int translate_pages(S390CPU *cpu, vaddr addr, int nr_pages,return 0 ; } +int s390_cpu_pv_mem_rw (S390CPU *cpu,unsigned int offset,void *hostbuf, +int len,bool is_write) +{ +int ret; + + ret =kvm_s390_mem_op_pv (cpu, offset, hostbuf, len, is_write); +return ret; +} +/** * s390_cpu_virt_mem_rw: * @laddr: the logical start address
Signature · 1 line
2.20.1
PATCH v3 4/17 s390x: protvirt: Add diag308 subcodes 8 - 10
In reply to Janosch FrankFor diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.
Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.
Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.
Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.
--- hw/s390x/ipl.c | 48 ++++++++++++++++++++++++++++++++++++++++++--- hw/s390x/ipl.h | 31 +++++++++++++++++++++++++++++ target/s390x/diag.c | 27 ++++++++++++++++++++++--- 3 files changed, 100 insertions(+), 6 deletions(-)
hw/s390x/ipl.c +45 -3
@@ -538,15 +538,56 @@ static bool is_virtio_scsi_device(IplParameterBlock *iplb)return is_virtio_ccw_device_of_type (iplb,VIRTIO_ID_SCSI ); } +int s390_ipl_pv_check_components (IplParameterBlock *iplb) +{ +int i; +IPLBlockPV *ipib_pv = &iplb->pv; + +if (ipib_pv->num_comp ==0 ) { +return -EINVAL ; + } + +for (i =0 ; i < ipib_pv->num_comp; i++) { + +/* Addr must be 4k aligned */ +if (ipib_pv->components[i].addr & ~TARGET_PAGE_MASK ) { +return -EINVAL ; + } + +/* Tweak prefix is monotonously increasing with each component */ +if (i < ipib_pv->num_comp -1 && + ipib_pv->components[i].tweak_pref > + ipib_pv->components[i +1 ].tweak_pref) { +return -EINVAL ; + } + } +return 1 ; +} +void s390_ipl_update_diag308 (IplParameterBlock *iplb) {S390IPLState *ipl =get_ipl_device (); - ipl->iplb = *iplb; - ipl->iplb_valid = true; +if (iplb->pbt ==5 ) { + ipl->iplb_pbt5 = *iplb; + ipl->iplb_valid_pbt5 = true; + }else { + ipl->iplb = *iplb; + ipl->iplb_valid = true; + } ipl->netboot =is_virtio_net_device (iplb); } +IplParameterBlock *s390_ipl_get_iplb_secure (void ) +{ +S390IPLState *ipl =get_ipl_device (); + +if (!ipl->iplb_valid_pbt5) { +return NULL ; + } +return &ipl->iplb_pbt5; +} +IplParameterBlock *s390_ipl_get_iplb (void ) {S390IPLState *ipl =get_ipl_device ();
@@ -561,7 +602,8 @@ void s390_ipl_reset_request(CPUState *cs, enum s390_reset reset_type) {S390IPLState *ipl =get_ipl_device (); -if (reset_type ==S390_RESET_EXTERNAL || reset_type ==S390_RESET_REIPL ) { +if (reset_type ==S390_RESET_EXTERNAL || reset_type ==S390_RESET_REIPL || + reset_type ==S390_RESET_PV ) {/* use CPU 0 for full resets */ ipl->reset_cpu_index =0 ; }else {
hw/s390x/ipl.h +31 -0
@@ -15,6 +15,23 @@#include "cpu.h" #include "hw/qdev-core.h" +struct IPLBlockPVComp { +uint64_t tweak_pref; +uint64_t addr; +uint64_t size; +}QEMU_PACKED ; +typedef struct IPLBlockPVComp IPLBlockPVComp ; + +struct IPLBlockPV { +uint8_t reserved[87 ]; +uint8_t version; +uint32_t num_comp; +uint64_t pv_header_addr; +uint64_t pv_header_len; +struct IPLBlockPVComp components[]; +}QEMU_PACKED ; +typedef struct IPLBlockPV IPLBlockPV ; +struct IplBlockCcw {uint8_t reserved0[85 ];uint8_t ssid;
@@ -71,6 +88,7 @@ union IplParameterBlock {union {IplBlockCcw ccw;IplBlockFcp fcp; +IPLBlockPV pv;IplBlockQemuScsi scsi; }; }QEMU_PACKED ;
@@ -84,9 +102,11 @@ union IplParameterBlock {typedef union IplParameterBlock IplParameterBlock ;int s390_ipl_set_loadparm (uint8_t *loadparm); +int s390_ipl_pv_check_components (IplParameterBlock *iplb);void s390_ipl_update_diag308 (IplParameterBlock *iplb);void s390_ipl_prepare_cpu (S390CPU *cpu);IplParameterBlock *s390_ipl_get_iplb (void ); +IplParameterBlock *s390_ipl_get_iplb_secure (void );enum s390_reset {/* default is a reset not triggered by a CPU e.g. issued by QMP */
@@ -94,6 +114,7 @@ enum s390_reset {S390_RESET_REIPL ,S390_RESET_MODIFIED_CLEAR ,S390_RESET_LOAD_NORMAL , + S390_RESET_PV, };void s390_ipl_reset_request (CPUState *cs,enum s390_reset reset_type);void s390_ipl_get_reset_request (CPUState **cs,enum s390_reset *reset_type);
@@ -133,6 +154,7 @@ struct S390IPLState {/*< private >*/ DeviceState parent_obj;IplParameterBlock iplb; +IplParameterBlock iplb_pbt5;QemuIplParameters qipl;uint64_t start_addr;uint64_t compat_start_addr;
@@ -140,6 +162,7 @@ struct S390IPLState {uint64_t compat_bios_start_addr;bool enforce_bios;bool iplb_valid; +bool iplb_valid_pbt5;bool netboot;/* reset related properties don't have to be migrated or reset */ enum s390_reset reset_type;
@@ -161,9 +184,11 @@ QEMU_BUILD_BUG_MSG(offsetof(S390IPLState, iplb) & 3, "alignment of iplb wrong");#define S390_IPL_TYPE_FCP 0x00#define S390_IPL_TYPE_CCW 0x02 +#define S390_IPL_TYPE_PV 0x05#define S390_IPL_TYPE_QEMU_SCSI 0xff#define S390_IPLB_HEADER_LEN 8 +#define S390_IPLB_MIN_PV_LEN 148#define S390_IPLB_MIN_CCW_LEN 200#define S390_IPLB_MIN_FCP_LEN 384#define S390_IPLB_MIN_QEMU_SCSI_LEN 200
@@ -185,4 +210,10 @@ static inline bool iplb_valid_fcp(IplParameterBlock *iplb) iplb->pbt == S390_IPL_TYPE_FCP; } +static inline bool iplb_valid_pv (IplParameterBlock *iplb) +{ +return be32_to_cpu (iplb->len) >=S390_IPLB_MIN_PV_LEN && + iplb->pbt ==S390_IPL_TYPE_PV ; +} +#endif
target/s390x/diag.c +24 -3
@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)#define DIAG_308_RC_OK 0x0001#define DIAG_308_RC_NO_CONF 0x0102#define DIAG_308_RC_INVALID 0x0402 +#define DIAG_308_RC_NO_PV_CONF 0x0a02 +#define DIAG_308_RC_INV_FOR_PV 0x0b02#define DIAG308_RESET_MOD_CLR 0#define DIAG308_RESET_LOAD_NORM 1
@@ -59,6 +61,9 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)#define DIAG308_LOAD_NORMAL_DUMP 4#define DIAG308_SET 5#define DIAG308_STORE 6 +#define DIAG308_PV_SET 8 +#define DIAG308_PV_STORE 9 +#define DIAG308_PV_START 10static int diag308_parm_check (CPUS390XState *env,uint64_t r1,uint64_t addr,uintptr_t ra,bool write)
@@ -105,6 +110,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)s390_ipl_reset_request (cs,S390_RESET_REIPL );break ;case DIAG308_SET : +case DIAG308_PV_SET :if (diag308_parm_check (env, r1, addr, ra, false)) {return ; }
@@ -117,7 +123,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); -if (!iplb_valid_ccw (iplb) && !iplb_valid_fcp (iplb)) { +if (!iplb_valid_ccw (iplb) && !iplb_valid_fcp (iplb) && + !(iplb_valid_pv (iplb) &&s390_ipl_pv_check_components (iplb) >=0 )) { env->regs[r1 +1 ] =DIAG_308_RC_INVALID ; goto out; }
@@ -128,17 +135,31 @@ out:g_free (iplb);return ;case DIAG308_STORE : +case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } - iplb =s390_ipl_get_iplb (); +if (subcode ==DIAG308_PV_STORE ) { + iplb =s390_ipl_get_iplb_secure (); + }else { + iplb =s390_ipl_get_iplb (); + }if (iplb) {cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); env->regs[r1 +1 ] =DIAG_308_RC_OK ; }else { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; } -return ; +break ; +case DIAG308_PV_START : + iplb =s390_ipl_get_iplb_secure (); +if (!iplb || !iplb_valid_pv (iplb)) { + env->regs[r1 +1 ] =DIAG_308_RC_NO_PV_CONF ; +return ; + } + +s390_ipl_reset_request (cs,S390_RESET_PV ); +break ;default :s390_program_interrupt (env,PGM_SPECIFICATION , ra);break ;
Signature · 1 line
2.20.1
PATCH v3 9/17 s390: protvirt: Move STSI data over SIDAD
In reply to Janosch FrankFor protected guests, we need to put the STSI emulation results into the SIDA, so SIE will write them into the guest at the next entry.
--- target/s390x/kvm.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
target/s390x/kvm.c +12 -3
@@ -1772,11 +1772,16 @@ static int handle_tsch(S390CPU *cpu)static void insert_stsi_3_2_2 (S390CPU *cpu,__u64 addr,uint8_t ar) { +CPUS390XState *env = &cpu->env;SysIB_322 sysib;int del; -if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu,0 , &sysib,sizeof (sysib)); + }else { +if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { +return ; + } }/* Shift the stack of Extended Names to prepare for our own data */ memmove(&sysib.ext_names[1], &sysib.ext_names[0],
@@ -1815,7 +1820,11 @@ static void insert_stsi_3_2_2(S390CPU *cpu, __u64 addr, uint8_t ar)/* Insert UUID */ memcpy (sysib.vm[0 ].uuid, &qemu_uuid,sizeof (sysib.vm[0 ].uuid)); -s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); +if (env->pv) { +s390_cpu_pv_mem_write (cpu,0 , &sysib,sizeof (sysib)); + }else { +s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); + } }static int handle_stsi (S390CPU *cpu)
Signature · 1 line
2.20.1
PATCH v3 16/17 s390x: protvirt: Handle SIGP store status correctly
In reply to Janosch FrankStatus storing is not done by QEMU anymore, but is handled by SIE.
--- target/s390x/helper.c | 4 ++++ target/s390x/sigp.c | 1 + 2 files changed, 5 insertions(+)
target/s390x/helper.c +4 -0
@@ -246,6 +246,10 @@ int s390_store_status(S390CPU *cpu, hwaddr addr, bool store_arch)hwaddr len =sizeof (*sa);int i; +if (cpu->env.pv) { +return 0 ; + } + sa =cpu_physical_memory_map (addr, &len,1 );if (!sa) {return -EFAULT ;
target/s390x/sigp.c +1 -0
@@ -497,6 +497,7 @@ void do_stop_interrupt(CPUS390XState *env)if (s390_cpu_set_state (S390_CPU_STATE_STOPPED , cpu) ==0 ) {qemu_system_shutdown_request (SHUTDOWN_CAUSE_GUEST_SHUTDOWN ); } +/* Storing will occur on next SIE entry for fmt 4 */ if (cpu->env.sigp_order ==SIGP_STOP_STORE_STATUS ) {s390_store_status (cpu,S390_STORE_STATUS_DEF_ADDR , true); }
Signature · 1 line
2.20.1
PATCH v3 13/17 s390x: protvirt: Move diag 308 data over SIDAD
In reply to Janosch FrankFor protected guests the IPIB is written/read to/from the satellite block, so we need to make those accesses virtual to make them go through KVM mem ops.
--- target/s390x/diag.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-)
target/s390x/diag.c +25 -7
@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,void handle_diag_308 (CPUS390XState *env,uint64_t r1,uint64_t r3,uintptr_t ra) {CPUState *cs =env_cpu (env); +S390CPU *cpu =S390_CPU (cs);uint64_t addr = env->regs[r1];uint64_t subcode = env->regs[r3];IplParameterBlock *iplb;
@@ -118,14 +119,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)if (diag308_parm_check (env, r1, addr, ra, false)) {return ; } + iplb =g_new0 (IplParameterBlock,1 ); -cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,sizeof (iplb->len)); + } +if (!iplb_valid_len (iplb)) { env->regs[r1 +1 ] =DIAG_308_RC_INVALID ; goto out; } -cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,be32_to_cpu (iplb->len)); + }if (!iplb_valid_ccw (iplb) && !iplb_valid_fcp (iplb) && !(iplb_valid_pv (iplb) &&s390_ipl_pv_check_components (iplb) >=0 )) {
@@ -137,23 +148,30 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) env->regs[r1 +1 ] =DIAG_308_RC_OK ; out:g_free (iplb); -return ; +break ;case DIAG308_STORE :case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } +if (subcode ==DIAG308_PV_STORE ) { iplb =s390_ipl_get_iplb_secure (); }else { iplb =s390_ipl_get_iplb (); } -if (iplb) { -cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); - env->regs[r1 +1 ] =DIAG_308_RC_OK ; - }else { +if (!iplb) { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; +return ; } + +if (!env->pv) { +cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_write (cpu,0 , iplb,be32_to_cpu (iplb->len)); + } + + env->regs[r1 +1 ] =DIAG_308_RC_OK ;break ;case DIAG308_PV_START : iplb =s390_ipl_get_iplb_secure ();
Signature · 1 line
2.20.1
PATCH v3 11/17 s390x: protvirt: SCLP interpretation
In reply to Janosch FrankSCLP for a protected guest is done over the SIDAD, so we need to use the s390_cpu_virt_mem_* functions to access the SIDAD instead of guest memory when reading/writing SCBs.
To not confuse the sclp emulation, we set 0x4000 as the SCCB address, since the function that injects the sclp external interrupt would reject a zero sccb address.
--- hw/s390x/sclp.c | 17 +++++++++++++++++ include/hw/s390x/sclp.h | 2 ++ target/s390x/kvm.c | 5 +++++ 3 files changed, 24 insertions(+)
hw/s390x/sclp.c +17 -0
@@ -193,6 +193,23 @@ static void sclp_execute(SCLPDevice *sclp, SCCB *sccb, uint32_t code) } } +#define SCLP_PV_DUMMY_ADDR 0x4000 +int sclp_service_call_protected (CPUS390XState *env,uint64_t sccb, +uint32_t code) +{ +SCLPDevice *sclp =get_sclp_device (); +SCLPDeviceClass *sclp_c =SCLP_GET_CLASS (sclp); +SCCB work_sccb; +hwaddr sccb_len =sizeof (SCCB ); + +s390_cpu_pv_mem_read (env_archcpu (env),0 , &work_sccb, sccb_len); + sclp_c->execute (sclp, &work_sccb, code); +s390_cpu_pv_mem_write (env_archcpu (env),0 , &work_sccb, +be16_to_cpu (work_sccb.h.length)); + sclp_c->service_interrupt (sclp,SCLP_PV_DUMMY_ADDR ); +return 0 ; +} +int sclp_service_call (CPUS390XState *env,uint64_t sccb,uint32_t code) {SCLPDevice *sclp =get_sclp_device ();
include/hw/s390x/sclp.h +2 -0
@@ -217,5 +217,7 @@ void s390_sclp_init(void);void sclp_service_interrupt (uint32_t sccb);void raise_irq_cpu_hotplug (void );int sclp_service_call (CPUS390XState *env,uint64_t sccb,uint32_t code); +int sclp_service_call_protected (CPUS390XState *env,uint64_t sccb, +uint32_t code);#endif
target/s390x/kvm.c +5 -0
@@ -1224,6 +1224,11 @@ static void kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run, sccb = env->regs[ipbh0 &0xf ]; code = env->regs[(ipbh0 &0xf0 ) >>4 ]; +if (run->s390_sieic.icptcode ==ICPT_PV_INSTR ) { +sclp_service_call_protected (env, sccb, code); +return ; + } + r =sclp_service_call (env, sccb, code);if (r <0 ) {kvm_s390_program_interrupt (cpu, -r);
Signature · 1 line
2.20.1
PATCH v3 15/17 s390x: protvirt: Move IO control structures over SIDA
In reply to Janosch FrankFor protected guests, we need to put the IO emulation results into the SIDA, so SIE will write them into the guest at the next entry.
--- target/s390x/ioinst.c | 87 ++++++++++++++++++++++++++++++------------- 1 file changed, 61 insertions(+), 26 deletions(-)
target/s390x/ioinst.c +61 -26
@@ -129,9 +129,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ; } -if (s390_cpu_virt_mem_read (cpu, addr, ar, &schib,sizeof (schib))) { -s390_cpu_virt_mem_handle_exc (cpu, ra); -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu, addr, &schib,sizeof (schib)); + }else { +if (s390_cpu_virt_mem_read (cpu, addr, ar, &schib,sizeof (schib))) { +s390_cpu_virt_mem_handle_exc (cpu, ra); +return ; + } }if (ioinst_disassemble_sch_ident (reg1, &m, &cssid, &ssid, &schid) || !ioinst_schib_valid (&schib)) {
@@ -186,9 +190,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ; } -if (s390_cpu_virt_mem_read (cpu, addr, ar, &orig_orb,sizeof (orb))) { -s390_cpu_virt_mem_handle_exc (cpu, ra); -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu, addr, &orig_orb,sizeof (orb)); + }else { +if (s390_cpu_virt_mem_read (cpu, addr, ar, &orig_orb,sizeof (orb))) { +s390_cpu_virt_mem_handle_exc (cpu, ra); +return ; + } }copy_orb_from_guest (&orb, &orig_orb); if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid) ||
@@ -222,14 +230,19 @@ void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra) cc =css_do_stcrw (&crw);/* 0 - crw stored, 1 - zeroes stored */ -if (s390_cpu_virt_mem_write (cpu, addr, ar, &crw,sizeof (crw)) ==0 ) { +if (env->pv) { +s390_cpu_pv_mem_write (cpu, addr, &crw,sizeof (crw));setcc (cpu, cc); }else { -if (cc ==0 ) { -/* Write failed: requeue CRW since STCRW is suppressing */ -css_undo_stcrw (&crw); +if (s390_cpu_virt_mem_write (cpu, addr, ar, &crw,sizeof (crw)) ==0 ) { +setcc (cpu, cc); + }else { +if (cc ==0 ) { +/* Write failed: requeue CRW since STCRW is suppressing */ +css_undo_stcrw (&crw); + } +s390_cpu_virt_mem_handle_exc (cpu, ra); } -s390_cpu_virt_mem_handle_exc (cpu, ra); } }
@@ -251,6 +264,9 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, }if (ioinst_disassemble_sch_ident (reg1, &m, &cssid, &ssid, &schid)) { +if (env->pv) { +return ; + }/* * As operand exceptions have a lower priority than access exceptions, * we check whether the memory area is writeable (injecting the
@@ -283,14 +299,19 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, } }if (cc !=3 ) { -if (s390_cpu_virt_mem_write (cpu, addr, ar, &schib, -sizeof (schib)) !=0 ) { -s390_cpu_virt_mem_handle_exc (cpu, ra); -return ; +if (env->pv) { +s390_cpu_pv_mem_write (cpu, addr, &schib,sizeof (schib)); + }else { +if (s390_cpu_virt_mem_write (cpu, addr, ar, &schib, +sizeof (schib)) !=0 ) { +s390_cpu_virt_mem_handle_exc (cpu, ra); +return ; + } } }else {/* Access exceptions have a higher priority than cc3 */ -if (s390_cpu_virt_mem_check_write (cpu, addr, ar,sizeof (schib)) !=0 ) { +if (!env->pv && +s390_cpu_virt_mem_check_write (cpu, addr, ar,sizeof (schib)) !=0 ) {s390_cpu_virt_mem_handle_exc (cpu, ra);return ; }
@@ -327,15 +348,20 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra) }/* 0 - status pending, 1 - not status pending, 3 - not operational */ if (cc !=3 ) { -if (s390_cpu_virt_mem_write (cpu, addr, ar, &irb, irb_len) !=0 ) { -s390_cpu_virt_mem_handle_exc (cpu, ra); -return -EFAULT ; +if (env->pv) { +s390_cpu_pv_mem_write (cpu, addr, &irb, irb_len); + }else { +if (s390_cpu_virt_mem_write (cpu, addr, ar, &irb, irb_len) !=0 ) { +s390_cpu_virt_mem_handle_exc (cpu, ra); +return -EFAULT ; + } }css_do_tsch_update_subch (sch); }else { irb_len =sizeof (irb) -sizeof (irb.emw);/* Access exceptions have a higher priority than cc3 */ -if (s390_cpu_virt_mem_check_write (cpu, addr, ar, irb_len) !=0 ) { +if (!env->pv && +s390_cpu_virt_mem_check_write (cpu, addr, ar, irb_len) !=0 ) {s390_cpu_virt_mem_handle_exc (cpu, ra);return -EFAULT ; }
@@ -633,9 +659,13 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra)* present CHSC sub-handlers ... if we ever need more, we should take * care of req->len here first. */ -if (s390_cpu_virt_mem_read (cpu, addr, reg, buf,sizeof (ChscReq))) { -s390_cpu_virt_mem_handle_exc (cpu, ra); -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu, addr, buf,sizeof (ChscReq)); + }else { +if (s390_cpu_virt_mem_read (cpu, addr, reg, buf,sizeof (ChscReq))) { +s390_cpu_virt_mem_handle_exc (cpu, ra); +return ; + } } req = (ChscReq *)buf; len =be16_to_cpu (req->len);
@@ -666,11 +696,16 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra)break ; } -if (!s390_cpu_virt_mem_write (cpu, addr + len, reg, res, -be16_to_cpu (res->len))) { +if (env->pv) { +s390_cpu_pv_mem_write (cpu, addr + len, res,be16_to_cpu (res->len));setcc (cpu,0 );/* Command execution complete */ }else { -s390_cpu_virt_mem_handle_exc (cpu, ra); +if (!s390_cpu_virt_mem_write (cpu, addr + len, reg, res, +be16_to_cpu (res->len))) { +setcc (cpu,0 );/* Command execution complete */ + }else { +s390_cpu_virt_mem_handle_exc (cpu, ra); + } } }
Signature · 1 line
2.20.1
PATCH v3 17/17 s390x: For now add unpack feature to GA1
In reply to Janosch Frank--- target/s390x/gen-features.c | 1 + 1 file changed, 1 insertion(+)
target/s390x/gen-features.c +1 -0
@@ -562,6 +562,7 @@ static uint16_t full_GEN15_GA1[] = {S390_FEAT_GROUP_MSA_EXT_9 ,S390_FEAT_GROUP_MSA_EXT_9_PCKMO ,S390_FEAT_ETOKEN , + S390_FEAT_UNPACK, };/* Default features (in order of release)
Signature · 1 line
2.20.1
PATCH v3 14/17 s390x: protvirt: Disable address checks for PV guest IO emulation
In reply to Janosch FrankIO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.
--- target/s390x/ioinst.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-)
target/s390x/ioinst.c +19 -7
@@ -17,6 +17,16 @@#include "trace.h" #include "hw/s390x/s390-pci-bus.h" +static uint64_t get_address_from_regs (CPUS390XState *env,uint32_t ipb, +uint8_t *ar) +{ +if (env->pv) { + *ar =0 ; +return 0 ; + } +return decode_basedisp_s (env, ipb, ar); +} +int ioinst_disassemble_sch_ident (uint32_t value,int *m,int *cssid,int *ssid,int *schid) {
@@ -114,7 +124,7 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)CPUS390XState *env = &cpu->env;uint8_t ar; - addr =decode_basedisp_s (env, ipb, &ar); + addr =get_address_from_regs (env, ipb, &ar);if (addr &3 ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ;
@@ -171,7 +181,7 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)CPUS390XState *env = &cpu->env;uint8_t ar; - addr =decode_basedisp_s (env, ipb, &ar); + addr =get_address_from_regs (env, ipb, &ar);if (addr &3 ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ;
@@ -203,7 +213,7 @@ void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)CPUS390XState *env = &cpu->env;uint8_t ar; - addr =decode_basedisp_s (env, ipb, &ar); + addr =get_address_from_regs (env, ipb, &ar);if (addr &3 ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ;
@@ -234,7 +244,7 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,CPUS390XState *env = &cpu->env;uint8_t ar; - addr =decode_basedisp_s (env, ipb, &ar); + addr =get_address_from_regs (env, ipb, &ar);if (addr &3 ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return ;
@@ -303,7 +313,7 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)return -EIO ; }trace_ioinst_sch_id ("tsch" , cssid, ssid, schid); - addr =decode_basedisp_s (env, ipb, &ar); + addr =get_address_from_regs (env, ipb, &ar);if (addr &3 ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);return -EIO ;
@@ -601,7 +611,7 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra) {ChscReq *req;ChscResp *res; -uint64_t addr; +uint64_t addr =0 ;int reg;uint16_t len;uint16_t command;
@@ -610,7 +620,9 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra)trace_ioinst ("chsc" ); reg = (ipb >>20 ) &0x00f ; - addr = env->regs[reg]; +if (!env->pv) { + addr = env->regs[reg]; + }/* Page boundary? */ if (addr &0xfff ) {s390_program_interrupt (env,PGM_SPECIFICATION , ra);
Signature · 1 line
2.20.1
PATCH v3 6/17 s390x: protvirt: Add migration blocker
In reply to Janosch FrankMigration is not yet supported.
--- hw/s390x/s390-virtio-ccw.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
hw/s390x/s390-virtio-ccw.c +16 -0
@@ -42,6 +42,9 @@#include "hw/s390x/tod.h" #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" +#include "migration/blocker.h" + +static Error *pv_mig_blocker;S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {
@@ -373,6 +376,7 @@ static void s390_machine_reset(MachineState *machine)CPUState *cs, *t;S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine); +static Error *local_err;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);
@@ -422,6 +426,17 @@ static void s390_machine_reset(MachineState *machine) }run_on_cpu (cs, s390_do_cpu_reset,RUN_ON_CPU_NULL ); +if (!pv_mig_blocker) { +error_setg (&pv_mig_blocker, +"protected VMs are currently not migrateable." ); + } +migrate_add_blocker (pv_mig_blocker, &local_err); +if (local_err) { +error_report_err (local_err); +error_free (pv_mig_blocker); +exit (1 ); + } +if (s390_machine_pv_secure (ms)) {CPU_FOREACH (t) {s390_pv_vcpu_destroy (t);
@@ -430,6 +445,7 @@ static void s390_machine_reset(MachineState *machine) ms->pv = false;s390_machine_inject_pv_error (cs); +migrate_del_blocker (pv_mig_blocker);s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu);return ; }
Signature · 1 line
2.20.1
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Subject: [PATCH v3 00/17] s390x: Protected Virtualization support
Message-id: [email protected]
Type: series
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
8ffbebb s390x: For now add unpack feature to GA1
a480bb0 s390x: protvirt: Handle SIGP store status correctly c5157a7 s390x: protvirt: Move IO control structures over SIDA cfa7ecd s390x: protvirt: Disable address checks for PV guest IO emulation 24e8dda s390x: protvirt: Move diag 308 data over SIDAD 1730605 s390x: protvirt: Set guest IPL PSW
476b4a4 s390x: protvirt: SCLP interpretation
4a0434d s390x: Add SIDA memory ops
f077e58 s390: protvirt: Move STSI data over SIDAD
b63f901 s390x: protvirt: KVM intercept changes
194b56c s390x: protvirt: Handle diag 308 subcodes 0,1,3,4 6424c01 s390x: protvirt: Add migration blocker
ef65f16 s390x: protvirt: Support unpack facility
439c439 s390x: protvirt: Add diag308 subcodes 8 - 10
4c50a4d Sync pv
c5e938d s390x: Add missing vcpu reset functions
820a1a9 Header sync
=== OUTPUT BEGIN ===
1/17 Checking commit 820a1a9f5c25 (Header sync)
ERROR: Missing Signed-off-by: line(s)
total: 1 errors, 0 warnings, 16 lines checked
Patch 1/17 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
2/17 Checking commit c5e938d73f53 (s390x: Add missing vcpu reset functions) 3/17 Checking commit 4c50a4d5dbb9 (Sync pv)
4/17 Checking commit 439c439526ae (s390x: protvirt: Add diag308 subcodes 8 - 10) 5/17 Checking commit ef65f167a065 (s390x: protvirt: Support unpack facility)
ERROR: braces {} are necessary for all arms of this statement
#79: FILE: hw/s390x/ipl.c:706:
+ if (rc)
[...]
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? #102:
new file mode 100644
ERROR: "foo* bar" should be "foo *bar"
#127: FILE: hw/s390x/pv.c:21:
+const char* cmd_names[] = {
ERROR: spaces required around that ':' (ctx:VxW)
#298: FILE: hw/s390x/pv.h:32:
+int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: }
^
total: 3 errors, 1 warnings, 410 lines checked
Patch 5/17 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
6/17 Checking commit 6424c0195f3e (s390x: protvirt: Add migration blocker) 7/17 Checking commit 194b56c59d1a (s390x: protvirt: Handle diag 308 subcodes 0,1,3,4) 8/17 Checking commit b63f90167c46 (s390x: protvirt: KVM intercept changes)
ERROR: switch and case should be at the same indent
#42: FILE: target/s390x/kvm.c:1696:
switch (icpt_code) {[...]
+ case ICPT_PV_INSTR:
+ case ICPT_PV_INSTR_NOTIFICATION:
total: 1 errors, 0 warnings, 16 lines checked
Patch 8/17 has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.
9/17 Checking commit f077e582c4b8 (s390: protvirt: Move STSI data over SIDAD)
10/17 Checking commit 4a0434d0a66b (s390x: Add SIDA memory ops)
11/17 Checking commit 476b4a40966d (s390x: protvirt: SCLP interpretation)
12/17 Checking commit 17306057f1e4 (s390x: protvirt: Set guest IPL PSW)
13/17 Checking commit 24e8dda2b422 (s390x: protvirt: Move diag 308 data over SIDAD) 14/17 Checking commit cfa7ecd534cd (s390x: protvirt: Disable address checks for PV guest IO emulation) 15/17 Checking commit c5157a79c674 (s390x: protvirt: Move IO control structures over SIDA) 16/17 Checking commit a480bb062c57 (s390x: protvirt: Handle SIGP store status correctly) 17/17 Checking commit 8ffbebb43d51 (s390x: For now add unpack feature to GA1)
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]
Re: [PATCH v3 02/17] s390x: Add missing vcpu reset functions
In reply to Janosch FrankQuoted 11 lines · On Fri, 14 Feb 2020 10:16:21 -0500 Janosch Frank <[email protected]> wrote:
Up to now we only had an ioctl to reset vcpu data QEMU couldn't reach for the initial reset, which was also called for the clear reset. To be architecture compliant, we also need to clear local interrupts on a normal reset.
Because of this and the upcoming protvirt support we need to add ioctls for the missing clear and normal resets.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>
Quoted 6 lines
--- target/s390x/cpu.c | 14 ++++++++++++-- target/s390x/kvm-stub.c | 10 +++++++++- target/s390x/kvm.c | 42 ++++++++++++++++++++++++++++++++-------- target/s390x/kvm_s390x.h | 4 +++- 4 files changed, 58 insertions(+), 12 deletions(-)
As the kernel interfaces are already merged upstream, I guess it makes sense to do a proper headers sync and merge this?
Re: [PATCH v3 02/17] s390x: Add missing vcpu reset functions
Quoted 21 lines · Cornelia Huck
Quoted 11 lines · On Fri, 14 Feb 2020 10:16:21 -0500 Janosch Frank <[email protected]> wrote:
Up to now we only had an ioctl to reset vcpu data QEMU couldn't reach for the initial reset, which was also called for the clear reset. To be architecture compliant, we also need to clear local interrupts on a normal reset.
Because of this and the upcoming protvirt support we need to add ioctls for the missing clear and normal resets.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>Quoted 6 lines
--- target/s390x/cpu.c | 14 ++++++++++++-- target/s390x/kvm-stub.c | 10 +++++++++- target/s390x/kvm.c | 42 ++++++++++++++++++++++++++++++++-------- target/s390x/kvm_s390x.h | 4 +++- 4 files changed, 58 insertions(+), 12 deletions(-)
Quoted 3 lines
As the kernel interfaces are already merged upstream, I guess it makes sense to do a proper headers sync and merge this?
Yes, that would be great
Quoted 7 lines · On Fri, 14 Feb 2020 10:16:19 -0500 Janosch Frank <[email protected]> wrote:
Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.
The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.
Does it make sense to start looking at this series now, or I should I wait until review on the kernel side has settled? (I've lost track of the state of the interfaces...)
Quoted 48 lines
V3:
* Use dedicated functions to access SIDA
* Smaller cleanups and segfault fixes
* Error reporting for Ultravisor calls
* Inject of RC of diag308 subcode 10 failsV2:
* Split out cleanups
* Internal PV state tracking
* Review feedbackJanosch Frank (17):
Header sync
s390x: Add missing vcpu reset functions
Sync pv
s390x: protvirt: Add diag308 subcodes 8 - 10
s390x: protvirt: Support unpack facility
s390x: protvirt: Add migration blocker
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390: protvirt: Move STSI data over SIDAD
s390x: Add SIDA memory ops
s390x: protvirt: SCLP interpretation
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Move IO control structures over SIDA
s390x: protvirt: Handle SIGP store status correctly
s390x: For now add unpack feature to GA1hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 80 +++++++++++++- hw/s390x/ipl.h | 33 ++++++ hw/s390x/pv.c | 160 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 40 +++++++ hw/s390x/s390-virtio-ccw.c | 136 ++++++++++++++++++++++- hw/s390x/sclp.c | 17 +++ include/hw/s390x/s390-virtio-ccw.h | 1 + include/hw/s390x/sclp.h | 2 + linux-headers/linux/kvm.h | 48 ++++++++- target/s390x/cpu.c | 41 +++++-- target/s390x/cpu.h | 8 +- target/s390x/cpu_features_def.inc.h | 1 + target/s390x/diag.c | 63 +++++++++-- target/s390x/gen-features.c | 1 + target/s390x/helper.c | 4 + target/s390x/ioinst.c | 113 ++++++++++++++------
Quoted 9 lines
target/s390x/kvm-stub.c | 10 +- target/s390x/kvm.c | 89 ++++++++++++++-- target/s390x/kvm_s390x.h | 6 +- target/s390x/mmu_helper.c | 9 ++ target/s390x/sigp.c | 1 + 22 files changed, 789 insertions(+), 75 deletions(-) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h
-ENODOC; can you add something under docs/ that describes how you configure a pv guest and what the initialization/teardown flow is?
Quoted 8 lines · Cornelia Huck
Quoted 5 lines · On Fri, 14 Feb 2020 10:16:19 -0500 Janosch Frank <[email protected]> wrote:
Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.
Quoted 6 lines
Quoted 2 lines
The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.
Does it make sense to start looking at this series now, or I should I wait until review on the kernel side has settled? (I've lost track of the state of the interfaces...)
There won't be too many change because of KVM review. Most noteworthy is the switch from vm/vcpu create to a global switch through the VM, but that's just merging IOCTLs.
Quoted 59 lines
Quoted 49 lines
V3:
* Use dedicated functions to access SIDA
* Smaller cleanups and segfault fixes
* Error reporting for Ultravisor calls
* Inject of RC of diag308 subcode 10 failsV2:
* Split out cleanups
* Internal PV state tracking
* Review feedbackJanosch Frank (17):
Header sync
s390x: Add missing vcpu reset functions
Sync pv
s390x: protvirt: Add diag308 subcodes 8 - 10
s390x: protvirt: Support unpack facility
s390x: protvirt: Add migration blocker
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390: protvirt: Move STSI data over SIDAD
s390x: Add SIDA memory ops
s390x: protvirt: SCLP interpretation
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Move IO control structures over SIDA
s390x: protvirt: Handle SIGP store status correctly
s390x: For now add unpack feature to GA1hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 80 +++++++++++++- hw/s390x/ipl.h | 33 ++++++ hw/s390x/pv.c | 160 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 40 +++++++ hw/s390x/s390-virtio-ccw.c | 136 ++++++++++++++++++++++- hw/s390x/sclp.c | 17 +++ include/hw/s390x/s390-virtio-ccw.h | 1 + include/hw/s390x/sclp.h | 2 + linux-headers/linux/kvm.h | 48 ++++++++- target/s390x/cpu.c | 41 +++++-- target/s390x/cpu.h | 8 +- target/s390x/cpu_features_def.inc.h | 1 + target/s390x/diag.c | 63 +++++++++-- target/s390x/gen-features.c | 1 + target/s390x/helper.c | 4 + target/s390x/ioinst.c | 113 ++++++++++++++------Quoted 9 lines
target/s390x/kvm-stub.c | 10 +- target/s390x/kvm.c | 89 ++++++++++++++-- target/s390x/kvm_s390x.h | 6 +- target/s390x/mmu_helper.c | 9 ++ target/s390x/sigp.c | 1 + 22 files changed, 789 insertions(+), 75 deletions(-) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h
Quoted 2 lines
-ENODOC; can you add something under docs/ that describes how you configure a pv guest and what the initialization/teardown flow is?
Sure, but could you give me a bit more detail?
What do you mean by configure?
Command line options?
Quoted 8 lines · On Tue, 18 Feb 2020 14:15:32 +0100 Janosch Frank <[email protected]> wrote:
Quoted 2 lines · Cornelia Huck
-ENODOC; can you add something under docs/ that describes how you configure a pv guest and what the initialization/teardown flow is?Sure, but could you give me a bit more detail?
What do you mean by configure?
Command line options?
Basically: What does someone using QEMU need to know if they want to set it up to run pv guests? So a quick overview, command line options, prereqs, etc.
(Maybe you can also lift some stuff from the kernel doc?)
Quoted 12 lines · Cornelia Huck
Quoted 8 lines · On Tue, 18 Feb 2020 14:15:32 +0100 Janosch Frank <[email protected]> wrote:
Quoted 2 lines · Cornelia Huck
-ENODOC; can you add something under docs/ that describes how you configure a pv guest and what the initialization/teardown flow is?Sure, but could you give me a bit more detail?
What do you mean by configure?
Command line options?
Quoted 6 lines
Basically: What does someone using QEMU need to know if they want to set it up to run pv guests? So a quick overview, command line options, prereqs, etc.
(Maybe you can also lift some stuff from the kernel doc?)
Ok, will do
Cornelia Huck
AppliedRe: [PATCH v3 02/17] s390x: Add missing vcpu reset functions
In reply to Janosch FrankQuoted 11 lines · On Fri, 14 Feb 2020 10:16:21 -0500 Janosch Frank <[email protected]> wrote:
Up to now we only had an ioctl to reset vcpu data QEMU couldn't reach for the initial reset, which was also called for the clear reset. To be architecture compliant, we also need to clear local interrupts on a normal reset.
Because of this and the upcoming protvirt support we need to add ioctls for the missing clear and normal resets.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>
Quoted 6 lines
--- target/s390x/cpu.c | 14 ++++++++++++-- target/s390x/kvm-stub.c | 10 +++++++++- target/s390x/kvm.c | 42 ++++++++++++++++++++++++++++++++-------- target/s390x/kvm_s390x.h | 4 +++- 4 files changed, 58 insertions(+), 12 deletions(-)
Thanks, applied (together with a headers update.)
Re: [PATCH v3 04/17] s390x: protvirt: Add diag308 subcodes 8 - 10
In reply to Janosch FrankQuoted 21 lines · On Fri, 14 Feb 2020 10:16:23 -0500 Janosch Frank <[email protected]> wrote:
For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.
Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.
Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.
Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/ipl.c | 48 ++++++++++++++++++++++++++++++++++++++++++--- hw/s390x/ipl.h | 31 +++++++++++++++++++++++++++++ target/s390x/diag.c | 27 ++++++++++++++++++++++--- 3 files changed, 100 insertions(+), 6 deletions(-)
(...)
hw/s390x/ipl.c static bool is_virtio_scsi_device(IplParameterBlock *iplb) · line 571
Quoted 7 lines
void
s390_ipl_update_diag308(IplParameterBlock *iplb){ S390IPLState *ipl = get_ipl_device();hw/s390x/ipl.c +1
-2- ipl->iplb = *iplb; - ipl->iplb_valid = true; +if (iplb->pbt ==5 ) {
Magic value; maybe introduce a #define or at least a comment?
Quoted 10 lines
hw/s390x/ipl.c +6
-0+ ipl->iplb_pbt5 = *iplb; + ipl->iplb_valid_pbt5 = true; + }else { + ipl->iplb = *iplb; + ipl->iplb_valid = true; + } ipl->netboot =is_virtio_net_device (iplb); }
hw/s390x/ipl.c struct S390IPLState { · line 157
Quoted 4 lines
hw/s390x/ipl.c +1
-0@@ -133,6 +154,7 @@ struct S390IPLState {/*< private >*/ DeviceState parent_obj;IplParameterBlock iplb; +IplParameterBlock iplb_pbt5;
Add /* for protected virtualization */ ?
Or, if this is not used for anything else, call it iplb_pv?
Quoted 3 lines
hw/s390x/ipl.h +0
-0QemuIplParameters qipl;uint64_t start_addr;uint64_t compat_start_addr;
(...)
target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 56
Quoted 10 lines
target/s390x/diag.c +2
-0@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)#define DIAG_308_RC_OK 0x0001#define DIAG_308_RC_NO_CONF 0x0102#define DIAG_308_RC_INVALID 0x0402 +#define DIAG_308_RC_NO_PV_CONF 0x0a02 +#define DIAG_308_RC_INV_FOR_PV 0x0b02
DIAG_308_RC_INVAL_FOR_PV ?
Quoted 3 lines
target/s390x/diag.c +0
-0#define DIAG308_RESET_MOD_CLR 0#define DIAG308_RESET_LOAD_NORM 1
(...)
Quoted 4 lines
target/s390x/diag.c +0
-0@@ -128,17 +135,31 @@ out:g_free (iplb);return ;case DIAG308_STORE :
target/s390x/diag.c out: · line 146
Quoted 10 lines
target/s390x/diag.c +6
-1+case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } - iplb =s390_ipl_get_iplb (); +if (subcode ==DIAG308_PV_STORE ) { + iplb =s390_ipl_get_iplb_secure (); + }else { + iplb =s390_ipl_get_iplb (); + }
iplb = (subcode == DIAG308_PV_STORE) ?
s390_ipl_get_iplb_secure() : s390_ipl_get_iplb();
Matter of taste, I guess.
Quoted 20 lines
target/s390x/diag.c +10
-1if (iplb) {cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); env->regs[r1 +1 ] =DIAG_308_RC_OK ; }else { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; } -return ; +break ; +case DIAG308_PV_START : + iplb =s390_ipl_get_iplb_secure (); +if (!iplb || !iplb_valid_pv (iplb)) { + env->regs[r1 +1 ] =DIAG_308_RC_NO_PV_CONF ; +return ; + } + +s390_ipl_reset_request (cs,S390_RESET_PV ); +break ;default :s390_program_interrupt (env,PGM_SPECIFICATION , ra);break ;
Re: [PATCH v3 05/17] s390x: protvirt: Support unpack facility
In reply to Janosch FrankQuoted 1 line · On Fri, 14 Feb 2020 10:16:24 -0500 Janosch Frank <[email protected]> wrote:
When a guest has saved a ipib of type 5 and call diagnose308 with
s/call/calls/
Quoted 30 lines
subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.
The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrityOnly after step 5 a protected VM is allowed to run.
Signed-off-by Janosch Frank <[email protected]>Signed-off-by Christian Borntraeger <[email protected]> [Changesto machine]
--- hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 32 ++++++ hw/s390x/ipl.h | 2 + hw/s390x/pv.c | 154 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 38 +++++++ hw/s390x/s390-virtio-ccw.c | 79 ++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 1 + target/s390x/cpu.c | 4 + target/s390x/cpu.h | 1 + target/s390x/cpu_features_def.inc.h | 1 + 10 files changed, 313 insertions(+) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h
(...)
hw/s390x/pv.c line 4
Quoted 10 lines
hw/s390x/pv.c new file +4
-0@@ -0,0 +1,154 @@ +/* +* Secure execution functions +* +* Copyright IBM Corp. 2019
Update the year?
Quoted 7 lines
hw/s390x/pv.c new file +7
-0+* Author(s): +* Janosch Frank <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/
(...)
hw/s390x/pv.c line 96
Quoted 3 lines
hw/s390x/pv.c new file +3
-0+void s390_pv_vm_destroy (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_DESTROY ,NULL );
Why does this exit()? Should Never Happen?
Quoted 7 lines
hw/s390x/pv.c new file +7
-0+} + +int s390_pv_vcpu_create (CPUState *cs) +{ +int rc; + + rc =s390_pv_cmd_vcpu (cs,KVM_PV_VCPU_CREATE ,NULL );
hw/s390x/pv.c line 113
Quoted 10 lines
hw/s390x/pv.c new file +10
-0+if (!rc) { +S390_CPU (cs)->env.pv = true; + } + +return rc; +} + +void s390_pv_vcpu_destroy (CPUState *cs) +{ +s390_pv_cmd_vcpu_exit (cs,KVM_PV_VCPU_DESTROY ,NULL );
dito
Quoted 2 lines
hw/s390x/pv.c new file +2
-0+S390_CPU (cs)->env.pv = false; +}
(...)
hw/s390x/pv.c line 143
Quoted 3 lines
hw/s390x/pv.c new file +3
-0+void s390_pv_perf_clear_reset (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_PREP_RESET ,NULL );
And here. Or is that because the machine should not be left around in an undefined state?
Quoted 17 lines
hw/s390x/pv.c new file +11
-0+} + +int s390_pv_verify (void ) +{ +return s390_pv_cmd (KVM_PV_VM_VERIFY ,NULL ); +} + +void s390_pv_unshare (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_UNSHARE_ALL ,NULL ); +}hw/s390x/pv.h new file +0
-0
hw/s390x/pv.h line 4
Quoted 4 lines
hw/s390x/pv.h new file +4
-0@@ -0,0 +1,38 @@ +/* +* Protected Virtualization header +* +* Copyright IBM Corp. 2019
Year++
Quoted 13 lines
hw/s390x/pv.h new file +13
-0+* Author(s): +* Janosch Frank <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ + +#ifndef HW_S390_PV_H +#define HW_S390_PV_H + +#ifdef CONFIG_KVM +int s390_pv_vm_create (void );
hw/s390x/pv.h line 27
Quoted 10 lines
hw/s390x/pv.h new file +10
-0+void s390_pv_vm_destroy (void ); +void s390_pv_vcpu_destroy (CPUState *cs); +int s390_pv_vcpu_create (CPUState *cs); +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length); +int s390_pv_unpack (uint64_t addr,uint64_t size,uint64_t tweak); +void s390_pv_perf_clear_reset (void ); +int s390_pv_verify (void ); +void s390_pv_unshare (void ); +#else +int s390_pv_vm_create (void ) {return 0 ; }
I'm wondering why you return 0 here (and below). These function should not be called for !KVM, but just to help catch logic error, use -EINVAL or so?
Quoted 24 lines
hw/s390x/pv.h new file +11
-0+void s390_pv_vm_destroy (void ) {} +void s390_pv_vcpu_destroy (CPUState *cs) {} +int s390_pv_vcpu_create (CPUState *cs) {return 0 ; } +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length) {return 0 ; } +int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: } +void s390_pv_perf_clear_reset (void ) {} +int s390_pv_verify (void ) {return 0 ; } +void s390_pv_unshare (void ) {} +#endif + +#endif /* HW_S390_PV_H */ hw/s390x/s390-virtio-ccw.c +1
-0@@ -41,6 +41,7 @@#include "hw/qdev-properties.h" #include "hw/s390x/tod.h" #include "sysemu/sysemu.h" +#include "hw/s390x/pv.h" S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {
hw/s390x/s390-virtio-ccw.c static void s390_create_sclpconsole(const char *type, Chardev *chardev) · line 248
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c +2
-0@@ -240,9 +241,11 @@ static void s390_create_sclpconsole(const char *type, Chardev *chardev)static void ccw_init (MachineState *machine) {int ret; +S390CcwMachineState *ms =S390_CCW_MACHINE (machine);VirtualCssBus *css_bus;DeviceState *dev; + ms->pv = false;
I'm wondering why you need to init this to false - isn't it already zeroed out?
Quoted 12 lines
hw/s390x/s390-virtio-ccw.c +5
-0s390_sclp_init ();/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram_size);@@ -318,10 +321,58 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu); } +static int s390_machine_pv_secure (S390CcwMachineState *ms) +{ +CPUState *t; +int rc; +
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 338
Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +10
-0+/* Create SE VM */ + rc =s390_pv_vm_create (); +if (rc) { +return rc; + } + +CPU_FOREACH (t) { + rc =s390_pv_vcpu_create (t); +if (rc) { +return rc;
No need to undo something on error?
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 347
Quoted 9 lines
hw/s390x/s390-virtio-ccw.c +9
-0+ } + } + + ms->pv = true; + +/* Set SE header and unpack */ + rc =s390_ipl_prepare_pv_header (); +if (rc) { +return rc;
Also here.
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 353
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +6
-0+ } + +/* Decrypt image */ + rc =s390_ipl_pv_unpack (); +if (rc) { +return rc;
And here.
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 358
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5
-0+ } + +/* Verify integrity */ + rc =s390_pv_verify (); +return rc;
And here.
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+}
(...)
Quoted 14 lines
target/s390x/cpu.c +2
-0@@ -37,6 +37,8 @@#include "sysemu/hw_accel.h" #include "hw/qdev-properties.h" #ifndef CONFIG_USER_ONLY +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/pv.h" #include "hw/boards.h" #include "sysemu/arch_init.h" #include "sysemu/sysemu.h"
target/s390x/cpu.c static void s390_cpu_realizefn(DeviceState *dev, Error **errp) · line 196
Quoted 4 lines
target/s390x/cpu.c +1
-0@@ -191,6 +193,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)#if !defined(CONFIG_USER_ONLY )MachineState *ms =MACHINE (qdev_get_machine ()); +S390CcwMachineState *ccw =S390_CCW_MACHINE (ms);
I find the variable name a bit confusing... maybe ccw_ms?
target/s390x/cpu.c static void s390_cpu_realizefn(DeviceState *dev, Error **errp) · line 211
Quoted 8 lines
target/s390x/cpu.c +1
-0unsigned int max_cpus = ms->smp.max_cpus;if (cpu->env.core_id >= max_cpus) { error_setg(&err, "Unable to add CPU with core-id: %" PRIu32@@ -205,6 +208,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp) goto out; } + cpu->env.pv = ccw->pv;
So, if you add a cpu, it will inherit the pv state of the machine... doesn't it need any setup?
Quoted 3 lines
target/s390x/cpu.c +0
-0/* sync cs->cpu_index and env->core_id. The latter is needed for TCG. */ cs->cpu_index = cpu->env.core_id;#endif
(...)
Re: [PATCH v3 06/17] s390x: protvirt: Add migration blocker
In reply to Janosch FrankQuoted 30 lines · On Fri, 14 Feb 2020 10:16:25 -0500 Janosch Frank <[email protected]> wrote:
Migration is not yet supported.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)hw/s390x/s390-virtio-ccw.c +4
-0@@ -42,6 +42,9 @@#include "hw/s390x/tod.h" #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" +#include "migration/blocker.h" + +static Error *pv_mig_blocker;S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {@@ -373,6 +376,7 @@ static void s390_machine_reset(MachineState *machine)CPUState *cs, *t;S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine); +static Error *local_err;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 433
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c +5
-0@@ -422,6 +426,17 @@ static void s390_machine_reset(MachineState *machine) }run_on_cpu (cs, s390_do_cpu_reset,RUN_ON_CPU_NULL ); +if (!pv_mig_blocker) { +error_setg (&pv_mig_blocker, +"protected VMs are currently not migrateable." ); + } +migrate_add_blocker (pv_mig_blocker, &local_err);
If I'm not lost in the context, that's during PV_RESET. I'm a bit confused why you'd add the blocker here?
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 437
Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +4
-0+if (local_err) { +error_report_err (local_err); +error_free (pv_mig_blocker); +exit (1 );
Why the exit()? Can't you fail the call?
Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +3
-0+ } +if (s390_machine_pv_secure (ms)) {CPU_FOREACH (t) {s390_pv_vcpu_destroy (t);@@ -430,6 +445,7 @@ static void s390_machine_reset(MachineState *machine) ms->pv = false;s390_machine_inject_pv_error (cs); +migrate_del_blocker (pv_mig_blocker);s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu);return ; }
Re: [PATCH v3 09/17] s390: protvirt: Move STSI data over SIDAD
In reply to Janosch FrankQuoted 14 lines · On Fri, 14 Feb 2020 10:16:28 -0500 Janosch Frank <[email protected]> wrote:
For protected guests, we need to put the STSI emulation results into the SIDA, so SIE will write them into the guest at the next entry.
Signed-off-by Janosch Frank <[email protected]>--- target/s390x/kvm.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)target/s390x/kvm.c +0
-0@@ -1772,11 +1772,16 @@ static int handle_tsch(S390CPU *cpu)
target/s390x/kvm.c static int handle_tsch(S390CPU *cpu) · line 1780
Quoted 10 lines
target/s390x/kvm.c +3
-2static void insert_stsi_3_2_2 (S390CPU *cpu,__u64 addr,uint8_t ar) { +CPUS390XState *env = &cpu->env;SysIB_322 sysib;int del; -if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu,0 , &sysib,sizeof (sysib));
This is only introduced by the next patch, right?
Quoted 20 lines
target/s390x/kvm.c +9
-1+ }else { +if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { +return ; + } }/* Shift the stack of Extended Names to prepare for our own data */ memmove(&sysib.ext_names[1], &sysib.ext_names[0],@@ -1815,7 +1820,11 @@ static void insert_stsi_3_2_2(S390CPU *cpu, __u64 addr, uint8_t ar)/* Insert UUID */ memcpy (sysib.vm[0 ].uuid, &qemu_uuid,sizeof (sysib.vm[0 ].uuid)); -s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); +if (env->pv) { +s390_cpu_pv_mem_write (cpu,0 , &sysib,sizeof (sysib)); + }else { +s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); + } }static int handle_stsi (S390CPU *cpu)
Re: [PATCH v3 13/17] s390x: protvirt: Move diag 308 data over SIDAD
In reply to Janosch FrankQuoted 3 lines · On Fri, 14 Feb 2020 10:16:32 -0500 Janosch Frank <[email protected]> wrote:
For protected guests the IPIB is written/read to/from the satellite block, so we need to make those accesses virtual to make them go through KVM mem ops.
Confused. What does 'make those accesses virtual' mean?
Quoted 19 lines
Signed-off-by Janosch Frank <[email protected]>--- target/s390x/diag.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-)target/s390x/diag.c +1
-0@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,void handle_diag_308 (CPUS390XState *env,uint64_t r1,uint64_t r3,uintptr_t ra) {CPUState *cs =env_cpu (env); +S390CPU *cpu =S390_CPU (cs);uint64_t addr = env->regs[r1];uint64_t subcode = env->regs[r3];IplParameterBlock *iplb;
target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 122
Quoted 4 lines
target/s390x/diag.c +1
-0@@ -118,14 +119,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)if (diag308_parm_check (env, r1, addr, ra, false)) {return ; } +
Whitespace.
Quoted 24 lines
target/s390x/diag.c +11
-2iplb =g_new0 (IplParameterBlock,1 ); -cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,sizeof (iplb->len)); + } +if (!iplb_valid_len (iplb)) { env->regs[r1 +1 ] =DIAG_308_RC_INVALID ; goto out; } -cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,be32_to_cpu (iplb->len)); + }if (!iplb_valid_ccw (iplb) && !iplb_valid_fcp (iplb) && !(iplb_valid_pv (iplb) &&s390_ipl_pv_check_components (iplb) >=0 )) {@@ -137,23 +148,30 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) env->regs[r1 +1 ] =DIAG_308_RC_OK ;
target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 157
Quoted 10 lines
target/s390x/diag.c +2
-1out:g_free (iplb); -return ; +break ;case DIAG308_STORE :case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } +
Whitespace.
Quoted 24 lines
target/s390x/diag.c +10
-4if (subcode ==DIAG308_PV_STORE ) { iplb =s390_ipl_get_iplb_secure (); }else { iplb =s390_ipl_get_iplb (); } -if (iplb) { -cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); - env->regs[r1 +1 ] =DIAG_308_RC_OK ; - }else { +if (!iplb) { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; +return ; } + +if (!env->pv) { +cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_write (cpu,0 , iplb,be32_to_cpu (iplb->len)); + } + + env->regs[r1 +1 ] =DIAG_308_RC_OK ;break ;case DIAG308_PV_START : iplb =s390_ipl_get_iplb_secure ();
Re: [PATCH v3 16/17] s390x: protvirt: Handle SIGP store status correctly
In reply to Janosch FrankQuoted 29 lines · On Fri, 14 Feb 2020 10:16:35 -0500 Janosch Frank <[email protected]> wrote:
Status storing is not done by QEMU anymore, but is handled by SIE.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>--- target/s390x/helper.c | 4 ++++ target/s390x/sigp.c | 1 + 2 files changed, 5 insertions(+)target/s390x/helper.c +4
-0@@ -246,6 +246,10 @@ int s390_store_status(S390CPU *cpu, hwaddr addr, bool store_arch)hwaddr len =sizeof (*sa);int i; +if (cpu->env.pv) { +return 0 ; + } + sa =cpu_physical_memory_map (addr, &len,1 );if (!sa) {return -EFAULT ;target/s390x/sigp.c +0
-0
target/s390x/sigp.c void do_stop_interrupt(CPUS390XState *env) · line 500
Quoted 4 lines
target/s390x/sigp.c +1
-0@@ -497,6 +497,7 @@ void do_stop_interrupt(CPUS390XState *env)if (s390_cpu_set_state (S390_CPU_STATE_STOPPED , cpu) ==0 ) {qemu_system_shutdown_request (SHUTDOWN_CAUSE_GUEST_SHUTDOWN ); } +/* Storing will occur on next SIE entry for fmt 4 */
What's fmt 4?
Quoted 3 lines
target/s390x/sigp.c +0
-0if (cpu->env.sigp_order ==SIGP_STOP_STORE_STATUS ) {s390_store_status (cpu,S390_STORE_STATUS_DEF_ADDR , true); }
Re: [PATCH v3 04/17] s390x: protvirt: Add diag308 subcodes 8 - 10
In reply to Cornelia HuckQuoted 31 lines · Cornelia Huck
Quoted 21 lines · On Fri, 14 Feb 2020 10:16:23 -0500 Janosch Frank <[email protected]> wrote:
For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.
Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.
Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.
Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/ipl.c | 48 ++++++++++++++++++++++++++++++++++++++++++--- hw/s390x/ipl.h | 31 +++++++++++++++++++++++++++++ target/s390x/diag.c | 27 ++++++++++++++++++++++--- 3 files changed, 100 insertions(+), 6 deletions(-)(...)
Quoted 4 lines
void
s390_ipl_update_diag308(IplParameterBlock *iplb){ S390IPLState *ipl = get_ipl_device();
Quoted 5 lines
hw/s390x/ipl.c static bool is_virtio_scsi_device(IplParameterBlock *iplb) · line 571
Quoted 3 lines
hw/s390x/ipl.c +1
-2- ipl->iplb = *iplb; - ipl->iplb_valid = true; +if (iplb->pbt ==5 ) {Magic value; maybe introduce a #define or at least a comment?
Well, there is already one defined, but it seems I never came around to use it -_-
Quoted 18 lines
Quoted 11 lines
hw/s390x/ipl.c +6
-0+ ipl->iplb_pbt5 = *iplb; + ipl->iplb_valid_pbt5 = true; + }else { + ipl->iplb = *iplb; + ipl->iplb_valid = true; + } ipl->netboot =is_virtio_net_device (iplb); }hw/s390x/ipl.c struct S390IPLState { · line 157
Quoted 4 lines
hw/s390x/ipl.c +1
-0@@ -133,6 +154,7 @@ struct S390IPLState {/*< private >*/ DeviceState parent_obj;IplParameterBlock iplb; +IplParameterBlock iplb_pbt5;Add /* for protected virtualization */ ?
Quoted 1 line
Or, if this is not used for anything else, call it
iplb_pv?
ack.
Quoted 7 lines
Quoted 4 lines
hw/s390x/ipl.h +0
-0QemuIplParameters qipl;uint64_t start_addr;uint64_t compat_start_addr;(...)
Quoted 12 lines
target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 56
Quoted 10 lines
target/s390x/diag.c +2
-0@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)#define DIAG_308_RC_OK 0x0001#define DIAG_308_RC_NO_CONF 0x0102#define DIAG_308_RC_INVALID 0x0402 +#define DIAG_308_RC_NO_PV_CONF 0x0a02 +#define DIAG_308_RC_INV_FOR_PV 0x0b02
DIAG_308_RC_INVAL_FOR_PV?
ack
Quoted 23 lines
Quoted 4 lines
target/s390x/diag.c +0
-0#define DIAG308_RESET_MOD_CLR 0#define DIAG308_RESET_LOAD_NORM 1(...)
Quoted 4 lines
target/s390x/diag.c +0
-0@@ -128,17 +135,31 @@ out:g_free (iplb);return ;case DIAG308_STORE :target/s390x/diag.c out: · line 146
Quoted 10 lines
target/s390x/diag.c +6
-1+case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } - iplb =s390_ipl_get_iplb (); +if (subcode ==DIAG308_PV_STORE ) { + iplb =s390_ipl_get_iplb_secure (); + }else { + iplb =s390_ipl_get_iplb (); + }iplb = (subcode ==
DIAG308_PV_STORE) ?
Quoted 3 lines
s390_ipl_get_iplb_secure() : s390_ipl_get_iplb();Matter of taste, I guess.
I'll have a look what I find nicer to read
Quoted 22 lines
Quoted 21 lines
target/s390x/diag.c +10
-1if (iplb) {cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); env->regs[r1 +1 ] =DIAG_308_RC_OK ; }else { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; } -return ; +break ; +case DIAG308_PV_START : + iplb =s390_ipl_get_iplb_secure (); +if (!iplb || !iplb_valid_pv (iplb)) { + env->regs[r1 +1 ] =DIAG_308_RC_NO_PV_CONF ; +return ; + } + +s390_ipl_reset_request (cs,S390_RESET_PV ); +break ;default :s390_program_interrupt (env,PGM_SPECIFICATION , ra);break ;
Re: [PATCH v3 05/17] s390x: protvirt: Support unpack facility
In reply to Cornelia HuckQuoted 40 lines · Cornelia Huck
Quoted 1 line · On Fri, 14 Feb 2020 10:16:24 -0500 Janosch Frank <[email protected]> wrote:
When a guest has saved a ipib of type 5 and call diagnose308 with
s/call/calls/
Quoted 30 lines
subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.
The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrityOnly after step 5 a protected VM is allowed to run.
Signed-off-by Janosch Frank <[email protected]>Signed-off-by Christian Borntraeger <[email protected]> [Changesto machine]
--- hw/s390x/Makefile.objs | 1 + hw/s390x/ipl.c | 32 ++++++ hw/s390x/ipl.h | 2 + hw/s390x/pv.c | 154 ++++++++++++++++++++++++++++ hw/s390x/pv.h | 38 +++++++ hw/s390x/s390-virtio-ccw.c | 79 ++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 1 + target/s390x/cpu.c | 4 + target/s390x/cpu.h | 1 + target/s390x/cpu_features_def.inc.h | 1 + 10 files changed, 313 insertions(+) create mode 100644 hw/s390x/pv.c create mode 100644 hw/s390x/pv.h(...)
Quoted 12 lines
hw/s390x/pv.c line 4
Quoted 10 lines
hw/s390x/pv.c new file +4
-0@@ -0,0 +1,154 @@ +/* +* Secure execution functions +* +* Copyright IBM Corp. 2019 Update the year?
ack.
Quoted 9 lines
hw/s390x/pv.c line 11
Quoted 8 lines
hw/s390x/pv.c new file +7
-0+* Author(s): +* Janosch Frank <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/
Quoted 7 lines
(...)
hw/s390x/pv.c line 96
Quoted 3 lines
hw/s390x/pv.c new file +3
-0+void s390_pv_vm_destroy (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_DESTROY ,NULL );Why does this
exit()? Should Never Happen?
Yes, and we can't recover from this.
Quoted 24 lines
Quoted 8 lines
hw/s390x/pv.c new file +7
-0+} + +int s390_pv_vcpu_create (CPUState *cs) +{ +int rc; + + rc =s390_pv_cmd_vcpu (cs,KVM_PV_VCPU_CREATE ,NULL );hw/s390x/pv.c line 113
Quoted 10 lines
hw/s390x/pv.c new file +10
-0+if (!rc) { +S390_CPU (cs)->env.pv = true; + } + +return rc; +} + +void s390_pv_vcpu_destroy (CPUState *cs) +{ +s390_pv_cmd_vcpu_exit (cs,KVM_PV_VCPU_DESTROY ,NULL );dito
hw/s390x/pv.c
Quoted 2 lines
hw/s390x/pv.c new file +2
-0+S390_CPU (cs)->env.pv = false; +}
Quoted 8 lines
(...)
hw/s390x/pv.c line 143
Quoted 3 lines
hw/s390x/pv.c new file +3
-0+void s390_pv_perf_clear_reset (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_PREP_RESET ,NULL );And here. Or is that because the machine should not be left around in an undefined state?
If it failed, we could only try again, there's no fixing the problem. So I chose to rather exit instead of looping around something which most likely will never recover after the first error.
Quoted 105 lines
Quoted 18 lines
hw/s390x/pv.c new file +11
-0+} + +int s390_pv_verify (void ) +{ +return s390_pv_cmd (KVM_PV_VM_VERIFY ,NULL ); +} + +void s390_pv_unshare (void ) +{ +s390_pv_cmd_exit (KVM_PV_VM_UNSHARE_ALL ,NULL ); +}hw/s390x/pv.h new file +0
-0hw/s390x/pv.h line 4
Quoted 4 lines
hw/s390x/pv.h new file +4
-0@@ -0,0 +1,38 @@ +/* +* Protected Virtualization header +* +* Copyright IBM Corp. 2019 Year++
Quoted 13 lines
hw/s390x/pv.h new file +13
-0+* Author(s): +* Janosch Frank <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ + +#ifndef HW_S390_PV_H +#define HW_S390_PV_H + +#ifdef CONFIG_KVM +int s390_pv_vm_create (void );hw/s390x/pv.h line 27
Quoted 10 lines
hw/s390x/pv.h new file +10
-0+void s390_pv_vm_destroy (void ); +void s390_pv_vcpu_destroy (CPUState *cs); +int s390_pv_vcpu_create (CPUState *cs); +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length); +int s390_pv_unpack (uint64_t addr,uint64_t size,uint64_t tweak); +void s390_pv_perf_clear_reset (void ); +int s390_pv_verify (void ); +void s390_pv_unshare (void ); +#else +int s390_pv_vm_create (void ) {return 0 ; }I'm wondering why you return 0 here (and below). These function should not be called for !KVM, but just to help catch logic error, use
-EINVALor so?Quoted 24 lines
hw/s390x/pv.h new file +11
-0+void s390_pv_vm_destroy (void ) {} +void s390_pv_vcpu_destroy (CPUState *cs) {} +int s390_pv_vcpu_create (CPUState *cs) {return 0 ; } +int s390_pv_set_sec_parms (uint64_t origin,uint64_t length) {return 0 ; } +int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: } +void s390_pv_perf_clear_reset (void ) {} +int s390_pv_verify (void ) {return 0 ; } +void s390_pv_unshare (void ) {} +#endif + +#endif /* HW_S390_PV_H */ hw/s390x/s390-virtio-ccw.c +1
-0@@ -41,6 +41,7 @@#include "hw/qdev-properties.h" #include "hw/s390x/tod.h" #include "sysemu/sysemu.h" +#include "hw/s390x/pv.h" S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {hw/s390x/s390-virtio-ccw.c static void s390_create_sclpconsole(const char *type, Chardev *chardev) · line 248
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c +2
-0@@ -240,9 +241,11 @@ static void s390_create_sclpconsole(const char *type, Chardev *chardev)static void ccw_init (MachineState *machine) {int ret; +S390CcwMachineState *ms =S390_CCW_MACHINE (machine);VirtualCssBus *css_bus;DeviceState *dev; + ms->pv = false;I'm wondering why you need to init this to false - isn't it already zeroed out?
Quoted 16 lines
hw/s390x/s390-virtio-ccw.c +9
-0s390_sclp_init ();/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram_size);@@ -318,10 +321,58 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu); } +static int s390_machine_pv_secure (S390CcwMachineState *ms) +{ +CPUState *t; +int rc; + +/* Create SE VM */ + rc =s390_pv_vm_create (); +if (rc) { +return rc;
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 338
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +6
-0+ } + +CPU_FOREACH (t) { + rc =s390_pv_vcpu_create (t); +if (rc) { +return rc;No need to undo something on error?
There have been changes in this area anyway, since Christian switched to one create/destroy instead of separate for vm and vcpu.
I'll update the error handling in the new state and send out the patches ssonish.
Quoted 73 lines
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 347
Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +9
-0+ } + } + + ms->pv = true; + +/* Set SE header and unpack */ + rc =s390_ipl_prepare_pv_header (); +if (rc) { +return rc;Also here.
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 353
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +6
-0+ } + +/* Decrypt image */ + rc =s390_ipl_pv_unpack (); +if (rc) { +return rc;And here.
hw/s390x/s390-virtio-ccw.c static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg) · line 358
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5
-0+ } + +/* Verify integrity */ + rc =s390_pv_verify (); +return rc;And here.
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+}(...)
Quoted 14 lines
target/s390x/cpu.c +2
-0@@ -37,6 +37,8 @@#include "sysemu/hw_accel.h" #include "hw/qdev-properties.h" #ifndef CONFIG_USER_ONLY +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/pv.h" #include "hw/boards.h" #include "sysemu/arch_init.h" #include "sysemu/sysemu.h" target/s390x/cpu.c static void s390_cpu_realizefn(DeviceState *dev, Error **errp) · line 196
Quoted 4 lines
target/s390x/cpu.c +1
-0@@ -191,6 +193,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)#if !defined(CONFIG_USER_ONLY )MachineState *ms =MACHINE (qdev_get_machine ()); +S390CcwMachineState *ccw =S390_CCW_MACHINE (ms);I find the variable name a bit confusing... maybe
ccw_ms?target/s390x/cpu.c static void s390_cpu_realizefn(DeviceState *dev, Error **errp) · line 211
Quoted 8 lines
target/s390x/cpu.c +1
-0unsigned int max_cpus = ms->smp.max_cpus;if (cpu->env.core_id >= max_cpus) { error_setg(&err, "Unable to add CPU with core-id: %" PRIu32@@ -205,6 +208,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp) goto out; } + cpu->env.pv = ccw->pv;So, if you add a cpu, it will inherit the pv state of the machine... doesn't it need any setup?
Quoted 3 lines
target/s390x/cpu.c +0
-0/* sync cs->cpu_index and env->core_id. The latter is needed for TCG. */ cs->cpu_index = cpu->env.core_id;#endif (...)
Re: [PATCH v3 06/17] s390x: protvirt: Add migration blocker
In reply to Cornelia HuckQuoted 36 lines · Cornelia Huck
Quoted 33 lines · On Fri, 14 Feb 2020 10:16:25 -0500 Janosch Frank <[email protected]> wrote:
Migration is not yet supported.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)hw/s390x/s390-virtio-ccw.c +4
-0@@ -42,6 +42,9 @@#include "hw/s390x/tod.h" #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" +#include "migration/blocker.h" + +static Error *pv_mig_blocker;S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {@@ -373,6 +376,7 @@ static void s390_machine_reset(MachineState *machine)CPUState *cs, *t;S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine); +static Error *local_err;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);@@ -422,6 +426,17 @@ static void s390_machine_reset(MachineState *machine) }run_on_cpu (cs, s390_do_cpu_reset,RUN_ON_CPU_NULL );
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 433
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5
-0+if (!pv_mig_blocker) { +error_setg (&pv_mig_blocker, +"protected VMs are currently not migrateable." ); + } +migrate_add_blocker (pv_mig_blocker, &local_err);If I'm not lost in the context, that's during
PV_RESET. I'm a bit confused why you'd add the blocker here?
Where would you want me to add it?
It's here where we switch into secure mode and I need to block before switching and unblock if it fails.
When having the blocker in diag.c, I'd have a hard time unblocking on a PV switch fail.
Quoted 7 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 437
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +4
-0+if (local_err) { +error_report_err (local_err); +error_free (pv_mig_blocker); +exit (1 );Why the
exit()? Can't you fail the call?
Well, if that fails and we go protected, I wouldn't be protected agains migrations, right?
Quoted 16 lines
Quoted 14 lines
hw/s390x/s390-virtio-ccw.c +3
-0+ } +if (s390_machine_pv_secure (ms)) {CPU_FOREACH (t) {s390_pv_vcpu_destroy (t);@@ -430,6 +445,7 @@ static void s390_machine_reset(MachineState *machine) ms->pv = false;s390_machine_inject_pv_error (cs); +migrate_del_blocker (pv_mig_blocker);s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu);return ; }
Re: [PATCH v3 09/17] s390: protvirt: Move STSI data over SIDAD
In reply to Cornelia HuckQuoted 23 lines · Cornelia Huck
Quoted 20 lines · On Fri, 14 Feb 2020 10:16:28 -0500 Janosch Frank <[email protected]> wrote:
For protected guests, we need to put the STSI emulation results into the SIDA, so SIE will write them into the guest at the next entry.
Signed-off-by Janosch Frank <[email protected]>--- target/s390x/kvm.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)target/s390x/kvm.c +1
-0@@ -1772,11 +1772,16 @@ static int handle_tsch(S390CPU *cpu)static void insert_stsi_3_2_2 (S390CPU *cpu,__u64 addr,uint8_t ar) { +CPUS390XState *env = &cpu->env;SysIB_322 sysib;int del;
Quoted 6 lines
target/s390x/kvm.c static int handle_tsch(S390CPU *cpu) · line 1780
Quoted 4 lines
target/s390x/kvm.c +2
-2-if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { -return ; +if (env->pv) { +s390_cpu_pv_mem_read (cpu,0 , &sysib,sizeof (sysib));This is only introduced by the next patch, right?
Ups, time to reorder.
Quoted 23 lines
Quoted 21 lines
target/s390x/kvm.c +9
-1+ }else { +if (s390_cpu_virt_mem_read (cpu, addr, ar, &sysib,sizeof (sysib))) { +return ; + } }/* Shift the stack of Extended Names to prepare for our own data */ memmove(&sysib.ext_names[1], &sysib.ext_names[0],@@ -1815,7 +1820,11 @@ static void insert_stsi_3_2_2(S390CPU *cpu, __u64 addr, uint8_t ar)/* Insert UUID */ memcpy (sysib.vm[0 ].uuid, &qemu_uuid,sizeof (sysib.vm[0 ].uuid)); -s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); +if (env->pv) { +s390_cpu_pv_mem_write (cpu,0 , &sysib,sizeof (sysib)); + }else { +s390_cpu_virt_mem_write (cpu, addr, ar, &sysib,sizeof (sysib)); + } }static int handle_stsi (S390CPU *cpu)
Re: [PATCH v3 13/17] s390x: protvirt: Move diag 308 data over SIDAD
In reply to Cornelia HuckQuoted 6 lines · Cornelia Huck
Quoted 3 lines · On Fri, 14 Feb 2020 10:16:32 -0500 Janosch Frank <[email protected]> wrote:
For protected guests the IPIB is written/read to/from the satellite block, so we need to make those accesses virtual to make them go through KVM mem ops.
..we need those accesses to go through s390_cpu_pv_mem_read/write() so we use the memop IOCTL to access the sattelite block.
Quoted 2 lines
Confused. What does 'make those accesses virtual' mean?
I need to update the patch description.
Virtual memory access was needed when I overloaded the normal memops to access the SIDA. Real access would have accessed guest memory directly which results in a guest crash for protected VMs.
Quoted 18 lines
Quoted 18 lines
Signed-off-by Janosch Frank <[email protected]>--- target/s390x/diag.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-)target/s390x/diag.c +1
-0@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,void handle_diag_308 (CPUS390XState *env,uint64_t r1,uint64_t r3,uintptr_t ra) {CPUState *cs =env_cpu (env); +S390CPU *cpu =S390_CPU (cs);uint64_t addr = env->regs[r1];uint64_t subcode = env->regs[r3];
Quoted 8 lines
target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 122
Quoted 6 lines
target/s390x/diag.c +1
-0IplParameterBlock *iplb;@@ -118,14 +119,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)if (diag308_parm_check (env, r1, addr, ra, false)) {return ; } +Whitespace.
Ack.
Quoted 21 lines
Quoted 21 lines
target/s390x/diag.c +11
-2iplb =g_new0 (IplParameterBlock,1 ); -cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,sizeof (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,sizeof (iplb->len)); + } +if (!iplb_valid_len (iplb)) { env->regs[r1 +1 ] =DIAG_308_RC_INVALID ; goto out; } -cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); +if (!env->pv) { +cpu_physical_memory_read (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_read (cpu,0 , iplb,be32_to_cpu (iplb->len)); + }
Quoted 16 lines
Quoted 4 lines
target/s390x/diag.c +0
-0if (!iplb_valid_ccw (iplb) && !iplb_valid_fcp (iplb) && !(iplb_valid_pv (iplb) &&s390_ipl_pv_check_components (iplb) >=0 )) {@@ -137,23 +148,30 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) env->regs[r1 +1 ] =DIAG_308_RC_OK ;target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 157
Quoted 10 lines
target/s390x/diag.c +2
-1out:g_free (iplb); -return ; +break ;case DIAG308_STORE :case DIAG308_PV_STORE :if (diag308_parm_check (env, r1, addr, ra, true)) {return ; } +Whitespace.
Ack.
Quoted 27 lines
Quoted 25 lines
target/s390x/diag.c +10
-4if (subcode ==DIAG308_PV_STORE ) { iplb =s390_ipl_get_iplb_secure (); }else { iplb =s390_ipl_get_iplb (); } -if (iplb) { -cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); - env->regs[r1 +1 ] =DIAG_308_RC_OK ; - }else { +if (!iplb) { env->regs[r1 +1 ] =DIAG_308_RC_NO_CONF ; +return ; } + +if (!env->pv) { +cpu_physical_memory_write (addr, iplb,be32_to_cpu (iplb->len)); + }else { +s390_cpu_pv_mem_write (cpu,0 , iplb,be32_to_cpu (iplb->len)); + } + + env->regs[r1 +1 ] =DIAG_308_RC_OK ;break ;case DIAG308_PV_START : iplb =s390_ipl_get_iplb_secure ();
Re: [PATCH v3 16/17] s390x: protvirt: Handle SIGP store status correctly
In reply to Cornelia HuckQuoted 30 lines · Cornelia Huck
Quoted 27 lines · On Fri, 14 Feb 2020 10:16:35 -0500 Janosch Frank <[email protected]> wrote:
Status storing is not done by QEMU anymore, but is handled by SIE.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>--- target/s390x/helper.c | 4 ++++ target/s390x/sigp.c | 1 + 2 files changed, 5 insertions(+)target/s390x/helper.c +4
-0@@ -246,6 +246,10 @@ int s390_store_status(S390CPU *cpu, hwaddr addr, bool store_arch)hwaddr len =sizeof (*sa);int i; +if (cpu->env.pv) { +return 0 ; + } + sa =cpu_physical_memory_map (addr, &len,1 );if (!sa) {return -EFAULT ;target/s390x/sigp.c +0
-1--- a/target/s390x/sigp.c
Quoted 8 lines
target/s390x/sigp.c void do_stop_interrupt(CPUS390XState *env) · line 500
Quoted 6 lines
+++
b/target/s390x/sigp.ctarget/s390x/sigp.c +1
-0@@ -497,6 +497,7 @@ void do_stop_interrupt(CPUS390XState *env)if (s390_cpu_set_state (S390_CPU_STATE_STOPPED , cpu) ==0 ) {qemu_system_shutdown_request (SHUTDOWN_CAUSE_GUEST_SHUTDOWN ); } +/* Storing will occur on next SIE entry for fmt 4 */ What's fmt 4?
PVMs are using a FMT 4 SCB instead of the normal format 2 one.
So I guess this should read:
/* Storing will occur on next SIE entry for protected VMs */
Quoted 6 lines
Quoted 4 lines
target/s390x/sigp.c +0
-0if (cpu->env.sigp_order ==SIGP_STOP_STORE_STATUS ) {s390_store_status (cpu,S390_STORE_STATUS_DEF_ADDR , true); }
Re: [PATCH v3 16/17] s390x: protvirt: Handle SIGP store status correctly
Quoted 40 lines · On Thu, 20 Feb 2020 12:30:28 +0100 Janosch Frank <[email protected]> wrote:
Quoted 30 lines · Cornelia Huck
Quoted 27 lines · On Fri, 14 Feb 2020 10:16:35 -0500 Janosch Frank <[email protected]> wrote:
Status storing is not done by QEMU anymore, but is handled by SIE.
Signed-off-by Janosch Frank <[email protected]>Reviewed-by Thomas Huth <[email protected]>--- target/s390x/helper.c | 4 ++++ target/s390x/sigp.c | 1 + 2 files changed, 5 insertions(+)target/s390x/helper.c +4
-0@@ -246,6 +246,10 @@ int s390_store_status(S390CPU *cpu, hwaddr addr, bool store_arch)hwaddr len =sizeof (*sa);int i; +if (cpu->env.pv) { +return 0 ; + } + sa =cpu_physical_memory_map (addr, &len,1 );if (!sa) {return -EFAULT ;target/s390x/sigp.c +0
-1--- a/target/s390x/sigp.cQuoted 8 lines
target/s390x/sigp.c void do_stop_interrupt(CPUS390XState *env) · line 500
Quoted 6 lines
+++
b/target/s390x/sigp.ctarget/s390x/sigp.c +1
-0@@ -497,6 +497,7 @@ void do_stop_interrupt(CPUS390XState *env)if (s390_cpu_set_state (S390_CPU_STATE_STOPPED , cpu) ==0 ) {qemu_system_shutdown_request (SHUTDOWN_CAUSE_GUEST_SHUTDOWN ); } +/* Storing will occur on next SIE entry for fmt 4 */ What's fmt 4?
Quoted 4 lines
PVMs are using a FMT 4 SCB instead of the normal format 2 one.
So I guess this should read:
/* Storing will occur on next SIE entry for protected VMs */
Better :)
Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
target/s390x/sigp.c +0
-0if (cpu->env.sigp_order ==SIGP_STOP_STORE_STATUS ) {s390_store_status (cpu,S390_STORE_STATUS_DEF_ADDR , true); }
Re: [PATCH v3 06/17] s390x: protvirt: Add migration blocker
In reply to Janosch FrankQuoted 59 lines · On Thu, 20 Feb 2020 12:24:23 +0100 Janosch Frank <[email protected]> wrote:
Quoted 36 lines · Cornelia Huck
Quoted 33 lines · On Fri, 14 Feb 2020 10:16:25 -0500 Janosch Frank <[email protected]> wrote:
Migration is not yet supported.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)hw/s390x/s390-virtio-ccw.c +4
-0@@ -42,6 +42,9 @@#include "hw/s390x/tod.h" #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" +#include "migration/blocker.h" + +static Error *pv_mig_blocker;S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {@@ -373,6 +376,7 @@ static void s390_machine_reset(MachineState *machine)CPUState *cs, *t;S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine); +static Error *local_err;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);@@ -422,6 +426,17 @@ static void s390_machine_reset(MachineState *machine) }run_on_cpu (cs, s390_do_cpu_reset,RUN_ON_CPU_NULL );Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 433
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5
-0+if (!pv_mig_blocker) { +error_setg (&pv_mig_blocker, +"protected VMs are currently not migrateable." ); + } +migrate_add_blocker (pv_mig_blocker, &local_err);If I'm not lost in the context, that's during
PV_RESET. I'm a bit confused why you'd add the blocker here?Where would you want me to add it?
It's here where we switch into secure mode and I need to block before switching and unblock if it fails.When having the blocker in
diag.c, I'd have a hard time unblocking on a PV switch fail.Quoted 6 lines
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +4
-0+if (local_err) { +error_report_err (local_err); +error_free (pv_mig_blocker); +exit (1 );
Quoted 4 lines
Quoted 1 line
Why the
exit()? Can't you fail the call?Well, if that fails and we go protected, I wouldn't be protected agains migrations, right?
No, I meant not go protected, if that's possible.
Quoted 5 lines
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 440
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +2
-0+ } +if (s390_machine_pv_secure (ms)) {
Ok, I think what confuses me is this call: it reads as if you actually tear down things if the machine is secure. Call it s390_machine_pv_make_secure() to make sure it is actively doing something and not checking a previously set value?
Quoted 14 lines
Quoted 12 lines
Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +1
-0CPU_FOREACH (t) {s390_pv_vcpu_destroy (t);@@ -430,6 +445,7 @@ static void s390_machine_reset(MachineState *machine) ms->pv = false;s390_machine_inject_pv_error (cs); +migrate_del_blocker (pv_mig_blocker);s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu);return ; }
Re: [PATCH v3 06/17] s390x: protvirt: Add migration blocker
Quoted 62 lines · Cornelia Huck
Quoted 59 lines · On Thu, 20 Feb 2020 12:24:23 +0100 Janosch Frank <[email protected]> wrote:
Quoted 36 lines · Cornelia Huck
Quoted 33 lines · On Fri, 14 Feb 2020 10:16:25 -0500 Janosch Frank <[email protected]> wrote:
Migration is not yet supported.
Signed-off-by Janosch Frank <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)hw/s390x/s390-virtio-ccw.c +4
-0@@ -42,6 +42,9 @@#include "hw/s390x/tod.h" #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" +#include "migration/blocker.h" + +static Error *pv_mig_blocker;S390CPU *s390_cpu_addr2state (uint16_t cpu_addr) {@@ -373,6 +376,7 @@ static void s390_machine_reset(MachineState *machine)CPUState *cs, *t;S390CPU *cpu;S390CcwMachineState *ms =S390_CCW_MACHINE (machine); +static Error *local_err;/* get the reset parameters, reset them once done */ s390_ipl_get_reset_request (&cs, &reset_type);@@ -422,6 +426,17 @@ static void s390_machine_reset(MachineState *machine) }run_on_cpu (cs, s390_do_cpu_reset,RUN_ON_CPU_NULL );Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 433
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5
-0+if (!pv_mig_blocker) { +error_setg (&pv_mig_blocker, +"protected VMs are currently not migrateable." ); + } +migrate_add_blocker (pv_mig_blocker, &local_err);If I'm not lost in the context, that's during
PV_RESET. I'm a bit confused why you'd add the blocker here?Where would you want me to add it?
It's here where we switch into secure mode and I need to block before switching and unblock if it fails.When having the blocker in
diag.c, I'd have a hard time unblocking on a PV switch fail.Quoted 6 lines
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +4
-0+if (local_err) { +error_report_err (local_err); +error_free (pv_mig_blocker); +exit (1 );
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Why the
exit()? Can't you fail the call?Well, if that fails and we go protected, I wouldn't be protected agains migrations, right?
No, I meant not go protected, if that's possible.
That would be an option, now that we have a proper d308 rc for such a thing.
Will add!
Quoted 11 lines
Quoted 6 lines
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 440
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +2
-0+ } +if (s390_machine_pv_secure (ms)) {Ok, I think what confuses me is this call: it reads as if you actually tear down things if the machine is secure. Call it
s390_machine_pv_make_secure()to make sure it is actively doing something and not checking a previously set value?
Ok, will use something along these lines
Quoted 16 lines
Quoted 15 lines
Quoted 13 lines
Quoted 11 lines
hw/s390x/s390-virtio-ccw.c +1
-0CPU_FOREACH (t) {s390_pv_vcpu_destroy (t);@@ -430,6 +445,7 @@ static void s390_machine_reset(MachineState *machine) ms->pv = false;s390_machine_inject_pv_error (cs); +migrate_del_blocker (pv_mig_blocker);s390_cpu_set_state (S390_CPU_STATE_OPERATING , cpu);return ; }