Skip to contentSkip to conversation

s390x: Protected Virtualization support

PATCH Superseded Janosch Frank 8 participants 90 messages Nov 20, 2019 – Feb 7, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover s390x: Protected Virtualization support

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Janosch Frank (15):
s390x: Cleanup cpu resets
s390x: Beautify diag308 handling
s390x: protvirt: Add diag308 subcodes 8 - 10
Header sync protvirt
s390x: protvirt: Sync PV state
s390x: protvirt: Support unpack facility
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390x: protvirt: SCLP interpretation
s390x: protvirt: Add new VCPU reset functions
RFC: s390x: Exit on vcpu reset error
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Handle SIGP store status correctly

 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  81 +++++++++++++++++-
 hw/s390x/ipl.h                      |  35 ++++++++
 hw/s390x/pv.c                       | 123 +++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  27 ++++++
 hw/s390x/s390-virtio-ccw.c          |  79 ++++++++++++++---
 hw/s390x/sclp.c                     |  16 ++++
 include/hw/s390x/sclp.h             |   2 +
 linux-headers/asm-s390/kvm.h        |   4 +-
 linux-headers/linux/kvm.h           |  43 ++++++++++
 target/s390x/cpu.c                  | 127 ++++++++++++++--------------
 target/s390x/cpu.h                  |   1 +
 target/s390x/cpu_features_def.inc.h |   1 +
 target/s390x/diag.c                 | 108 +++++++++++++++++------
 target/s390x/ioinst.c               |  46 ++++++----
 target/s390x/kvm-stub.c             |  10 ++-
 target/s390x/kvm.c                  |  58 +++++++++++--
 target/s390x/kvm_s390x.h            |   4 +-
 target/s390x/sigp.c                 |   7 +-
 19 files changed, 640 insertions(+), 133 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/15 s390x: Cleanup cpu resets

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)
hw/s390x/s390-virtio-ccw.c +3 -0
index d3edeef0ad..c1d1440272 100644
@@ -348,6 +348,9 @@ static void s390_machine_reset(MachineState *machine)
         break;
     case S390_RESET_LOAD_NORMAL:
         CPU_FOREACH(t) {
+            if (t == cs) {
+                continue;
+            }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
         subsystem_reset();
target/s390x/cpu.c +49 -62

145 lines not on this page · raw message

Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

Quoted 2 lines · On Wed, 20 Nov 2019 06:43:20 -0500 Janosch Frank <[email protected]> wrote:

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

Doing that makes sense.

Quoted 12 lines
---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)
hw/s390x/s390-virtio-ccw.c +0 -0
index d3edeef0ad..c1d1440272 100644

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 353

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +3 -0
@@ -348,6 +348,9 @@ static void s390_machine_reset(MachineState *machine)
         break;
     case S390_RESET_LOAD_NORMAL:
         CPU_FOREACH(t) {
+            if (t == cs) {
+                continue;
+            }

Hm, why is this needed now?

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 85

Quoted 12 lines
hw/s390x/s390-virtio-ccw.c +0 -0
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
         subsystem_reset();
target/s390x/cpu.c +0 -1
index 3abe7e80fd..10d5b915d8 100644
@@ -82,67 +82,53 @@ static void s390_cpu_load_normal(CPUState *s)
 }
 #endif
 
-/* S390CPUClass::cpu_reset() */

Not sure if it would be worth keeping these comments near by the calling functions.

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 89

Quoted 6 lines
target/s390x/cpu.c +5 -1
-static void s390_cpu_reset(CPUState *s)
+enum {
+    S390_CPU_RESET_NORMAL,
+    S390_CPU_RESET_INITIAL,
+    S390_CPU_RESET_CLEAR,
+};

Maybe make this into a proper type, so you can use type checking?

(...)

The diff is a bit hard to read, but the change seems fine at a glance.

#3 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

Quoted 15 lines · Cornelia Huck
Quoted 2 lines · On Wed, 20 Nov 2019 06:43:20 -0500 Janosch Frank <[email protected]> wrote:

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

Doing that makes sense.

Quoted 7 lines
---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)
Quoted 13 lines

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 353

Quoted 11 lines
hw/s390x/s390-virtio-ccw.c +3 -0
index d3edeef0ad..c1d1440272 100644
@@ -348,6 +348,9 @@ static void s390_machine_reset(MachineState *machine)
         break;
     case S390_RESET_LOAD_NORMAL:
         CPU_FOREACH(t) {
+            if (t == cs) {
+                continue;
+            }

Hm, why is this needed now?

The Ultravisor checks which reset is done to which cpu.
So blindly resetting the calling cpu with a normal reset to then do a clear/initial reset will return an error.

Quoted 17 lines

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 85

Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +0 -0
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
         subsystem_reset();
target/s390x/cpu.c +0 -1
index 3abe7e80fd..10d5b915d8 100644
@@ -82,67 +82,53 @@ static void s390_cpu_load_normal(CPUState *s)
 }
 #endif
 
-/* S390CPUClass::cpu_reset() */

Not sure if it would be worth keeping these comments near by the calling functions.

Quoted 8 lines

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 89

Quoted 6 lines
target/s390x/cpu.c +5 -1
-static void s390_cpu_reset(CPUState *s)
+enum {
+    S390_CPU_RESET_NORMAL,
+    S390_CPU_RESET_INITIAL,
+    S390_CPU_RESET_CLEAR,
+};

Maybe make this into a proper type, so you can use type checking?

Ok

Quoted 5 lines

(...)

The diff is a bit hard to read, but the change seems fine at a glance.

#4 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

Quoted 30 lines · On Thu, 21 Nov 2019 12:32:38 +0100 Janosch Frank <[email protected]> wrote:
Quoted 15 lines · Cornelia Huck
Quoted 2 lines · On Wed, 20 Nov 2019 06:43:20 -0500 Janosch Frank <[email protected]> wrote:

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

Doing that makes sense.

Quoted 7 lines
---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)
Quoted 13 lines

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 353

Quoted 11 lines
hw/s390x/s390-virtio-ccw.c +3 -0
index d3edeef0ad..c1d1440272 100644
@@ -348,6 +348,9 @@ static void s390_machine_reset(MachineState *machine)
         break;
     case S390_RESET_LOAD_NORMAL:
         CPU_FOREACH(t) {
+            if (t == cs) {
+                continue;
+            }  

Hm, why is this needed now?

Quoted 3 lines

The Ultravisor checks which reset is done to which cpu.
So blindly resetting the calling cpu with a normal reset to then do a clear/initial reset will return an error.

Let's split this change out, then? The rest of the patch is a simple refactoring.

#5 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

In reply to Janosch Frank
Quoted 8 lines · Janosch Frank

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)

[...]

Quoted 7 lines
target/s390x/cpu.c +0 -0
index 3abe7e80fd..10d5b915d8 100644
@@ -82,67 +82,53 @@ static void s390_cpu_load_normal(CPUState *s)
 }
 #endif

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 91

Quoted 10 lines
target/s390x/cpu.c +7 -2
 
-/* S390CPUClass::cpu_reset() */
-static void s390_cpu_reset(CPUState *s)
+enum {
+    S390_CPU_RESET_NORMAL,
+    S390_CPU_RESET_INITIAL,
+    S390_CPU_RESET_CLEAR,
+};
+
+static void s390_cpu_reset(CPUState *s, uint8_t type)

Please give the enum a name and use that instead of uint8_t for "type". Or at least make it an "int". uint8_t is not really appropriate here.

Quoted 11 lines
target/s390x/cpu.c +0 -3
 {
     S390CPU *cpu = S390_CPU(s);
     S390CPUClass *scc = S390_CPU_GET_CLASS(cpu);
     CPUS390XState *env = &cpu->env;
 
-    env->pfault_token = -1UL;
-    env->bpbc = false;
     scc->parent_reset(s);
     cpu->env.sigp_order = 0;
     s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu);
-}

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 104

Quoted 10 lines
target/s390x/cpu.c +4 -5
 
-/* S390CPUClass::initial_reset() */
-static void s390_cpu_initial_reset(CPUState *s)
-{
-    S390CPU *cpu = S390_CPU(s);
-    CPUS390XState *env = &cpu->env;
+    /* Set initial values after clearing */
+    switch (type) {
+    case S390_CPU_RESET_CLEAR:
+        /* Fallthrough will clear the rest */

I think you could drop the above comment, since /* Fallthrough */ two lines later should be enough.

Quoted 37 lines
target/s390x/cpu.c +18 -18
+        memset(env, 0, offsetof(CPUS390XState, start_initial_reset_fields));
+        /* Fallthrough */
+    case S390_CPU_RESET_INITIAL:
+        memset(&env->start_initial_reset_fields, 0,
+               offsetof(CPUS390XState, end_reset_fields) -
+               offsetof(CPUS390XState, start_initial_reset_fields));
+        /* architectured initial values for CR 0 and 14 */
+        env->cregs[0] = CR0_RESET;
+        env->cregs[14] = CR14_RESET;
 
-    s390_cpu_reset(s);
-    /* initial reset does not clear everything! */
-    memset(&env->start_initial_reset_fields, 0,
-        offsetof(CPUS390XState, end_reset_fields) -
-        offsetof(CPUS390XState, start_initial_reset_fields));
-
-    /* architectured initial values for CR 0 and 14 */
-    env->cregs[0] = CR0_RESET;
-    env->cregs[14] = CR14_RESET;
-
-    /* architectured initial value for Breaking-Event-Address register */
-    env->gbea = 1;
-
-    env->pfault_token = -1UL;
-
-    /* tininess for underflow is detected before rounding */
-    set_float_detect_tininess(float_tininess_before_rounding,
-                              &env->fpu_status);
+        /* architectured initial value for Breaking-Event-Address register */
+        env->gbea = 1;
+        /* tininess for underflow is detected before rounding */
+        set_float_detect_tininess(float_tininess_before_rounding,
+                                  &env->fpu_status);
+        /* Fallthrough */
+    case S390_CPU_RESET_NORMAL:
+        env->pfault_token = -1UL;
+        env->bpbc = false;

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 131

Quoted 10 lines
target/s390x/cpu.c +5 -2
+        break;
+    }
 
     /* Reset state inside the kernel that we cannot access yet from QEMU. */
-    if (kvm_enabled()) {
-        kvm_s390_reset_vcpu(cpu);
+    if (kvm_enabled() && (type == S390_CPU_RESET_CLEAR ||
+                          type == S390_CPU_RESET_INITIAL)) {
+            kvm_s390_reset_vcpu(cpu);
     }

Why don't you simply move that into the switch-case statement, too?

[...]

Anyway, re-using code is of course a good idea, but I wonder whether it would be nicer to keep most things in place, and then simply chain the functions like this:

static void s390_cpu_reset_normal(CPUState *s)
{
...
}

static void s390_cpu_reset_initial(CPUState *s)

{
    ...
    s390_cpu_reset_normal(s);
    ...
}

static void s390_cpu_reset_clear(CPUState *s)

{
    ...
    s390_cpu_reset_initial()
    ...
}

Just my 0.02 €, but at least for me, that's easier to understand than a switch-case statement with fallthroughs inbetween.

Thomas

#6 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

Quoted 18 lines · Thomas Huth
Quoted 8 lines · Janosch Frank

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

---
 hw/s390x/s390-virtio-ccw.c |   3 +
 target/s390x/cpu.c         | 111 ++++++++++++++++---------------------
 2 files changed, 52 insertions(+), 62 deletions(-)

[...]

Quoted 8 lines
target/s390x/cpu.c +0 -0
index 3abe7e80fd..10d5b915d8 100644
@@ -82,67 +82,53 @@ static void s390_cpu_load_normal(CPUState *s)
 }
 #endif
 
Quoted 12 lines

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 91

Quoted 9 lines
target/s390x/cpu.c +7 -2
-/* S390CPUClass::cpu_reset() */
-static void s390_cpu_reset(CPUState *s)
+enum {
+    S390_CPU_RESET_NORMAL,
+    S390_CPU_RESET_INITIAL,
+    S390_CPU_RESET_CLEAR,
+};
+
+static void s390_cpu_reset(CPUState *s, uint8_t type)

Please give the enum a name and use that instead of uint8_t for "type". Or at least make it an "int". uint8_t is not really appropriate here.

Sure

Quoted 66 lines
Quoted 12 lines
target/s390x/cpu.c +0 -3
 {
     S390CPU *cpu = S390_CPU(s);
     S390CPUClass *scc = S390_CPU_GET_CLASS(cpu);
     CPUS390XState *env = &cpu->env;
 
-    env->pfault_token = -1UL;
-    env->bpbc = false;
     scc->parent_reset(s);
     cpu->env.sigp_order = 0;
     s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu);
-}

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 104

Quoted 10 lines
target/s390x/cpu.c +4 -5
 
-/* S390CPUClass::initial_reset() */
-static void s390_cpu_initial_reset(CPUState *s)
-{
-    S390CPU *cpu = S390_CPU(s);
-    CPUS390XState *env = &cpu->env;
+    /* Set initial values after clearing */
+    switch (type) {
+    case S390_CPU_RESET_CLEAR:
+        /* Fallthrough will clear the rest */

I think you could drop the above comment, since /* Fallthrough */ two lines later should be enough.

Quoted 40 lines
target/s390x/cpu.c +20 -18
+        memset(env, 0, offsetof(CPUS390XState, start_initial_reset_fields));
+        /* Fallthrough */
+    case S390_CPU_RESET_INITIAL:
+        memset(&env->start_initial_reset_fields, 0,
+               offsetof(CPUS390XState, end_reset_fields) -
+               offsetof(CPUS390XState, start_initial_reset_fields));
+        /* architectured initial values for CR 0 and 14 */
+        env->cregs[0] = CR0_RESET;
+        env->cregs[14] = CR14_RESET;
 
-    s390_cpu_reset(s);
-    /* initial reset does not clear everything! */
-    memset(&env->start_initial_reset_fields, 0,
-        offsetof(CPUS390XState, end_reset_fields) -
-        offsetof(CPUS390XState, start_initial_reset_fields));
-
-    /* architectured initial values for CR 0 and 14 */
-    env->cregs[0] = CR0_RESET;
-    env->cregs[14] = CR14_RESET;
-
-    /* architectured initial value for Breaking-Event-Address register */
-    env->gbea = 1;
-
-    env->pfault_token = -1UL;
-
-    /* tininess for underflow is detected before rounding */
-    set_float_detect_tininess(float_tininess_before_rounding,
-                              &env->fpu_status);
+        /* architectured initial value for Breaking-Event-Address register */
+        env->gbea = 1;
+        /* tininess for underflow is detected before rounding */
+        set_float_detect_tininess(float_tininess_before_rounding,
+                                  &env->fpu_status);
+        /* Fallthrough */
+    case S390_CPU_RESET_NORMAL:
+        env->pfault_token = -1UL;
+        env->bpbc = false;
+        break;
+    }
 
Quoted 9 lines

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 131

Quoted 7 lines
target/s390x/cpu.c +3 -2
     /* Reset state inside the kernel that we cannot access yet from QEMU. */
-    if (kvm_enabled()) {
-        kvm_s390_reset_vcpu(cpu);
+    if (kvm_enabled() && (type == S390_CPU_RESET_CLEAR ||
+                          type == S390_CPU_RESET_INITIAL)) {
+            kvm_s390_reset_vcpu(cpu);
     }

Why don't you simply move that into the switch-case statement, too?

There was a reason for that, time to load it from cold storage...

Quoted 6 lines

[...]

Anyway, re-using code is of course a good idea, but I wonder whether it would be nicer to keep most things in place, and then simply chain the functions like this:

I tried that and I prefer the version in the patch.

Quoted 25 lines

static void s390_cpu_reset_normal(CPUState *s)

{
   ...
}

static void s390_cpu_reset_initial(CPUState *s)

{
    ...
    s390_cpu_reset_normal(s);
    ...
}

static void s390_cpu_reset_clear(CPUState *s)

{
    ...
    s390_cpu_reset_initial()
    ...
}

Just my 0.02 €, but at least for me, that's easier to understand than a switch-case statement with fallthroughs inbetween.

Thomas

#7 lore raw

Re: [PATCH 01/15] s390x: Cleanup cpu resets

Quoted 4 lines · Janosch Frank
Quoted 3 lines · Thomas Huth
Quoted 2 lines · Janosch Frank

Let's move the resets into one function and switch by type, so we can use fallthroughs for shared reset actions.

[...]

Quoted 50 lines
Quoted 40 lines
Quoted 40 lines
target/s390x/cpu.c +20 -18
+        memset(env, 0, offsetof(CPUS390XState, start_initial_reset_fields));
+        /* Fallthrough */
+    case S390_CPU_RESET_INITIAL:
+        memset(&env->start_initial_reset_fields, 0,
+               offsetof(CPUS390XState, end_reset_fields) -
+               offsetof(CPUS390XState, start_initial_reset_fields));
+        /* architectured initial values for CR 0 and 14 */
+        env->cregs[0] = CR0_RESET;
+        env->cregs[14] = CR14_RESET;
 
-    s390_cpu_reset(s);
-    /* initial reset does not clear everything! */
-    memset(&env->start_initial_reset_fields, 0,
-        offsetof(CPUS390XState, end_reset_fields) -
-        offsetof(CPUS390XState, start_initial_reset_fields));
-
-    /* architectured initial values for CR 0 and 14 */
-    env->cregs[0] = CR0_RESET;
-    env->cregs[14] = CR14_RESET;
-
-    /* architectured initial value for Breaking-Event-Address register */
-    env->gbea = 1;
-
-    env->pfault_token = -1UL;
-
-    /* tininess for underflow is detected before rounding */
-    set_float_detect_tininess(float_tininess_before_rounding,
-                              &env->fpu_status);
+        /* architectured initial value for Breaking-Event-Address register */
+        env->gbea = 1;
+        /* tininess for underflow is detected before rounding */
+        set_float_detect_tininess(float_tininess_before_rounding,
+                                  &env->fpu_status);
+        /* Fallthrough */
+    case S390_CPU_RESET_NORMAL:
+        env->pfault_token = -1UL;
+        env->bpbc = false;
+        break;
+    }
Quoted 9 lines

target/s390x/cpu.c static void s390_cpu_load_normal(CPUState *s) · line 131

Quoted 7 lines
     /* Reset state inside the kernel that we cannot access yet from QEMU. */
target/s390x/cpu.c +3 -2
-    if (kvm_enabled()) {
-        kvm_s390_reset_vcpu(cpu);
+    if (kvm_enabled() && (type == S390_CPU_RESET_CLEAR ||
+                          type == S390_CPU_RESET_INITIAL)) {
+            kvm_s390_reset_vcpu(cpu);
     }

Why don't you simply move that into the switch-case statement, too?

Quoted 1 line

There was a reason for that, time to load it from cold storage...

I just noticed that you rework this in patch 10/15, so it indeed makes sense to keep it outside of the switch-case-statement above...

Quoted 7 lines
Quoted 5 lines

[...]

Anyway, re-using code is of course a good idea, but I wonder whether it would be nicer to keep most things in place, and then simply chain the functions like this:

I tried that and I prefer the version in the patch.

... and with patch 10 in mind, it indeed also makes more sense to *not* use the chaining that I've suggested. So never mind, your switch with the fallthrough statements is just fine.

Thomas

#8 lore raw

PATCH 2/15 s390x: Beautify diag308 handling

In reply to Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
 target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +32 -22
index 53c2f81f2a..067c667ba7 100644
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
+                              uintptr_t ra, bool write)
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;
+    }
+    return 0;
+}
+
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         return;
     }
 
-    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
     }
 
     switch (subcode) {
-    case 0:
+    case DIAG308_RES_MOD_CLR:
         s390_ipl_reset_request(cs, S390_RESET_MODIFIED_CLEAR);
         break;
-    case 1:
+    case DIAG308_RES_LOAD_NORM:
         s390_ipl_reset_request(cs, S390_RESET_LOAD_NORMAL);
         break;
-    case 3:
+    case DIAG308_LOAD_CLEAR:
+        /* Well we still lack the clearing bit... */
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
-    case 5:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), false,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_SET:
+        if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
         iplb = g_new0(IplParameterBlock, 1);
@@ -110,15 +127,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 out:
         g_free(iplb);
         return;
-    case 6:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), true,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_STORE:
+        if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
         iplb = s390_ipl_get_iplb();
Signature · 1 line
2.20.1
Reviewed-by on v1 2/15
#9 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

Quoted 15 lines · On Wed, 20 Nov 2019 06:43:21 -0500 Janosch Frank <[email protected]> wrote:

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
 target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +0 -0
index 53c2f81f2a..067c667ba7 100644

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 57

Quoted 5 lines
target/s390x/diag.c +2 -0
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1

Maybe s/RES/RESET/ ? Not that much longer, but clearer IMHO.

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 61

Quoted 4 lines
target/s390x/diag.c +4 -0
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6

(...)

Really more readable afterwards.

#10 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

Quoted 21 lines · Cornelia Huck
Quoted 18 lines · On Wed, 20 Nov 2019 06:43:21 -0500 Janosch Frank <[email protected]> wrote:

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
 target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +0 -0
index 53c2f81f2a..067c667ba7 100644
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
 
Quoted 4 lines

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 57

Quoted 2 lines
target/s390x/diag.c +2 -0
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1

Maybe s/RES/RESET/ ? Not that much longer, but clearer IMHO.

Sure

Quoted 11 lines

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 61

Quoted 5 lines
target/s390x/diag.c +4 -0
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6

(...)

Really more readable afterwards.

Thanks!

Reviewed-by on v1 2/15
#11 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

In reply to Janosch Frank
Quoted 45 lines · Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
  target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
  1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +23 -0
index 53c2f81f2a..067c667ba7 100644
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
  #define DIAG_308_RC_NO_CONF         0x0102
  #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
+                              uintptr_t ra, bool write)
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;
+    }
+    return 0;
+}
+
  void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
  {
      CPUState *cs = env_cpu(env);

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 91

Quoted 5 lines
target/s390x/diag.c +1 -1
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
          return;
      }
 
-    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {

Strange, the default case in the switch was basically dead code.

Quoted 41 lines
target/s390x/diag.c +6 -16
          s390_program_interrupt(env, PGM_SPECIFICATION, ra);
          return;
      }
 
      switch (subcode) {
-    case 0:
+    case DIAG308_RES_MOD_CLR:
          s390_ipl_reset_request(cs, S390_RESET_MODIFIED_CLEAR);
          break;
-    case 1:
+    case DIAG308_RES_LOAD_NORM:
          s390_ipl_reset_request(cs, S390_RESET_LOAD_NORMAL);
          break;
-    case 3:
+    case DIAG308_LOAD_CLEAR:
+        /* Well we still lack the clearing bit... */
          s390_ipl_reset_request(cs, S390_RESET_REIPL);
          break;
-    case 5:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), false,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_SET:
+        if (diag308_parm_check(env, r1, addr, ra, false)) {
              return;
          }
          iplb = g_new0(IplParameterBlock, 1);
@@ -110,15 +127,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
  out:
          g_free(iplb);
          return;
-    case 6:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 134

Quoted 10 lines
target/s390x/diag.c +2 -5
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), true,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_STORE:
+        if (diag308_parm_check(env, r1, addr, ra, true)) {
              return;
          }
          iplb = s390_ipl_get_iplb();
Signature · 3 lines
Thanks,

David / dhildenb
#12 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

Quoted 99 lines · David Hildenbrand
Quoted 45 lines · Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
  target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
  1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +23 -0
index 53c2f81f2a..067c667ba7 100644
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
  #define DIAG_308_RC_NO_CONF         0x0102
  #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
+                              uintptr_t ra, bool write)
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;
+    }
+    return 0;
+}
+
  void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
  {
      CPUState *cs = env_cpu(env);

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 91

Quoted 5 lines
target/s390x/diag.c +1 -1
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
          return;
      }
 
-    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {

Strange, the default case in the switch was basically dead code.

Quoted 45 lines
target/s390x/diag.c +6 -21
          s390_program_interrupt(env, PGM_SPECIFICATION, ra);
          return;
      }
 
      switch (subcode) {
-    case 0:
+    case DIAG308_RES_MOD_CLR:
          s390_ipl_reset_request(cs, S390_RESET_MODIFIED_CLEAR);
          break;
-    case 1:
+    case DIAG308_RES_LOAD_NORM:
          s390_ipl_reset_request(cs, S390_RESET_LOAD_NORMAL);
          break;
-    case 3:
+    case DIAG308_LOAD_CLEAR:
+        /* Well we still lack the clearing bit... */
          s390_ipl_reset_request(cs, S390_RESET_REIPL);
          break;
-    case 5:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), false,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_SET:
+        if (diag308_parm_check(env, r1, addr, ra, false)) {
              return;
          }
          iplb = g_new0(IplParameterBlock, 1);
@@ -110,15 +127,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
  out:
          g_free(iplb);
          return;
-    case 6:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), true,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
Quoted 8 lines

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 134

Quoted 6 lines
target/s390x/diag.c +2 -0
+    case DIAG308_STORE:
+        if (diag308_parm_check(env, r1, addr, ra, true)) {
              return;
          }
          iplb = s390_ipl_get_iplb();

Thanks!

#13 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

In reply to David Hildenbrand
Quoted 47 lines · David Hildenbrand
Quoted 46 lines · Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
  target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------

  1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +0 -0
index 53c2f81f2a..067c667ba7 100644
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t
 r1, uint64_t r3)
   #define DIAG_308_RC_NO_CONF         0x0102
   #define DIAG_308_RC_INVALID         0x0402

  +#define DIAG308_RES_MOD_CLR        0

target/s390x/diag.c +7 -0
+#define DIAG308_RES_LOAD_NORM        1
+#define DIAG308_LOAD_CLEAR        3
+#define DIAG308_LOAD_NORMAL_DUMP    4
+#define DIAG308_SET            5
+#define DIAG308_STORE            6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1,

uint64_t addr,
+                              uintptr_t ra, bool write)

target/s390x/diag.c +14 -0
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;
+    }
+    return 0;
+}
+

  void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
  {
      CPUState *cs = env_cpu(env);

Quoted 8 lines

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t · line 92

Quoted 6 lines
target/s390x/diag.c +1 -0
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t
 r1, uint64_t r3, uintptr_t ra)
           return;
       }
   -    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {

Strange, the default case in the switch was basically dead code.

Not really, case 2 and case 4 are not handled there. We even had a funny bug some months ago, where a guest could terminate QEMU with one of these codes, see commit 37dbd1f4d4805edcd18d94eb202bb3461b3cd52d.

Thomas

Reviewed-by on v1 2/15
#14 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

In reply to Janosch Frank
Quoted 15 lines · Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
 target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +0 -0
index 53c2f81f2a..067c667ba7 100644

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 57

Quoted 5 lines
target/s390x/diag.c +2 -0
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1

I think I'd also prefer RESET instead of RES here.

Quoted 7 lines
target/s390x/diag.c +7 -0
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
+                              uintptr_t ra, bool write)

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 74

Quoted 10 lines
target/s390x/diag.c +10 -0
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;

or maybe -EFAULT ? ;-)

Quoted 54 lines
target/s390x/diag.c +11 -17
+    }
+    return 0;
+}
+
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         return;
     }
 
-    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
     }
 
     switch (subcode) {
-    case 0:
+    case DIAG308_RES_MOD_CLR:
         s390_ipl_reset_request(cs, S390_RESET_MODIFIED_CLEAR);
         break;
-    case 1:
+    case DIAG308_RES_LOAD_NORM:
         s390_ipl_reset_request(cs, S390_RESET_LOAD_NORMAL);
         break;
-    case 3:
+    case DIAG308_LOAD_CLEAR:
+        /* Well we still lack the clearing bit... */
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
-    case 5:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), false,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_SET:
+        if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
         iplb = g_new0(IplParameterBlock, 1);
@@ -110,15 +127,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 out:
         g_free(iplb);
         return;
-    case 6:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 134

Quoted 10 lines
target/s390x/diag.c +2 -5
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), true,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_STORE:
+        if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
         iplb = s390_ipl_get_iplb();

With RESET instead of RES:

#15 lore raw

Re: [PATCH 02/15] s390x: Beautify diag308 handling

Quoted 35 lines · Thomas Huth
Quoted 15 lines · Janosch Frank

Let's improve readability by:
* Using constants for the subcodes
* Moving parameter checking into a function
* Removing subcode > 6 check as the default case catches that

---
 target/s390x/diag.c | 54 +++++++++++++++++++++++++++------------------
 1 file changed, 32 insertions(+), 22 deletions(-)
target/s390x/diag.c +0 -0
index 53c2f81f2a..067c667ba7 100644

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 57

Quoted 5 lines
target/s390x/diag.c +2 -0
@@ -53,6 +53,29 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
 
+#define DIAG308_RES_MOD_CLR		0
+#define DIAG308_RES_LOAD_NORM		1

I think I'd also prefer RESET instead of RES here.

Quoted 11 lines
target/s390x/diag.c +11 -0
+#define DIAG308_LOAD_CLEAR		3
+#define DIAG308_LOAD_NORMAL_DUMP	4
+#define DIAG308_SET			5
+#define DIAG308_STORE			6
+
+static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
+                              uintptr_t ra, bool write)
+{
+    if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
+        s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+        return -EINVAL;
Quoted 8 lines

target/s390x/diag.c int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3) · line 74

Quoted 6 lines
target/s390x/diag.c +6 -0
+    }
+    if (!address_space_access_valid(&address_space_memory, addr,
+                                    sizeof(IplParameterBlock), write,
+                                    MEMTXATTRS_UNSPECIFIED)) {
+        s390_program_interrupt(env, PGM_ADDRESSING, ra);
+        return -EINVAL;

or maybe -EFAULT ? ;-)

Honestly, I'm asking myself why I'm even bothering to return specific error codes when all I check is rc != 0. So I guess I just make it -1 for all errors.

Quoted 70 lines
Quoted 55 lines
target/s390x/diag.c +11 -17
+    }
+    return 0;
+}
+
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
@@ -65,30 +88,24 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         return;
     }
 
-    if ((subcode & ~0x0ffffULL) || (subcode > 6)) {
+    if (subcode & ~0x0ffffULL) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
     }
 
     switch (subcode) {
-    case 0:
+    case DIAG308_RES_MOD_CLR:
         s390_ipl_reset_request(cs, S390_RESET_MODIFIED_CLEAR);
         break;
-    case 1:
+    case DIAG308_RES_LOAD_NORM:
         s390_ipl_reset_request(cs, S390_RESET_LOAD_NORMAL);
         break;
-    case 3:
+    case DIAG308_LOAD_CLEAR:
+        /* Well we still lack the clearing bit... */
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
-    case 5:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), false,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_SET:
+        if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
         iplb = g_new0(IplParameterBlock, 1);
@@ -110,15 +127,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 out:
         g_free(iplb);
         return;
-    case 6:
-        if ((r1 & 1) || (addr & 0x0fffULL)) {
-            s390_program_interrupt(env, PGM_SPECIFICATION, ra);
-            return;

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 134

Quoted 10 lines
target/s390x/diag.c +2 -5
-        }
-        if (!address_space_access_valid(&address_space_memory, addr,
-                                        sizeof(IplParameterBlock), true,
-                                        MEMTXATTRS_UNSPECIFIED)) {
-            s390_program_interrupt(env, PGM_ADDRESSING, ra);
+    case DIAG308_STORE:
+        if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
         iplb = s390_ipl_get_iplb();

With RESET instead of RES:

#16 lore raw

PATCH 3/15 s390x: protvirt: Add diag308 subcodes 8 - 10

In reply to Janosch Frank

For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.

Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.

Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.

Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.

---
 hw/s390x/ipl.c      | 48 ++++++++++++++++++++++++++++++++++++++++++---
 hw/s390x/ipl.h      | 33 +++++++++++++++++++++++++++++++
 target/s390x/diag.c | 26 ++++++++++++++++++++++--
 3 files changed, 102 insertions(+), 5 deletions(-)
hw/s390x/ipl.c +45 -3
index ca544d64c5..a077926f36 100644
@@ -529,15 +529,56 @@ static bool is_virtio_scsi_device(IplParameterBlock *iplb)
     return is_virtio_ccw_device_of_type(iplb, VIRTIO_ID_SCSI);
 }
 
+int s390_ipl_pv_check_comp(IplParameterBlock *iplb)
+{
+    int i;
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    if (ipib_pv->num_comp == 0) {
+        return -EINVAL;
+    }
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+
+        /* Addr must be 4k aligned */
+        if (ipib_pv->components[i].addr & ~TARGET_PAGE_MASK) {
+            return -EINVAL;
+        }
+
+        /* Tweak prefix is monotonously increasing with each component */
+        if (i < ipib_pv->num_comp - 1 &&
+            ipib_pv->components[i].tweak_pref >
+            ipib_pv->components[i + 1].tweak_pref) {
+            return -EINVAL;
+        }
+    }
+    return 1;
+}
+
 void s390_ipl_update_diag308(IplParameterBlock *iplb)
 {
     S390IPLState *ipl = get_ipl_device();
 
-    ipl->iplb = *iplb;
-    ipl->iplb_valid = true;
+    if (iplb->pbt == 5) {
+        ipl->iplb_pbt5 = *iplb;
+        ipl->iplb_valid_pbt5 = true;
+    } else {
+        ipl->iplb = *iplb;
+        ipl->iplb_valid = true;
+    }
     ipl->netboot = is_virtio_net_device(iplb);
 }
 
+IplParameterBlock *s390_ipl_get_iplb_secure(void)
+{
+    S390IPLState *ipl = get_ipl_device();
+
+    if (!ipl->iplb_valid_pbt5) {
+        return NULL;
+    }
+    return &ipl->iplb_pbt5;
+}
+
 IplParameterBlock *s390_ipl_get_iplb(void)
 {
     S390IPLState *ipl = get_ipl_device();
@@ -552,7 +593,8 @@ void s390_ipl_reset_request(CPUState *cs, enum s390_reset reset_type)
 {
     S390IPLState *ipl = get_ipl_device();
 
-    if (reset_type == S390_RESET_EXTERNAL || reset_type == S390_RESET_REIPL) {
+    if (reset_type == S390_RESET_EXTERNAL || reset_type == S390_RESET_REIPL ||
+        reset_type == S390_RESET_PV) {
         /* use CPU 0 for full resets */
         ipl->reset_cpu_index = 0;
     } else {
hw/s390x/ipl.h +33 -0
index d4813105db..7b8a493509 100644
@@ -15,6 +15,24 @@
 #include "cpu.h"
 #include "hw/qdev-core.h"
 
+struct IPLBlockPVComp {
+    uint64_t tweak_pref;
+    uint64_t addr;
+    uint64_t size;
+} QEMU_PACKED;
+typedef struct IPLBlockPVComp IPLBlockPVComp;
+
+struct IPLBlockPV {
+    uint8_t  reserved[84];
+    uint8_t  reserved67[3];
+    uint8_t  version;
+    uint32_t num_comp;
+    uint64_t pv_header_addr;
+    uint64_t pv_header_len;
+    struct IPLBlockPVComp components[];
+} QEMU_PACKED;
+typedef struct IPLBlockPV IPLBlockPV;
+
 struct IplBlockCcw {
     uint8_t  reserved0[85];
     uint8_t  ssid;
@@ -71,6 +89,7 @@ union IplParameterBlock {
         union {
             IplBlockCcw ccw;
             IplBlockFcp fcp;
+            IPLBlockPV pv;
             IplBlockQemuScsi scsi;
         };
     } QEMU_PACKED;
@@ -84,9 +103,11 @@ union IplParameterBlock {
 typedef union IplParameterBlock IplParameterBlock;
 
 int s390_ipl_set_loadparm(uint8_t *loadparm);
+int s390_ipl_pv_check_comp(IplParameterBlock *iplb);
 void s390_ipl_update_diag308(IplParameterBlock *iplb);
 void s390_ipl_prepare_cpu(S390CPU *cpu);
 IplParameterBlock *s390_ipl_get_iplb(void);
+IplParameterBlock *s390_ipl_get_iplb_secure(void);
 
 enum s390_reset {
     /* default is a reset not triggered by a CPU e.g. issued by QMP */
@@ -94,6 +115,7 @@ enum s390_reset {
     S390_RESET_REIPL,
     S390_RESET_MODIFIED_CLEAR,
     S390_RESET_LOAD_NORMAL,
+    S390_RESET_PV,
 };
 void s390_ipl_reset_request(CPUState *cs, enum s390_reset reset_type);
 void s390_ipl_get_reset_request(CPUState **cs, enum s390_reset *reset_type);
@@ -133,6 +155,7 @@ struct S390IPLState {
     /*< private >*/
     DeviceState parent_obj;
     IplParameterBlock iplb;
+    IplParameterBlock iplb_pbt5;
     QemuIplParameters qipl;
     uint64_t start_addr;
     uint64_t compat_start_addr;
@@ -140,6 +163,7 @@ struct S390IPLState {
     uint64_t compat_bios_start_addr;
     bool enforce_bios;
     bool iplb_valid;
+    bool iplb_valid_pbt5;
     bool netboot;
     /* reset related properties don't have to be migrated or reset */
     enum s390_reset reset_type;
@@ -161,9 +185,11 @@ QEMU_BUILD_BUG_MSG(offsetof(S390IPLState, iplb) & 3, "alignment of iplb wrong");
 
 #define S390_IPL_TYPE_FCP 0x00
 #define S390_IPL_TYPE_CCW 0x02
+#define S390_IPL_TYPE_PV 0x05
 #define S390_IPL_TYPE_QEMU_SCSI 0xff
 
 #define S390_IPLB_HEADER_LEN 8
+#define S390_IPLB_MIN_PV_LEN 148
 #define S390_IPLB_MIN_CCW_LEN 200
 #define S390_IPLB_MIN_FCP_LEN 384
 #define S390_IPLB_MIN_QEMU_SCSI_LEN 200
@@ -185,4 +211,11 @@ static inline bool iplb_valid_fcp(IplParameterBlock *iplb)
            iplb->pbt == S390_IPL_TYPE_FCP;
 }
 
+static inline bool iplb_valid_se(IplParameterBlock *iplb)
+{
+    return be32_to_cpu(iplb->len) >= S390_IPLB_MIN_PV_LEN &&
+           iplb->pbt == S390_IPL_TYPE_PV;
+}
+
+
 #endif
target/s390x/diag.c +24 -2
index 067c667ba7..32049bb4ee 100644
@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_OK              0x0001
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
+#define DIAG_308_RC_NO_PV_CONF      0x0a02
+#define DIAG_308_RC_INV_FOR_PV      0x0b02
 
 #define DIAG308_RES_MOD_CLR		0
 #define DIAG308_RES_LOAD_NORM		1
@@ -59,6 +61,9 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG308_LOAD_NORMAL_DUMP	4
 #define DIAG308_SET			5
 #define DIAG308_STORE			6
+#define DIAG308_PV_SET			8
+#define DIAG308_PV_STORE		9
+#define DIAG308_PV_START		10
 
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
@@ -105,6 +110,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
     case DIAG308_SET:
+    case DIAG308_PV_SET: /* Set SE parms */
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
@@ -117,7 +123,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 
         cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
 
-        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb)) {
+        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
+            !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
@@ -128,10 +135,15 @@ out:
         g_free(iplb);
         return;
     case DIAG308_STORE:
+    case DIAG308_PV_STORE: /* Get SE parms */
         if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
-        iplb = s390_ipl_get_iplb();
+        if (subcode == DIAG308_PV_STORE) {
+            iplb = s390_ipl_get_iplb_secure();
+        } else {
+            iplb = s390_ipl_get_iplb();
+        }
         if (iplb) {
             cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
             env->regs[r1 + 1] = DIAG_308_RC_OK;
@@ -139,6 +151,16 @@ out:
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;
         }
         return;
+        break;
+    case DIAG308_PV_START: /* SE start */
+        iplb = s390_ipl_get_iplb_secure();
+        if (!iplb_valid_se(iplb)) {
+            env->regs[r1 + 1] = DIAG_308_RC_NO_PV_CONF;
+            return;
+        }
+
+        s390_ipl_reset_request(cs, S390_RESET_PV);
+        break;
     default:
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         break;
Signature · 1 line
2.20.1
#17 lore raw

Re: [PATCH 03/15] s390x: protvirt: Add diag308 subcodes 8 - 10

Quoted 27 lines · On Wed, 20 Nov 2019 06:43:22 -0500 Janosch Frank <[email protected]> wrote:

For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.

Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.

Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.

Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.

---
 hw/s390x/ipl.c      | 48 ++++++++++++++++++++++++++++++++++++++++++---
 hw/s390x/ipl.h      | 33 +++++++++++++++++++++++++++++++
 target/s390x/diag.c | 26 ++++++++++++++++++++++--
 3 files changed, 102 insertions(+), 5 deletions(-)
hw/s390x/ipl.c +0 -0
index ca544d64c5..a077926f36 100644

hw/s390x/ipl.c static bool is_virtio_scsi_device(IplParameterBlock *iplb) · line 532

Quoted 4 lines
hw/s390x/ipl.c +1 -0
@@ -529,15 +529,56 @@ static bool is_virtio_scsi_device(IplParameterBlock *iplb)
     return is_virtio_ccw_device_of_type(iplb, VIRTIO_ID_SCSI);
 }
 
+int s390_ipl_pv_check_comp(IplParameterBlock *iplb)

s390_ipl_pv_check_components() ?

Quoted 25 lines
hw/s390x/ipl.c +25 -0
+{
+    int i;
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    if (ipib_pv->num_comp == 0) {
+        return -EINVAL;
+    }
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+
+        /* Addr must be 4k aligned */
+        if (ipib_pv->components[i].addr & ~TARGET_PAGE_MASK) {
+            return -EINVAL;
+        }
+
+        /* Tweak prefix is monotonously increasing with each component */
+        if (i < ipib_pv->num_comp - 1 &&
+            ipib_pv->components[i].tweak_pref >
+            ipib_pv->components[i + 1].tweak_pref) {
+            return -EINVAL;
+        }
+    }
+    return 1;
+}
+

(...)

hw/s390x/ipl.c static inline bool iplb_valid_fcp(IplParameterBlock *iplb) · line 214

Quoted 5 lines
hw/s390x/ipl.c +1 -0
@@ -185,4 +211,11 @@ static inline bool iplb_valid_fcp(IplParameterBlock *iplb)
            iplb->pbt == S390_IPL_TYPE_FCP;
 }
 
+static inline bool iplb_valid_se(IplParameterBlock *iplb)

iplb_valid_pv() ?

Quoted 7 lines
hw/s390x/ipl.c +6 -0
+{
+    return be32_to_cpu(iplb->len) >= S390_IPLB_MIN_PV_LEN &&
+           iplb->pbt == S390_IPL_TYPE_PV;
+}
+
+
 #endif

(...)

hw/s390x/ipl.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 113

Quoted 5 lines
hw/s390x/ipl.c +1 -0
@@ -105,6 +110,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
     case DIAG308_SET:
+    case DIAG308_PV_SET: /* Set SE parms */

PV parms?

('SE' makes me think of 'service element' :)

Quoted 14 lines
hw/s390x/ipl.c +2 -1
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
@@ -117,7 +123,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 
         cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
 
-        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb)) {
+        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
+            !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }

hw/s390x/ipl.c out: · line 138

Quoted 4 lines
hw/s390x/ipl.c +1 -0
@@ -128,10 +135,15 @@ out:
         g_free(iplb);
         return;
     case DIAG308_STORE:
+    case DIAG308_PV_STORE: /* Get SE parms */

Same here.

Quoted 13 lines
hw/s390x/ipl.c +5 -1
         if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
-        iplb = s390_ipl_get_iplb();
+        if (subcode == DIAG308_PV_STORE) {
+            iplb = s390_ipl_get_iplb_secure();
+        } else {
+            iplb = s390_ipl_get_iplb();
+        }
         if (iplb) {
             cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
             env->regs[r1 + 1] = DIAG_308_RC_OK;

hw/s390x/ipl.c out: · line 155

Quoted 5 lines
hw/s390x/ipl.c +2 -0
@@ -139,6 +151,16 @@ out:
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;
         }
         return;
+        break;
+    case DIAG308_PV_START: /* SE start */

Also here :)

hw/s390x/ipl.c out: · line 166

Quoted 11 lines
hw/s390x/ipl.c +8 -0
+        iplb = s390_ipl_get_iplb_secure();
+        if (!iplb_valid_se(iplb)) {
+            env->regs[r1 + 1] = DIAG_308_RC_NO_PV_CONF;
+            return;
+        }
+
+        s390_ipl_reset_request(cs, S390_RESET_PV);
+        break;
     default:
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         break;

Did not spot anything else.

#18 lore raw

Re: [PATCH 03/15] s390x: protvirt: Add diag308 subcodes 8 - 10

In reply to Janosch Frank
Quoted 17 lines · Janosch Frank

For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.

Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.

Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.

Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.

---

[...]

Quoted 8 lines
hw/s390x/ipl.h +0 -0
index d4813105db..7b8a493509 100644
@@ -15,6 +15,24 @@
 #include "cpu.h"
 #include "hw/qdev-core.h"
 

hw/s390x/ipl.h line 27

Quoted 10 lines
hw/s390x/ipl.h +10 -0
+struct IPLBlockPVComp {
+    uint64_t tweak_pref;
+    uint64_t addr;
+    uint64_t size;
+} QEMU_PACKED;
+typedef struct IPLBlockPVComp IPLBlockPVComp;
+
+struct IPLBlockPV {
+    uint8_t  reserved[84];
+    uint8_t  reserved67[3];

What does the "67" mean here?

hw/s390x/ipl.h line 34

Quoted 7 lines
hw/s390x/ipl.h +7 -0
+    uint8_t  version;
+    uint32_t num_comp;
+    uint64_t pv_header_addr;
+    uint64_t pv_header_len;
+    struct IPLBlockPVComp components[];
+} QEMU_PACKED;
+typedef struct IPLBlockPV IPLBlockPV;

Given the fact that we had quite some headaches with QEMU_PACKED structs in the past already, and the structs seem to be naturally aligned ... what about dropping the QEMU_PACKED here and using QEMU_BUILD_BUG() to assert that the struct has the correct size?

[...]

hw/s390x/ipl.h static inline bool iplb_valid_fcp(IplParameterBlock *iplb) · line 220

Quoted 11 lines
hw/s390x/ipl.h +7 -0
@@ -185,4 +211,11 @@ static inline bool iplb_valid_fcp(IplParameterBlock *iplb)
            iplb->pbt == S390_IPL_TYPE_FCP;
 }
 
+static inline bool iplb_valid_se(IplParameterBlock *iplb)
+{
+    return be32_to_cpu(iplb->len) >= S390_IPLB_MIN_PV_LEN &&
+           iplb->pbt == S390_IPL_TYPE_PV;
+}
+
+

Drop one empty line?

Quoted 60 lines
hw/s390x/ipl.h +0 -0
 #endif
target/s390x/diag.c +14 -2
index 067c667ba7..32049bb4ee 100644
@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_OK              0x0001
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
+#define DIAG_308_RC_NO_PV_CONF      0x0a02
+#define DIAG_308_RC_INV_FOR_PV      0x0b02
 
 #define DIAG308_RES_MOD_CLR		0
 #define DIAG308_RES_LOAD_NORM		1
@@ -59,6 +61,9 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG308_LOAD_NORMAL_DUMP	4
 #define DIAG308_SET			5
 #define DIAG308_STORE			6
+#define DIAG308_PV_SET			8
+#define DIAG308_PV_STORE		9
+#define DIAG308_PV_START		10
 
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
@@ -105,6 +110,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
     case DIAG308_SET:
+    case DIAG308_PV_SET: /* Set SE parms */
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
@@ -117,7 +123,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 
         cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
 
-        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb)) {
+        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
+            !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
@@ -128,10 +135,15 @@ out:
         g_free(iplb);
         return;
     case DIAG308_STORE:
+    case DIAG308_PV_STORE: /* Get SE parms */
         if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
-        iplb = s390_ipl_get_iplb();
+        if (subcode == DIAG308_PV_STORE) {
+            iplb = s390_ipl_get_iplb_secure();
+        } else {
+            iplb = s390_ipl_get_iplb();
+        }
         if (iplb) {
             cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
             env->regs[r1 + 1] = DIAG_308_RC_OK;

target/s390x/diag.c out: · line 154

Quoted 4 lines
target/s390x/diag.c +1 -0
@@ -139,6 +151,16 @@ out:
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;
         }
         return;
+        break;

<hamletmode>
To return or to break, that's the question...
</hamletmode>

... please choose one of the two.

target/s390x/diag.c out: · line 166

Quoted 13 lines
target/s390x/diag.c +9 -0
+    case DIAG308_PV_START: /* SE start */
+        iplb = s390_ipl_get_iplb_secure();
+        if (!iplb_valid_se(iplb)) {
+            env->regs[r1 + 1] = DIAG_308_RC_NO_PV_CONF;
+            return;
+        }
+
+        s390_ipl_reset_request(cs, S390_RESET_PV);
+        break;
     default:
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         break;

Thomas

#19 lore raw

Re: [PATCH 03/15] s390x: protvirt: Add diag308 subcodes 8 - 10

Quoted 27 lines · Thomas Huth
Quoted 17 lines · Janosch Frank

For diag308 subcodes 8 - 10 we have a new ipib of type 5. The ipib holds the address and length of the secure execution header, as well as a list of guest components.

Each component is a block of memory, for example kernel or initrd, which needs to be decrypted by the Ultravisor in order to run a protected VM. The secure execution header instructs the Ultravisor on how to handle the protected VM and its components.

Subcodes 8 and 9 are similiar to 5 and 6 and subcode 10 will finally start the protected guest.

Subcodes 8-10 are not valid in protected mode, we have to do a subcode 3 and then the 8 and 10 combination for a protected reboot.

---

[...]

Quoted 8 lines
hw/s390x/ipl.h +0 -0
index d4813105db..7b8a493509 100644
@@ -15,6 +15,24 @@
 #include "cpu.h"
 #include "hw/qdev-core.h"
 
Quoted 12 lines

hw/s390x/ipl.h line 27

Quoted 10 lines
hw/s390x/ipl.h +10 -0
+struct IPLBlockPVComp {
+    uint64_t tweak_pref;
+    uint64_t addr;
+    uint64_t size;
+} QEMU_PACKED;
+typedef struct IPLBlockPVComp IPLBlockPVComp;
+
+struct IPLBlockPV {
+    uint8_t  reserved[84];
+    uint8_t  reserved67[3];

What does the "67" mean here?

Was compacted to reserved[87]

Quoted 13 lines

hw/s390x/ipl.h line 34

Quoted 8 lines
hw/s390x/ipl.h +7 -0
+    uint8_t  version;
+    uint32_t num_comp;
+    uint64_t pv_header_addr;
+    uint64_t pv_header_len;
+    struct IPLBlockPVComp components[];
+} QEMU_PACKED;
+typedef struct IPLBlockPV IPLBlockPV;

Given the fact that we had quite some headaches with QEMU_PACKED structs in the past already, and the structs seem to be naturally aligned ... what about dropping the QEMU_PACKED here and using QEMU_BUILD_BUG() to assert that the struct has the correct size?

As the struct in which IPLBlockPV is embeded are also packed, removing the PACKED annotation results in this:

/mnt/dev/qemu/hw/s390x/ipl.c:544:27: error: taking address of packed member of ‘struct <anonymous>’ may result in an unaligned pointer value [-Werror=address-of-packed-member]

Quoted 6 lines

[...]

Quoted 4 lines
hw/s390x/ipl.h +0 -0
@@ -185,4 +211,11 @@ static inline bool iplb_valid_fcp(IplParameterBlock *iplb)
            iplb->pbt == S390_IPL_TYPE_FCP;
 }
 
Quoted 9 lines

hw/s390x/ipl.h static inline bool iplb_valid_fcp(IplParameterBlock *iplb) · line 220

Quoted 7 lines
hw/s390x/ipl.h +7 -0
+static inline bool iplb_valid_se(IplParameterBlock *iplb)
+{
+    return be32_to_cpu(iplb->len) >= S390_IPLB_MIN_PV_LEN &&
+           iplb->pbt == S390_IPL_TYPE_PV;
+}
+
+

Drop one empty line?

ack.

Quoted 89 lines
Quoted 61 lines
hw/s390x/ipl.h +0 -0
 #endif
target/s390x/diag.c +14 -2
index 067c667ba7..32049bb4ee 100644
@@ -52,6 +52,8 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG_308_RC_OK              0x0001
 #define DIAG_308_RC_NO_CONF         0x0102
 #define DIAG_308_RC_INVALID         0x0402
+#define DIAG_308_RC_NO_PV_CONF      0x0a02
+#define DIAG_308_RC_INV_FOR_PV      0x0b02
 
 #define DIAG308_RES_MOD_CLR		0
 #define DIAG308_RES_LOAD_NORM		1
@@ -59,6 +61,9 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 #define DIAG308_LOAD_NORMAL_DUMP	4
 #define DIAG308_SET			5
 #define DIAG308_STORE			6
+#define DIAG308_PV_SET			8
+#define DIAG308_PV_STORE		9
+#define DIAG308_PV_START		10
 
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
@@ -105,6 +110,7 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         s390_ipl_reset_request(cs, S390_RESET_REIPL);
         break;
     case DIAG308_SET:
+    case DIAG308_PV_SET: /* Set SE parms */
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
@@ -117,7 +123,8 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 
         cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
 
-        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb)) {
+        if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
+            !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
@@ -128,10 +135,15 @@ out:
         g_free(iplb);
         return;
     case DIAG308_STORE:
+    case DIAG308_PV_STORE: /* Get SE parms */
         if (diag308_parm_check(env, r1, addr, ra, true)) {
             return;
         }
-        iplb = s390_ipl_get_iplb();
+        if (subcode == DIAG308_PV_STORE) {
+            iplb = s390_ipl_get_iplb_secure();
+        } else {
+            iplb = s390_ipl_get_iplb();
+        }
         if (iplb) {
             cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
             env->regs[r1 + 1] = DIAG_308_RC_OK;

target/s390x/diag.c out: · line 154

Quoted 4 lines
target/s390x/diag.c +1 -0
@@ -139,6 +151,16 @@ out:
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;
         }
         return;
+        break;

<hamletmode>
To return or to break, that's the question...
</hamletmode>

... please choose one of the two.

target/s390x/diag.c out: · line 166

Quoted 13 lines
target/s390x/diag.c +9 -0
+    case DIAG308_PV_START: /* SE start */
+        iplb = s390_ipl_get_iplb_secure();
+        if (!iplb_valid_se(iplb)) {
+            env->regs[r1 + 1] = DIAG_308_RC_NO_PV_CONF;
+            return;
+        }
+
+        s390_ipl_reset_request(cs, S390_RESET_PV);
+        break;
     default:
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         break;

Thomas

#20 lore raw

PATCH 4/15 Header sync protvirt

In reply to Janosch Frank

Let's sync all the protvirt header changes

---
 linux-headers/asm-s390/kvm.h |  3 ++-
 linux-headers/linux/kvm.h    | 42 ++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
linux-headers/asm-s390/kvm.h +2 -1
index 03ab5968c7..41976d33f0 100644
@@ -255,7 +255,8 @@ struct kvm_sync_regs {
 	__u8  reserved[512];	/* for future vector expansion */
 	__u32 fpc;		/* valid on KVM_SYNC_VRS or KVM_SYNC_FPRS */
 	__u8 bpbc : 1;		/* bp mode */
-	__u8 reserved2 : 7;
+	__u8 pv : 1;		/* pv mode */
+	__u8 reserved2 : 6;
 	__u8 padding1[51];	/* riccb needs to be 64byte aligned */
 	__u8 riccb[64];		/* runtime instrumentation controls block */
 	__u8 padding2[192];	/* sdnx needs to be 256byte aligned */
linux-headers/linux/kvm.h +42 -0

54 lines not on this page · raw message

Signature · 1 line
2.20.1
#21 lore raw

Re: [PATCH 04/15] Header sync protvirt

Quoted 14 lines · On Wed, 20 Nov 2019 06:43:23 -0500 Janosch Frank <[email protected]> wrote:

Let's sync all the protvirt header changes

---
 linux-headers/asm-s390/kvm.h |  3 ++-
 linux-headers/linux/kvm.h    | 42 ++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
linux-headers/linux/kvm.h +0 -0
index 18892d6541..d031051601 100644

linux-headers/linux/kvm.h struct kvm_ppc_resize_hpt { · line 999

Quoted 5 lines
linux-headers/linux/kvm.h +2 -0
@@ -995,6 +995,8 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_SVE 170
 #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
+#define KVM_CAP_S390_PROTECTED 180
+#define KVM_CAP_S390_VCPU_RESETS 181

Hm, where does this cap come from? I did not see it in the kernel patches.

Quoted 3 lines
linux-headers/linux/kvm.h +0 -0
 #ifdef KVM_CAP_IRQ_ROUTING
 
#22 lore raw

Re: [PATCH 04/15] Header sync protvirt

Quoted 12 lines · Cornelia Huck
Quoted 8 lines · On Wed, 20 Nov 2019 06:43:23 -0500 Janosch Frank <[email protected]> wrote:

Let's sync all the protvirt header changes

---
 linux-headers/asm-s390/kvm.h |  3 ++-
 linux-headers/linux/kvm.h    | 42 ++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
Quoted 13 lines

linux-headers/linux/kvm.h struct kvm_ppc_resize_hpt { · line 999

Quoted 10 lines
linux-headers/linux/kvm.h +2 -0
index 18892d6541..d031051601 100644
@@ -995,6 +995,8 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_SVE 170
 #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
+#define KVM_CAP_S390_PROTECTED 180
+#define KVM_CAP_S390_VCPU_RESETS 181

Hm, where does this cap come from? I did not see it in the kernel patches.

It'll be part of the V1 once I send it out.
Pierre found the problem while reviewing QEMU, so I fixed it in both repos.

Quoted 5 lines
Quoted 4 lines
linux-headers/linux/kvm.h +0 -0
 #ifdef KVM_CAP_IRQ_ROUTING
 
#23 lore raw

Re: [PATCH 04/15] Header sync protvirt

Quoted 27 lines · On Thu, 21 Nov 2019 14:12:21 +0100 Janosch Frank <[email protected]> wrote:
Quoted 12 lines · Cornelia Huck
Quoted 8 lines · On Wed, 20 Nov 2019 06:43:23 -0500 Janosch Frank <[email protected]> wrote:

Let's sync all the protvirt header changes

---
 linux-headers/asm-s390/kvm.h |  3 ++-
 linux-headers/linux/kvm.h    | 42 ++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 1 deletion(-)
Quoted 13 lines

linux-headers/linux/kvm.h struct kvm_ppc_resize_hpt { · line 999

Quoted 10 lines
linux-headers/linux/kvm.h +2 -0
index 18892d6541..d031051601 100644
@@ -995,6 +995,8 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_ARM_SVE 170
 #define KVM_CAP_ARM_PTRAUTH_ADDRESS 171
 #define KVM_CAP_ARM_PTRAUTH_GENERIC 172
+#define KVM_CAP_S390_PROTECTED 180
+#define KVM_CAP_S390_VCPU_RESETS 181  

Hm, where does this cap come from? I did not see it in the kernel patches.

Quoted 2 lines

It'll be part of the V1 once I send it out.
Pierre found the problem while reviewing QEMU, so I fixed it in both repos.

Ok, that cap probably makes sense. It was just surprising to find :)

#24 lore raw

PATCH 5/15 s390x: protvirt: Sync PV state

In reply to Janosch Frank

We do not always have the SIE intercept code handy at each place where we do emulation. Unfortunately emulation for secure guests often differ slightly from normal emulation and we need to make decisions based on the protected state of the VCPU.

Let's sync the protected state and make it available.

---
 linux-headers/asm-s390/kvm.h | 1 +
 target/s390x/cpu.h           | 1 +
 target/s390x/kvm.c           | 4 ++++
 3 files changed, 6 insertions(+)
linux-headers/asm-s390/kvm.h +1 -0
index 41976d33f0..7c46cf6078 100644
@@ -231,6 +231,7 @@ struct kvm_guest_debug_arch {
 #define KVM_SYNC_GSCB   (1UL << 9)
 #define KVM_SYNC_BPBC   (1UL << 10)
 #define KVM_SYNC_ETOKEN (1UL << 11)
+#define KVM_SYNC_PV	(1UL << 12)
 /* length and alignment of the sdnx as a power of two */
 #define SDNXC 8
 #define SDNXL (1UL << SDNXC)
target/s390x/cpu.h +1 -0
index 17460ed7b3..a787221772 100644
@@ -116,6 +116,7 @@ struct CPUS390XState {
 
     /* Fields up to this point are cleared by a CPU reset */
     struct {} end_reset_fields;
+    bool pv; /* protected virtualization */
 
 #if !defined(CONFIG_USER_ONLY)
     uint32_t core_id; /* PoP "CPU address", same as cpu_index */
target/s390x/kvm.c +4 -0
index c24c869e77..418154ccfe 100644
@@ -676,6 +676,10 @@ int kvm_arch_get_registers(CPUState *cs)
         env->etoken_extension = cs->kvm_run->s.regs.etoken_extension;
     }
 
+    if (can_sync_regs(cs, KVM_SYNC_PV)) {
+        env->pv = !!cs->kvm_run->s.regs.pv;
+    }
+
     /* pfault parameters */
     if (can_sync_regs(cs, KVM_SYNC_PFAULT)) {
         env->pfault_token = cs->kvm_run->s.regs.pft;
Signature · 1 line
2.20.1
#25 lore raw

Re: [PATCH 05/15] s390x: protvirt: Sync PV state

Quoted 19 lines · On Wed, 20 Nov 2019 06:43:24 -0500 Janosch Frank <[email protected]> wrote:

We do not always have the SIE intercept code handy at each place where we do emulation. Unfortunately emulation for secure guests often differ slightly from normal emulation and we need to make decisions based on the protected state of the VCPU.

Let's sync the protected state and make it available.

---
 linux-headers/asm-s390/kvm.h | 1 +
 target/s390x/cpu.h           | 1 +
 target/s390x/kvm.c           | 4 ++++
 3 files changed, 6 insertions(+)
linux-headers/asm-s390/kvm.h +0 -0
index 41976d33f0..7c46cf6078 100644

linux-headers/asm-s390/kvm.h struct kvm_guest_debug_arch { · line 234

Quoted 4 lines
linux-headers/asm-s390/kvm.h +1 -0
@@ -231,6 +231,7 @@ struct kvm_guest_debug_arch {
 #define KVM_SYNC_GSCB   (1UL << 9)
 #define KVM_SYNC_BPBC   (1UL << 10)
 #define KVM_SYNC_ETOKEN (1UL << 11)
+#define KVM_SYNC_PV	(1UL << 12)

That should go into the previous patch (will be picked up by header sync).

Quoted 20 lines
linux-headers/asm-s390/kvm.h +0 -0
 /* length and alignment of the sdnx as a power of two */
 #define SDNXC 8
 #define SDNXL (1UL << SDNXC)
target/s390x/cpu.h +1 -0
index 17460ed7b3..a787221772 100644
@@ -116,6 +116,7 @@ struct CPUS390XState {
 
     /* Fields up to this point are cleared by a CPU reset */
     struct {} end_reset_fields;
+    bool pv; /* protected virtualization */
 
 #if !defined(CONFIG_USER_ONLY)
     uint32_t core_id; /* PoP "CPU address", same as cpu_index */
target/s390x/kvm.c +0 -0
index c24c869e77..418154ccfe 100644

target/s390x/kvm.c int kvm_arch_get_registers(CPUState *cs) · line 685

Quoted 10 lines
target/s390x/kvm.c +4 -0
@@ -676,6 +676,10 @@ int kvm_arch_get_registers(CPUState *cs)
         env->etoken_extension = cs->kvm_run->s.regs.etoken_extension;
     }
 
+    if (can_sync_regs(cs, KVM_SYNC_PV)) {
+        env->pv = !!cs->kvm_run->s.regs.pv;
+    }
+
     /* pfault parameters */
     if (can_sync_regs(cs, KVM_SYNC_PFAULT)) {
         env->pfault_token = cs->kvm_run->s.regs.pft;

As you add a new field to the cpu state... you probably can't migrate protected guests, so you don't need a new vmstate subsection?

#26 lore raw

Re: [PATCH 05/15] s390x: protvirt: Sync PV state

Quoted 17 lines · Cornelia Huck
Quoted 14 lines · On Wed, 20 Nov 2019 06:43:24 -0500 Janosch Frank <[email protected]> wrote:

We do not always have the SIE intercept code handy at each place where we do emulation. Unfortunately emulation for secure guests often differ slightly from normal emulation and we need to make decisions based on the protected state of the VCPU.

Let's sync the protected state and make it available.

---
 linux-headers/asm-s390/kvm.h | 1 +
 target/s390x/cpu.h           | 1 +
 target/s390x/kvm.c           | 4 ++++
 3 files changed, 6 insertions(+)
Quoted 12 lines

linux-headers/asm-s390/kvm.h struct kvm_guest_debug_arch { · line 234

Quoted 9 lines
linux-headers/asm-s390/kvm.h +1 -0
index 41976d33f0..7c46cf6078 100644
@@ -231,6 +231,7 @@ struct kvm_guest_debug_arch {
 #define KVM_SYNC_GSCB   (1UL << 9)
 #define KVM_SYNC_BPBC   (1UL << 10)
 #define KVM_SYNC_ETOKEN (1UL << 11)
+#define KVM_SYNC_PV	(1UL << 12)

That should go into the previous patch (will be picked up by header sync).

Yes, will be fixed in a second

Quoted 24 lines
Quoted 24 lines
linux-headers/asm-s390/kvm.h +0 -0
 /* length and alignment of the sdnx as a power of two */
 #define SDNXC 8
 #define SDNXL (1UL << SDNXC)
target/s390x/cpu.h +1 -0
index 17460ed7b3..a787221772 100644
@@ -116,6 +116,7 @@ struct CPUS390XState {
 
     /* Fields up to this point are cleared by a CPU reset */
     struct {} end_reset_fields;
+    bool pv; /* protected virtualization */
 
 #if !defined(CONFIG_USER_ONLY)
     uint32_t core_id; /* PoP "CPU address", same as cpu_index */
target/s390x/kvm.c +0 -0
index c24c869e77..418154ccfe 100644
@@ -676,6 +676,10 @@ int kvm_arch_get_registers(CPUState *cs)
         env->etoken_extension = cs->kvm_run->s.regs.etoken_extension;
     }
 
Quoted 11 lines

target/s390x/kvm.c int kvm_arch_get_registers(CPUState *cs) · line 685

Quoted 7 lines
target/s390x/kvm.c +4 -0
+    if (can_sync_regs(cs, KVM_SYNC_PV)) {
+        env->pv = !!cs->kvm_run->s.regs.pv;
+    }
+
     /* pfault parameters */
     if (can_sync_regs(cs, KVM_SYNC_PFAULT)) {
         env->pfault_token = cs->kvm_run->s.regs.pft;

As you add a new field to the cpu state... you probably can't migrate protected guests, so you don't need a new vmstate subsection?

We can't migrate currently, but it's on out todo list

#27 lore raw

Re: [PATCH 05/15] s390x: protvirt: Sync PV state

In reply to Janosch Frank
Quoted 43 lines · Janosch Frank

We do not always have the SIE intercept code handy at each place where we do emulation. Unfortunately emulation for secure guests often differ slightly from normal emulation and we need to make decisions based on the protected state of the VCPU.

Let's sync the protected state and make it available.

---
 linux-headers/asm-s390/kvm.h | 1 +
 target/s390x/cpu.h           | 1 +
 target/s390x/kvm.c           | 4 ++++
 3 files changed, 6 insertions(+)
linux-headers/asm-s390/kvm.h +1 -0
index 41976d33f0..7c46cf6078 100644
@@ -231,6 +231,7 @@ struct kvm_guest_debug_arch {
 #define KVM_SYNC_GSCB   (1UL << 9)
 #define KVM_SYNC_BPBC   (1UL << 10)
 #define KVM_SYNC_ETOKEN (1UL << 11)
+#define KVM_SYNC_PV	(1UL << 12)
 /* length and alignment of the sdnx as a power of two */
 #define SDNXC 8
 #define SDNXL (1UL << SDNXC)
target/s390x/cpu.h +1 -0
index 17460ed7b3..a787221772 100644
@@ -116,6 +116,7 @@ struct CPUS390XState {
 
     /* Fields up to this point are cleared by a CPU reset */
     struct {} end_reset_fields;
+    bool pv; /* protected virtualization */
 
 #if !defined(CONFIG_USER_ONLY)
     uint32_t core_id; /* PoP "CPU address", same as cpu_index */
target/s390x/kvm.c +0 -0
index c24c869e77..418154ccfe 100644

target/s390x/kvm.c int kvm_arch_get_registers(CPUState *cs) · line 681

Quoted 6 lines
target/s390x/kvm.c +3 -0
@@ -676,6 +676,10 @@ int kvm_arch_get_registers(CPUState *cs)
         env->etoken_extension = cs->kvm_run->s.regs.etoken_extension;
     }
 
+    if (can_sync_regs(cs, KVM_SYNC_PV)) {
+        env->pv = !!cs->kvm_run->s.regs.pv;
+    }

Does this really need to be sync'ed each time during kvm_arch_get_registers()? I mean, this is not a state that continuously changes ... so when I guest enters PV mode, I assume that we have to do some stuff in QEMU anyway, so we could set the "pv = true" in that case. And during reset, we could clear it again. Or do I miss something?

Thomas

#28 lore raw

PATCH 6/15 s390x: protvirt: Support unpack facility

In reply to Janosch Frank

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  33 ++++++++
 hw/s390x/ipl.h                      |   2 +
 hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  26 ++++++
 hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
 target/s390x/cpu_features_def.inc.h |   1 +
 7 files changed, 216 insertions(+), 10 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
hw/s390x/Makefile.objs +1 -0
index 94e57113d8..568bab9711 100644
@@ -31,6 +31,7 @@ obj-y += tod-qemu.o
 obj-$(CONFIG_KVM) += tod-kvm.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
 obj-$(CONFIG_KVM) += s390-stattrib-kvm.o s390-mchk.o
+obj-$(CONFIG_KVM) += pv.o
 obj-y += s390-ccw.o
 obj-y += ap-device.o
 obj-y += ap-bridge.o
hw/s390x/ipl.c +33 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
 #include "qemu/cutils.h"
 #include "qemu/option.h"
 #include "exec/exec-all.h"
+#include "pv.h"
 
 #define KERN_IMAGE_START                0x010000UL
 #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
     cpu_physical_memory_unmap(addr, len, 1, len);
 }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}
+
 void s390_ipl_prepare_cpu(S390CPU *cpu)
 {
     S390IPLState *ipl = get_ipl_device();
hw/s390x/ipl.h +2 -0
index 7b8a493509..e848602c16 100644
@@ -105,6 +105,8 @@ typedef union IplParameterBlock IplParameterBlock;
 int s390_ipl_set_loadparm(uint8_t *loadparm);
 int s390_ipl_pv_check_comp(IplParameterBlock *iplb);
 void s390_ipl_update_diag308(IplParameterBlock *iplb);
+int s390_ipl_prepare_pv_header(void);
+int s390_ipl_pv_unpack(void);
 void s390_ipl_prepare_cpu(S390CPU *cpu);
 IplParameterBlock *s390_ipl_get_iplb(void);
 IplParameterBlock *s390_ipl_get_iplb_secure(void);
hw/s390x/pv.c new file +118 -0

118 lines not on this page · raw message

hw/s390x/pv.h new file +26 -0
new file mode 100644
index 0000000000..eb074e4bc9
@@ -0,0 +1,26 @@
+/*
+ * Protected Virtualization header
+ *
+ * Copyright IBM Corp. 2019
+ * Author(s):
+ *  Janosch Frank <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_PV_H
+#define HW_S390_PV_H
+
+int s390_pv_vm_create(void);
+int s390_pv_vm_destroy(void);
+int s390_pv_vcpu_destroy(CPUState *cs);
+int s390_pv_vcpu_create(CPUState *cs);
+int s390_pv_set_sec_parms(uint64_t origin, uint64_t length);
+int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak);
+int s390_pv_perf_clear_reset(void);
+int s390_pv_verify(void);
+int s390_pv_unshare(void);
+
+#endif /* HW_S390_PV_H */
hw/s390x/s390-virtio-ccw.c +35 -10

76 lines not on this page · raw message

target/s390x/cpu_features_def.inc.h +1 -0
index 31dff0d84e..60db28351d 100644
@@ -107,6 +107,7 @@ DEF_FEAT(DEFLATE_BASE, "deflate-base", STFL, 151, "Deflate-conversion facility (
 DEF_FEAT(VECTOR_PACKED_DECIMAL_ENH, "vxpdeh", STFL, 152, "Vector-Packed-Decimal-Enhancement Facility")
 DEF_FEAT(MSA_EXT_9, "msa9-base", STFL, 155, "Message-security-assist-extension-9 facility (excluding subfunctions)")
 DEF_FEAT(ETOKEN, "etoken", STFL, 156, "Etoken facility")
+DEF_FEAT(UNPACK, "unpack", STFL, 161, "Unpack facility")
 
 /* Features exposed via SCLP SCCB Byte 80 - 98  (bit numbers relative to byte-80) */
 DEF_FEAT(SIE_GSLS, "gsls", SCLP_CONF_CHAR, 40, "SIE: Guest-storage-limit-suppression facility")
Signature · 1 line
2.20.1
#29 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 32 lines · On Wed, 20 Nov 2019 06:43:25 -0500 Janosch Frank <[email protected]> wrote:

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  33 ++++++++
 hw/s390x/ipl.h                      |   2 +
 hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  26 ++++++
 hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
 target/s390x/cpu_features_def.inc.h |   1 +
 7 files changed, 216 insertions(+), 10 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
hw/s390x/Makefile.objs +0 -0
index 94e57113d8..568bab9711 100644

hw/s390x/Makefile.objs obj-y += tod-qemu.o · line 34

Quoted 4 lines
hw/s390x/Makefile.objs +1 -0
@@ -31,6 +31,7 @@ obj-y += tod-qemu.o
 obj-$(CONFIG_KVM) += tod-kvm.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
 obj-$(CONFIG_KVM) += s390-stattrib-kvm.o s390-mchk.o
+obj-$(CONFIG_KVM) += pv.o

As this is kvm only...

Quoted 34 lines
hw/s390x/Makefile.objs +0 -0
 obj-y += s390-ccw.o
 obj-y += ap-device.o
 obj-y += ap-bridge.o
hw/s390x/ipl.c +16 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
 #include "qemu/cutils.h"
 #include "qemu/option.h"
 #include "exec/exec-all.h"
+#include "pv.h"
 
 #define KERN_IMAGE_START                0x010000UL
 #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
     cpu_physical_memory_unmap(addr, len, 1, len);
 }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+

hw/s390x/ipl.c static void s390_ipl_prepare_qipl(S390CPU *cpu) · line 696

Quoted 10 lines
hw/s390x/ipl.c +10 -0
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);

...you probably need a stub version of the pv functions as well, right?

Quoted 10 lines
hw/s390x/ipl.c +7 -0
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}
+
 void s390_ipl_prepare_cpu(S390CPU *cpu)
 {
     S390IPLState *ipl = get_ipl_device();
#30 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 80 lines · Cornelia Huck
Quoted 32 lines · On Wed, 20 Nov 2019 06:43:25 -0500 Janosch Frank <[email protected]> wrote:

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  33 ++++++++
 hw/s390x/ipl.h                      |   2 +
 hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  26 ++++++
 hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
 target/s390x/cpu_features_def.inc.h |   1 +
 7 files changed, 216 insertions(+), 10 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
hw/s390x/Makefile.objs +0 -0
index 94e57113d8..568bab9711 100644

hw/s390x/Makefile.objs obj-y += tod-qemu.o · line 34

Quoted 4 lines
hw/s390x/Makefile.objs +1 -0
@@ -31,6 +31,7 @@ obj-y += tod-qemu.o
 obj-$(CONFIG_KVM) += tod-kvm.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
 obj-$(CONFIG_KVM) += s390-stattrib-kvm.o s390-mchk.o
+obj-$(CONFIG_KVM) += pv.o

As this is kvm only...

Quoted 38 lines
hw/s390x/Makefile.objs +0 -0
 obj-y += s390-ccw.o
 obj-y += ap-device.o
 obj-y += ap-bridge.o
hw/s390x/ipl.c +20 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
 #include "qemu/cutils.h"
 #include "qemu/option.h"
 #include "exec/exec-all.h"
+#include "pv.h"
 
 #define KERN_IMAGE_START                0x010000UL
 #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
     cpu_physical_memory_unmap(addr, len, 1, len);
 }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
Quoted 8 lines

hw/s390x/ipl.c static void s390_ipl_prepare_qipl(S390CPU *cpu) · line 696

Quoted 6 lines
hw/s390x/ipl.c +6 -0
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);

...you probably need a stub version of the pv functions as well, right?

Yes, I'm working on it

Quoted 12 lines
Quoted 11 lines
hw/s390x/ipl.c +7 -0
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}
+
 void s390_ipl_prepare_cpu(S390CPU *cpu)
 {
     S390IPLState *ipl = get_ipl_device();
#31 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

In reply to Janosch Frank
Quoted 309 lines · Janosch Frank

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
  hw/s390x/Makefile.objs              |   1 +
  hw/s390x/ipl.c                      |  33 ++++++++
  hw/s390x/ipl.h                      |   2 +
  hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
  hw/s390x/pv.h                       |  26 ++++++
  hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
  target/s390x/cpu_features_def.inc.h |   1 +
  7 files changed, 216 insertions(+), 10 deletions(-)

create mode 100644 hw/s390x/pv.c
create mode 100644 hw/s390x/pv.h

hw/s390x/Makefile.objs +1 -0
index 94e57113d8..568bab9711 100644
@@ -31,6 +31,7 @@ obj-y += tod-qemu.o
  obj-$(CONFIG_KVM) += tod-kvm.o
  obj-$(CONFIG_KVM) += s390-skeys-kvm.o
  obj-$(CONFIG_KVM) += s390-stattrib-kvm.o s390-mchk.o
+obj-$(CONFIG_KVM) += pv.o
  obj-y += s390-ccw.o
  obj-y += ap-device.o
  obj-y += ap-bridge.o
hw/s390x/ipl.c +33 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
  #include "qemu/cutils.h"
  #include "qemu/option.h"
  #include "exec/exec-all.h"
+#include "pv.h"
 
  #define KERN_IMAGE_START                0x010000UL
  #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
      cpu_physical_memory_unmap(addr, len, 1, len);
  }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}
+
  void s390_ipl_prepare_cpu(S390CPU *cpu)
  {
      S390IPLState *ipl = get_ipl_device();
hw/s390x/ipl.h +2 -0
index 7b8a493509..e848602c16 100644
@@ -105,6 +105,8 @@ typedef union IplParameterBlock IplParameterBlock;
  int s390_ipl_set_loadparm(uint8_t *loadparm);
  int s390_ipl_pv_check_comp(IplParameterBlock *iplb);
  void s390_ipl_update_diag308(IplParameterBlock *iplb);
+int s390_ipl_prepare_pv_header(void);
+int s390_ipl_pv_unpack(void);
  void s390_ipl_prepare_cpu(S390CPU *cpu);
  IplParameterBlock *s390_ipl_get_iplb(void);
  IplParameterBlock *s390_ipl_get_iplb_secure(void);
hw/s390x/pv.c new file +118 -0
new file mode 100644
index 0000000000..0218070322
@@ -0,0 +1,118 @@
+/*
+ * Secure execution functions
+ *
+ * Copyright IBM Corp. 2019
+ * Author(s):
+ *  Janosch Frank <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include <sys/ioctl.h>
+
+#include <linux/kvm.h>
+
+#include "qemu/error-report.h"
+#include "sysemu/kvm.h"
+#include "pv.h"
+
+static int s390_pv_cmd(uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vm_ioctl(kvm_state, KVM_S390_PV_COMMAND, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+static int s390_pv_cmd_vcpu(CPUState *cs, uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vcpu_ioctl(cs, KVM_S390_PV_COMMAND_VCPU, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV VCPU command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+int s390_pv_vm_create(void)
+{
+    return s390_pv_cmd(KVM_PV_VM_CREATE, NULL);
+}
+
+int s390_pv_vm_destroy(void)
+{
+    return s390_pv_cmd(KVM_PV_VM_DESTROY, NULL);
+}
+
+int s390_pv_vcpu_create(CPUState *cs)
+{
+    return s390_pv_cmd_vcpu(cs, KVM_PV_VCPU_CREATE, NULL);
+}
+
+int s390_pv_vcpu_destroy(CPUState *cs)
+{
+    S390CPU *cpu = S390_CPU(cs);
+    CPUS390XState *env = &cpu->env;
+    int rc;
+
+    rc = s390_pv_cmd_vcpu(cs, KVM_PV_VCPU_DESTROY, NULL);
+    if (!rc) {
+        env->pv = false;
+    }
+    return rc;
+}
+
+int s390_pv_set_sec_parms(uint64_t origin, uint64_t length)
+{
+    struct kvm_s390_pv_sec_parm args = {
+        .origin = origin,
+        .length = length,
+    };
+
+    return s390_pv_cmd(KVM_PV_VM_SET_SEC_PARMS, &args);
+}
+
+/*
+ * Called for each component in the SE type IPL parameter block 0.
+ */
+int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak)
+{
+    struct kvm_s390_pv_unp args = {
+        .addr = addr,
+        .size = size,
+        .tweak = tweak,
+    };
+
+    return s390_pv_cmd(KVM_PV_VM_UNPACK, &args);
+}
+
+int s390_pv_perf_clear_reset(void)
+{
+    return s390_pv_cmd(KVM_PV_VM_PERF_CLEAR_RESET, NULL);
+}
+
+int s390_pv_verify(void)
+{
+    return s390_pv_cmd(KVM_PV_VM_VERIFY, NULL);
+}
+
+int s390_pv_unshare(void)
+{
+    return s390_pv_cmd(KVM_PV_VM_UNSHARE, NULL);
+}
hw/s390x/pv.h new file +26 -0
new file mode 100644
index 0000000000..eb074e4bc9
@@ -0,0 +1,26 @@
+/*
+ * Protected Virtualization header
+ *
+ * Copyright IBM Corp. 2019
+ * Author(s):
+ *  Janosch Frank <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_PV_H
+#define HW_S390_PV_H
+
+int s390_pv_vm_create(void);
+int s390_pv_vm_destroy(void);
+int s390_pv_vcpu_destroy(CPUState *cs);
+int s390_pv_vcpu_create(CPUState *cs);
+int s390_pv_set_sec_parms(uint64_t origin, uint64_t length);
+int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak);
+int s390_pv_perf_clear_reset(void);
+int s390_pv_verify(void);
+int s390_pv_unshare(void);
+
+#endif /* HW_S390_PV_H */
hw/s390x/s390-virtio-ccw.c +6 -10
index c1d1440272..7262453616 100644
@@ -41,6 +41,7 @@
  #include "hw/qdev-properties.h"
  #include "hw/s390x/tod.h"
  #include "sysemu/sysemu.h"
+#include "hw/s390x/pv.h"
 
  S390CPU *s390_cpu_addr2state(uint16_t cpu_addr)
  {
@@ -322,6 +323,7 @@ static void s390_machine_reset(MachineState *machine)
  {
      enum s390_reset reset_type;
      CPUState *cs, *t;
+    S390CPU *cpu;
 
      /* get the reset parameters, reset them once done */
      s390_ipl_get_reset_request(&cs, &reset_type);
@@ -329,16 +331,10 @@ static void s390_machine_reset(MachineState *machine)
      /* all CPUs are paused and synchronized at this point */
      s390_cmma_reset();
 
-    switch (reset_type) {
-    case S390_RESET_EXTERNAL:
-    case S390_RESET_REIPL:
-        qemu_devices_reset();
-        s390_crypto_reset();
+    cpu = S390_CPU(cs);
 
-        /* configure and start the ipl CPU only */
-        run_on_cpu(cs, s390_do_cpu_ipl, RUN_ON_CPU_NULL);
-        break;
-    case S390_RESET_MODIFIED_CLEAR:
+    switch (reset_type) {
+    case S390_RESET_MODIFIED_CLEAR:  /* Subcode 0 */
          CPU_FOREACH(t) {
              run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
          }
@@ -346,7 +342,7 @@ static void s390_machine_reset(MachineState *machine)
          s390_crypto_reset();
          run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
          break;
-    case S390_RESET_LOAD_NORMAL:
+    case S390_RESET_LOAD_NORMAL: /* Subcode 1*/
          CPU_FOREACH(t) {
              if (t == cs) {
                  continue;

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 362

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +7 -0
@@ -357,6 +353,35 @@ static void s390_machine_reset(MachineState *machine)
          run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
          run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
          break;
+    case S390_RESET_EXTERNAL:
+    case S390_RESET_REIPL: /* Subcode 4 */
+        qemu_devices_reset();
+        s390_crypto_reset();
+        /* configure and start the ipl CPU only */
+        run_on_cpu(cs, s390_do_cpu_ipl, RUN_ON_CPU_NULL);
+        break;

Is there a way to modify this patch to not change unrelated code that heavily? Makes it harder to review.

Quoted 38 lines
hw/s390x/s390-virtio-ccw.c +22 -0
+    case S390_RESET_PV: /* Subcode 10 */
+        subsystem_reset();
+        s390_crypto_reset();
+
+        CPU_FOREACH(t) {
+            run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
+        }
+
+        /* Create SE VM */
+        s390_pv_vm_create();
+        CPU_FOREACH(t) {
+            s390_pv_vcpu_create(t);
+        }
+
+        /* Set SE header and unpack */
+        s390_ipl_prepare_pv_header();
+        /* Decrypt image */
+        s390_ipl_pv_unpack();
+        /* Verify integrity */
+        s390_pv_verify();
+        s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
+        break;
      default:
          g_assert_not_reached();
      }
target/s390x/cpu_features_def.inc.h +1 -0
index 31dff0d84e..60db28351d 100644
@@ -107,6 +107,7 @@ DEF_FEAT(DEFLATE_BASE, "deflate-base", STFL, 151, "Deflate-conversion facility (
  DEF_FEAT(VECTOR_PACKED_DECIMAL_ENH, "vxpdeh", STFL, 152, "Vector-Packed-Decimal-Enhancement Facility")
  DEF_FEAT(MSA_EXT_9, "msa9-base", STFL, 155, "Message-security-assist-extension-9 facility (excluding subfunctions)")
  DEF_FEAT(ETOKEN, "etoken", STFL, 156, "Etoken facility")
+DEF_FEAT(UNPACK, "unpack", STFL, 161, "Unpack facility")
 
  /* Features exposed via SCLP SCCB Byte 80 - 98  (bit numbers relative to byte-80) */
  DEF_FEAT(SIE_GSLS, "gsls", SCLP_CONF_CHAR, 40, "SIE: Guest-storage-limit-suppression facility")
Signature · 3 lines
Thanks,

David / dhildenb
#32 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 16 lines · David Hildenbrand

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 362

Quoted 11 lines · Janosch Frank
hw/s390x/s390-virtio-ccw.c +7 -0
@@ -357,6 +353,35 @@ static void s390_machine_reset(MachineState *machine)
          run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
          run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
          break;
+    case S390_RESET_EXTERNAL:
+    case S390_RESET_REIPL: /* Subcode 4 */
+        qemu_devices_reset();
+        s390_crypto_reset();
+        /* configure and start the ipl CPU only */
+        run_on_cpu(cs, s390_do_cpu_ipl, RUN_ON_CPU_NULL);
+        break;

Is there a way to modify this patch to not change unrelated code that heavily? Makes it harder to review.

https://github.com/frankjaa/qemu/commit/8c53d5c8a6bbcc53496c7a2877c7cbffc435b708

And please trim your emails.

Quoted 41 lines
Quoted 39 lines
hw/s390x/s390-virtio-ccw.c +22 -0
+    case S390_RESET_PV: /* Subcode 10 */
+        subsystem_reset();
+        s390_crypto_reset();
+
+        CPU_FOREACH(t) {
+            run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
+        }
+
+        /* Create SE VM */
+        s390_pv_vm_create();
+        CPU_FOREACH(t) {
+            s390_pv_vcpu_create(t);
+        }
+
+        /* Set SE header and unpack */
+        s390_ipl_prepare_pv_header();
+        /* Decrypt image */
+        s390_ipl_pv_unpack();
+        /* Verify integrity */
+        s390_pv_verify();
+        s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
+        break;
      default:
          g_assert_not_reached();
      }
target/s390x/cpu_features_def.inc.h +1 -0
index 31dff0d84e..60db28351d 100644
@@ -107,6 +107,7 @@ DEF_FEAT(DEFLATE_BASE, "deflate-base", STFL, 151, "Deflate-conversion facility (
  DEF_FEAT(VECTOR_PACKED_DECIMAL_ENH, "vxpdeh", STFL, 152, "Vector-Packed-Decimal-Enhancement Facility")
  DEF_FEAT(MSA_EXT_9, "msa9-base", STFL, 155, "Message-security-assist-extension-9 facility (excluding subfunctions)")
  DEF_FEAT(ETOKEN, "etoken", STFL, 156, "Etoken facility")
+DEF_FEAT(UNPACK, "unpack", STFL, 161, "Unpack facility")
 
  /* Features exposed via SCLP SCCB Byte 80 - 98  (bit numbers relative to byte-80) */
  DEF_FEAT(SIE_GSLS, "gsls", SCLP_CONF_CHAR, 40, "SIE: Guest-storage-limit-suppression facility")
#33 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 18 lines · Janosch Frank
Quoted 16 lines · David Hildenbrand

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 362

Quoted 11 lines · Janosch Frank
hw/s390x/s390-virtio-ccw.c +7 -0
@@ -357,6 +353,35 @@ static void s390_machine_reset(MachineState *machine)
           run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
           run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
           break;
+    case S390_RESET_EXTERNAL:
+    case S390_RESET_REIPL: /* Subcode 4 */
+        qemu_devices_reset();
+        s390_crypto_reset();
+        /* configure and start the ipl CPU only */
+        run_on_cpu(cs, s390_do_cpu_ipl, RUN_ON_CPU_NULL);
+        break;

Is there a way to modify this patch to not change unrelated code that heavily? Makes it harder to review.

Quoted 4 lines

https://github.com/frankjaa/qemu/commit/8c53d5c8a6bbcc53496c7a2877c7cbffc435b708

And please trim your emails.

If you use Thunderbird I suggest QuoteCollapse ... because nobody got time for that ;)

Signature · 3 lines
Thanks,

David / dhildenb
#34 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 4 lines · David Hildenbrand
Quoted 2 lines

And please trim your emails.

If you use Thunderbird I suggest QuoteCollapse ... because nobody got time for that ;)

neat.

#35 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 10 lines · Christian Borntraeger
Quoted 4 lines · David Hildenbrand
Quoted 2 lines

And please trim your emails.

If you use Thunderbird I suggest QuoteCollapse ... because nobody got time for that ;)

neat.

Yeah, seems like I'm already too old-school for fancy addons at my young age ;-)

#36 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

In reply to Janosch Frank
Quoted 49 lines · On Wed, 20 Nov 2019 06:43:25 -0500 Janosch Frank <[email protected]> wrote:

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  33 ++++++++
 hw/s390x/ipl.h                      |   2 +
 hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  26 ++++++
 hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
 target/s390x/cpu_features_def.inc.h |   1 +
 7 files changed, 216 insertions(+), 10 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
hw/s390x/pv.c new file +21 -0
+static int s390_pv_cmd(uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vm_ioctl(kvm_state, KVM_S390_PV_COMMAND, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+static int s390_pv_cmd_vcpu(CPUState *cs, uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,

hw/s390x/pv.c line 51

Quoted 10 lines
hw/s390x/pv.c new file +10 -0
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vcpu_ioctl(cs, KVM_S390_PV_COMMAND_VCPU, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV VCPU command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}

If you do a hard exit for any rc != 0 anyway, returning rc does not sound very useful :)

#37 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 57 lines · Cornelia Huck
Quoted 54 lines · On Wed, 20 Nov 2019 06:43:25 -0500 Janosch Frank <[email protected]> wrote:

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---
 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  33 ++++++++
 hw/s390x/ipl.h                      |   2 +
 hw/s390x/pv.c                       | 118 ++++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  26 ++++++
 hw/s390x/s390-virtio-ccw.c          |  45 ++++++++---
 target/s390x/cpu_features_def.inc.h |   1 +
 7 files changed, 216 insertions(+), 10 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
hw/s390x/pv.c new file +26 -0
+static int s390_pv_cmd(uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vm_ioctl(kvm_state, KVM_S390_PV_COMMAND, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+static int s390_pv_cmd_vcpu(CPUState *cs, uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vcpu_ioctl(cs, KVM_S390_PV_COMMAND_VCPU, &pv_cmd);
+    if (rc) {
Quoted 9 lines

hw/s390x/pv.c line 51

Quoted 5 lines
hw/s390x/pv.c new file +5 -0
+        error_report("KVM PV VCPU command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}

If you do a hard exit for any rc != 0 anyway, returning rc does not sound very useful :)

Yeah, that's also what Pierre said.
I'll need to rethink if there's actually a call where we could recover from an error and if not make everything void.

#38 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

In reply to Janosch Frank
Quoted 16 lines · Janosch Frank

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---

[...]

Quoted 37 lines
hw/s390x/ipl.c +22 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
 #include "qemu/cutils.h"
 #include "qemu/option.h"
 #include "exec/exec-all.h"
+#include "pv.h"
 
 #define KERN_IMAGE_START                0x010000UL
 #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
     cpu_physical_memory_unmap(addr, len, 1, len);
 }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+

hw/s390x/ipl.c static void s390_ipl_prepare_qipl(S390CPU *cpu) · line 702

Quoted 10 lines
hw/s390x/ipl.c +10 -0
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}

For both functions, s390_ipl_prepare_pv_header() and s390_ipl_pv_unpack(), you're ignoring the return code at the calling site, so errors go completely unnoticed. I suggest to either do an error_report() here or at the calling site...

Quoted 47 lines
hw/s390x/pv.c new file +41 -0
new file mode 100644
index 0000000000..0218070322
@@ -0,0 +1,118 @@
+/*
+ * Secure execution functions
+ *
+ * Copyright IBM Corp. 2019
+ * Author(s):
+ *  Janosch Frank <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include <sys/ioctl.h>
+
+#include <linux/kvm.h>
+
+#include "qemu/error-report.h"
+#include "sysemu/kvm.h"
+#include "pv.h"
+
+static int s390_pv_cmd(uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vm_ioctl(kvm_state, KVM_S390_PV_COMMAND, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+static int s390_pv_cmd_vcpu(CPUState *cs, uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,

hw/s390x/pv.c line 51

Quoted 10 lines
hw/s390x/pv.c new file +10 -0
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vcpu_ioctl(cs, KVM_S390_PV_COMMAND_VCPU, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV VCPU command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}

... or even better, since these functions exit on error anyway, make these functions and all the others "void" instead?

Thomas

#39 lore raw

Re: [PATCH 06/15] s390x: protvirt: Support unpack facility

Quoted 132 lines · Thomas Huth
Quoted 16 lines · Janosch Frank

When a guest has saved a ipib of type 5 and call diagnose308 with subcode 10, we have to setup the protected processing environment via Ultravisor calls. The calls are done by KVM and are exposed via an API.

The following steps are necessary:
1. Create a VM (register it with the Ultravisor)
2. Create secure CPUs for all of our current cpus
3. Forward the secure header to the Ultravisor (has all information on how to decrypt the image and VM information)
4. Protect image pages from the host and decrypt them
5. Verify the image integrity

Only after step 5 a protected VM is allowed to run.

---

[...]

Quoted 37 lines
hw/s390x/ipl.c +22 -0
index a077926f36..50501fcd27 100644
@@ -33,6 +33,7 @@
 #include "qemu/cutils.h"
 #include "qemu/option.h"
 #include "exec/exec-all.h"
+#include "pv.h"
 
 #define KERN_IMAGE_START                0x010000UL
 #define LINUX_MAGIC_ADDR                0x010008UL
@@ -668,6 +669,38 @@ static void s390_ipl_prepare_qipl(S390CPU *cpu)
     cpu_physical_memory_unmap(addr, len, 1, len);
 }
 
+int s390_ipl_prepare_pv_header(void)
+{
+    int rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+    void *hdr = g_malloc(ipib_pv->pv_header_len);
+
+    cpu_physical_memory_read(ipib_pv->pv_header_addr, hdr,
+                             ipib_pv->pv_header_len);
+    rc = s390_pv_set_sec_parms((uint64_t)hdr,
+                               ipib_pv->pv_header_len);
+    g_free(hdr);
+    return rc;
+}
+
+int s390_ipl_pv_unpack(void)
+{
+    int i, rc;
+    IplParameterBlock *iplb = s390_ipl_get_iplb_secure();
+    IPLBlockPV *ipib_pv = &iplb->pv;
+

hw/s390x/ipl.c static void s390_ipl_prepare_qipl(S390CPU *cpu) · line 702

Quoted 10 lines
hw/s390x/ipl.c +10 -0
+    for (i = 0; i < ipib_pv->num_comp; i++) {
+        rc = s390_pv_unpack(ipib_pv->components[i].addr,
+                            TARGET_PAGE_ALIGN(ipib_pv->components[i].size),
+                            ipib_pv->components[i].tweak_pref);
+        if (rc) {
+            return rc;
+        }
+    }
+    return rc;
+}

For both functions, s390_ipl_prepare_pv_header() and s390_ipl_pv_unpack(), you're ignoring the return code at the calling site, so errors go completely unnoticed. I suggest to either do an error_report() here or at the calling site...

Quoted 47 lines
hw/s390x/pv.c new file +41 -0
new file mode 100644
index 0000000000..0218070322
@@ -0,0 +1,118 @@
+/*
+ * Secure execution functions
+ *
+ * Copyright IBM Corp. 2019
+ * Author(s):
+ *  Janosch Frank <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include <sys/ioctl.h>
+
+#include <linux/kvm.h>
+
+#include "qemu/error-report.h"
+#include "sysemu/kvm.h"
+#include "pv.h"
+
+static int s390_pv_cmd(uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vm_ioctl(kvm_state, KVM_S390_PV_COMMAND, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}
+
+static int s390_pv_cmd_vcpu(CPUState *cs, uint32_t cmd, void *data)
+{
+    int rc;
+    struct kvm_pv_cmd pv_cmd = {
+        .cmd = cmd,

hw/s390x/pv.c line 51

Quoted 10 lines
hw/s390x/pv.c new file +10 -0
+        .data = (uint64_t)data,
+    };
+
+    rc = kvm_vcpu_ioctl(cs, KVM_S390_PV_COMMAND_VCPU, &pv_cmd);
+    if (rc) {
+        error_report("KVM PV VCPU command failed cmd: %d rc: %d", cmd, rc);
+        exit(1);
+    }
+    return rc;
+}

... or even better, since these functions exit on error anyway, make these functions and all the others "void" instead?

Quoted 2 lines

Thomas

You're not the first stating this idea :-)
We might get a return code for diag308 that signals that we failed to start the secure guest and then I might need them again.

If we don't get a rc, I'll definitely make all functions void, if we do I'll make nearly all void:)

#40 lore raw

PATCH 7/15 s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

In reply to Janosch Frank

Now that we know the protection state off the cpus, we can start handling all diag 308 subcodes in the protected state.

For subcodes 0 and 1 we need to unshare all pages before continuing, so the guest doesn't accidently expose data when dumping.

For subcode 3/4 we tear down the protected VM and reboot into unprotected mode. We do not provide a secure reboot.

Before we can do the unshare calls, we need to mark all cpus as stopped.

---
 hw/s390x/s390-virtio-ccw.c | 30 +++++++++++++++++++++++++++---
 target/s390x/diag.c        |  4 ++++
 2 files changed, 31 insertions(+), 3 deletions(-)
hw/s390x/s390-virtio-ccw.c +27 -3
index 7262453616..6fd50b4c42 100644
@@ -319,11 +319,26 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)
     s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
 }
 
+static void s390_pv_prepare_reset(CPUS390XState *env)
+{
+    CPUState *cs;
+
+    if (!env->pv) {
+        return;
+    }
+    CPU_FOREACH(cs) {
+        s390_cpu_set_state(S390_CPU_STATE_STOPPED, S390_CPU(cs));
+    }
+    s390_pv_unshare();
+    s390_pv_perf_clear_reset();
+}
+
 static void s390_machine_reset(MachineState *machine)
 {
     enum s390_reset reset_type;
     CPUState *cs, *t;
     S390CPU *cpu;
+    CPUS390XState *env;
 
     /* get the reset parameters, reset them once done */
     s390_ipl_get_reset_request(&cs, &reset_type);
@@ -332,29 +347,38 @@ static void s390_machine_reset(MachineState *machine)
     s390_cmma_reset();
 
     cpu = S390_CPU(cs);
+    env = &cpu->env;
 
     switch (reset_type) {
     case S390_RESET_MODIFIED_CLEAR:  /* Subcode 0 */
+        subsystem_reset();
+        s390_crypto_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
-        s390_crypto_reset();
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:
     case S390_RESET_REIPL: /* Subcode 4 */
+        if (env->pv) {
+            CPU_FOREACH(t) {
+                s390_pv_vcpu_destroy(t);
+            }
+            s390_pv_vm_destroy();
+        }
         qemu_devices_reset();
         s390_crypto_reset();
         /* configure and start the ipl CPU only */
target/s390x/diag.c +4 -0
index 32049bb4ee..db6d79cef3 100644
@@ -68,6 +68,10 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
 {
+    /* Handled by the Ultravisor */
+    if (env->pv) {
+        return 0;
+    }
     if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EINVAL;
Signature · 1 line
2.20.1
#41 lore raw

Re: [PATCH 07/15] s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

Quoted 2 lines · On Wed, 20 Nov 2019 06:43:26 -0500 Janosch Frank <[email protected]> wrote:

Now that we know the protection state off the cpus, we can start handling all diag 308 subcodes in the protected state.

"As we now have access to the protection state of the cpus, we can implement special handling of diag 308 subcodes for cpus in the protected state."

?

Quoted 53 lines

For subcodes 0 and 1 we need to unshare all pages before continuing, so the guest doesn't accidently expose data when dumping.

For subcode 3/4 we tear down the protected VM and reboot into unprotected mode. We do not provide a secure reboot.

Before we can do the unshare calls, we need to mark all cpus as stopped.

---
 hw/s390x/s390-virtio-ccw.c | 30 +++++++++++++++++++++++++++---
 target/s390x/diag.c        |  4 ++++
 2 files changed, 31 insertions(+), 3 deletions(-)
hw/s390x/s390-virtio-ccw.c +16 -0
index 7262453616..6fd50b4c42 100644
@@ -319,11 +319,26 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)
     s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
 }
 
+static void s390_pv_prepare_reset(CPUS390XState *env)
+{
+    CPUState *cs;
+
+    if (!env->pv) {
+        return;
+    }
+    CPU_FOREACH(cs) {
+        s390_cpu_set_state(S390_CPU_STATE_STOPPED, S390_CPU(cs));
+    }
+    s390_pv_unshare();
+    s390_pv_perf_clear_reset();
+}
+
 static void s390_machine_reset(MachineState *machine)
 {
     enum s390_reset reset_type;
     CPUState *cs, *t;
     S390CPU *cpu;
+    CPUS390XState *env;
 
     /* get the reset parameters, reset them once done */
     s390_ipl_get_reset_request(&cs, &reset_type);
@@ -332,29 +347,38 @@ static void s390_machine_reset(MachineState *machine)
     s390_cmma_reset();
 
     cpu = S390_CPU(cs);
+    env = &cpu->env;
 

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 342

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +3 -2
     switch (reset_type) {
     case S390_RESET_MODIFIED_CLEAR:  /* Subcode 0 */
+        subsystem_reset();
+        s390_crypto_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
-        s390_crypto_reset();

This also switches the order in which different parts are reset for normal guests. I presume that order doesn't matter here?

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 362

Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +0 -0
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/

missing blank before */ (introduced in a previous patch)

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 374

Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +2 -1
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:

Annotate this with the subcode as well? (in the patch introducing it)

Quoted 25 lines
hw/s390x/s390-virtio-ccw.c +6 -0
     case S390_RESET_REIPL: /* Subcode 4 */
+        if (env->pv) {
+            CPU_FOREACH(t) {
+                s390_pv_vcpu_destroy(t);
+            }
+            s390_pv_vm_destroy();
+        }
         qemu_devices_reset();
         s390_crypto_reset();
         /* configure and start the ipl CPU only */
target/s390x/diag.c +4 -0
index 32049bb4ee..db6d79cef3 100644
@@ -68,6 +68,10 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
 {
+    /* Handled by the Ultravisor */
+    if (env->pv) {
+        return 0;
+    }
     if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EINVAL;
#42 lore raw

Re: [PATCH 07/15] s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

Quoted 11 lines · Cornelia Huck
Quoted 2 lines · On Wed, 20 Nov 2019 06:43:26 -0500 Janosch Frank <[email protected]> wrote:

Now that we know the protection state off the cpus, we can start handling all diag 308 subcodes in the protected state.

"As we now have access to the protection state of the cpus, we can implement special handling of diag 308 subcodes for cpus in the protected state."

?

Sure

Quoted 54 lines
Quoted 54 lines

For subcodes 0 and 1 we need to unshare all pages before continuing, so the guest doesn't accidently expose data when dumping.

For subcode 3/4 we tear down the protected VM and reboot into unprotected mode. We do not provide a secure reboot.

Before we can do the unshare calls, we need to mark all cpus as stopped.

---
 hw/s390x/s390-virtio-ccw.c | 30 +++++++++++++++++++++++++++---
 target/s390x/diag.c        |  4 ++++
 2 files changed, 31 insertions(+), 3 deletions(-)
hw/s390x/s390-virtio-ccw.c +16 -0
index 7262453616..6fd50b4c42 100644
@@ -319,11 +319,26 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)
     s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
 }
 
+static void s390_pv_prepare_reset(CPUS390XState *env)
+{
+    CPUState *cs;
+
+    if (!env->pv) {
+        return;
+    }
+    CPU_FOREACH(cs) {
+        s390_cpu_set_state(S390_CPU_STATE_STOPPED, S390_CPU(cs));
+    }
+    s390_pv_unshare();
+    s390_pv_perf_clear_reset();
+}
+
 static void s390_machine_reset(MachineState *machine)
 {
     enum s390_reset reset_type;
     CPUState *cs, *t;
     S390CPU *cpu;
+    CPUS390XState *env;
 
     /* get the reset parameters, reset them once done */
     s390_ipl_get_reset_request(&cs, &reset_type);
@@ -332,29 +347,38 @@ static void s390_machine_reset(MachineState *machine)
     s390_cmma_reset();
 
     cpu = S390_CPU(cs);
+    env = &cpu->env;
 
Quoted 13 lines

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 342

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +3 -2
     switch (reset_type) {
     case S390_RESET_MODIFIED_CLEAR:  /* Subcode 0 */
+        subsystem_reset();
+        s390_crypto_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             run_on_cpu(t, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
-        s390_crypto_reset();

This also switches the order in which different parts are reset for normal guests. I presume that order doesn't matter here?

I don't think that the order is specified in the POP, it is however specified for protected VMs...

Quoted 6 lines

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 362

Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +0 -0
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/

missing blank before */ (introduced in a previous patch)

Will fix

Quoted 16 lines
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +2 -0
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 374

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +0 -1
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:
Quoted 1 line

Annotate this with the subcode as well? (in the patch introducing it)

Sure

Quoted 27 lines
Quoted 26 lines
hw/s390x/s390-virtio-ccw.c +6 -0
     case S390_RESET_REIPL: /* Subcode 4 */
+        if (env->pv) {
+            CPU_FOREACH(t) {
+                s390_pv_vcpu_destroy(t);
+            }
+            s390_pv_vm_destroy();
+        }
         qemu_devices_reset();
         s390_crypto_reset();
         /* configure and start the ipl CPU only */
target/s390x/diag.c +4 -0
index 32049bb4ee..db6d79cef3 100644
@@ -68,6 +68,10 @@ int handle_diag_288(CPUS390XState *env, uint64_t r1, uint64_t r3)
 static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
                               uintptr_t ra, bool write)
 {
+    /* Handled by the Ultravisor */
+    if (env->pv) {
+        return 0;
+    }
     if ((r1 & 1) || (addr & ~TARGET_PAGE_MASK)) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EINVAL;
#43 lore raw

Re: [PATCH 07/15] s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

In reply to Cornelia Huck
Quoted 25 lines · Cornelia Huck
Quoted 3 lines · On Wed, 20 Nov 2019 06:43:26 -0500 Janosch Frank <[email protected]> wrote:
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/

missing blank before */ (introduced in a previous patch)

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 374

Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +2 -1
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:
Quoted 1 line

Annotate this with the subcode as well? (in the patch introducing it)

I think this has no diag 308 subcode and is triggered by qemu

#44 lore raw

Re: [PATCH 07/15] s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

Quoted 26 lines · On Thu, 21 Nov 2019 15:04:31 +0100 Janosch Frank <[email protected]> wrote:
Quoted 25 lines · Cornelia Huck
Quoted 3 lines · On Wed, 20 Nov 2019 06:43:26 -0500 Janosch Frank <[email protected]> wrote:
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/  

missing blank before */ (introduced in a previous patch)

Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +2 -1
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:  
Quoted 3 lines
Quoted 1 line

Annotate this with the subcode as well? (in the patch introducing it)

I think this has no diag 308 subcode and is triggered by qemu

-ENOCOFFE

But even more reason to annotate this with the fact that this is triggered by QEMU :)

#45 lore raw

Re: [PATCH 07/15] s390x: protvirt: Handle diag 308 subcodes 0,1,3,4

Quoted 29 lines · Cornelia Huck
Quoted 26 lines · On Thu, 21 Nov 2019 15:04:31 +0100 Janosch Frank <[email protected]> wrote:
Quoted 25 lines · Cornelia Huck
Quoted 3 lines · On Wed, 20 Nov 2019 06:43:26 -0500 Janosch Frank <[email protected]> wrote:
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_LOAD_NORMAL: /* Subcode 1*/  

missing blank before */ (introduced in a previous patch)

Quoted 13 lines
hw/s390x/s390-virtio-ccw.c +2 -1
+        subsystem_reset();
+        s390_pv_prepare_reset(env);
         CPU_FOREACH(t) {
             if (t == cs) {
                 continue;
             }
             run_on_cpu(t, s390_do_cpu_reset, RUN_ON_CPU_NULL);
         }
-        subsystem_reset();
         run_on_cpu(cs, s390_do_cpu_initial_reset, RUN_ON_CPU_NULL);
         run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     case S390_RESET_EXTERNAL:  
Quoted 9 lines
Quoted 3 lines
Quoted 1 line

Annotate this with the subcode as well? (in the patch introducing it)

I think this has no diag 308 subcode and is triggered by qemu

-ENOCOFFE

But even more reason to annotate this with the fact that this is triggered by QEMU :)

You're too late with that idea :)
I just split out the reordering and the annotation into a new commit:

https://github.com/frankjaa/qemu/commit/8c53d5c8a6bbcc53496c7a2877c7cbffc435b708

#46 lore raw

PATCH 8/15 s390x: protvirt: KVM intercept changes

In reply to Janosch Frank

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +6 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c
 
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);
             break;
         case ICPT_PROGRAM:
Signature · 1 line
2.20.1
#47 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

Quoted 21 lines · On Wed, 20 Nov 2019 06:43:27 -0500 Janosch Frank <[email protected]> wrote:

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +0 -0
index 418154ccfe..58251c0229 100644

target/s390x/kvm.c line 119

Quoted 5 lines
target/s390x/kvm.c +2 -0
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c

_NOTIF ?

Quoted 12 lines
target/s390x/kvm.c +1 -0
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 

target/s390x/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 342

Quoted 4 lines
target/s390x/kvm.c +1 -0
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);

You don't seem to do anything with this yet?

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 10 lines
target/s390x/kvm.c +2 -0
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);

Doesn't handle_instruction() want to know whether it got a request for emulation vs a notification?

Quoted 2 lines
target/s390x/kvm.c +0 -0
             break;
         case ICPT_PROGRAM:
#48 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

Quoted 19 lines · Cornelia Huck
Quoted 16 lines · On Wed, 20 Nov 2019 06:43:27 -0500 Janosch Frank <[email protected]> wrote:

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
Quoted 12 lines

target/s390x/kvm.c line 119

Quoted 10 lines
target/s390x/kvm.c +2 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c

_NOTIF ?

Yeah, forgot about that

Quoted 12 lines
Quoted 12 lines
target/s390x/kvm.c +1 -0
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
Quoted 7 lines

target/s390x/kvm.c int kvm_arch_init(MachineState *ms, KVMState *s) · line 342

Quoted 5 lines
target/s390x/kvm.c +1 -0
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);

You don't seem to do anything with this yet?

No, I'm still a bit in the dark about how we want to tie protvirt into qemu.

Quoted 14 lines

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 11 lines
target/s390x/kvm.c +2 -0
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);

Doesn't handle_instruction() want to know whether it got a request for emulation vs a notification?

Currently not, the sclp patch looks at the vcpu run icptcode to figure out what's going on.

Quoted 5 lines
Quoted 3 lines
target/s390x/kvm.c +0 -0
             break;
         case ICPT_PROGRAM:
#49 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

In reply to Janosch Frank
Quoted 46 lines · Janosch Frank

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +4 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c
 
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 6 lines
target/s390x/kvm.c +2 -0
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);

Even if this works by default, my gut feeling tells me that it would be safer and cleaner to have a separate handler for this... Otherwise we might get surprising results if future machine generations intercept/notify for more or different instructions, I guess?

However, it's just a gut feeling ... I really don't have much experience with this PV stuff yet ... what do the others here think?

Thomas

#50 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

Quoted 59 lines · Thomas Huth
Quoted 46 lines · Janosch Frank

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +4 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c
 
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 6 lines
target/s390x/kvm.c +2 -0
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);

Even if this works by default, my gut feeling tells me that it would be safer and cleaner to have a separate handler for this... Otherwise we might get surprising results if future machine generations intercept/notify for more or different instructions, I guess?

Quoted 4 lines

However, it's just a gut feeling ... I really don't have much experience with this PV stuff yet ... what do the others here think?

Thomas

Adding a handle_instruction_pv doesn't hurt me too much. The default case can then do an error_report() and exit(1);

PV was designed in a way that we can re-use as much code as possible, so I tried using the normal instruction handlers and only change as little as possible in the instructions themselves.

#51 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

Quoted 66 lines · On Thu, 28 Nov 2019 17:38:19 +0100 Janosch Frank <[email protected]> wrote:
Quoted 59 lines · Thomas Huth
Quoted 46 lines · Janosch Frank

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +4 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c
 
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 6 lines
target/s390x/kvm.c +2 -0
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);  

Even if this works by default, my gut feeling tells me that it would be safer and cleaner to have a separate handler for this... Otherwise we might get surprising results if future machine generations intercept/notify for more or different instructions, I guess?

Quoted 4 lines

However, it's just a gut feeling ... I really don't have much experience with this PV stuff yet ... what do the others here think?

Thomas

Quoted 6 lines

Adding a handle_instruction_pv doesn't hurt me too much. The default case can then do an error_report() and exit(1);

PV was designed in a way that we can re-use as much code as possible, so I tried using the normal instruction handlers and only change as little as possible in the instructions themselves.

I think we could argue that handling 4 and 104 in the same function makes sense; but the 108 notification should really be separate, I think. From what I've seen, the expectation of what the hypervisor needs to do is just something else in this case ("hey, I did something; just to let you know").

Is the set of instructions you get a 104 for always supposed to be a subset of the instructions you get a 4 for? I'd expect it to be so.

#52 lore raw

Re: [PATCH 08/15] s390x: protvirt: KVM intercept changes

Quoted 72 lines · Cornelia Huck
Quoted 69 lines · On Thu, 28 Nov 2019 17:38:19 +0100 Janosch Frank <[email protected]> wrote:
Quoted 59 lines · Thomas Huth
Quoted 46 lines · Janosch Frank

Secure guests no longer intercept with code 4 for an instruction interception. Instead they have codes 104 and 108 for secure instruction interception and secure instruction notification respectively.

The 104 mirrors the 4, but the 108 is a notification, that something happened and the hypervisor might need to adjust its tracking data to that fact. An example for that is the set prefix notification interception, where KVM only reads the new prefix, but does not update the prefix in the state description.

---
 target/s390x/kvm.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/s390x/kvm.c +4 -0
index 418154ccfe..58251c0229 100644
@@ -115,6 +115,8 @@
 #define ICPT_CPU_STOP                   0x28
 #define ICPT_OPEREXC                    0x2c
 #define ICPT_IO                         0x40
+#define ICPT_PV_INSTR                   0x68
+#define ICPT_PV_INSTR_NOT               0x6c
 
 #define NR_LOCAL_IRQS 32
 /*
@@ -151,6 +153,7 @@ static int cap_s390_irq;
 static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
+static int cap_protvirt;
 
 static int active_cmma;
 
@@ -336,6 +339,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF);
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
+    cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {

target/s390x/kvm.c static int handle_intercept(S390CPU *cpu) · line 1673

Quoted 6 lines
target/s390x/kvm.c +2 -0
@@ -1664,6 +1668,8 @@ static int handle_intercept(S390CPU *cpu)
             (long)cs->kvm_run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
+        case ICPT_PV_INSTR:
+        case ICPT_PV_INSTR_NOT:
             r = handle_instruction(cpu, run);  

Even if this works by default, my gut feeling tells me that it would be safer and cleaner to have a separate handler for this... Otherwise we might get surprising results if future machine generations intercept/notify for more or different instructions, I guess?

Quoted 4 lines

However, it's just a gut feeling ... I really don't have much experience with this PV stuff yet ... what do the others here think?

Thomas

Adding a handle_instruction_pv doesn't hurt me too much. The default case can then do an error_report() and exit(1);

Quoted 6 lines
Quoted 3 lines

PV was designed in a way that we can re-use as much code as possible, so I tried using the normal instruction handlers and only change as little as possible in the instructions themselves.

I think we could argue that handling 4 and 104 in the same function makes sense; but the 108 notification should really be separate, I

In my latest answer to Thomas I stated that we could move to a separate pv instruction handler. I just had another look and rediscovered, that it would mean a lot more changes. I would need to duplicate the ipa/b parsing and for diagnose even the base+disp parsing.

So yes, I'd like to treat the 104 like the 4 intercept...

Quoted 3 lines

think. From what I've seen, the expectation of what the hypervisor needs to do is just something else in this case ("hey, I did something; just to let you know").

We can remove the notification from QEMU, as far is I know, we moved the instruction that used this path to a 104 code.

Quoted 4 lines

Is the set of instructions you get a 104 for always supposed to be a subset of the instructions you get a 4 for? I'd expect it to be so.

Yes
I'll ask if we'll get a new code for instructions that are only valid in PV mode; currently there are none.

#53 lore raw

PATCH 9/15 s390x: protvirt: SCLP interpretation

In reply to Janosch Frank

SCLP for a protected guest is done over the SIDAD, so we need to use the s390_cpu_virt_mem_* functions to access the SIDAD instead of guest memory when reading/writing SCBs.

To not confuse the sclp emulation, we set 0x42000 as the address, but ignore it for reading/writing the SCCB.

---
 hw/s390x/sclp.c         | 16 ++++++++++++++++
 include/hw/s390x/sclp.h |  2 ++
 target/s390x/kvm.c      |  8 +++++++-
 3 files changed, 25 insertions(+), 1 deletion(-)
hw/s390x/sclp.c +16 -0
index f57ce7b739..00d08adc7f 100644
@@ -193,6 +193,22 @@ static void sclp_execute(SCLPDevice *sclp, SCCB *sccb, uint32_t code)
     }
 }
 
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code)
+{
+    SCLPDevice *sclp = get_sclp_device();
+    SCLPDeviceClass *sclp_c = SCLP_GET_CLASS(sclp);
+    SCCB work_sccb;
+    hwaddr sccb_len = sizeof(SCCB);
+
+    s390_cpu_virt_mem_read(env_archcpu(env), 0, 0, &work_sccb, sccb_len);
+    sclp_c->execute(sclp, &work_sccb, code);
+    s390_cpu_virt_mem_write(env_archcpu(env), 0, 0, &work_sccb,
+                            be16_to_cpu(work_sccb.h.length));
+    sclp_c->service_interrupt(sclp, sccb);
+    return 0;
+}
+
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code)
 {
     SCLPDevice *sclp = get_sclp_device();
include/hw/s390x/sclp.h +2 -0
index c54413b78c..c0a3faa37d 100644
@@ -217,5 +217,7 @@ void s390_sclp_init(void);
 void sclp_service_interrupt(uint32_t sccb);
 void raise_irq_cpu_hotplug(void);
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code);
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code);
 
 #endif
target/s390x/kvm.c +7 -1
index 58251c0229..0f2458b553 100644
@@ -1172,7 +1172,13 @@ static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run,
     sccb = env->regs[ipbh0 & 0xf];
     code = env->regs[(ipbh0 & 0xf0) >> 4];
 
-    r = sclp_service_call(env, sccb, code);
+    if (run->s390_sieic.icptcode == ICPT_PV_INSTR ||
+        run->s390_sieic.icptcode == ICPT_PV_INSTR_NOT) {
+        r = sclp_service_call_protected(env, 0x42000, code);
+    } else {
+        r = sclp_service_call(env, sccb, code);
+    }
+
     if (r < 0) {
         kvm_s390_program_interrupt(cpu, -r);
     } else {
Signature · 1 line
2.20.1
#54 lore raw

Re: [PATCH 09/15] s390x: protvirt: SCLP interpretation

Quoted 58 lines · On Wed, 20 Nov 2019 06:43:28 -0500 Janosch Frank <[email protected]> wrote:

SCLP for a protected guest is done over the SIDAD, so we need to use the s390_cpu_virt_mem_* functions to access the SIDAD instead of guest memory when reading/writing SCBs.

To not confuse the sclp emulation, we set 0x42000 as the address, but ignore it for reading/writing the SCCB.

---
 hw/s390x/sclp.c         | 16 ++++++++++++++++
 include/hw/s390x/sclp.h |  2 ++
 target/s390x/kvm.c      |  8 +++++++-
 3 files changed, 25 insertions(+), 1 deletion(-)
hw/s390x/sclp.c +16 -0
index f57ce7b739..00d08adc7f 100644
@@ -193,6 +193,22 @@ static void sclp_execute(SCLPDevice *sclp, SCCB *sccb, uint32_t code)
     }
 }
 
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code)
+{
+    SCLPDevice *sclp = get_sclp_device();
+    SCLPDeviceClass *sclp_c = SCLP_GET_CLASS(sclp);
+    SCCB work_sccb;
+    hwaddr sccb_len = sizeof(SCCB);
+
+    s390_cpu_virt_mem_read(env_archcpu(env), 0, 0, &work_sccb, sccb_len);
+    sclp_c->execute(sclp, &work_sccb, code);
+    s390_cpu_virt_mem_write(env_archcpu(env), 0, 0, &work_sccb,
+                            be16_to_cpu(work_sccb.h.length));
+    sclp_c->service_interrupt(sclp, sccb);
+    return 0;
+}
+
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code)
 {
     SCLPDevice *sclp = get_sclp_device();
include/hw/s390x/sclp.h +2 -0
index c54413b78c..c0a3faa37d 100644
@@ -217,5 +217,7 @@ void s390_sclp_init(void);
 void sclp_service_interrupt(uint32_t sccb);
 void raise_irq_cpu_hotplug(void);
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code);
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code);
 
 #endif
target/s390x/kvm.c +0 -0
index 58251c0229..0f2458b553 100644

target/s390x/kvm.c static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run, · line 1177

Quoted 7 lines
target/s390x/kvm.c +3 -1
@@ -1172,7 +1172,13 @@ static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run,
     sccb = env->regs[ipbh0 & 0xf];
     code = env->regs[(ipbh0 & 0xf0) >> 4];
 
-    r = sclp_service_call(env, sccb, code);
+    if (run->s390_sieic.icptcode == ICPT_PV_INSTR ||
+        run->s390_sieic.icptcode == ICPT_PV_INSTR_NOT) {
+        r = sclp_service_call_protected(env, 0x42000, code);

I fear that confuses the reader instead of the emulation :)

Especially as you end up passing that value to
sclp_c->service_interrupt()...

Quoted 7 lines
target/s390x/kvm.c +4 -0
+    } else {
+        r = sclp_service_call(env, sccb, code);
+    }
+
     if (r < 0) {
         kvm_s390_program_interrupt(cpu, -r);
     } else {
#55 lore raw

Re: [PATCH 09/15] s390x: protvirt: SCLP interpretation

Quoted 71 lines · Cornelia Huck
Quoted 58 lines · On Wed, 20 Nov 2019 06:43:28 -0500 Janosch Frank <[email protected]> wrote:

SCLP for a protected guest is done over the SIDAD, so we need to use the s390_cpu_virt_mem_* functions to access the SIDAD instead of guest memory when reading/writing SCBs.

To not confuse the sclp emulation, we set 0x42000 as the address, but ignore it for reading/writing the SCCB.

---
 hw/s390x/sclp.c         | 16 ++++++++++++++++
 include/hw/s390x/sclp.h |  2 ++
 target/s390x/kvm.c      |  8 +++++++-
 3 files changed, 25 insertions(+), 1 deletion(-)
hw/s390x/sclp.c +16 -0
index f57ce7b739..00d08adc7f 100644
@@ -193,6 +193,22 @@ static void sclp_execute(SCLPDevice *sclp, SCCB *sccb, uint32_t code)
     }
 }
 
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code)
+{
+    SCLPDevice *sclp = get_sclp_device();
+    SCLPDeviceClass *sclp_c = SCLP_GET_CLASS(sclp);
+    SCCB work_sccb;
+    hwaddr sccb_len = sizeof(SCCB);
+
+    s390_cpu_virt_mem_read(env_archcpu(env), 0, 0, &work_sccb, sccb_len);
+    sclp_c->execute(sclp, &work_sccb, code);
+    s390_cpu_virt_mem_write(env_archcpu(env), 0, 0, &work_sccb,
+                            be16_to_cpu(work_sccb.h.length));
+    sclp_c->service_interrupt(sclp, sccb);
+    return 0;
+}
+
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code)
 {
     SCLPDevice *sclp = get_sclp_device();
include/hw/s390x/sclp.h +2 -0
index c54413b78c..c0a3faa37d 100644
@@ -217,5 +217,7 @@ void s390_sclp_init(void);
 void sclp_service_interrupt(uint32_t sccb);
 void raise_irq_cpu_hotplug(void);
 int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code);
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code);
 
 #endif
target/s390x/kvm.c +0 -0
index 58251c0229..0f2458b553 100644

target/s390x/kvm.c static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run, · line 1177

Quoted 7 lines
target/s390x/kvm.c +3 -1
@@ -1172,7 +1172,13 @@ static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run,
     sccb = env->regs[ipbh0 & 0xf];
     code = env->regs[(ipbh0 & 0xf0) >> 4];
 
-    r = sclp_service_call(env, sccb, code);
+    if (run->s390_sieic.icptcode == ICPT_PV_INSTR ||
+        run->s390_sieic.icptcode == ICPT_PV_INSTR_NOT) {
+        r = sclp_service_call_protected(env, 0x42000, code);

I fear that confuses the reader instead of the emulation :)

Quoted 2 lines

Especially as you end up passing that value to
sclp_c->service_interrupt()...

Pierre has some more opinions on that, so I'll let him present his planned changes to this patch :)

Quoted 9 lines
Quoted 8 lines
target/s390x/kvm.c +4 -0
+    } else {
+        r = sclp_service_call(env, sccb, code);
+    }
+
     if (r < 0) {
         kvm_s390_program_interrupt(cpu, -r);
     } else {
#56 lore raw

Re: [PATCH 09/15] s390x: protvirt: SCLP interpretation

Quoted 71 lines · Janosch Frank
Quoted 70 lines · Cornelia Huck
Quoted 58 lines · On Wed, 20 Nov 2019 06:43:28 -0500 Janosch Frank <[email protected]> wrote:

SCLP for a protected guest is done over the SIDAD, so we need to use the s390_cpu_virt_mem_* functions to access the SIDAD instead of guest memory when reading/writing SCBs.

To not confuse the sclp emulation, we set 0x42000 as the address, but ignore it for reading/writing the SCCB.

---
  hw/s390x/sclp.c         | 16 ++++++++++++++++
  include/hw/s390x/sclp.h |  2 ++
  target/s390x/kvm.c      |  8 +++++++-
  3 files changed, 25 insertions(+), 1 deletion(-)
hw/s390x/sclp.c +16 -0
index f57ce7b739..00d08adc7f 100644
@@ -193,6 +193,22 @@ static void sclp_execute(SCLPDevice *sclp, SCCB *sccb, uint32_t code)
      }
  }
 
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code)
+{
+    SCLPDevice *sclp = get_sclp_device();
+    SCLPDeviceClass *sclp_c = SCLP_GET_CLASS(sclp);
+    SCCB work_sccb;
+    hwaddr sccb_len = sizeof(SCCB);
+
+    s390_cpu_virt_mem_read(env_archcpu(env), 0, 0, &work_sccb, sccb_len);
+    sclp_c->execute(sclp, &work_sccb, code);
+    s390_cpu_virt_mem_write(env_archcpu(env), 0, 0, &work_sccb,
+                            be16_to_cpu(work_sccb.h.length));
+    sclp_c->service_interrupt(sclp, sccb);
+    return 0;
+}
+
  int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code)
  {
      SCLPDevice *sclp = get_sclp_device();
include/hw/s390x/sclp.h +2 -0
index c54413b78c..c0a3faa37d 100644
@@ -217,5 +217,7 @@ void s390_sclp_init(void);
  void sclp_service_interrupt(uint32_t sccb);
  void raise_irq_cpu_hotplug(void);
  int sclp_service_call(CPUS390XState *env, uint64_t sccb, uint32_t code);
+int sclp_service_call_protected(CPUS390XState *env, uint64_t sccb,
+                                uint32_t code);
 
  #endif
target/s390x/kvm.c +0 -0
index 58251c0229..0f2458b553 100644

target/s390x/kvm.c static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run, · line 1177

Quoted 7 lines
target/s390x/kvm.c +3 -1
@@ -1172,7 +1172,13 @@ static int kvm_sclp_service_call(S390CPU *cpu, struct kvm_run *run,
      sccb = env->regs[ipbh0 & 0xf];
      code = env->regs[(ipbh0 & 0xf0) >> 4];
 
-    r = sclp_service_call(env, sccb, code);
+    if (run->s390_sieic.icptcode == ICPT_PV_INSTR ||
+        run->s390_sieic.icptcode == ICPT_PV_INSTR_NOT) {
+        r = sclp_service_call_protected(env, 0x42000, code);

I fear that confuses the reader instead of the emulation :)

Quoted 4 lines
Quoted 2 lines

Especially as you end up passing that value to
sclp_c->service_interrupt()...

Pierre has some more opinions on that, so I'll let him present his planned changes to this patch :)

Hum, sorry but I can not present any thing at the moment. I will study the problem, it will take some time.

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines
target/s390x/kvm.c +4 -0
+    } else {
+        r = sclp_service_call(env, sccb, code);
+    }
+
      if (r < 0) {
          kvm_s390_program_interrupt(cpu, -r);
      } else {
Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#57 lore raw

PATCH 10/15 s390x: protvirt: Add new VCPU reset functions

In reply to Janosch Frank

CPU resets for protected guests need to be done via Ultravisor calls. Hence we need a way to issue these calls for each reset.

As we formerly had only one reset function and it was called for initial, as well as for the clear reset, we now need a new interface.

---
 target/s390x/cpu.c       | 15 ++++++++++++---
 target/s390x/kvm-stub.c  | 10 +++++++++-
 target/s390x/kvm.c       | 38 ++++++++++++++++++++++++++++++++------
 target/s390x/kvm_s390x.h |  4 +++-
 4 files changed, 56 insertions(+), 11 deletions(-)
target/s390x/cpu.c +12 -3
index 10d5b915d8..ac04fdbba9 100644
@@ -125,9 +125,18 @@ static void s390_cpu_reset(CPUState *s, uint8_t type)
     }
 
     /* Reset state inside the kernel that we cannot access yet from QEMU. */
-    if (kvm_enabled() && (type == S390_CPU_RESET_CLEAR ||
-                          type == S390_CPU_RESET_INITIAL)) {
-            kvm_s390_reset_vcpu(cpu);
+    if (kvm_enabled()) {
+        switch (type) {
+        case S390_CPU_RESET_CLEAR:
+            kvm_s390_reset_vcpu_clear(cpu);
+            break;
+        case S390_CPU_RESET_INITIAL:
+            kvm_s390_reset_vcpu_initial(cpu);
+            break;
+        case S390_CPU_RESET_NORMAL:
+            kvm_s390_reset_vcpu_normal(cpu);
+            break;
+        }
     }
 
 #if defined(CONFIG_USER_ONLY)
target/s390x/kvm-stub.c +9 -1
index 5152e2bdf1..c4cd497f85 100644
@@ -83,7 +83,15 @@ void kvm_s390_cmma_reset(void)
 {
 }
 
-void kvm_s390_reset_vcpu(S390CPU *cpu)
+void kvm_s390_reset_vcpu_initial(S390CPU *cpu)
+{
+}
+
+void kvm_s390_reset_vcpu_clear(S390CPU *cpu)
+{
+}
+
+void kvm_s390_reset_vcpu_normal(S390CPU *cpu)
 {
 }
 
target/s390x/kvm.c +32 -6
index 0f2458b553..190400df55 100644
@@ -154,6 +154,7 @@ static int cap_ri;
 static int cap_gs;
 static int cap_hpage_1m;
 static int cap_protvirt;
+static int cap_vcpu_resets;
 
 static int active_cmma;
 
@@ -340,6 +341,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     cap_mem_op = kvm_check_extension(s, KVM_CAP_S390_MEM_OP);
     cap_s390_irq = kvm_check_extension(s, KVM_CAP_S390_INJECT_IRQ);
     cap_protvirt = kvm_check_extension(s, KVM_CAP_S390_PROTECTED);
+    cap_vcpu_resets = kvm_check_extension(s, KVM_CAP_S390_VCPU_RESETS);
 
     if (!kvm_check_extension(s, KVM_CAP_S390_GMAP)
         || !kvm_check_extension(s, KVM_CAP_S390_COW)) {
@@ -401,20 +403,44 @@ int kvm_arch_destroy_vcpu(CPUState *cs)
     return 0;
 }
 
-void kvm_s390_reset_vcpu(S390CPU *cpu)
+static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type)
 {
     CPUState *cs = CPU(cpu);
 
-    /* The initial reset call is needed here to reset in-kernel
-     * vcpu data that we can't access directly from QEMU
-     * (i.e. with older kernels which don't support sync_regs/ONE_REG).
-     * Before this ioctl cpu_synchronize_state() is called in common kvm
-     * code (kvm-all) */
+    /*
+     * The reset call is needed here to reset in-kernel vcpu data that
+     * we can't access directly from QEMU (i.e. with older kernels
+     * which don't support sync_regs/ONE_REG).  Before this ioctl
+     * cpu_synchronize_state() is called in common kvm code
+     * (kvm-all).
+     */
+    if (cap_vcpu_resets) {
+        if (kvm_vcpu_ioctl(cs, KVM_S390_VCPU_RESET, type)) {
+            error_report("CPU reset type %ld failed on CPU %i",
+                         type, cs->cpu_index);
+        }
+        return;
+    }
     if (kvm_vcpu_ioctl(cs, KVM_S390_INITIAL_RESET, NULL)) {
         error_report("Initial CPU reset failed on CPU %i", cs->cpu_index);
     }
 }
 
+void kvm_s390_reset_vcpu_initial(S390CPU *cpu)
+{
+    kvm_s390_reset_vcpu(cpu, KVM_S390_VCPU_RESET_INITIAL);
+}
+
+void kvm_s390_reset_vcpu_clear(S390CPU *cpu)
+{
+    kvm_s390_reset_vcpu(cpu, KVM_S390_VCPU_RESET_CLEAR);
+}
+
+void kvm_s390_reset_vcpu_normal(S390CPU *cpu)
+{
+    kvm_s390_reset_vcpu(cpu, KVM_S390_VCPU_RESET_NORMAL);
+}
+
 static int can_sync_regs(CPUState *cs, int regs)
 {
     return cap_sync_regs && (cs->kvm_run->kvm_valid_regs & regs) == regs;
target/s390x/kvm_s390x.h +3 -1
index caf985955b..0b21789796 100644
@@ -34,7 +34,9 @@ int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
                                     int vq, bool assign);
 int kvm_s390_cmma_active(void);
 void kvm_s390_cmma_reset(void);
-void kvm_s390_reset_vcpu(S390CPU *cpu);
+void kvm_s390_reset_vcpu_clear(S390CPU *cpu);
+void kvm_s390_reset_vcpu_normal(S390CPU *cpu);
+void kvm_s390_reset_vcpu_initial(S390CPU *cpu);
 int kvm_s390_set_mem_limit(uint64_t new_limit, uint64_t *hw_limit);
 void kvm_s390_set_max_pagesize(uint64_t pagesize, Error **errp);
 void kvm_s390_crypto_reset(void);
Signature · 1 line
2.20.1
#58 lore raw

PATCH 11/15 RFC: s390x: Exit on vcpu reset error

In reply to Janosch Frank

If a vcpu is not properly reset it might be better to just end the VM.

---
 target/s390x/kvm.c | 2 ++
 1 file changed, 2 insertions(+)
target/s390x/kvm.c +2 -0
index 190400df55..0210b54157 100644
@@ -418,11 +418,13 @@ static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type)
         if (kvm_vcpu_ioctl(cs, KVM_S390_VCPU_RESET, type)) {
             error_report("CPU reset type %ld failed on CPU %i",
                          type, cs->cpu_index);
+            exit(1);
         }
         return;
     }
     if (kvm_vcpu_ioctl(cs, KVM_S390_INITIAL_RESET, NULL)) {
         error_report("Initial CPU reset failed on CPU %i", cs->cpu_index);
+        exit(1);
     }
 }
 
Signature · 1 line
2.20.1
Reviewed-by on v1 11/15
#59 lore raw

Re: [PATCH 11/15] RFC: s390x: Exit on vcpu reset error

Quoted 16 lines · Janosch Frank

If a vcpu is not properly reset it might be better to just end the VM.

---
  target/s390x/kvm.c | 2 ++
  1 file changed, 2 insertions(+)
target/s390x/kvm.c +0 -0
index 190400df55..0210b54157 100644
@@ -418,11 +418,13 @@ static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type)
          if (kvm_vcpu_ioctl(cs, KVM_S390_VCPU_RESET, type)) {
              error_report("CPU reset type %ld failed on CPU %i",
                           type, cs->cpu_index);

target/s390x/kvm.c static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type) · line 430

Quoted 10 lines
target/s390x/kvm.c +2 -0
+            exit(1);
          }
          return;
      }
      if (kvm_vcpu_ioctl(cs, KVM_S390_INITIAL_RESET, NULL)) {
          error_report("Initial CPU reset failed on CPU %i", cs->cpu_index);
+        exit(1);
      }
  }
 

According to the comment in include/qapi/error.h

"Please don't error_setg(&error_fatal, ...), use error_report() and exit(), because that's more obvious."

This is the right thing to do.

... and it's a fairly pathological thing to happen either way.

Signature · 3 lines
Thanks,

David / dhildenb
#60 lore raw

Re: [PATCH 11/15] RFC: s390x: Exit on vcpu reset error

Quoted 30 lines · David Hildenbrand
Quoted 16 lines · Janosch Frank

If a vcpu is not properly reset it might be better to just end the VM.

---
  target/s390x/kvm.c | 2 ++
  1 file changed, 2 insertions(+)
target/s390x/kvm.c +0 -0
index 190400df55..0210b54157 100644
@@ -418,11 +418,13 @@ static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type)
          if (kvm_vcpu_ioctl(cs, KVM_S390_VCPU_RESET, type)) {
              error_report("CPU reset type %ld failed on CPU %i",
                           type, cs->cpu_index);

target/s390x/kvm.c static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type) · line 430

Quoted 10 lines
target/s390x/kvm.c +2 -0
+            exit(1);
          }
          return;
      }
      if (kvm_vcpu_ioctl(cs, KVM_S390_INITIAL_RESET, NULL)) {
          error_report("Initial CPU reset failed on CPU %i", cs->cpu_index);
+        exit(1);
      }
  }
 

According to the comment in include/qapi/error.h

Quoted 9 lines

"Please don't error_setg(&error_fatal, ...), use error_report() and exit(), because that's more obvious."

This is the right thing to do.

... and it's a fairly pathological thing to happen either way.

Do we want to have that separate or should I squash it into the reset changes?

#61 lore raw

Re: [PATCH 11/15] RFC: s390x: Exit on vcpu reset error

Quoted 36 lines · Janosch Frank
Quoted 35 lines · David Hildenbrand
Quoted 16 lines · Janosch Frank

If a vcpu is not properly reset it might be better to just end the VM.

---
   target/s390x/kvm.c | 2 ++
   1 file changed, 2 insertions(+)
target/s390x/kvm.c +0 -0
index 190400df55..0210b54157 100644
@@ -418,11 +418,13 @@ static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type)
           if (kvm_vcpu_ioctl(cs, KVM_S390_VCPU_RESET, type)) {
               error_report("CPU reset type %ld failed on CPU %i",
                            type, cs->cpu_index);

target/s390x/kvm.c static void kvm_s390_reset_vcpu(S390CPU *cpu, unsigned long type) · line 430

Quoted 10 lines
target/s390x/kvm.c +2 -0
+            exit(1);
           }
           return;
       }
       if (kvm_vcpu_ioctl(cs, KVM_S390_INITIAL_RESET, NULL)) {
           error_report("Initial CPU reset failed on CPU %i", cs->cpu_index);
+        exit(1);
       }
   }
 

According to the comment in include/qapi/error.h

"Please don't error_setg(&error_fatal, ...), use error_report() and exit(), because that's more obvious."

This is the right thing to do.

Quoted 9 lines
Quoted 4 lines

... and it's a fairly pathological thing to happen either way.

Do we want to have that separate or should I squash it into the reset changes?

I' keep it separated.

Signature · 3 lines
Thanks,

David / dhildenb
#62 lore raw

PATCH 12/15 s390x: protvirt: Set guest IPL PSW

In reply to Janosch Frank

Handling of CPU reset and setting of the IPL psw from guest storage at offset 0 is done by a Ultravisor call. Let's only fetch it if necessary.

---
 hw/s390x/pv.c              | 5 +++++
 hw/s390x/pv.h              | 1 +
 hw/s390x/s390-virtio-ccw.c | 3 ++-
 linux-headers/linux/kvm.h  | 1 +
 target/s390x/cpu.c         | 9 ++++++++-
 5 files changed, 17 insertions(+), 2 deletions(-)
hw/s390x/pv.c +5 -0
index 0218070322..106252833f 100644
@@ -88,6 +88,11 @@ int s390_pv_set_sec_parms(uint64_t origin, uint64_t length)
     return s390_pv_cmd(KVM_PV_VM_SET_SEC_PARMS, &args);
 }
 
+int s390_pv_set_ipl_psw(CPUState *cs)
+{
+    return s390_pv_cmd_vcpu(cs, KVM_PV_VCPU_SET_IPL_PSW, NULL);
+}
+
 /*
  * Called for each component in the SE type IPL parameter block 0.
  */
hw/s390x/pv.h +1 -0
index eb074e4bc9..e670c67270 100644
@@ -18,6 +18,7 @@ int s390_pv_vm_destroy(void);
 int s390_pv_vcpu_destroy(CPUState *cs);
 int s390_pv_vcpu_create(CPUState *cs);
 int s390_pv_set_sec_parms(uint64_t origin, uint64_t length);
+int s390_pv_set_ipl_psw(CPUState *cs);
 int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak);
 int s390_pv_perf_clear_reset(void);
 int s390_pv_verify(void);
hw/s390x/s390-virtio-ccw.c +2 -1
index 6fd50b4c42..e020b92854 100644
@@ -404,7 +404,8 @@ static void s390_machine_reset(MachineState *machine)
         s390_ipl_pv_unpack();
         /* Verify integrity */
         s390_pv_verify();
-        s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
+        env->pv = true;
+        run_on_cpu(cs, s390_do_cpu_load_normal, RUN_ON_CPU_NULL);
         break;
     default:
         g_assert_not_reached();
linux-headers/linux/kvm.h +1 -0
index d031051601..b5c7552016 100644
@@ -1476,6 +1476,7 @@ enum pv_cmd_id {
 	KVM_PV_VM_UNSHARE,
 	KVM_PV_VCPU_CREATE,
 	KVM_PV_VCPU_DESTROY,
+	KVM_PV_VCPU_SET_IPL_PSW,
 };
 
 struct kvm_pv_cmd {
target/s390x/cpu.c +8 -1
index ac04fdbba9..c004987a87 100644
@@ -37,6 +37,7 @@
 #include "sysemu/hw_accel.h"
 #include "hw/qdev-properties.h"
 #ifndef CONFIG_USER_ONLY
+#include "hw/s390x/pv.h"
 #include "hw/boards.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/sysemu.h"
@@ -76,7 +77,13 @@ static bool s390_cpu_has_work(CPUState *cs)
 static void s390_cpu_load_normal(CPUState *s)
 {
     S390CPU *cpu = S390_CPU(s);
-    cpu->env.psw.addr = ldl_phys(s->as, 4) & PSW_MASK_ESA_ADDR;
+    CPUS390XState *env = &cpu->env;
+
+    if (!env->pv) {
+        cpu->env.psw.addr = ldl_phys(s->as, 4) & PSW_MASK_ESA_ADDR;
+    } else {
+        s390_pv_set_ipl_psw(s);
+    }
     cpu->env.psw.mask = PSW_MASK_32 | PSW_MASK_64;
     s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 12/15
#63 lore raw

Re: [PATCH 12/15] s390x: protvirt: Set guest IPL PSW

Quoted 12 lines · Janosch Frank

Handling of CPU reset and setting of the IPL psw from guest storage at offset 0 is done by a Ultravisor call. Let's only fetch it if necessary.

---
 hw/s390x/pv.c              | 5 +++++
 hw/s390x/pv.h              | 1 +
 hw/s390x/s390-virtio-ccw.c | 3 ++-
 linux-headers/linux/kvm.h  | 1 +
 target/s390x/cpu.c         | 9 ++++++++-
 5 files changed, 17 insertions(+), 2 deletions(-)

[...]

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 407

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +1 -1
index 6fd50b4c42..e020b92854 100644
@@ -404,7 +404,8 @@ static void s390_machine_reset(MachineState *machine)
         s390_ipl_pv_unpack();
         /* Verify integrity */
         s390_pv_verify();
-        s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
+        env->pv = true;

Ah, here you set env->pv = true manually ... so maybe that's another good reason to get rid of patch 05/15 ("s390x: protvirt: Sync PV state") in the end?

#64 lore raw

Re: [PATCH 12/15] s390x: protvirt: Set guest IPL PSW

Quoted 14 lines · Thomas Huth
Quoted 12 lines · Janosch Frank

Handling of CPU reset and setting of the IPL psw from guest storage at offset 0 is done by a Ultravisor call. Let's only fetch it if necessary.

---
 hw/s390x/pv.c              | 5 +++++
 hw/s390x/pv.h              | 1 +
 hw/s390x/s390-virtio-ccw.c | 3 ++-
 linux-headers/linux/kvm.h  | 1 +
 target/s390x/cpu.c         | 9 ++++++++-
 5 files changed, 17 insertions(+), 2 deletions(-)

[...]

Quoted 14 lines

hw/s390x/s390-virtio-ccw.c static void s390_machine_reset(MachineState *machine) · line 407

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +1 -1
index 6fd50b4c42..e020b92854 100644
@@ -404,7 +404,8 @@ static void s390_machine_reset(MachineState *machine)
         s390_ipl_pv_unpack();
         /* Verify integrity */
         s390_pv_verify();
-        s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
+        env->pv = true;

Ah, here you set env->pv = true manually ... so maybe that's another good reason to get rid of patch 05/15 ("s390x: protvirt: Sync PV state") in the end?

Given that QEMU triggers the state changes that would make sense.

Quoted 2 lines

Thanks!

Quoted 2 lines
#65 lore raw

PATCH 13/15 s390x: protvirt: Move diag 308 data over SIDAD

In reply to Janosch Frank

For protected guests the IPIB is written/read to/from the sattelite block, so we need to make those accesses virtual to make them go through KBM mem ops.

---
 target/s390x/diag.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)
target/s390x/diag.c +23 -3
index db6d79cef3..d96d8bdc6c 100644
@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
+    S390CPU *cpu = S390_CPU(cs);
     uint64_t addr =  env->regs[r1];
     uint64_t subcode = env->regs[r3];
     IplParameterBlock *iplb;
@@ -118,14 +119,27 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
+
         iplb = g_new0(IplParameterBlock, 1);
-        cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, sizeof(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);
+        }
+
         if (!iplb_valid_len(iplb)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
 
-        cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, be32_to_cpu(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);
+        }
+
 
         if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
             !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
@@ -149,7 +163,13 @@ out:
             iplb = s390_ipl_get_iplb();
         }
         if (iplb) {
-            cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
+            if (!env->pv) {
+                cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
+            } else {
+                s390_cpu_virt_mem_write(cpu, 0, 0, iplb,
+                                        be32_to_cpu(iplb->len));
+                s390_cpu_virt_mem_handle_exc(cpu, ra);
+            }
             env->regs[r1 + 1] = DIAG_308_RC_OK;
         } else {
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;
Signature · 1 line
2.20.1
#66 lore raw

Re: [PATCH 13/15] s390x: protvirt: Move diag 308 data over SIDAD

Quoted 1 line · Janosch Frank

For protected guests the IPIB is written/read to/from the sattelite

satellite

Quoted 2 lines

block, so we need to make those accesses virtual to make them go through KBM mem ops.

What's KBM ?

Quoted 19 lines
---
 target/s390x/diag.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)
target/s390x/diag.c +1 -0
index db6d79cef3..d96d8bdc6c 100644
@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
+    S390CPU *cpu = S390_CPU(cs);
     uint64_t addr =  env->regs[r1];
     uint64_t subcode = env->regs[r3];
     IplParameterBlock *iplb;
@@ -118,14 +119,27 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         if (diag308_parm_check(env, r1, addr, ra, false)) {

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 128

Quoted 10 lines
target/s390x/diag.c +6 -1
             return;
         }
+
         iplb = g_new0(IplParameterBlock, 1);
-        cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, sizeof(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);

I'm looking forward to protected virt support in TCG ;-)

Quoted 24 lines
target/s390x/diag.c +9 -2
+        }
+
         if (!iplb_valid_len(iplb)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
 
-        cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, be32_to_cpu(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);
+        }
+
 
         if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
             !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
@@ -149,7 +163,13 @@ out:
             iplb = s390_ipl_get_iplb();
         }
         if (iplb) {
-            cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));

target/s390x/diag.c out: · line 175

Quoted 10 lines
target/s390x/diag.c +7 -0
+            if (!env->pv) {
+                cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
+            } else {
+                s390_cpu_virt_mem_write(cpu, 0, 0, iplb,
+                                        be32_to_cpu(iplb->len));
+                s390_cpu_virt_mem_handle_exc(cpu, ra);
+            }
             env->regs[r1 + 1] = DIAG_308_RC_OK;
         } else {
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;

I wonder whether we maybe want to have some wrapper functions for these reads and writes, something like:

void s390_cpu_physical_memory_write(...)

{
    if (!env->pv) {
        cpu_physical_memory_write(...);
    } else {
        s390_cpu_virt_mem_write(...);
    }
}

?

Thomas

#67 lore raw

Re: [PATCH 13/15] s390x: protvirt: Move diag 308 data over SIDAD

Quoted 9 lines · Thomas Huth
Quoted 1 line · Janosch Frank

For protected guests the IPIB is written/read to/from the sattelite

satellite

Quoted 2 lines

block, so we need to make those accesses virtual to make them go through KBM mem ops.

What's KBM ?

A totally new hypervisor that's much faster than KVM because b comes before v in the alphabet.

Quoted 24 lines
Quoted 24 lines
---
 target/s390x/diag.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)
target/s390x/diag.c +2 -0
index db6d79cef3..d96d8bdc6c 100644
@@ -88,6 +88,7 @@ static int diag308_parm_check(CPUS390XState *env, uint64_t r1, uint64_t addr,
 void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
 {
     CPUState *cs = env_cpu(env);
+    S390CPU *cpu = S390_CPU(cs);
     uint64_t addr =  env->regs[r1];
     uint64_t subcode = env->regs[r3];
     IplParameterBlock *iplb;
@@ -118,14 +119,27 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra)
         if (diag308_parm_check(env, r1, addr, ra, false)) {
             return;
         }
+
         iplb = g_new0(IplParameterBlock, 1);
Quoted 8 lines

target/s390x/diag.c void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3, uintptr_t ra) · line 128

Quoted 6 lines
target/s390x/diag.c +5 -1
-        cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, sizeof(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, sizeof(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);

I'm looking forward to protected virt support in TCG ;-)

Who doesn't?

Quoted 30 lines
Quoted 30 lines
target/s390x/diag.c +15 -2
+        }
+
         if (!iplb_valid_len(iplb)) {
             env->regs[r1 + 1] = DIAG_308_RC_INVALID;
             goto out;
         }
 
-        cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        if (!env->pv) {
+            cpu_physical_memory_read(addr, iplb, be32_to_cpu(iplb->len));
+        } else {
+            s390_cpu_virt_mem_read(cpu, 0, 0, iplb, be32_to_cpu(iplb->len));
+            s390_cpu_virt_mem_handle_exc(cpu, ra);
+        }
+
 
         if (!iplb_valid_ccw(iplb) && !iplb_valid_fcp(iplb) &&
             !(iplb_valid_se(iplb) && s390_ipl_pv_check_comp(iplb) >= 0)) {
@@ -149,7 +163,13 @@ out:
             iplb = s390_ipl_get_iplb();
         }
         if (iplb) {
-            cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
+            if (!env->pv) {
+                cpu_physical_memory_write(addr, iplb, be32_to_cpu(iplb->len));
+            } else {
+                s390_cpu_virt_mem_write(cpu, 0, 0, iplb,
+                                        be32_to_cpu(iplb->len));
+                s390_cpu_virt_mem_handle_exc(cpu, ra);
Quoted 8 lines

target/s390x/diag.c out: · line 175

Quoted 5 lines
target/s390x/diag.c +1 -0
+            }
             env->regs[r1 + 1] = DIAG_308_RC_OK;
         } else {
             env->regs[r1 + 1] = DIAG_308_RC_NO_CONF;

I wonder whether we maybe want to have some wrapper functions for these reads and writes, something like:

diag308 is currently the only instruction where we would need it.

sclp needs a new handler, because we can skip lots of checks if pv is enabled.
IO instructions are already using logical addressing

Quoted 14 lines

void s390_cpu_physical_memory_write(...)

{
    if (!env->pv) {
        cpu_physical_memory_write(...);
    } else {
        s390_cpu_virt_mem_write(...);
    }
}

?

Thomas

#68 lore raw

Re: [PATCH 13/15] s390x: protvirt: Move diag 308 data over SIDAD

Quoted 6 lines · Janosch Frank
Quoted 5 lines · Thomas Huth
Quoted 1 line · Janosch Frank

For protected guests the IPIB is written/read to/from the sattelite

satellite

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

block, so we need to make those accesses virtual to make them go through KBM mem ops.

What's KBM ?

A totally new hypervisor that's much faster than KVM because b comes before v in the alphabet.

Sounds more lime IBM is slowly taking over KVM :D

Signature · 3 lines
Thanks,

David / dhildenb
#69 lore raw

PATCH 14/15 s390x: protvirt: Disable address checks for PV guest IO emulation

In reply to Janosch Frank

IO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.

---
 target/s390x/ioinst.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)
target/s390x/ioinst.c +29 -17
index c437a1d8c6..d3bd422ddd 100644
@@ -110,11 +110,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     SCHIB schib;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -167,11 +169,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     ORB orig_orb, orb;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -198,12 +202,14 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
 void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     CRW crw;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -228,13 +234,15 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
 {
     int cssid, ssid, schid, m;
     SubchDev *sch;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     SCHIB schib;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -294,16 +302,18 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     IRB irb;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc, irb_len;
-    uint8_t ar;
+    uint8_t ar = 0;
 
     if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid)) {
         s390_program_interrupt(env, PGM_OPERAND, ra);
         return -EIO;
     }
     trace_ioinst_sch_id("tsch", cssid, ssid, schid);
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EIO;
@@ -601,7 +611,7 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     ChscReq *req;
     ChscResp *res;
-    uint64_t addr;
+    uint64_t addr = 0;
     int reg;
     uint16_t len;
     uint16_t command;
@@ -610,7 +620,9 @@ void ioinst_handle_chsc(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 
     trace_ioinst("chsc");
     reg = (ipb >> 20) & 0x00f;
-    addr = env->regs[reg];
+    if (!env->pv) {
+        addr = env->regs[reg];
+    }
     /* Page boundary? */
     if (addr & 0xfff) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
Signature · 1 line
2.20.1
Reviewed-by on v1 14/15
#70 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

Quoted 95 lines · Janosch Frank

IO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.

---
 target/s390x/ioinst.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)
target/s390x/ioinst.c +22 -14
index c437a1d8c6..d3bd422ddd 100644
@@ -110,11 +110,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     SCHIB schib;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -167,11 +169,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     ORB orig_orb, orb;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -198,12 +202,14 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
 void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     CRW crw;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -228,13 +234,15 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
 {
     int cssid, ssid, schid, m;
     SubchDev *sch;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     SCHIB schib;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -294,16 +302,18 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     IRB irb;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc, irb_len;
-    uint8_t ar;
+    uint8_t ar = 0;
 
     if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid)) {
         s390_program_interrupt(env, PGM_OPERAND, ra);

target/s390x/ioinst.c int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra) · line 319

Quoted 10 lines
target/s390x/ioinst.c +3 -1
         return -EIO;
     }
     trace_ioinst_sch_id("tsch", cssid, ssid, schid);
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EIO;

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

Anyway,

#71 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

Quoted 103 lines · Thomas Huth
Quoted 102 lines · Janosch Frank

IO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.

---
 target/s390x/ioinst.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)
target/s390x/ioinst.c +25 -15
index c437a1d8c6..d3bd422ddd 100644
@@ -110,11 +110,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     SCHIB schib;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -167,11 +169,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     ORB orig_orb, orb;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -198,12 +202,14 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
 void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     CRW crw;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -228,13 +234,15 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
 {
     int cssid, ssid, schid, m;
     SubchDev *sch;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     SCHIB schib;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -294,16 +302,18 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     IRB irb;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc, irb_len;
-    uint8_t ar;
+    uint8_t ar = 0;
 
     if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid)) {
         s390_program_interrupt(env, PGM_OPERAND, ra);
         return -EIO;
     }
     trace_ioinst_sch_id("tsch", cssid, ssid, schid);
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
Quoted 8 lines

target/s390x/ioinst.c int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra) · line 319

Quoted 3 lines
target/s390x/ioinst.c +0 -0
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EIO;

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

I'd like to keep decode_basedisp_s() as is, but how about a static function in ioinst.c called something like get_address_from_regs()?

It'll call decode_basedisp_s() or return 0.

Quoted 4 lines

Anyway,

Thanks!

#72 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

Quoted 11 lines · On Thu, 28 Nov 2019 17:10:38 +0100 Janosch Frank <[email protected]> wrote:
Quoted 4 lines · Thomas Huth

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

I'd like to keep decode_basedisp_s() as is, but how about a static function in ioinst.c called something like get_address_from_regs()?

It'll call decode_basedisp_s() or return 0.

We could do something like that; but do we ever get there for other instruction formats as well? It feels a bit odd to single out this one.

#73 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

Quoted 10 lines · Cornelia Huck
Quoted 7 lines · On Thu, 28 Nov 2019 17:10:38 +0100 Janosch Frank <[email protected]> wrote:
Quoted 4 lines · Thomas Huth

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

Quoted 8 lines
Quoted 4 lines

I'd like to keep decode_basedisp_s() as is, but how about a static function in ioinst.c called something like get_address_from_regs()?

It'll call decode_basedisp_s() or return 0.

We could do something like that; but do we ever get there for other instruction formats as well? It feels a bit odd to single out this one.

sclp is rre (register address) and diag308 is rs-a (r1 is the address).

#74 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

In reply to Janosch Frank
Quoted 113 lines · Janosch Frank
Quoted 103 lines · Thomas Huth
Quoted 102 lines · Janosch Frank

IO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.

---
 target/s390x/ioinst.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)
target/s390x/ioinst.c +25 -15
index c437a1d8c6..d3bd422ddd 100644
@@ -110,11 +110,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     SCHIB schib;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -167,11 +169,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     ORB orig_orb, orb;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -198,12 +202,14 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
 void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     CRW crw;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -228,13 +234,15 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
 {
     int cssid, ssid, schid, m;
     SubchDev *sch;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     SCHIB schib;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -294,16 +302,18 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     IRB irb;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc, irb_len;
-    uint8_t ar;
+    uint8_t ar = 0;
 
     if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid)) {
         s390_program_interrupt(env, PGM_OPERAND, ra);
         return -EIO;
     }
     trace_ioinst_sch_id("tsch", cssid, ssid, schid);
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
Quoted 8 lines

target/s390x/ioinst.c int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra) · line 319

Quoted 3 lines
target/s390x/ioinst.c +0 -0
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EIO;

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

Quoted 4 lines

I'd like to keep decode_basedisp_s() as is, but how about a static function in ioinst.c called something like get_address_from_regs()?

It'll call decode_basedisp_s() or return 0.

Sounds fine for me, too!

Thomas

#75 lore raw

Re: [PATCH 14/15] s390x: protvirt: Disable address checks for PV guest IO emulation

In reply to Thomas Huth
Quoted 103 lines · Thomas Huth
Quoted 102 lines · Janosch Frank

IO instruction data is routed through SIDAD for protected guests, so adresses do not need to be checked, as this is kernel memory.

---
 target/s390x/ioinst.c | 46 +++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 17 deletions(-)
target/s390x/ioinst.c +25 -15
index c437a1d8c6..d3bd422ddd 100644
@@ -110,11 +110,13 @@ void ioinst_handle_msch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     SCHIB schib;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -167,11 +169,13 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     ORB orig_orb, orb;
-    uint64_t addr;
+    uint64_t addr = 0;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -198,12 +202,14 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
 void ioinst_handle_stcrw(S390CPU *cpu, uint32_t ipb, uintptr_t ra)
 {
     CRW crw;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -228,13 +234,15 @@ void ioinst_handle_stsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb,
 {
     int cssid, ssid, schid, m;
     SubchDev *sch;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc;
     SCHIB schib;
     CPUS390XState *env = &cpu->env;
-    uint8_t ar;
+    uint8_t ar = 0;
 
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return;
@@ -294,16 +302,18 @@ int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra)
     int cssid, ssid, schid, m;
     SubchDev *sch;
     IRB irb;
-    uint64_t addr;
+    uint64_t addr = 0;
     int cc, irb_len;
-    uint8_t ar;
+    uint8_t ar = 0;
 
     if (ioinst_disassemble_sch_ident(reg1, &m, &cssid, &ssid, &schid)) {
         s390_program_interrupt(env, PGM_OPERAND, ra);
         return -EIO;
     }
     trace_ioinst_sch_id("tsch", cssid, ssid, schid);
-    addr = decode_basedisp_s(env, ipb, &ar);
+    if (!env->pv) {
+        addr = decode_basedisp_s(env, ipb, &ar);
+    }
Quoted 8 lines

target/s390x/ioinst.c int ioinst_handle_tsch(S390CPU *cpu, uint64_t reg1, uint32_t ipb, uintptr_t ra) · line 319

Quoted 3 lines
target/s390x/ioinst.c +0 -0
     if (addr & 3) {
         s390_program_interrupt(env, PGM_SPECIFICATION, ra);
         return -EIO;

Would it make sense to hide all these changes in decode_basedisp_s() instead? ... so that decode_basedisp_s() returns 0 if env->pv == true ? ... or are there still cases where we need real values from decode_basedisp_s() in case of env->pv==true?

Pierre already suggested that, I'll look into it. Hopefully we have no addr != 0 or addr > 2 * PAGE_SIZE checks. Because of that it might make more sense to just rip out the checks.

Quoted 4 lines

Anyway,

#76 lore raw

PATCH 15/15 s390x: protvirt: Handle SIGP store status correctly

In reply to Janosch Frank

Status storing is obviously not done by qemu anymore.

---
 target/s390x/sigp.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
target/s390x/sigp.c +5 -2
index 2ce22d4dc1..68634d694a 100644
@@ -144,7 +144,9 @@ static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg)
     case S390_CPU_STATE_STOPPED:
         /* already stopped, just store the status */
         cpu_synchronize_state(cs);
-        s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        if (!cpu->env.pv) {
+            s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        }
         break;
     }
     si->cc = SIGP_CC_ORDER_CODE_ACCEPTED;
@@ -497,7 +499,8 @@ void do_stop_interrupt(CPUS390XState *env)
     if (s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu) == 0) {
         qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
     }
-    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS) {
+    /* Storing will occur on next SIE entry for fmt 4 */
+    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS && !env->pv) {
         s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
     }
     env->sigp_order = 0;
Signature · 1 line
2.20.1
#77 lore raw

Re: [PATCH 15/15] s390x: protvirt: Handle SIGP store status correctly

Quoted 12 lines · Janosch Frank

Status storing is obviously not done by qemu anymore.

---
  target/s390x/sigp.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
target/s390x/sigp.c +0 -0
index 2ce22d4dc1..68634d694a 100644

target/s390x/sigp.c static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg) · line 149

Quoted 7 lines
target/s390x/sigp.c +3 -1
@@ -144,7 +144,9 @@ static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg)
      case S390_CPU_STATE_STOPPED:
          /* already stopped, just store the status */
          cpu_synchronize_state(cs);
-        s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        if (!cpu->env.pv) {
+            s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        }

Confused, how is that case handled? The CPU is already stopped, so it won't be run again (consequently, next SIE entry can't store it). Who will store the status?

Quoted 14 lines
target/s390x/sigp.c +2 -1
          break;
      }
      si->cc = SIGP_CC_ORDER_CODE_ACCEPTED;
@@ -497,7 +499,8 @@ void do_stop_interrupt(CPUS390XState *env)
      if (s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu) == 0) {
          qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
      }
-    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS) {
+    /* Storing will occur on next SIE entry for fmt 4 */
+    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS && !env->pv) {
          s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
      }
      env->sigp_order = 0;
Signature · 3 lines
Thanks,

David / dhildenb
#78 lore raw

Re: [PATCH 15/15] s390x: protvirt: Handle SIGP store status correctly

Quoted 8 lines · David Hildenbrand
Quoted 7 lines · Janosch Frank

Status storing is obviously not done by qemu anymore.

---
  target/s390x/sigp.c | 7 +++++--
  1 file changed, 5 insertions(+), 2 deletions(-)
Quoted 16 lines

target/s390x/sigp.c static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg) · line 149

Quoted 12 lines
target/s390x/sigp.c +3 -1
index 2ce22d4dc1..68634d694a 100644
@@ -144,7 +144,9 @@ static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg)
      case S390_CPU_STATE_STOPPED:
          /* already stopped, just store the status */
          cpu_synchronize_state(cs);
-        s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        if (!cpu->env.pv) {
+            s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        }

Confused, how is that case handled? The CPU is already stopped, so it won't be run again (consequently, next SIE entry can't store it). Who will store the status?

Firmware does some magic

Quoted 17 lines
Quoted 15 lines
target/s390x/sigp.c +2 -1
          break;
      }
      si->cc = SIGP_CC_ORDER_CODE_ACCEPTED;
@@ -497,7 +499,8 @@ void do_stop_interrupt(CPUS390XState *env)
      if (s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu) == 0) {
          qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
      }
-    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS) {
+    /* Storing will occur on next SIE entry for fmt 4 */
+    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS && !env->pv) {
          s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
      }
      env->sigp_order = 0;
Reviewed-by on v1 15/15
#79 lore raw

Re: [PATCH 15/15] s390x: protvirt: Handle SIGP store status correctly

In reply to Janosch Frank
Quoted 24 lines · Janosch Frank

Status storing is obviously not done by qemu anymore.

---
 target/s390x/sigp.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
target/s390x/sigp.c +3 -1
index 2ce22d4dc1..68634d694a 100644
@@ -144,7 +144,9 @@ static void sigp_stop_and_store_status(CPUState *cs, run_on_cpu_data arg)
     case S390_CPU_STATE_STOPPED:
         /* already stopped, just store the status */
         cpu_synchronize_state(cs);
-        s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        if (!cpu->env.pv) {
+            s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
+        }
         break;
     }
     si->cc = SIGP_CC_ORDER_CODE_ACCEPTED;

target/s390x/sigp.c void do_stop_interrupt(CPUS390XState *env) · line 506

Quoted 9 lines
target/s390x/sigp.c +2 -1
@@ -497,7 +499,8 @@ void do_stop_interrupt(CPUS390XState *env)
     if (s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu) == 0) {
         qemu_system_shutdown_request(SHUTDOWN_CAUSE_GUEST_SHUTDOWN);
     }
-    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS) {
+    /* Storing will occur on next SIE entry for fmt 4 */
+    if (cpu->env.sigp_order == SIGP_STOP_STORE_STATUS && !env->pv) {
         s390_store_status(cpu, S390_STORE_STATUS_DEF_ADDR, true);
     }
     env->sigp_order = 0;

Maybe move the check into the s390_store_status() function instead?

Anyway,

#80 lore raw
In reply to Janosch Frank

On Wed, 20 Nov 2019 06:43:19 -0500
Janosch Frank <[email protected]> wrote:

Do you have a branch with this somewhere?

Quoted 11 lines

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Any other cleanups than in the first two patches? I.e., anything that could be picked up independently?

Quoted 41 lines

Janosch Frank (15):
s390x: Cleanup cpu resets
s390x: Beautify diag308 handling
s390x: protvirt: Add diag308 subcodes 8 - 10
Header sync protvirt
s390x: protvirt: Sync PV state
s390x: protvirt: Support unpack facility
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390x: protvirt: SCLP interpretation
s390x: protvirt: Add new VCPU reset functions
RFC: s390x: Exit on vcpu reset error
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Handle SIGP store status correctly

 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  81 +++++++++++++++++-
 hw/s390x/ipl.h                      |  35 ++++++++
 hw/s390x/pv.c                       | 123 +++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  27 ++++++
 hw/s390x/s390-virtio-ccw.c          |  79 ++++++++++++++---
 hw/s390x/sclp.c                     |  16 ++++
 include/hw/s390x/sclp.h             |   2 +
 linux-headers/asm-s390/kvm.h        |   4 +-
 linux-headers/linux/kvm.h           |  43 ++++++++++
 target/s390x/cpu.c                  | 127 ++++++++++++++--------------
 target/s390x/cpu.h                  |   1 +
 target/s390x/cpu_features_def.inc.h |   1 +
 target/s390x/diag.c                 | 108 +++++++++++++++++------
 target/s390x/ioinst.c               |  46 ++++++----
 target/s390x/kvm-stub.c             |  10 ++-
 target/s390x/kvm.c                  |  58 +++++++++++--
 target/s390x/kvm_s390x.h            |   4 +-
 target/s390x/sigp.c                 |   7 +-
 19 files changed, 640 insertions(+), 133 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
#81 lore raw
Quoted 13 lines · Cornelia Huck

On Wed, 20 Nov 2019 06:43:19 -0500
Janosch Frank <[email protected]> wrote:

Do you have a branch with this somewhere?

Quoted 8 lines

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

Quoted 6 lines
Quoted 3 lines

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Any other cleanups than in the first two patches? I.e., anything that could be picked up independently?

Maybe patch #11, but that's RFC

Quoted 44 lines
Quoted 42 lines

Janosch Frank (15):
s390x: Cleanup cpu resets
s390x: Beautify diag308 handling
s390x: protvirt: Add diag308 subcodes 8 - 10
Header sync protvirt
s390x: protvirt: Sync PV state
s390x: protvirt: Support unpack facility
s390x: protvirt: Handle diag 308 subcodes 0,1,3,4
s390x: protvirt: KVM intercept changes
s390x: protvirt: SCLP interpretation
s390x: protvirt: Add new VCPU reset functions
RFC: s390x: Exit on vcpu reset error
s390x: protvirt: Set guest IPL PSW
s390x: protvirt: Move diag 308 data over SIDAD
s390x: protvirt: Disable address checks for PV guest IO emulation
s390x: protvirt: Handle SIGP store status correctly

 hw/s390x/Makefile.objs              |   1 +
 hw/s390x/ipl.c                      |  81 +++++++++++++++++-
 hw/s390x/ipl.h                      |  35 ++++++++
 hw/s390x/pv.c                       | 123 +++++++++++++++++++++++++++
 hw/s390x/pv.h                       |  27 ++++++
 hw/s390x/s390-virtio-ccw.c          |  79 ++++++++++++++---
 hw/s390x/sclp.c                     |  16 ++++
 include/hw/s390x/sclp.h             |   2 +
 linux-headers/asm-s390/kvm.h        |   4 +-
 linux-headers/linux/kvm.h           |  43 ++++++++++
 target/s390x/cpu.c                  | 127 ++++++++++++++--------------
 target/s390x/cpu.h                  |   1 +
 target/s390x/cpu_features_def.inc.h |   1 +
 target/s390x/diag.c                 | 108 +++++++++++++++++------
 target/s390x/ioinst.c               |  46 ++++++----
 target/s390x/kvm-stub.c             |  10 ++-
 target/s390x/kvm.c                  |  58 +++++++++++--
 target/s390x/kvm_s390x.h            |   4 +-
 target/s390x/sigp.c                 |   7 +-
 19 files changed, 640 insertions(+), 133 deletions(-)
 create mode 100644 hw/s390x/pv.c
 create mode 100644 hw/s390x/pv.h
#84 lore raw
In reply to Janosch Frank
Quoted 11 lines · Janosch Frank

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Can you give some guidance on how management applications including libvirt & layers above (oVirt, OpenStack, etc) would/should use this feature ? What new command line / monitor calls are needed, and what feature restrictions are there on its use ?

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#85 lore raw
Quoted 13 lines · Daniel P. Berrangé
Quoted 11 lines · Janosch Frank

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Quoted 8 lines

Can you give some guidance on how management applications including libvirt & layers above (oVirt, OpenStack, etc) would/should use this feature ? What new command line / monitor calls are needed, and what feature restrictions are there on its use ?

Regards,
Daniel

Hey Daniel,

management applications generally do not need to know about this feature. Most of the magic is in the guest image, which boots up in a certain way to become a protected machine.

The requirements for that to happen are:
* Machine/firmware support
* KVM & QEMU support
* IO only with iommu
* Guest needs to use IO bounce buffers
* A kernel image or a kernel on a disk that was prepared with special tooling

Such VMs are started like any other VM and run a short "normal" stub that will prepare some things and then requests to be protected.

Most of the restrictions are memory related and might be lifted in the future:
* No paging
* No migration
* No huge page backings
* No collaborative memory management

There are no monitor changes or cmd additions currently. We're trying to insert protected VMs into the normal VM flow as much as possible. You can even do a memory dump without any segfault or protection exception for QEMU, however the guest's memory content will be unreadable because it's encrypted.

#86 lore raw
Quoted 23 lines · Janosch Frank
Quoted 9 lines · Daniel P. Berrangé
Quoted 8 lines · Janosch Frank

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

Quoted 8 lines
Quoted 3 lines

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Can you give some guidance on how management applications including libvirt & layers above (oVirt, OpenStack, etc) would/should use this feature ? What new command line / monitor calls are needed, and what feature restrictions are there on its use ?

management applications generally do not need to know about this feature. Most of the magic is in the guest image, which boots up in a certain way to become a protected machine.

Quoted 7 lines

The requirements for that to happen are:
* Machine/firmware support
* KVM & QEMU support
* IO only with iommu
* Guest needs to use IO bounce buffers
* A kernel image or a kernel on a disk that was prepared with special tooling

If the user has a guest image that's expecting to run in protected machine mode, presumably this will fail to boot if run on a host which doesn't support this feature ?

As a mgmt app I think there will be a need to be able to determine whether a host + QEMU combo is actually able to support protected machines. If the mgmt app is given an image and the users says it required protected mode, then the mgmt app needs to know which host(s) are able to run it.

Doing version number checks is not particularly desirable, so is there a way libvirt can determine if QEMU + host in general supports protected machines, so that we can report this feature to mgmt apps ?

If a guest has booted & activated protected mode is there any way for libvirt to query that status ? This would allow the mgmt app to know that the guest is not going to be migratable thereafter.

Is there any way to prevent a guest from using protected mode even if QEMU supports it ? eg the mgmt app may want to be able to guarantee that all VMs are migratable, so don't want a guest OS secretly activating protected mode which blocks migration.

Quoted 7 lines

Such VMs are started like any other VM and run a short "normal" stub that will prepare some things and then requests to be protected.

Most of the restrictions are memory related and might be lifted in the future:
* No paging
* No migration

Presumably QEMU is going to set a migration blocker when a guest activates protected mode ?

Quoted 8 lines

* No huge page backings
* No collaborative memory management

There are no monitor changes or cmd additions currently. We're trying to insert protected VMs into the normal VM flow as much as possible. You can even do a memory dump without any segfault or protection exception for QEMU, however the guest's memory content will be unreadable because it's encrypted.

Is there any way to securely acquire a key needed to interpret this, or is the memory dump completely useless ?

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#87 lore raw
Quoted 32 lines · Daniel P. Berrangé
Quoted 23 lines · Janosch Frank
Quoted 9 lines · Daniel P. Berrangé
Quoted 8 lines · Janosch Frank

Most of the QEMU changes for PV are related to the new IPL type with subcodes 8 - 10 and the execution of the necessary Ultravisor calls to IPL secure guests. Note that we can only boot into secure mode from normal mode, i.e. stfle 161 is not active in secure mode.

The other changes related to data gathering for emulation and disabling addressing checks in secure mode, as well as CPU resets.

Quoted 8 lines
Quoted 3 lines

While working on this I sprinkled in some cleanups, as we sometimes significantly increase line count of some functions and they got unreadable.

Can you give some guidance on how management applications including libvirt & layers above (oVirt, OpenStack, etc) would/should use this feature ? What new command line / monitor calls are needed, and what feature restrictions are there on its use ?

management applications generally do not need to know about this feature. Most of the magic is in the guest image, which boots up in a certain way to become a protected machine.

Quoted 7 lines

The requirements for that to happen are:
* Machine/firmware support
* KVM & QEMU support
* IO only with iommu
* Guest needs to use IO bounce buffers
* A kernel image or a kernel on a disk that was prepared with special tooling

Quoted 3 lines

If the user has a guest image that's expecting to run in protected machine mode, presumably this will fail to boot if run on a host which doesn't support this feature ?

Yes, the guest will lack stfle facility 161 and KVM will report a specification exception on the diagnose subcode 8 - 10.

Quoted 10 lines

As a mgmt app I think there will be a need to be able to determine whether a host + QEMU combo is actually able to support protected machines. If the mgmt app is given an image and the users says it required protected mode, then the mgmt app needs to know which host(s) are able to run it.

Doing version number checks is not particularly desirable, so is there a way libvirt can determine if QEMU + host in general supports protected machines, so that we can report this feature to mgmt apps ?

I thought that would be visible via the cpu model by checking for the unpack facility (161)?
Time for somebody else to explain that.

@Viktor @Boris: This one's for you.

Quoted 5 lines

If a guest has booted & activated protected mode is there any way for libvirt to query that status ? This would allow the mgmt app to know that the guest is not going to be migratable thereafter.

Currently not

Quoted 5 lines

Is there any way to prevent a guest from using protected mode even if QEMU supports it ? eg the mgmt app may want to be able to guarantee that all VMs are migratable, so don't want a guest OS secretly activating protected mode which blocks migration.

Not enabling facility 161 is enough.

Quoted 11 lines
Quoted 8 lines

Such VMs are started like any other VM and run a short "normal" stub that will prepare some things and then requests to be protected.

Most of the restrictions are memory related and might be lifted in the future:
* No paging
* No migration

Presumably QEMU is going to set a migration blocker when a guest activates protected mode ?

Well, that's stuff I still need to figure out :)

Quoted 4 lines
Quoted 3 lines

* No huge page backings
* No collaborative memory management

Quoted 8 lines
Quoted 5 lines

There are no monitor changes or cmd additions currently. We're trying to insert protected VMs into the normal VM flow as much as possible. You can even do a memory dump without any segfault or protection exception for QEMU, however the guest's memory content will be unreadable because it's encrypted.

Is there any way to securely acquire a key needed to interpret this, or is the memory dump completely useless ?

It's part of the design, but not yet implemented.

Quoted 4 lines

Regards,
Daniel

#88 lore raw

On 11/29/19 3:02 PM, Janosch Frank wrote:
[...]

Quoted 11 lines
Quoted 10 lines

As a mgmt app I think there will be a need to be able to determine whether a host + QEMU combo is actually able to support protected machines. If the mgmt app is given an image and the users says it required protected mode, then the mgmt app needs to know which host(s) are able to run it.

Doing version number checks is not particularly desirable, so is there a way libvirt can determine if QEMU + host in general supports protected machines, so that we can report this feature to mgmt apps ?

Quoted 7 lines

I thought that would be visible via the cpu model by checking for the unpack facility (161)?
Time for somebody else to explain that.

@Viktor @Boris: This one's for you.

Right, a management app could check the supported CPU model, with something like virsh domcapabilities. The domain's CPU model would have to require the 'unpack' facility. So, in theory any management app establishing CPU model compatibility using the libvirt APIs should be able to find appropriate hosts. [...]

Signature · 2 lines
Kind Regards,
   Viktor
#89 lore raw
In reply to Janosch Frank
Quoted 8 lines · Janosch Frank
Quoted 4 lines · Daniel P. Berrangé

Is there any way to prevent a guest from using protected mode even if QEMU supports it ? eg the mgmt app may want to be able to guarantee that all VMs are migratable, so don't want a guest OS secretly activating protected mode which blocks migration.

Not enabling facility 161 is enough.

Is this facility enabled by default in any scenario ?

What happens if the feature is enabled & QEMU is also coinfigured to use huge pages or does not have memory pinned into RAM, given that those features are said to be incompatible ?

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

Such VMs are started like any other VM and run a short "normal" stub that will prepare some things and then requests to be protected.

Most of the restrictions are memory related and might be lifted in the future:
* No paging
* No migration

Quoted 8 lines
Quoted 2 lines

Presumably QEMU is going to set a migration blocker when a guest activates protected mode ?

Well, that's stuff I still need to figure out :)

Quoted 3 lines
Quoted 3 lines

* No huge page backings
* No collaborative memory management

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|