Skip to contentSkip to conversation

Generalize start-powered-off property from ARM

PATCH Superseded Thiago Jung Bauermann 7 participants 43 messages Jul 23, 2020 – Aug 18, 2020

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover Generalize start-powered-off property from ARM

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

The only problem is that I was only able to test these changes on a ppc64le pseries KVM guest, so except for patches 2 and 3, all others are only build-tested. Also, my grasp of QOM lifecycle is basically non-existant so please be aware of that when reviewing this series.

The last patch may be wrong, as pointed out by Eduardo, so I marked it as RFC. It may make sense to drop it.

Applies cleanly on yesterday's master.

Changes since v2:

General:
- Added Philippe's Reviewed-by to some of the patches.

Patch "ppc/spapr: Use start-powered-off CPUState property"
- Set the CPUState::start_powered_off variable directly rather than using object_property_set_bool(). Suggested by Philippe.

Patch "sparc/sun4m: Remove main_cpu_reset()"
- New patch. Suggested by Philippe.

Patch "sparc/sun4m: Use start-powered-off CPUState property"
- Remove secondary_cpu_reset(). Suggested by Philippe.
- Remove setting of cs->halted = 1 from cpu_devinit(). Suggested by Philippe.

Patch "Don't set CPUState::halted in cpu_devinit()"
- Squashed into previous patch. Suggested by Philippe.

Patch "sparc/sun4m: Use one cpu_reset() function for main and secondary CPUs"
- Dropped.

Patch "target/s390x: Use start-powered-off CPUState property"
- Set the CPUState::start_powered_off variable directly rather than using object_property_set_bool(). Suggested by Philippe.
- Mention in the commit message Eduardo's observation that before this patch, the code didn't set cs->halted on reset.

Thiago Jung Bauermann (8):
target/arm: Move start-powered-off property to generic CPUState target/arm: Move setting of CPU halted state to generic code ppc/spapr: Use start-powered-off CPUState property ppc/e500: Use start-powered-off CPUState property mips/cps: Use start-powered-off CPUState property sparc/sun4m: Remove main_cpu_reset()
sparc/sun4m: Use start-powered-off CPUState property target/s390x: Use start-powered-off CPUState property

 exec.c                  |  1 +
 hw/core/cpu.c           |  2 +-
 hw/mips/cps.c           |  6 +++---
 hw/ppc/e500.c           | 10 +++++++---
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 hw/sparc/sun4m.c        | 28 ++--------------------------
 include/hw/core/cpu.h   |  4 ++++
 target/arm/cpu.c        |  4 +---
 target/arm/cpu.h        |  3 ---
 target/arm/kvm32.c      |  2 +-
 target/arm/kvm64.c      |  2 +-
 target/s390x/cpu.c      |  2 +-
 12 files changed, 27 insertions(+), 47 deletions(-)
#1 lore raw

PATCH v3 1/8 target/arm: Move start-powered-off property to generic CPUState

There are other platforms which also have CPUs that start powered off, so generalize the start-powered-off property so that it can be used by them.

Note that ARMv7MState also has a property of the same name but this patch doesn't change it because that class isn't a subclass of CPUState so it wouldn't be a trivial change.

This change should not cause any change in behavior.

---
 exec.c                | 1 +
 include/hw/core/cpu.h | 4 ++++
 target/arm/cpu.c      | 5 ++---
 target/arm/cpu.h      | 3 ---
 target/arm/kvm32.c    | 2 +-
 target/arm/kvm64.c    | 2 +-
 6 files changed, 9 insertions(+), 8 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

exec.c +1 -0
index 6f381f98e2..82e82fab09 100644
@@ -899,6 +899,7 @@ Property cpu_common_props[] = {
     DEFINE_PROP_LINK("memory", CPUState, memory, TYPE_MEMORY_REGION,
                      MemoryRegion *),
 #endif
+    DEFINE_PROP_BOOL("start-powered-off", CPUState, start_powered_off, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 
include/hw/core/cpu.h +4 -0
index 8f145733ce..9fc2696db5 100644
@@ -374,6 +374,10 @@ struct CPUState {
     bool created;
     bool stop;
     bool stopped;
+
+    /* Should CPU start in powered-off state? */
+    bool start_powered_off;
+
     bool unplug;
     bool crash_occurred;
     bool exit_request;
target/arm/cpu.c +2 -3
index 111579554f..ec65c7653f 100644
@@ -174,8 +174,8 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR1] = cpu->isar.mvfr1;
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
-    cpu->power_state = cpu->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = cpu->start_powered_off;
+    cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
+    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
@@ -2182,7 +2182,6 @@ static const ARMCPUInfo arm_cpus[] = {
 };
 
 static Property arm_cpu_properties[] = {
-    DEFINE_PROP_BOOL("start-powered-off", ARMCPU, start_powered_off, false),
     DEFINE_PROP_UINT32("psci-conduit", ARMCPU, psci_conduit, 0),
     DEFINE_PROP_UINT64("midr", ARMCPU, midr, 0),
     DEFINE_PROP_UINT64("mp-affinity", ARMCPU,
target/arm/cpu.h +0 -3
index 9e8ed423ea..a925d26996 100644
@@ -810,9 +810,6 @@ struct ARMCPU {
      */
     uint32_t psci_version;
 
-    /* Should CPU start in PSCI powered-off state? */
-    bool start_powered_off;
-
     /* Current power state, access guarded by BQL */
     ARMPSCIState power_state;
 
target/arm/kvm32.c +1 -1
index 0af46b41c8..1f2b8f8b7a 100644
@@ -218,7 +218,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
target/arm/kvm64.c +1 -1
index 1169237905..f8a6d905fb 100644
@@ -775,7 +775,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
Reviewed-by on v3 1/8
#2 lore raw

Re: [PATCH v3 1/8] target/arm: Move start-powered-off property to generic CPUState

Quoted 7 lines · Thiago Jung Bauermann

There are other platforms which also have CPUs that start powered off, so generalize the start-powered-off property so that it can be used by them.

Note that ARMv7MState also has a property of the same name but this patch doesn't change it because that class isn't a subclass of CPUState so it wouldn't be a trivial change.

Quoted 5 lines

This change should not cause any change in behavior.

Quoted 103 lines
---
 exec.c                | 1 +
 include/hw/core/cpu.h | 4 ++++
 target/arm/cpu.c      | 5 ++---
 target/arm/cpu.h      | 3 ---
 target/arm/kvm32.c    | 2 +-
 target/arm/kvm64.c    | 2 +-
 6 files changed, 9 insertions(+), 8 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

exec.c +1 -0
index 6f381f98e2..82e82fab09 100644
@@ -899,6 +899,7 @@ Property cpu_common_props[] = {
     DEFINE_PROP_LINK("memory", CPUState, memory, TYPE_MEMORY_REGION,
                      MemoryRegion *),
 #endif
+    DEFINE_PROP_BOOL("start-powered-off", CPUState, start_powered_off, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 
include/hw/core/cpu.h +4 -0
index 8f145733ce..9fc2696db5 100644
@@ -374,6 +374,10 @@ struct CPUState {
     bool created;
     bool stop;
     bool stopped;
+
+    /* Should CPU start in powered-off state? */
+    bool start_powered_off;
+
     bool unplug;
     bool crash_occurred;
     bool exit_request;
target/arm/cpu.c +2 -3
index 111579554f..ec65c7653f 100644
@@ -174,8 +174,8 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR1] = cpu->isar.mvfr1;
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
-    cpu->power_state = cpu->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = cpu->start_powered_off;
+    cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
+    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
@@ -2182,7 +2182,6 @@ static const ARMCPUInfo arm_cpus[] = {
 };
 
 static Property arm_cpu_properties[] = {
-    DEFINE_PROP_BOOL("start-powered-off", ARMCPU, start_powered_off, false),
     DEFINE_PROP_UINT32("psci-conduit", ARMCPU, psci_conduit, 0),
     DEFINE_PROP_UINT64("midr", ARMCPU, midr, 0),
     DEFINE_PROP_UINT64("mp-affinity", ARMCPU,
target/arm/cpu.h +0 -3
index 9e8ed423ea..a925d26996 100644
@@ -810,9 +810,6 @@ struct ARMCPU {
      */
     uint32_t psci_version;
 
-    /* Should CPU start in PSCI powered-off state? */
-    bool start_powered_off;
-
     /* Current power state, access guarded by BQL */
     ARMPSCIState power_state;
 
target/arm/kvm32.c +1 -1
index 0af46b41c8..1f2b8f8b7a 100644
@@ -218,7 +218,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
target/arm/kvm64.c +1 -1
index 1169237905..f8a6d905fb 100644
@@ -775,7 +775,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#3 lore raw

Re: [PATCH v3 1/8] target/arm: Move start-powered-off property to generic CPUState

Quoted 8 lines · David Gibson
Quoted 7 lines · Thiago Jung Bauermann

There are other platforms which also have CPUs that start powered off, so generalize the start-powered-off property so that it can be used by them.

Note that ARMv7MState also has a property of the same name but this patch doesn't change it because that class isn't a subclass of CPUState so it wouldn't be a trivial change.

Quoted 7 lines
Quoted 5 lines

This change should not cause any change in behavior.

Thanks! Sory about the extra work.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
Reviewed-by
#4 lore raw

Re: [PATCH v3 1/8] target/arm: Move start-powered-off property to generic CPUState

In reply to David Gibson
Quoted 8 lines · On Thu, 23 Jul 2020 13:06:09 +1000 David Gibson <[email protected]>…
Quoted 7 lines · Thiago Jung Bauermann

There are other platforms which also have CPUs that start powered off, so generalize the start-powered-off property so that it can be used by them.

Note that ARMv7MState also has a property of the same name but this patch doesn't change it because that class isn't a subclass of CPUState so it wouldn't be a trivial change.

Quoted 8 lines
Quoted 5 lines

This change should not cause any change in behavior.

Quoted 104 lines
Quoted 103 lines
---
 exec.c                | 1 +
 include/hw/core/cpu.h | 4 ++++
 target/arm/cpu.c      | 5 ++---
 target/arm/cpu.h      | 3 ---
 target/arm/kvm32.c    | 2 +-
 target/arm/kvm64.c    | 2 +-
 6 files changed, 9 insertions(+), 8 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

exec.c +1 -0
index 6f381f98e2..82e82fab09 100644
@@ -899,6 +899,7 @@ Property cpu_common_props[] = {
     DEFINE_PROP_LINK("memory", CPUState, memory, TYPE_MEMORY_REGION,
                      MemoryRegion *),
 #endif
+    DEFINE_PROP_BOOL("start-powered-off", CPUState, start_powered_off, false),
     DEFINE_PROP_END_OF_LIST(),
 };
 
include/hw/core/cpu.h +4 -0
index 8f145733ce..9fc2696db5 100644
@@ -374,6 +374,10 @@ struct CPUState {
     bool created;
     bool stop;
     bool stopped;
+
+    /* Should CPU start in powered-off state? */
+    bool start_powered_off;
+
     bool unplug;
     bool crash_occurred;
     bool exit_request;
target/arm/cpu.c +2 -3
index 111579554f..ec65c7653f 100644
@@ -174,8 +174,8 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR1] = cpu->isar.mvfr1;
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
-    cpu->power_state = cpu->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = cpu->start_powered_off;
+    cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
+    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
@@ -2182,7 +2182,6 @@ static const ARMCPUInfo arm_cpus[] = {
 };
 
 static Property arm_cpu_properties[] = {
-    DEFINE_PROP_BOOL("start-powered-off", ARMCPU, start_powered_off, false),
     DEFINE_PROP_UINT32("psci-conduit", ARMCPU, psci_conduit, 0),
     DEFINE_PROP_UINT64("midr", ARMCPU, midr, 0),
     DEFINE_PROP_UINT64("mp-affinity", ARMCPU,
target/arm/cpu.h +0 -3
index 9e8ed423ea..a925d26996 100644
@@ -810,9 +810,6 @@ struct ARMCPU {
      */
     uint32_t psci_version;
 
-    /* Should CPU start in PSCI powered-off state? */
-    bool start_powered_off;
-
     /* Current power state, access guarded by BQL */
     ARMPSCIState power_state;
 
target/arm/kvm32.c +1 -1
index 0af46b41c8..1f2b8f8b7a 100644
@@ -218,7 +218,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
target/arm/kvm64.c +1 -1
index 1169237905..f8a6d905fb 100644
@@ -775,7 +775,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
     /* Determine init features for this CPU */
     memset(cpu->kvm_init_features, 0, sizeof(cpu->kvm_init_features));
-    if (cpu->start_powered_off) {
+    if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
     if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
#5 lore raw

Re: [PATCH v3 1/8] target/arm: Move start-powered-off property to generic CPUState

Quoted 13 lines · Greg Kurz
Quoted 10 lines · On Thu, 23 Jul 2020 13:06:09 +1000 David Gibson <[email protected]>…
Quoted 9 lines · Thiago Jung Bauermann

There are other platforms which also have CPUs that start powered off, so generalize the start-powered-off property so that it can be used by them.

Note that ARMv7MState also has a property of the same name but this patch doesn't change it because that class isn't a subclass of CPUState so it wouldn't be a trivial change.

This change should not cause any change in behavior.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Thanks!

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#6 lore raw

PATCH v3 2/8 target/arm: Move setting of CPU halted state to generic code

In reply to Thiago Jung Bauermann

This change is in a separate patch because it's not so obvious that it won't cause a regression.

---
 hw/core/cpu.c    | 2 +-
 target/arm/cpu.c | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

NB: I wasn't able to run this patch on an ARM machine. I did run it on a ppc64le pseries KVM guest.

hw/core/cpu.c +1 -1
index 594441a150..71bb7859f1 100644
@@ -258,7 +258,7 @@ static void cpu_common_reset(DeviceState *dev)
     }
 
     cpu->interrupt_request = 0;
-    cpu->halted = 0;
+    cpu->halted = cpu->start_powered_off;
     cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
     atomic_set(&cpu->icount_decr_ptr->u32, 0);
target/arm/cpu.c +0 -1
index ec65c7653f..b6c65e4df6 100644
@@ -175,7 +175,6 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
     cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
Reviewed-by on v3 2/8
#7 lore raw

Re: [PATCH v3 2/8] target/arm: Move setting of CPU halted state to generic code

Quoted 6 lines · Thiago Jung Bauermann

This change is in a separate patch because it's not so obvious that it won't cause a regression.

Quoted 34 lines
---
 hw/core/cpu.c    | 2 +-
 target/arm/cpu.c | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

NB: I wasn't able to run this patch on an ARM machine. I did run it on a ppc64le pseries KVM guest.

hw/core/cpu.c +1 -1
index 594441a150..71bb7859f1 100644
@@ -258,7 +258,7 @@ static void cpu_common_reset(DeviceState *dev)
     }
 
     cpu->interrupt_request = 0;
-    cpu->halted = 0;
+    cpu->halted = cpu->start_powered_off;
     cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
     atomic_set(&cpu->icount_decr_ptr->u32, 0);
target/arm/cpu.c +0 -1
index ec65c7653f..b6c65e4df6 100644
@@ -175,7 +175,6 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
     cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Reviewed-by on v3 2/8
#8 lore raw

Re: [PATCH v3 2/8] target/arm: Move setting of CPU halted state to generic code

In reply to Thiago Jung Bauermann
Quoted 7 lines · On Wed, 22 Jul 2020 23:56:51 -0300 Thiago Jung Bauermann…

This change is in a separate patch because it's not so obvious that it won't cause a regression.

---

Quoted 32 lines
 hw/core/cpu.c    | 2 +-
 target/arm/cpu.c | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

NB: I wasn't able to run this patch on an ARM machine. I did run it on a ppc64le pseries KVM guest.

hw/core/cpu.c +1 -1
index 594441a150..71bb7859f1 100644
@@ -258,7 +258,7 @@ static void cpu_common_reset(DeviceState *dev)
     }
 
     cpu->interrupt_request = 0;
-    cpu->halted = 0;
+    cpu->halted = cpu->start_powered_off;
     cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
     atomic_set(&cpu->icount_decr_ptr->u32, 0);
target/arm/cpu.c +0 -1
index ec65c7653f..b6c65e4df6 100644
@@ -175,7 +175,6 @@ static void arm_cpu_reset(DeviceState *dev)
     env->vfp.xregs[ARM_VFP_MVFR2] = cpu->isar.mvfr2;
 
     cpu->power_state = s->start_powered_off ? PSCI_OFF : PSCI_ON;
-    s->halted = s->start_powered_off;
 
     if (arm_feature(env, ARM_FEATURE_IWMMXT)) {
         env->iwmmxt.cregs[ARM_IWMMXT_wCID] = 0x69051000 | 'Q';
#9 lore raw

PATCH v3 3/8 ppc/spapr: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann

PowerPC sPAPR CPUs start in the halted state, and spapr_reset_vcpu() attempts to implement this by setting CPUState::halted to 1. But that's too late for the case of hotplugged CPUs in a machine configure with 2 or more threads per core.

By then, other parts of QEMU have already caused the vCPU to run in an unitialized state a couple of times. For example, ppc_cpu_reset() calls ppc_tlb_invalidate_all(), which ends up calling async_run_on_cpu(). This kicks the new vCPU while it has CPUState::halted = 0, causing QEMU to issue a KVM_RUN ioctl on the new vCPU before the guest is able to make the start-cpu RTAS call to initialize its register state.

This problem doesn't seem to cause visible issues for regular guests, but on a secure guest running under the Ultravisor it does. The Ultravisor relies on being able to snoop on the start-cpu RTAS call to map vCPUs to guests, and this issue causes it to see a stray vCPU that doesn't belong to any guest.

Fix by setting the start-powered-off CPUState property in spapr_create_vcpu(), which makes cpu_common_reset() initialize CPUState::halted to 1 at an earlier moment.

---
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

NB: Tested on ppc64le pseries KVM guest with two threads per core. Hot-plugging additional cores doesn't cause the bug described above anymore.

hw/ppc/spapr_cpu_core.c +5 -5
index c4f47dcc04..2125fdac34 100644
@@ -36,11 +36,6 @@ static void spapr_reset_vcpu(PowerPCCPU *cpu)
 
     cpu_reset(cs);
 
-    /* All CPUs start halted.  CPU0 is unhalted from the machine level
-     * reset code and the rest are explicitly started up by the guest
-     * using an RTAS call */
-    cs->halted = 1;
-
     env->spr[SPR_HIOR] = 0;
 
     lpcr = env->spr[SPR_LPCR];
@@ -274,6 +269,11 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    /*
+     * All CPUs start halted. CPU0 is unhalted from the machine level reset code
+     * and the rest are explicitly started up by the guest using an RTAS call.
+     */
+    cs->start_powered_off = true;
     cs->cpu_index = cc->core_id + i;
     spapr_set_vcpu_id(cpu, cs->cpu_index, &local_err);
     if (local_err) {
Acked-by on v3 3/8
#10 lore raw

Re: [PATCH v3 3/8] ppc/spapr: Use start-powered-off CPUState property

On Wed, Jul 22, 2020 at 11:56:52PM -0300, Thiago Jung Bauermann wrote:
65;6003;1c> PowerPC sPAPR CPUs start in the halted state, and spapr_reset_vcpu()

Quoted 17 lines

attempts to implement this by setting CPUState::halted to 1. But that's too late for the case of hotplugged CPUs in a machine configure with 2 or more threads per core.

By then, other parts of QEMU have already caused the vCPU to run in an unitialized state a couple of times. For example, ppc_cpu_reset() calls ppc_tlb_invalidate_all(), which ends up calling async_run_on_cpu(). This kicks the new vCPU while it has CPUState::halted = 0, causing QEMU to issue a KVM_RUN ioctl on the new vCPU before the guest is able to make the start-cpu RTAS call to initialize its register state.

This problem doesn't seem to cause visible issues for regular guests, but on a secure guest running under the Ultravisor it does. The Ultravisor relies on being able to snoop on the start-cpu RTAS call to map vCPUs to guests, and this issue causes it to see a stray vCPU that doesn't belong to any guest.

Quoted 6 lines

Fix by setting the start-powered-off CPUState property in spapr_create_vcpu(), which makes cpu_common_reset() initialize CPUState::halted to 1 at an earlier moment.

Quoted 37 lines
---
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

NB: Tested on ppc64le pseries KVM guest with two threads per core. Hot-plugging additional cores doesn't cause the bug described above anymore.

hw/ppc/spapr_cpu_core.c +5 -5
index c4f47dcc04..2125fdac34 100644
@@ -36,11 +36,6 @@ static void spapr_reset_vcpu(PowerPCCPU *cpu)
 
     cpu_reset(cs);
 
-    /* All CPUs start halted.  CPU0 is unhalted from the machine level
-     * reset code and the rest are explicitly started up by the guest
-     * using an RTAS call */
-    cs->halted = 1;
-
     env->spr[SPR_HIOR] = 0;
 
     lpcr = env->spr[SPR_LPCR];
@@ -274,6 +269,11 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    /*
+     * All CPUs start halted. CPU0 is unhalted from the machine level reset code
+     * and the rest are explicitly started up by the guest using an RTAS call.
+     */
+    cs->start_powered_off = true;
     cs->cpu_index = cc->core_id + i;
     spapr_set_vcpu_id(cpu, cs->cpu_index, &local_err);
     if (local_err) {
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Reviewed-by on v3 3/8
#11 lore raw

Re: [PATCH v3 3/8] ppc/spapr: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann
Quoted 18 lines · On Wed, 22 Jul 2020 23:56:52 -0300 Thiago Jung Bauermann…

PowerPC sPAPR CPUs start in the halted state, and spapr_reset_vcpu() attempts to implement this by setting CPUState::halted to 1. But that's too late for the case of hotplugged CPUs in a machine configure with 2 or more threads per core.

By then, other parts of QEMU have already caused the vCPU to run in an unitialized state a couple of times. For example, ppc_cpu_reset() calls ppc_tlb_invalidate_all(), which ends up calling async_run_on_cpu(). This kicks the new vCPU while it has CPUState::halted = 0, causing QEMU to issue a KVM_RUN ioctl on the new vCPU before the guest is able to make the start-cpu RTAS call to initialize its register state.

This problem doesn't seem to cause visible issues for regular guests, but on a secure guest running under the Ultravisor it does. The Ultravisor relies on being able to snoop on the start-cpu RTAS call to map vCPUs to guests, and this issue causes it to see a stray vCPU that doesn't belong to any guest.

Quoted 7 lines

Fix by setting the start-powered-off CPUState property in spapr_create_vcpu(), which makes cpu_common_reset() initialize CPUState::halted to 1 at an earlier moment.

---

Thanks for doing this ! I remember seeing partly initialized CPUs being kicked in the past, which is clearly wrong but I never got time to find a proper fix (especially since it didn't seem to break anything).

Quoted 35 lines
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

NB: Tested on ppc64le pseries KVM guest with two threads per core. Hot-plugging additional cores doesn't cause the bug described above anymore.

hw/ppc/spapr_cpu_core.c +5 -5
index c4f47dcc04..2125fdac34 100644
@@ -36,11 +36,6 @@ static void spapr_reset_vcpu(PowerPCCPU *cpu)
 
     cpu_reset(cs);
 
-    /* All CPUs start halted.  CPU0 is unhalted from the machine level
-     * reset code and the rest are explicitly started up by the guest
-     * using an RTAS call */
-    cs->halted = 1;
-
     env->spr[SPR_HIOR] = 0;
 
     lpcr = env->spr[SPR_LPCR];
@@ -274,6 +269,11 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    /*
+     * All CPUs start halted. CPU0 is unhalted from the machine level reset code
+     * and the rest are explicitly started up by the guest using an RTAS call.
+     */
+    cs->start_powered_off = true;
     cs->cpu_index = cc->core_id + i;
     spapr_set_vcpu_id(cpu, cs->cpu_index, &local_err);
     if (local_err) {
#12 lore raw

Re: [PATCH v3 3/8] ppc/spapr: Use start-powered-off CPUState property

Quoted 29 lines · Greg Kurz
Quoted 18 lines · On Wed, 22 Jul 2020 23:56:52 -0300 Thiago Jung Bauermann…

PowerPC sPAPR CPUs start in the halted state, and spapr_reset_vcpu() attempts to implement this by setting CPUState::halted to 1. But that's too late for the case of hotplugged CPUs in a machine configure with 2 or more threads per core.

By then, other parts of QEMU have already caused the vCPU to run in an unitialized state a couple of times. For example, ppc_cpu_reset() calls ppc_tlb_invalidate_all(), which ends up calling async_run_on_cpu(). This kicks the new vCPU while it has CPUState::halted = 0, causing QEMU to issue a KVM_RUN ioctl on the new vCPU before the guest is able to make the start-cpu RTAS call to initialize its register state.

This problem doesn't seem to cause visible issues for regular guests, but on a secure guest running under the Ultravisor it does. The Ultravisor relies on being able to snoop on the start-cpu RTAS call to map vCPUs to guests, and this issue causes it to see a stray vCPU that doesn't belong to any guest.

Quoted 7 lines

Fix by setting the start-powered-off CPUState property in spapr_create_vcpu(), which makes cpu_common_reset() initialize CPUState::halted to 1 at an earlier moment.

---

Quoted 5 lines

Thanks for doing this ! I remember seeing partly initialized CPUs being kicked in the past, which is clearly wrong but I never got time to find a proper fix (especially since it didn't seem to break anything).

Thanks for confirming the issue, and for reviewing the code.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
Reviewed-by on v3 3/8
#13 lore raw

Re: [PATCH v3 3/8] ppc/spapr: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann
Quoted 51 lines · Thiago Jung Bauermann

PowerPC sPAPR CPUs start in the halted state, and spapr_reset_vcpu() attempts to implement this by setting CPUState::halted to 1. But that's too late for the case of hotplugged CPUs in a machine configure with 2 or more threads per core.

By then, other parts of QEMU have already caused the vCPU to run in an unitialized state a couple of times. For example, ppc_cpu_reset() calls ppc_tlb_invalidate_all(), which ends up calling async_run_on_cpu(). This kicks the new vCPU while it has CPUState::halted = 0, causing QEMU to issue a KVM_RUN ioctl on the new vCPU before the guest is able to make the start-cpu RTAS call to initialize its register state.

This problem doesn't seem to cause visible issues for regular guests, but on a secure guest running under the Ultravisor it does. The Ultravisor relies on being able to snoop on the start-cpu RTAS call to map vCPUs to guests, and this issue causes it to see a stray vCPU that doesn't belong to any guest.

Fix by setting the start-powered-off CPUState property in spapr_create_vcpu(), which makes cpu_common_reset() initialize CPUState::halted to 1 at an earlier moment.

---
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

NB: Tested on ppc64le pseries KVM guest with two threads per core. Hot-plugging additional cores doesn't cause the bug described above anymore.

hw/ppc/spapr_cpu_core.c +0 -5
index c4f47dcc04..2125fdac34 100644
@@ -36,11 +36,6 @@ static void spapr_reset_vcpu(PowerPCCPU *cpu)
 
     cpu_reset(cs);
 
-    /* All CPUs start halted.  CPU0 is unhalted from the machine level
-     * reset code and the rest are explicitly started up by the guest
-     * using an RTAS call */
-    cs->halted = 1;
-
     env->spr[SPR_HIOR] = 0;
 
     lpcr = env->spr[SPR_LPCR];
@@ -274,6 +269,11 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
 

hw/ppc/spapr_cpu_core.c static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp) · line 279

Quoted 10 lines
hw/ppc/spapr_cpu_core.c +5 -0
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    /*
+     * All CPUs start halted. CPU0 is unhalted from the machine level reset code
+     * and the rest are explicitly started up by the guest using an RTAS call.
+     */
+    cs->start_powered_off = true;
     cs->cpu_index = cc->core_id + i;
     spapr_set_vcpu_id(cpu, cs->cpu_index, &local_err);
     if (local_err) {
#14 lore raw

PATCH v3 4/8 ppc/e500: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in ppce500_cpu_reset_sec(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

---
 hw/ppc/e500.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/ppc/e500.c +7 -3
index ab9884e315..dda71bc05d 100644
@@ -704,9 +704,6 @@ static void ppce500_cpu_reset_sec(void *opaque)
 
     cpu_reset(cs);
 
-    /* Secondary CPU starts in halted state for now. Needs to change when
-       implementing non-kernel boot. */
-    cs->halted = 1;
     cs->exception_index = EXCP_HLT;
 }
 
@@ -897,6 +894,13 @@ void ppce500_init(MachineState *machine)
         } else {
             /* Secondary CPUs */
             qemu_register_reset(ppce500_cpu_reset_sec, cpu);
+
+            /*
+             * Secondary CPU starts in halted state for now. Needs to change
+             * when implementing non-kernel boot.
+             */
+            object_property_set_bool(OBJECT(cs), "start-powered-off", true,
+                                     &error_abort);
         }
     }
 
Acked-by on v3 4/8
#15 lore raw

Re: [PATCH v3 4/8] ppc/e500: Use start-powered-off CPUState property

Quoted 6 lines · Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in ppce500_cpu_reset_sec(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 36 lines
---
 hw/ppc/e500.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/ppc/e500.c +7 -3
index ab9884e315..dda71bc05d 100644
@@ -704,9 +704,6 @@ static void ppce500_cpu_reset_sec(void *opaque)
 
     cpu_reset(cs);
 
-    /* Secondary CPU starts in halted state for now. Needs to change when
-       implementing non-kernel boot. */
-    cs->halted = 1;
     cs->exception_index = EXCP_HLT;
 }
 
@@ -897,6 +894,13 @@ void ppce500_init(MachineState *machine)
         } else {
             /* Secondary CPUs */
             qemu_register_reset(ppce500_cpu_reset_sec, cpu);
+
+            /*
+             * Secondary CPU starts in halted state for now. Needs to change
+             * when implementing non-kernel boot.
+             */
+            object_property_set_bool(OBJECT(cs), "start-powered-off", true,
+                                     &error_abort);
         }
     }
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#16 lore raw

PATCH v3 5/8 mips/cps: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in main_cpu_reset(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

---
 hw/mips/cps.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/mips/cps.c +3 -3
index 615e1a1ad2..d5b6c78019 100644
@@ -52,9 +52,6 @@ static void main_cpu_reset(void *opaque)
     CPUState *cs = CPU(cpu);
 
     cpu_reset(cs);
-
-    /* All VPs are halted on reset. Leave powering up to CPC. */
-    cs->halted = 1;
 }
 
 static bool cpu_mips_itu_supported(CPUMIPSState *env)
@@ -89,6 +86,9 @@ static void mips_cps_realize(DeviceState *dev, Error **errp)
             env->itc_tag = mips_itu_get_tag_region(&s->itu);
             env->itu = &s->itu;
         }
+        /* All VPs are halted on reset. Leave powering up to CPC. */
+        object_property_set_bool(OBJECT(cpu), "start-powered-off", true,
+                                 &error_abort);
         qemu_register_reset(main_cpu_reset, cpu);
     }
 
Reviewed-by on v3 5/8
#17 lore raw

Re: [PATCH v3 5/8] mips/cps: Use start-powered-off CPUState property

Quoted 6 lines · Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in main_cpu_reset(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 32 lines
---
 hw/mips/cps.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/mips/cps.c +3 -3
index 615e1a1ad2..d5b6c78019 100644
@@ -52,9 +52,6 @@ static void main_cpu_reset(void *opaque)
     CPUState *cs = CPU(cpu);
 
     cpu_reset(cs);
-
-    /* All VPs are halted on reset. Leave powering up to CPC. */
-    cs->halted = 1;
 }
 
 static bool cpu_mips_itu_supported(CPUMIPSState *env)
@@ -89,6 +86,9 @@ static void mips_cps_realize(DeviceState *dev, Error **errp)
             env->itc_tag = mips_itu_get_tag_region(&s->itu);
             env->itu = &s->itu;
         }
+        /* All VPs are halted on reset. Leave powering up to CPC. */
+        object_property_set_bool(OBJECT(cpu), "start-powered-off", true,
+                                 &error_abort);
         qemu_register_reset(main_cpu_reset, cpu);
     }
 
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#18 lore raw

PATCH v3 6/8 sparc/sun4m: Remove main_cpu_reset()

In reply to Thiago Jung Bauermann

We rely on cpu_common_reset() to set cs->halted to 0, so main_cpu_reset() is pointless.

---
 hw/sparc/sun4m.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/sparc/sun4m.c +1 -12
index 9be930415f..f1d92df781 100644
@@ -218,15 +218,6 @@ static void dummy_cpu_set_irq(void *opaque, int irq, int level)
 {
 }
 
-static void main_cpu_reset(void *opaque)
-{
-    SPARCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-
-    cpu_reset(cs);
-    cs->halted = 0;
-}
-
 static void secondary_cpu_reset(void *opaque)
 {
     SPARCCPU *cpu = opaque;
@@ -827,9 +818,7 @@ static void cpu_devinit(const char *cpu_type, unsigned int id,
     env = &cpu->env;
 
     cpu_sparc_set_id(env, id);
-    if (id == 0) {
-        qemu_register_reset(main_cpu_reset, cpu);
-    } else {
+    if (id != 0) {
         qemu_register_reset(secondary_cpu_reset, cpu);
         cs = CPU(cpu);
         cs->halted = 1;
#19 lore raw

Re: [PATCH v3 6/8] sparc/sun4m: Remove main_cpu_reset()

Quoted 5 lines · Thiago Jung Bauermann

We rely on cpu_common_reset() to set cs->halted to 0, so main_cpu_reset() is pointless.

Quoted 39 lines
---
 hw/sparc/sun4m.c | 13 +------------
 1 file changed, 1 insertion(+), 12 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/sparc/sun4m.c +1 -12
index 9be930415f..f1d92df781 100644
@@ -218,15 +218,6 @@ static void dummy_cpu_set_irq(void *opaque, int irq, int level)
 {
 }
 
-static void main_cpu_reset(void *opaque)
-{
-    SPARCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-
-    cpu_reset(cs);
-    cs->halted = 0;
-}
-
 static void secondary_cpu_reset(void *opaque)
 {
     SPARCCPU *cpu = opaque;
@@ -827,9 +818,7 @@ static void cpu_devinit(const char *cpu_type, unsigned int id,
     env = &cpu->env;
 
     cpu_sparc_set_id(env, id);
-    if (id == 0) {
-        qemu_register_reset(main_cpu_reset, cpu);
-    } else {
+    if (id != 0) {
         qemu_register_reset(secondary_cpu_reset, cpu);
         cs = CPU(cpu);
         cs->halted = 1;
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#20 lore raw

PATCH v3 7/8 sparc/sun4m: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in secondary_cpu_reset(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

This makes secondary_cpu_reset() unnecessary, so remove it.

Also remove setting of cs->halted from cpu_devinit(), which seems out of place when compared to similar code in other architectures (e.g., ppce500_init() in hw/ppc/e500.c).

---
 hw/sparc/sun4m.c | 17 ++---------------
 1 file changed, 2 insertions(+), 15 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/sparc/sun4m.c +2 -15
index f1d92df781..fd74e516bb 100644
@@ -218,15 +218,6 @@ static void dummy_cpu_set_irq(void *opaque, int irq, int level)
 {
 }
 
-static void secondary_cpu_reset(void *opaque)
-{
-    SPARCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-
-    cpu_reset(cs);
-    cs->halted = 1;
-}
-
 static void cpu_halt_signal(void *opaque, int irq, int level)
 {
     if (level && current_cpu) {
@@ -810,7 +801,6 @@ static const TypeInfo ram_info = {
 static void cpu_devinit(const char *cpu_type, unsigned int id,
                         uint64_t prom_addr, qemu_irq **cpu_irqs)
 {
-    CPUState *cs;
     SPARCCPU *cpu;
     CPUSPARCState *env;
 
@@ -818,11 +808,8 @@ static void cpu_devinit(const char *cpu_type, unsigned int id,
     env = &cpu->env;
 
     cpu_sparc_set_id(env, id);
-    if (id != 0) {
-        qemu_register_reset(secondary_cpu_reset, cpu);
-        cs = CPU(cpu);
-        cs->halted = 1;
-    }
+    object_property_set_bool(OBJECT(cpu), "start-powered-off", id != 0,
+                             &error_abort);
     *cpu_irqs = qemu_allocate_irqs(cpu_set_irq, cpu, MAX_PILS);
     env->prom_addr = prom_addr;
 }
Reviewed-by on v3 7/8
#21 lore raw

Re: [PATCH v3 7/8] sparc/sun4m: Use start-powered-off CPUState property

Quoted 4 lines · Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in secondary_cpu_reset(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 8 lines

This makes secondary_cpu_reset() unnecessary, so remove it.

Also remove setting of cs->halted from cpu_devinit(), which seems out of place when compared to similar code in other architectures (e.g., ppce500_init() in hw/ppc/e500.c).

Quoted 50 lines
---
 hw/sparc/sun4m.c | 17 ++---------------
 1 file changed, 2 insertions(+), 15 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/sparc/sun4m.c +2 -15
index f1d92df781..fd74e516bb 100644
@@ -218,15 +218,6 @@ static void dummy_cpu_set_irq(void *opaque, int irq, int level)
 {
 }
 
-static void secondary_cpu_reset(void *opaque)
-{
-    SPARCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-
-    cpu_reset(cs);
-    cs->halted = 1;
-}
-
 static void cpu_halt_signal(void *opaque, int irq, int level)
 {
     if (level && current_cpu) {
@@ -810,7 +801,6 @@ static const TypeInfo ram_info = {
 static void cpu_devinit(const char *cpu_type, unsigned int id,
                         uint64_t prom_addr, qemu_irq **cpu_irqs)
 {
-    CPUState *cs;
     SPARCCPU *cpu;
     CPUSPARCState *env;
 
@@ -818,11 +808,8 @@ static void cpu_devinit(const char *cpu_type, unsigned int id,
     env = &cpu->env;
 
     cpu_sparc_set_id(env, id);
-    if (id != 0) {
-        qemu_register_reset(secondary_cpu_reset, cpu);
-        cs = CPU(cpu);
-        cs->halted = 1;
-    }
+    object_property_set_bool(OBJECT(cpu), "start-powered-off", id != 0,
+                             &error_abort);
     *cpu_irqs = qemu_allocate_irqs(cpu_set_irq, cpu, MAX_PILS);
     env->prom_addr = prom_addr;
 }
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
Reviewed-by on v3 7/8
#22 lore raw

Re: [PATCH v3 7/8] sparc/sun4m: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann
Quoted 52 lines · Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in secondary_cpu_reset(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

This makes secondary_cpu_reset() unnecessary, so remove it.

Also remove setting of cs->halted from cpu_devinit(), which seems out of place when compared to similar code in other architectures (e.g., ppce500_init() in hw/ppc/e500.c).

---
 hw/sparc/sun4m.c | 17 ++---------------
 1 file changed, 2 insertions(+), 15 deletions(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

hw/sparc/sun4m.c +0 -10
index f1d92df781..fd74e516bb 100644
@@ -218,15 +218,6 @@ static void dummy_cpu_set_irq(void *opaque, int irq, int level)
 {
 }
 
-static void secondary_cpu_reset(void *opaque)
-{
-    SPARCCPU *cpu = opaque;
-    CPUState *cs = CPU(cpu);
-
-    cpu_reset(cs);
-    cs->halted = 1;
-}
-
 static void cpu_halt_signal(void *opaque, int irq, int level)
 {
     if (level && current_cpu) {
@@ -810,7 +801,6 @@ static const TypeInfo ram_info = {
 static void cpu_devinit(const char *cpu_type, unsigned int id,
                         uint64_t prom_addr, qemu_irq **cpu_irqs)
 {
-    CPUState *cs;
     SPARCCPU *cpu;
     CPUSPARCState *env;
 
@@ -818,11 +808,8 @@ static void cpu_devinit(const char *cpu_type, unsigned int id,
     env = &cpu->env;
 
     cpu_sparc_set_id(env, id);

hw/sparc/sun4m.c static void cpu_devinit(const char *cpu_type, unsigned int id, · line 815

Quoted 10 lines
hw/sparc/sun4m.c +2 -5
-    if (id != 0) {
-        qemu_register_reset(secondary_cpu_reset, cpu);
-        cs = CPU(cpu);
-        cs->halted = 1;
-    }
+    object_property_set_bool(OBJECT(cpu), "start-powered-off", id != 0,
+                             &error_abort);
     *cpu_irqs = qemu_allocate_irqs(cpu_set_irq, cpu, MAX_PILS);
     env->prom_addr = prom_addr;
 }
#23 lore raw

RFC PATCH v3 8/8 target/s390x: Use start-powered-off CPUState property

In reply to Thiago Jung Bauermann

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

target/s390x/cpu.c +1 -1
index 08eb674d22..73d7d6007e 100644
@@ -291,7 +291,7 @@ static void s390_cpu_initfn(Object *obj)
     S390CPU *cpu = S390_CPU(obj);
 
     cpu_set_cpustate_pointers(cpu);
-    cs->halted = 1;
+    cs->start_powered_off = true;
     cs->exception_index = EXCP_HLT;
 #if !defined(CONFIG_USER_ONLY)
     object_property_add(obj, "crash-information", "GuestPanicInformation",
#24 lore raw

Re: [RFC PATCH v3 8/8] target/s390x: Use start-powered-off CPUState property

Quoted 6 lines · On Wed, 22 Jul 2020 23:56:57 -0300 Thiago Jung Bauermann…

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

I think that should be fine, as we change the cpu state to STOPPED in the reset function, which sets halted to 1.

Quoted 8 lines
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

No noticeable difference under kvm, but running under tcg seems a bit more sluggish than usual, and I saw some pausing on reboot (after the bios handover to the kernel). Not sure if it were just flukes on my laptop, would appreciate if someone else could give it a go.

Quoted 15 lines
target/s390x/cpu.c +1 -1
index 08eb674d22..73d7d6007e 100644
@@ -291,7 +291,7 @@ static void s390_cpu_initfn(Object *obj)
     S390CPU *cpu = S390_CPU(obj);
 
     cpu_set_cpustate_pointers(cpu);
-    cs->halted = 1;
+    cs->start_powered_off = true;
     cs->exception_index = EXCP_HLT;
 #if !defined(CONFIG_USER_ONLY)
     object_property_add(obj, "crash-information", "GuestPanicInformation",
#25 lore raw

Re: [RFC PATCH v3 8/8] target/s390x: Use start-powered-off CPUState property

Hi,

Quoted 7 lines · Cornelia Huck
Quoted 4 lines · On Wed, 22 Jul 2020 23:56:57 -0300 Thiago Jung Bauermann…

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 5 lines
Quoted 2 lines

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

I think that should be fine, as we change the cpu state to STOPPED in the reset function, which sets halted to 1.

Nice, thanks for checking.

Quoted 6 lines
Quoted 6 lines
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 7 lines
Quoted 2 lines

NB: I was only able to test that this patch builds. I wasn't able to run it.

No noticeable difference under kvm, but running under tcg seems a bit more sluggish than usual, and I saw some pausing on reboot (after the bios handover to the kernel). Not sure if it were just flukes on my laptop, would appreciate if someone else could give it a go.

I tried today setting up a TCG guest, but didn't have success yet. Will try some more tomorrow.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#26 lore raw

Re: [RFC PATCH v3 8/8] target/s390x: Use start-powered-off CPUState property

Quoted 11 lines · On Tue, 28 Jul 2020 21:51:33 -0300 Thiago Jung Bauermann…

Hi,

Quoted 7 lines · Cornelia Huck
Quoted 4 lines · On Wed, 22 Jul 2020 23:56:57 -0300 Thiago Jung Bauermann…

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 21 lines
Quoted 5 lines
Quoted 2 lines

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

I think that should be fine, as we change the cpu state to STOPPED in the reset function, which sets halted to 1.

Nice, thanks for checking.

Quoted 13 lines
Quoted 8 lines
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

No noticeable difference under kvm, but running under tcg seems a bit more sluggish than usual, and I saw some pausing on reboot (after the bios handover to the kernel). Not sure if it were just flukes on my laptop, would appreciate if someone else could give it a go.

Experimented a bit with it again. There's a pause when switching from the bios to the kernel (after the load reset normal has been done, I guess), which is always there, but seems to get more noticeable with this patch (varying wildly, but seems longer on average.) Hard to pin down, and I don't really see a reason why that should happen, as we should end up with halted == 1 in any case. Might still be a fluke, even though I see it both on my laptop and on an LPAR (when running under tcg; not seen under kvm, which is much faster anyway.)

Quoted 4 lines

I tried today setting up a TCG guest, but didn't have success yet. Will try some more tomorrow.

I'm also looking a bit at the other s390 folks :)

Acked-by
#27 lore raw

Re: [RFC PATCH v3 8/8] target/s390x: Use start-powered-off CPUState property

Quoted 36 lines · On Thu, 30 Jul 2020 11:45:41 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 11 lines · On Tue, 28 Jul 2020 21:51:33 -0300 Thiago Jung Bauermann…

Hi,

Quoted 7 lines · Cornelia Huck
Quoted 4 lines · On Wed, 22 Jul 2020 23:56:57 -0300 Thiago Jung Bauermann…

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 21 lines
Quoted 5 lines
Quoted 2 lines

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

I think that should be fine, as we change the cpu state to STOPPED in the reset function, which sets halted to 1.

Nice, thanks for checking.

Quoted 13 lines
Quoted 8 lines
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

No noticeable difference under kvm, but running under tcg seems a bit more sluggish than usual, and I saw some pausing on reboot (after the bios handover to the kernel). Not sure if it were just flukes on my laptop, would appreciate if someone else could give it a go.

Quoted 8 lines

Experimented a bit with it again. There's a pause when switching from the bios to the kernel (after the load reset normal has been done, I guess), which is always there, but seems to get more noticeable with this patch (varying wildly, but seems longer on average.) Hard to pin down, and I don't really see a reason why that should happen, as we should end up with halted == 1 in any case. Might still be a fluke, even though I see it both on my laptop and on an LPAR (when running under tcg; not seen under kvm, which is much faster anyway.)

Tried again, the pause now seems comparable to the pause prior to this series. Might depend on the phase of the moon.

I ran kvm unit tests on it, and it looks good. So, I'm reasonable confident that this is fine, really just seems to be a fluke.

#28 lore raw

Re: [RFC PATCH v3 8/8] target/s390x: Use start-powered-off CPUState property

Quoted 48 lines · Cornelia Huck
Quoted 36 lines · On Thu, 30 Jul 2020 11:45:41 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 11 lines · On Tue, 28 Jul 2020 21:51:33 -0300 Thiago Jung Bauermann…

Hi,

Quoted 7 lines · Cornelia Huck
Quoted 4 lines · On Wed, 22 Jul 2020 23:56:57 -0300 Thiago Jung Bauermann…

Instead of setting CPUState::halted to 1 in s390_cpu_initfn(), use the start-powered-off property which makes cpu_common_reset() initialize it to 1 in common code.

Quoted 21 lines
Quoted 5 lines
Quoted 2 lines

Note that this changes behavior by setting cs->halted to 1 on reset, which didn't happen before.

I think that should be fine, as we change the cpu state to STOPPED in the reset function, which sets halted to 1.

Nice, thanks for checking.

Quoted 13 lines
Quoted 8 lines
---
 target/s390x/cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

NB: I was only able to test that this patch builds. I wasn't able to run it.

No noticeable difference under kvm, but running under tcg seems a bit more sluggish than usual, and I saw some pausing on reboot (after the bios handover to the kernel). Not sure if it were just flukes on my laptop, would appreciate if someone else could give it a go.

Quoted 8 lines

Experimented a bit with it again. There's a pause when switching from the bios to the kernel (after the load reset normal has been done, I guess), which is always there, but seems to get more noticeable with this patch (varying wildly, but seems longer on average.) Hard to pin down, and I don't really see a reason why that should happen, as we should end up with halted == 1 in any case. Might still be a fluke, even though I see it both on my laptop and on an LPAR (when running under tcg; not seen under kvm, which is much faster anyway.)

Quoted 7 lines

Tried again, the pause now seems comparable to the pause prior to this series. Might depend on the phase of the moon.

I ran kvm unit tests on it, and it looks good. So, I'm reasonable confident that this is fine, really just seems to be a fluke.

Ah, that's a relief. Thank you very much for looking into this!

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#29 lore raw
In reply to Thiago Jung Bauermann
Quoted 9 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Since this series fixes a bug is it eligible for 5.1, at least the patches that were already approved by the appropriate maintainers?

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#30 lore raw
Quoted 9 lines · Thiago Jung Bauermann
Quoted 6 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

Quoted 6 lines
Quoted 3 lines

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Since this series fixes a bug is it eligible for 5.1, at least the patches that were already approved by the appropriate maintainers?

Ok by me.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#31 lore raw

Le jeu. 30 juil. 2020 03:00, David Gibson <[email protected]> a écrit :

Quoted 12 lines · Thiago Jung Bauermann
Quoted 11 lines
Quoted 8 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's

Quoted 8 lines

still in

Quoted 4 lines
Quoted 1 line

an unitialized state (more details in patch 3).

Since this series fixes a bug is it eligible for 5.1, at least the patches that were already approved by the appropriate maintainers?

Ok by me.

Maybe just the arm generalization and ppc fix for 5.1, delaying all not bugfix to 5.2?

Quoted 7 lines

--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_

                                | _way_ _around_!

http://www.ozlabs.org/~dgibson

#32 lore raw
Quoted 18 lines · Philippe Mathieu-Daudé

Le jeu. 30 juil. 2020 03:00, David Gibson <[email protected]> a écrit :

Quoted 15 lines · Thiago Jung Bauermann
Quoted 11 lines
Quoted 8 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's

still in

Quoted 2 lines
Quoted 1 line

an unitialized state (more details in patch 3).

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Since this series fixes a bug is it eligible for 5.1, at least the patches that were already approved by the appropriate maintainers?

Ok by me.

Maybe just the arm generalization and ppc fix for 5.1, delaying all not bugfix to 5.2?

That would be great.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#33 lore raw
Quoted 23 lines · Thiago Jung Bauermann
Quoted 21 lines · Philippe Mathieu-Daudé

Le jeu. 30 juil. 2020 03:00, David Gibson <[email protected]> a écrit :

Quoted 18 lines · Thiago Jung Bauermann
Quoted 11 lines
Quoted 8 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's

still in

Quoted 4 lines
Quoted 1 line

an unitialized state (more details in patch 3).

Since this series fixes a bug is it eligible for 5.1, at least the patches that were already approved by the appropriate maintainers?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Ok by me.

Maybe just the arm generalization and ppc fix for 5.1, delaying all not bugfix to 5.2?

That would be great.

Any news on this? Is there something I should be doing? I saw -rc3 today but not these patches.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#34 lore raw
Quoted 2 lines · Thiago Jung Bauermann

Any news on this? Is there something I should be doing? I saw -rc3 today but not these patches.

Sorry, you've missed the bus for 5.1 at this point. I'd assumed that the relevant bits of the patchset would go into a PPC pullreq if it was important for 5.1.

As I understand it, this isn't a regression from 5.0, right?

thanks
-- PMM

#35 lore raw
Quoted 10 lines · Peter Maydell
Quoted 2 lines · Thiago Jung Bauermann

Any news on this? Is there something I should be doing? I saw -rc3 today but not these patches.

Sorry, you've missed the bus for 5.1 at this point. I'd assumed that the relevant bits of the patchset would go into a PPC pullreq if it was important for 5.1.

As I understand it, this isn't a regression from 5.0, right?

Right, it isn't.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#36 lore raw
In reply to Peter Maydell
Quoted 8 lines · Peter Maydell
Quoted 2 lines · Thiago Jung Bauermann

Any news on this? Is there something I should be doing? I saw -rc3 today but not these patches.

Sorry, you've missed the bus for 5.1 at this point. I'd assumed that the relevant bits of the patchset would go into a PPC pullreq if it was important for 5.1.

Ah, bother. Meanwhile I assumed that since it was cross-target it was going in separately, so I didn't take it through my tree.

Oh well, I guess we apply ASAP in 5.2 and we'll do a backport downstream.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#37 lore raw
Quoted 7 lines · David Gibson
Quoted 5 lines · Peter Maydell
Quoted 2 lines · Thiago Jung Bauermann

Any news on this? Is there something I should be doing? I saw -rc3 today but not these patches.

Quoted 6 lines
Quoted 3 lines

Sorry, you've missed the bus for 5.1 at this point. I'd assumed that the relevant bits of the patchset would go into a PPC pullreq if it was important for 5.1.

Ah, bother. Meanwhile I assumed that since it was cross-target it was going in separately, so I didn't take it through my tree.

This kind of confusion is why it's a good idea to list "this ought to be fixed for the release" issues on the Planning wiki page. That way I will see them when I'm looking for patches that might have been forgotten or that I need to wait for a pullreq for before tagging...

-- PMM

Acked-by on the v3 series
#38 lore raw
In reply to Thiago Jung Bauermann
Quoted 7 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

Quoted 7 lines

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

for the Arm bits if you want to take the bug-fixing parts of this series in via some other tree. (I think they've all been reviewed.)

thanks
-- PMM

#39 lore raw
In reply to Thiago Jung Bauermann
Quoted 19 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

The only problem is that I was only able to test these changes on a ppc64le pseries KVM guest, so except for patches 2 and 3, all others are only build-tested. Also, my grasp of QOM lifecycle is basically non-existant so please be aware of that when reviewing this series.

Quoted 4 lines

The last patch may be wrong, as pointed out by Eduardo, so I marked it as RFC. It may make sense to drop it.

Applies cleanly on yesterday's master.

This series appears to break the Travis build for a MIPS target:

Unexpected error in qdev_prop_set_after_realize() at /home/travis/build/dgibson/qemu/hw/core/qdev-properties.c:30:
qemu-system-mips64el: Attempt to set property 'start-powered-off' on anonymous device (type 'I6400-mips64-cpu') after it was realized Broken pipe
/home/travis/build/dgibson/qemu/tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped) Aborted (core dumped)
ERROR qom-test - too few tests run (expected 8, got 0) /home/travis/build/dgibson/qemu/tests/Makefile.include:650: recipe for target 'check-qtest-mips64el' failed

Quoted 54 lines

Changes since v2:

General:
- Added Philippe's Reviewed-by to some of the patches.

Patch "ppc/spapr: Use start-powered-off CPUState property"
- Set the CPUState::start_powered_off variable directly rather than using object_property_set_bool(). Suggested by Philippe.

Patch "sparc/sun4m: Remove main_cpu_reset()"
- New patch. Suggested by Philippe.

Patch "sparc/sun4m: Use start-powered-off CPUState property"
- Remove secondary_cpu_reset(). Suggested by Philippe.
- Remove setting of cs->halted = 1 from cpu_devinit(). Suggested by Philippe.

Patch "Don't set CPUState::halted in cpu_devinit()"
- Squashed into previous patch. Suggested by Philippe.

Patch "sparc/sun4m: Use one cpu_reset() function for main and secondary CPUs"
- Dropped.

Patch "target/s390x: Use start-powered-off CPUState property"
- Set the CPUState::start_powered_off variable directly rather than using object_property_set_bool(). Suggested by Philippe.
- Mention in the commit message Eduardo's observation that before this patch, the code didn't set cs->halted on reset.

Thiago Jung Bauermann (8):
target/arm: Move start-powered-off property to generic CPUState target/arm: Move setting of CPU halted state to generic code ppc/spapr: Use start-powered-off CPUState property ppc/e500: Use start-powered-off CPUState property mips/cps: Use start-powered-off CPUState property sparc/sun4m: Remove main_cpu_reset()
sparc/sun4m: Use start-powered-off CPUState property target/s390x: Use start-powered-off CPUState property

 exec.c                  |  1 +
 hw/core/cpu.c           |  2 +-
 hw/mips/cps.c           |  6 +++---
 hw/ppc/e500.c           | 10 +++++++---
 hw/ppc/spapr_cpu_core.c | 10 +++++-----
 hw/sparc/sun4m.c        | 28 ++--------------------------
 include/hw/core/cpu.h   |  4 ++++
 target/arm/cpu.c        |  4 +---
 target/arm/cpu.h        |  3 ---
 target/arm/kvm32.c      |  2 +-
 target/arm/kvm64.c      |  2 +-
 target/s390x/cpu.c      |  2 +-
 12 files changed, 27 insertions(+), 47 deletions(-)
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#40 lore raw
Quoted 27 lines · David Gibson
Quoted 19 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

The only problem is that I was only able to test these changes on a ppc64le pseries KVM guest, so except for patches 2 and 3, all others are only build-tested. Also, my grasp of QOM lifecycle is basically non-existant so please be aware of that when reviewing this series.

Quoted 4 lines

The last patch may be wrong, as pointed out by Eduardo, so I marked it as RFC. It may make sense to drop it.

Applies cleanly on yesterday's master.

This series appears to break the Travis build for a MIPS target:

Quoted 7 lines

Unexpected error in qdev_prop_set_after_realize() at /home/travis/build/dgibson/qemu/hw/core/qdev-properties.c:30:
qemu-system-mips64el: Attempt to set property 'start-powered-off' on anonymous device (type 'I6400-mips64-cpu') after it was realized Broken pipe
/home/travis/build/dgibson/qemu/tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped) Aborted (core dumped)
ERROR qom-test - too few tests run (expected 8, got 0) /home/travis/build/dgibson/qemu/tests/Makefile.include:650: recipe for target 'check-qtest-mips64el' failed

Good catch. hw/mips/cps.c, hw/ppc/e500.c and hw/sparc/sun4m.c are incorrectly setting the property after the cpu is realized because the cpu is created with cpu_create(). We need to create them with object_initialize_child() and realize them manually with qdev_realize().

#41 lore raw
Quoted 36 lines · Philippe Mathieu-Daudé
Quoted 27 lines · David Gibson
Quoted 19 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

The only problem is that I was only able to test these changes on a ppc64le pseries KVM guest, so except for patches 2 and 3, all others are only build-tested. Also, my grasp of QOM lifecycle is basically non-existant so please be aware of that when reviewing this series.

Quoted 4 lines

The last patch may be wrong, as pointed out by Eduardo, so I marked it as RFC. It may make sense to drop it.

Applies cleanly on yesterday's master.

This series appears to break the Travis build for a MIPS target:

Quoted 7 lines

Unexpected error in qdev_prop_set_after_realize() at /home/travis/build/dgibson/qemu/hw/core/qdev-properties.c:30:
qemu-system-mips64el: Attempt to set property 'start-powered-off' on anonymous device (type 'I6400-mips64-cpu') after it was realized Broken pipe
/home/travis/build/dgibson/qemu/tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped) Aborted (core dumped)
ERROR qom-test - too few tests run (expected 8, got 0) /home/travis/build/dgibson/qemu/tests/Makefile.include:650: recipe for target 'check-qtest-mips64el' failed

Quoted 4 lines

Good catch. hw/mips/cps.c, hw/ppc/e500.c and hw/sparc/sun4m.c are incorrectly setting the property after the cpu is realized because the cpu is created with cpu_create(). We need to create them with object_initialize_child() and realize them manually with qdev_realize().

Thiago, can you fix that up and repost please.

Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#42 lore raw
Quoted 37 lines · David Gibson
Quoted 36 lines · Philippe Mathieu-Daudé
Quoted 34 lines · David Gibson
Quoted 19 lines · Thiago Jung Bauermann

The ARM code has a start-powered-off property in ARMCPU, which is a subclass of CPUState. This property causes arm_cpu_reset() to set CPUState::halted to 1, signalling that the CPU should start in a halted state. Other architectures also have code which aim to achieve the same effect, but without using a property.

The ppc/spapr version has a bug where QEMU does a KVM_RUN on the vcpu before cs->halted is set to 1, causing the vcpu to run while it's still in an unitialized state (more details in patch 3).

Peter Maydell mentioned the ARM start-powered-off property and Eduardo Habkost suggested making it generic, so this patch series does that, for all cases which I was able to find via grep in the code.

The only problem is that I was only able to test these changes on a ppc64le pseries KVM guest, so except for patches 2 and 3, all others are only build-tested. Also, my grasp of QOM lifecycle is basically non-existant so please be aware of that when reviewing this series.

Quoted 4 lines

The last patch may be wrong, as pointed out by Eduardo, so I marked it as RFC. It may make sense to drop it.

Applies cleanly on yesterday's master.

This series appears to break the Travis build for a MIPS target:

Unexpected error in qdev_prop_set_after_realize() at /home/travis/build/dgibson/qemu/hw/core/qdev-properties.c:30:
qemu-system-mips64el: Attempt to set property 'start-powered-off' on anonymous device (type 'I6400-mips64-cpu') after it was realized Broken pipe
/home/travis/build/dgibson/qemu/tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal 6 (Aborted) (core dumped) Aborted (core dumped)
ERROR qom-test - too few tests run (expected 8, got 0) /home/travis/build/dgibson/qemu/tests/Makefile.include:650: recipe for target 'check-qtest-mips64el' failed

Quoted 4 lines
Quoted 4 lines

Good catch. hw/mips/cps.c, hw/ppc/e500.c and hw/sparc/sun4m.c are incorrectly setting the property after the cpu is realized because the cpu is created with cpu_create(). We need to create them with object_initialize_child() and realize them manually with qdev_realize().

Does it have to be with object_initialize_child()? I found very few examples of CPUs initialized with that function (e.g., atmega.c, rx62n.c, nrf51_soc.c).

A more direct substitute for cpu_create() would be object_new(). I provide an example at the end of this email. What do you think?

I'm trying to test it with make docker-test-misc@debian-mips64el-cross, but still having some trouble with that. Is there another way to reproduce this issue?

Quoted 1 line

Thiago, can you fix that up and repost please.

Ok, I'm working on it.

--
Thiago Jung Bauermann
IBM Linux Technology Center

hw/mips/cps.c +9 -1
index d5b6c78019..be85357dc0 100644
@@ -73,7 +73,9 @@ static void mips_cps_realize(DeviceState *dev, Error **errp)
     bool saar_present = false;
 
     for (i = 0; i < s->num_vp; i++) {
-        cpu = MIPS_CPU(cpu_create(s->cpu_type));
+        Error *err = NULL;
+
+        cpu = MIPS_CPU(object_new(s->cpu_type));
 
         /* Init internal devices */
         cpu_mips_irq_init_cpu(cpu);
@@ -90,6 +92,12 @@ static void mips_cps_realize(DeviceState *dev, Error **errp)
         object_property_set_bool(OBJECT(cpu), "start-powered-off", true,
                                  &error_abort);
         qemu_register_reset(main_cpu_reset, cpu);
+
+        if (!qdev_realize(DEVICE(cpu), NULL, &err)) {
+            error_report_err(err);
+            object_unref(OBJECT(cpu));
+            exit(EXIT_FAILURE);
+        }
     }
 
     cpu = MIPS_CPU(first_cpu);