Skip to contentSkip to conversation

target-ppc/spapr: Add FWNMI support in QEMU for PowerKM guests

PATCH Superseded Ganesh Goudar 3 participants 22 messages Jan 2, 2020 – Jan 9, 2020

v18 of 21 Superseded Latest is v21 →

lore
Share & tools
lore
#0 lore raw

v18 cover target-ppc/spapr: Add FWNMI support in QEMU for PowerKM guests

This patch set adds support for FWNMI in PowerKVM guests.

System errors such as SLB multihit and memory errors that cannot be corrected by hardware is passed on to the kernel for handling by raising machine check exception (an NMI). Upon such machine check exceptions, if the address in error belongs to guest then KVM invokes guests' 0x200 interrupt vector if the guest is not FWNMI capable. For FWNMI capable guest KVM passes the control to QEMU by exiting the guest.

This patch series adds functionality to QEMU to pass on such machine check exceptions to the FWNMI capable guest kernel by building an error log and invoking the guest registered machine check handling routine.

The KVM changes are now part of the upstream kernel (commit e20bbd3d). This series contain QEMU changes.

Change Log v18:
- Dynamically create the Error object before adding it as blocker
- In apply hook check if the fwnmi calls are already registered and if kvm supports fwnmi before registering the fwnmi calls.
- In rtas_ibm_nmi_register() test the feature flag before attempting to get the RTAS address
- Introduce a bool member "fwnmi_calls_registered" to check if the fwnmi calls are registered and use the same in needed hook to save the state during migration.

Change Log v17:
- Add fwnmi cap to migration state
- Reprhase the commit message in patch 2/7

Change Log v16:
- Fixed coding style problems

Change Log v15:
- Removed cap_ppc_fwnmi
- Moved fwnmi registeration to .apply hook
- Assume SLOF has allocated enough room for rtas error log
- Using ARRAY_SIZE to end the loop
- Do not set FWNMI cap in post_load, now its done in .apply hook

Change Log v14:
- Feature activation moved to a separate patch
- Fixed issues with migration blocker

Change Log v13:
- Minor fixes (mostly nits)
- Moved FWNMI guest registration check from patch 4 to 3.

Change Log v12:
- Rebased to latest ppc-for-4.2 (SHA b1e8156743)

Change Log v11:
- Moved FWNMI SPAPR cap defaults to 4.2 class option
- Fixed issues with handling fwnmi KVM capability

Change Log v10:
- Reshuffled the patch sequence + minor fixes

Change Log v9:
- Fixed kvm cap and spapr cap issues

Change Log v8:
- Added functionality to check FWNMI capability during VM migration

---

Aravinda Prasad (7):
Wrapper function to wait on condition for the main loop mutex
ppc: spapr: Introduce FWNMI capability
target/ppc: Handle NMI guest exit
target/ppc: Build rtas error log upon an MCE
ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS
calls
migration: Include migration support for machine check handling
ppc: spapr: Activate the FWNMI functionality

 cpus.c                   |   5 +
 hw/ppc/spapr.c           |  52 ++++++++
 hw/ppc/spapr_caps.c      |  24 ++++
 hw/ppc/spapr_events.c    | 273 +++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_rtas.c      |  88 +++++++++++++
 include/hw/ppc/spapr.h   |  27 +++-
 include/qemu/main-loop.h |   8 ++
 target/ppc/kvm.c         |  24 ++++
 target/ppc/kvm_ppc.h     |   8 ++
 target/ppc/trace-events  |   1 +
 10 files changed, 508 insertions(+), 2 deletions(-)
Signature · 1 line
2.24.0.155.gd9f6f3b619
#1 lore raw

PATCH v18 1/7 Wrapper function to wait on condition for the main loop mutex

From: Aravinda Prasad <[email protected]>

Introduce a wrapper function to wait on condition for the main loop mutex. This function atomically releases the main loop mutex and causes the calling thread to block on the condition. This wrapper is required because qemu_global_mutex is a static variable.

---
 cpus.c                   | 5 +++++
 include/qemu/main-loop.h | 8 ++++++++
 2 files changed, 13 insertions(+)
cpus.c +5 -0
index b472378b70..79388d2b0f 100644
@@ -1835,6 +1835,11 @@ void qemu_mutex_unlock_iothread(void)
     qemu_mutex_unlock(&qemu_global_mutex);
 }
 
+void qemu_cond_wait_iothread(QemuCond *cond)
+{
+    qemu_cond_wait(cond, &qemu_global_mutex);
+}
+
 static bool all_vcpus_paused(void)
 {
     CPUState *cpu;
include/qemu/main-loop.h +8 -0
index f6ba78ea73..a6d20b0719 100644
@@ -295,6 +295,14 @@ void qemu_mutex_lock_iothread_impl(const char *file, int line);
  */
 void qemu_mutex_unlock_iothread(void);
 
+/*
+ * qemu_cond_wait_iothread: Wait on condition for the main loop mutex
+ *
+ * This function atomically releases the main loop mutex and causes
+ * the calling thread to block on the condition.
+ */
+void qemu_cond_wait_iothread(QemuCond *cond);
+
 /* internal interfaces */
 
 void qemu_fd_register(int fd);
Signature · 1 line
2.17.2
#2 lore raw

Re: [PATCH v18 1/7] Wrapper function to wait on condition for the main loop mutex

Quoted 8 lines · On Thu, 2 Jan 2020 13:21:05 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Introduce a wrapper function to wait on condition for the main loop mutex. This function atomically releases the main loop mutex and causes the calling thread to block on the condition. This wrapper is required because qemu_global_mutex is a static variable.

Quoted 4 lines

---

This should have your Signed-off-by: tag as well.

Quoted 39 lines
 cpus.c                   | 5 +++++
 include/qemu/main-loop.h | 8 ++++++++
 2 files changed, 13 insertions(+)
cpus.c +5 -0
index b472378b70..79388d2b0f 100644
@@ -1835,6 +1835,11 @@ void qemu_mutex_unlock_iothread(void)
     qemu_mutex_unlock(&qemu_global_mutex);
 }
 
+void qemu_cond_wait_iothread(QemuCond *cond)
+{
+    qemu_cond_wait(cond, &qemu_global_mutex);
+}
+
 static bool all_vcpus_paused(void)
 {
     CPUState *cpu;
include/qemu/main-loop.h +8 -0
index f6ba78ea73..a6d20b0719 100644
@@ -295,6 +295,14 @@ void qemu_mutex_lock_iothread_impl(const char *file, int line);
  */
 void qemu_mutex_unlock_iothread(void);
 
+/*
+ * qemu_cond_wait_iothread: Wait on condition for the main loop mutex
+ *
+ * This function atomically releases the main loop mutex and causes
+ * the calling thread to block on the condition.
+ */
+void qemu_cond_wait_iothread(QemuCond *cond);
+
 /* internal interfaces */
 
 void qemu_fd_register(int fd);
author
#3 lore raw

Re: [PATCH v18 1/7] Wrapper function to wait on condition for the main loop mutex

Quoted 11 lines · Greg Kurz
Quoted 8 lines · On Thu, 2 Jan 2020 13:21:05 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Introduce a wrapper function to wait on condition for the main loop mutex. This function atomically releases the main loop mutex and causes the calling thread to block on the condition. This wrapper is required because qemu_global_mutex is a static variable.

Quoted 5 lines
Quoted 4 lines

---

This should have your Signed-off-by: tag as well.

Sure, thanks

Quoted 40 lines
Quoted 40 lines
  cpus.c                   | 5 +++++
  include/qemu/main-loop.h | 8 ++++++++
  2 files changed, 13 insertions(+)
cpus.c +5 -0
index b472378b70..79388d2b0f 100644
@@ -1835,6 +1835,11 @@ void qemu_mutex_unlock_iothread(void)
      qemu_mutex_unlock(&qemu_global_mutex);
  }
 
+void qemu_cond_wait_iothread(QemuCond *cond)
+{
+    qemu_cond_wait(cond, &qemu_global_mutex);
+}
+
  static bool all_vcpus_paused(void)
  {
      CPUState *cpu;
include/qemu/main-loop.h +8 -0
index f6ba78ea73..a6d20b0719 100644
@@ -295,6 +295,14 @@ void qemu_mutex_lock_iothread_impl(const char *file, int line);
   */
  void qemu_mutex_unlock_iothread(void);
 
+/*
+ * qemu_cond_wait_iothread: Wait on condition for the main loop mutex
+ *
+ * This function atomically releases the main loop mutex and causes
+ * the calling thread to block on the condition.
+ */
+void qemu_cond_wait_iothread(QemuCond *cond);
+
  /* internal interfaces */
 
  void qemu_fd_register(int fd);
#4 lore raw

PATCH v18 2/7 ppc: spapr: Introduce FWNMI capability

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

Introduce fwnmi an spapr capability and validate it against the kernels existing capability by trying to enable it.

[eliminate cap_ppc_fwnmi, add fwnmi cap to migration state
and reprhase the commit message]

---
 hw/ppc/spapr.c         |  2 ++
 hw/ppc/spapr_caps.c    | 18 ++++++++++++++++++
 include/hw/ppc/spapr.h |  5 ++++-
 target/ppc/kvm.c       |  8 ++++++++
 target/ppc/kvm_ppc.h   |  6 ++++++
 5 files changed, 38 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +2 -0
index f11422fc41..c91e64aad0 100644
@@ -1994,6 +1994,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_dtb,
         &vmstate_spapr_cap_large_decr,
         &vmstate_spapr_cap_ccf_assist,
+        &vmstate_spapr_cap_fwnmi,
         NULL
     }
 };
@@ -4400,6 +4401,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF;
     smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON;
     smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF;
+    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
     spapr_caps_add_properties(smc, &error_abort);
     smc->irq = &spapr_irq_dual;
     smc->dr_phb_enabled = true;
hw/ppc/spapr_caps.c +18 -0
index 481dfd2a27..3001098601 100644
@@ -496,6 +496,14 @@ static void cap_ccf_assist_apply(SpaprMachineState *spapr, uint8_t val,
     }
 }
 
+static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
+                                Error **errp)
+{
+    if (!val) {
+        return; /* Disabled by default */
+    }
+}
+
 SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
     [SPAPR_CAP_HTM] = {
         .name = "htm",
@@ -595,6 +603,15 @@ SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
         .type = "bool",
         .apply = cap_ccf_assist_apply,
     },
+    [SPAPR_CAP_FWNMI_MCE] = {
+        .name = "fwnmi-mce",
+        .description = "Handle fwnmi machine check exceptions",
+        .index = SPAPR_CAP_FWNMI_MCE,
+        .get = spapr_cap_get_bool,
+        .set = spapr_cap_set_bool,
+        .type = "bool",
+        .apply = cap_fwnmi_mce_apply,
+    },
 };
 
 static SpaprCapabilities default_caps_with_cpu(SpaprMachineState *spapr,
@@ -734,6 +751,7 @@ SPAPR_CAP_MIG_STATE(hpt_maxpagesize, SPAPR_CAP_HPT_MAXPAGESIZE);
 SPAPR_CAP_MIG_STATE(nested_kvm_hv, SPAPR_CAP_NESTED_KVM_HV);
 SPAPR_CAP_MIG_STATE(large_decr, SPAPR_CAP_LARGE_DECREMENTER);
 SPAPR_CAP_MIG_STATE(ccf_assist, SPAPR_CAP_CCF_ASSIST);
+SPAPR_CAP_MIG_STATE(fwnmi, SPAPR_CAP_FWNMI_MCE);
 
 void spapr_caps_init(SpaprMachineState *spapr)
 {
include/hw/ppc/spapr.h +4 -1
index 61f005c6f6..7bc5fc3a9e 100644
@@ -79,8 +79,10 @@ typedef enum {
 #define SPAPR_CAP_LARGE_DECREMENTER     0x08
 /* Count Cache Flush Assist HW Instruction */
 #define SPAPR_CAP_CCF_ASSIST            0x09
+/* FWNMI machine check handling */
+#define SPAPR_CAP_FWNMI_MCE             0x0A
 /* Num Caps */
-#define SPAPR_CAP_NUM                   (SPAPR_CAP_CCF_ASSIST + 1)
+#define SPAPR_CAP_NUM                   (SPAPR_CAP_FWNMI_MCE + 1)
 
 /*
  * Capability Values
@@ -869,6 +871,7 @@ extern const VMStateDescription vmstate_spapr_cap_hpt_maxpagesize;
 extern const VMStateDescription vmstate_spapr_cap_nested_kvm_hv;
 extern const VMStateDescription vmstate_spapr_cap_large_decr;
 extern const VMStateDescription vmstate_spapr_cap_ccf_assist;
+extern const VMStateDescription vmstate_spapr_cap_fwnmi;
 
 static inline uint8_t spapr_get_cap(SpaprMachineState *spapr, int cap)
 {
target/ppc/kvm.c +8 -0
index d1c334f0e3..518de7e4b7 100644
@@ -2060,6 +2060,14 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
     }
 }
 
+int kvmppc_set_fwnmi(void)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(first_cpu);
+    CPUState *cs = CPU(cpu);
+
+    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
+}
+
 int kvmppc_smt_threads(void)
 {
     return cap_ppc_smt ? cap_ppc_smt : 1;
target/ppc/kvm_ppc.h +6 -0
index f22daabf51..eedb15d48e 100644
@@ -27,6 +27,7 @@ void kvmppc_enable_h_page_init(void);
 void kvmppc_set_papr(PowerPCCPU *cpu);
 int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
 void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
+int kvmppc_set_fwnmi(void);
 int kvmppc_smt_threads(void);
 void kvmppc_error_append_smt_possible_hint(Error *const *errp);
 int kvmppc_set_smt_threads(int smt);
@@ -159,6 +160,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
 {
 }
 
+static inline int kvmppc_set_fwnmi(void)
+{
+    return -1;
+}
+
 static inline int kvmppc_smt_threads(void)
 {
     return 1;
Signature · 1 line
2.17.2
#5 lore raw

Re: [PATCH v18 2/7] ppc: spapr: Introduce FWNMI capability

Quoted 4 lines · On Thu, 2 Jan 2020 13:21:06 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Introduce fwnmi an spapr capability and validate it against the kernels existing capability by trying to enable it.

I see this patch indeed adds a KVM helper to do that but it isn't called before some later patch in this series. Please update the changelog to make it clear this is just a preparatory patch that doesn't change any existing behavior.

Quoted 4 lines

[eliminate cap_ppc_fwnmi, add fwnmi cap to migration state
and reprhase the commit message]

Personal changes and S-o-b are usually appended to the existing ones...

Quoted 1 line

... i.e. here.

Quoted 138 lines
---
 hw/ppc/spapr.c         |  2 ++
 hw/ppc/spapr_caps.c    | 18 ++++++++++++++++++
 include/hw/ppc/spapr.h |  5 ++++-
 target/ppc/kvm.c       |  8 ++++++++
 target/ppc/kvm_ppc.h   |  6 ++++++
 5 files changed, 38 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +2 -0
index f11422fc41..c91e64aad0 100644
@@ -1994,6 +1994,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_dtb,
         &vmstate_spapr_cap_large_decr,
         &vmstate_spapr_cap_ccf_assist,
+        &vmstate_spapr_cap_fwnmi,
         NULL
     }
 };
@@ -4400,6 +4401,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF;
     smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON;
     smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF;
+    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
     spapr_caps_add_properties(smc, &error_abort);
     smc->irq = &spapr_irq_dual;
     smc->dr_phb_enabled = true;
hw/ppc/spapr_caps.c +18 -0
index 481dfd2a27..3001098601 100644
@@ -496,6 +496,14 @@ static void cap_ccf_assist_apply(SpaprMachineState *spapr, uint8_t val,
     }
 }
 
+static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
+                                Error **errp)
+{
+    if (!val) {
+        return; /* Disabled by default */
+    }
+}
+
 SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
     [SPAPR_CAP_HTM] = {
         .name = "htm",
@@ -595,6 +603,15 @@ SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
         .type = "bool",
         .apply = cap_ccf_assist_apply,
     },
+    [SPAPR_CAP_FWNMI_MCE] = {
+        .name = "fwnmi-mce",
+        .description = "Handle fwnmi machine check exceptions",
+        .index = SPAPR_CAP_FWNMI_MCE,
+        .get = spapr_cap_get_bool,
+        .set = spapr_cap_set_bool,
+        .type = "bool",
+        .apply = cap_fwnmi_mce_apply,
+    },
 };
 
 static SpaprCapabilities default_caps_with_cpu(SpaprMachineState *spapr,
@@ -734,6 +751,7 @@ SPAPR_CAP_MIG_STATE(hpt_maxpagesize, SPAPR_CAP_HPT_MAXPAGESIZE);
 SPAPR_CAP_MIG_STATE(nested_kvm_hv, SPAPR_CAP_NESTED_KVM_HV);
 SPAPR_CAP_MIG_STATE(large_decr, SPAPR_CAP_LARGE_DECREMENTER);
 SPAPR_CAP_MIG_STATE(ccf_assist, SPAPR_CAP_CCF_ASSIST);
+SPAPR_CAP_MIG_STATE(fwnmi, SPAPR_CAP_FWNMI_MCE);
 
 void spapr_caps_init(SpaprMachineState *spapr)
 {
include/hw/ppc/spapr.h +4 -1
index 61f005c6f6..7bc5fc3a9e 100644
@@ -79,8 +79,10 @@ typedef enum {
 #define SPAPR_CAP_LARGE_DECREMENTER     0x08
 /* Count Cache Flush Assist HW Instruction */
 #define SPAPR_CAP_CCF_ASSIST            0x09
+/* FWNMI machine check handling */
+#define SPAPR_CAP_FWNMI_MCE             0x0A
 /* Num Caps */
-#define SPAPR_CAP_NUM                   (SPAPR_CAP_CCF_ASSIST + 1)
+#define SPAPR_CAP_NUM                   (SPAPR_CAP_FWNMI_MCE + 1)
 
 /*
  * Capability Values
@@ -869,6 +871,7 @@ extern const VMStateDescription vmstate_spapr_cap_hpt_maxpagesize;
 extern const VMStateDescription vmstate_spapr_cap_nested_kvm_hv;
 extern const VMStateDescription vmstate_spapr_cap_large_decr;
 extern const VMStateDescription vmstate_spapr_cap_ccf_assist;
+extern const VMStateDescription vmstate_spapr_cap_fwnmi;
 
 static inline uint8_t spapr_get_cap(SpaprMachineState *spapr, int cap)
 {
target/ppc/kvm.c +8 -0
index d1c334f0e3..518de7e4b7 100644
@@ -2060,6 +2060,14 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
     }
 }
 
+int kvmppc_set_fwnmi(void)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(first_cpu);
+    CPUState *cs = CPU(cpu);
+
+    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
+}
+
 int kvmppc_smt_threads(void)
 {
     return cap_ppc_smt ? cap_ppc_smt : 1;
target/ppc/kvm_ppc.h +6 -0
index f22daabf51..eedb15d48e 100644
@@ -27,6 +27,7 @@ void kvmppc_enable_h_page_init(void);
 void kvmppc_set_papr(PowerPCCPU *cpu);
 int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
 void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
+int kvmppc_set_fwnmi(void);
 int kvmppc_smt_threads(void);
 void kvmppc_error_append_smt_possible_hint(Error *const *errp);
 int kvmppc_set_smt_threads(int smt);
@@ -159,6 +160,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
 {
 }
 
+static inline int kvmppc_set_fwnmi(void)
+{
+    return -1;
+}
+
 static inline int kvmppc_smt_threads(void)
 {
     return 1;
author
#6 lore raw

Re: [PATCH v18 2/7] ppc: spapr: Introduce FWNMI capability

Quoted 11 lines · Greg Kurz
Quoted 4 lines · On Thu, 2 Jan 2020 13:21:06 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Introduce fwnmi an spapr capability and validate it against the kernels existing capability by trying to enable it.

I see this patch indeed adds a KVM helper to do that but it isn't called before some later patch in this series. Please update the changelog to make it clear this is just a preparatory patch that doesn't change any existing behavior.

Ok, ill update the commit message

Quoted 9 lines
Quoted 4 lines

[eliminate cap_ppc_fwnmi, add fwnmi cap to migration state
and reprhase the commit message]

Personal changes and S-o-b are usually appended to the existing ones...

Quoted 1 line

... i.e. here.

ok

Quoted 139 lines
Quoted 139 lines
---
  hw/ppc/spapr.c         |  2 ++
  hw/ppc/spapr_caps.c    | 18 ++++++++++++++++++
  include/hw/ppc/spapr.h |  5 ++++-
  target/ppc/kvm.c       |  8 ++++++++
  target/ppc/kvm_ppc.h   |  6 ++++++
  5 files changed, 38 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +2 -0
index f11422fc41..c91e64aad0 100644
@@ -1994,6 +1994,7 @@ static const VMStateDescription vmstate_spapr = {
          &vmstate_spapr_dtb,
          &vmstate_spapr_cap_large_decr,
          &vmstate_spapr_cap_ccf_assist,
+        &vmstate_spapr_cap_fwnmi,
          NULL
      }
  };
@@ -4400,6 +4401,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
      smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF;
      smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON;
      smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF;
+    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
      spapr_caps_add_properties(smc, &error_abort);
      smc->irq = &spapr_irq_dual;
      smc->dr_phb_enabled = true;
hw/ppc/spapr_caps.c +18 -0
index 481dfd2a27..3001098601 100644
@@ -496,6 +496,14 @@ static void cap_ccf_assist_apply(SpaprMachineState *spapr, uint8_t val,
      }
  }
 
+static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
+                                Error **errp)
+{
+    if (!val) {
+        return; /* Disabled by default */
+    }
+}
+
  SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
      [SPAPR_CAP_HTM] = {
          .name = "htm",
@@ -595,6 +603,15 @@ SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
          .type = "bool",
          .apply = cap_ccf_assist_apply,
      },
+    [SPAPR_CAP_FWNMI_MCE] = {
+        .name = "fwnmi-mce",
+        .description = "Handle fwnmi machine check exceptions",
+        .index = SPAPR_CAP_FWNMI_MCE,
+        .get = spapr_cap_get_bool,
+        .set = spapr_cap_set_bool,
+        .type = "bool",
+        .apply = cap_fwnmi_mce_apply,
+    },
  };
 
  static SpaprCapabilities default_caps_with_cpu(SpaprMachineState *spapr,
@@ -734,6 +751,7 @@ SPAPR_CAP_MIG_STATE(hpt_maxpagesize, SPAPR_CAP_HPT_MAXPAGESIZE);
  SPAPR_CAP_MIG_STATE(nested_kvm_hv, SPAPR_CAP_NESTED_KVM_HV);
  SPAPR_CAP_MIG_STATE(large_decr, SPAPR_CAP_LARGE_DECREMENTER);
  SPAPR_CAP_MIG_STATE(ccf_assist, SPAPR_CAP_CCF_ASSIST);
+SPAPR_CAP_MIG_STATE(fwnmi, SPAPR_CAP_FWNMI_MCE);
 
  void spapr_caps_init(SpaprMachineState *spapr)
  {
include/hw/ppc/spapr.h +4 -1
index 61f005c6f6..7bc5fc3a9e 100644
@@ -79,8 +79,10 @@ typedef enum {
  #define SPAPR_CAP_LARGE_DECREMENTER     0x08
  /* Count Cache Flush Assist HW Instruction */
  #define SPAPR_CAP_CCF_ASSIST            0x09
+/* FWNMI machine check handling */
+#define SPAPR_CAP_FWNMI_MCE             0x0A
  /* Num Caps */
-#define SPAPR_CAP_NUM                   (SPAPR_CAP_CCF_ASSIST + 1)
+#define SPAPR_CAP_NUM                   (SPAPR_CAP_FWNMI_MCE + 1)
 
  /*
   * Capability Values
@@ -869,6 +871,7 @@ extern const VMStateDescription vmstate_spapr_cap_hpt_maxpagesize;
  extern const VMStateDescription vmstate_spapr_cap_nested_kvm_hv;
  extern const VMStateDescription vmstate_spapr_cap_large_decr;
  extern const VMStateDescription vmstate_spapr_cap_ccf_assist;
+extern const VMStateDescription vmstate_spapr_cap_fwnmi;
 
  static inline uint8_t spapr_get_cap(SpaprMachineState *spapr, int cap)
  {
target/ppc/kvm.c +8 -0
index d1c334f0e3..518de7e4b7 100644
@@ -2060,6 +2060,14 @@ void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
      }
  }
 
+int kvmppc_set_fwnmi(void)
+{
+    PowerPCCPU *cpu = POWERPC_CPU(first_cpu);
+    CPUState *cs = CPU(cpu);
+
+    return kvm_vcpu_enable_cap(cs, KVM_CAP_PPC_FWNMI, 0);
+}
+
  int kvmppc_smt_threads(void)
  {
      return cap_ppc_smt ? cap_ppc_smt : 1;
target/ppc/kvm_ppc.h +6 -0
index f22daabf51..eedb15d48e 100644
@@ -27,6 +27,7 @@ void kvmppc_enable_h_page_init(void);
  void kvmppc_set_papr(PowerPCCPU *cpu);
  int kvmppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr);
  void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy);
+int kvmppc_set_fwnmi(void);
  int kvmppc_smt_threads(void);
  void kvmppc_error_append_smt_possible_hint(Error *const *errp);
  int kvmppc_set_smt_threads(int smt);
@@ -159,6 +160,11 @@ static inline void kvmppc_set_mpic_proxy(PowerPCCPU *cpu, int mpic_proxy)
  {
  }
 
+static inline int kvmppc_set_fwnmi(void)
+{
+    return -1;
+}
+
  static inline int kvmppc_smt_threads(void)
  {
      return 1;
#7 lore raw

PATCH v18 3/7 target/ppc: Handle NMI guest exit

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

Memory error such as bit flips that cannot be corrected by hardware are passed on to the kernel for handling. If the memory address in error belongs to guest then the guest kernel is responsible for taking suitable action. Patch [1] enhances KVM to exit guest with exit reason set to KVM_EXIT_NMI in such cases. This patch handles KVM_EXIT_NMI exit.

[1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
(e20bbd3d and related commits)

---
 hw/ppc/spapr.c          |  8 ++++++++
 hw/ppc/spapr_events.c   | 37 +++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h  | 10 ++++++++++
 target/ppc/kvm.c        | 14 ++++++++++++++
 target/ppc/kvm_ppc.h    |  2 ++
 target/ppc/trace-events |  1 +
 6 files changed, 72 insertions(+)
hw/ppc/spapr.c +8 -0
index c91e64aad0..975d7da734 100644
@@ -1679,6 +1679,12 @@ static void spapr_machine_reset(MachineState *machine)
     first_ppc_cpu->env.gpr[5] = 0;
 
     spapr->cas_reboot = false;
+
+    spapr->mc_status = -1;
+    spapr->guest_machine_check_addr = -1;
+
+    /* Signal all vCPUs waiting on this condition */
+    qemu_cond_broadcast(&spapr->mc_delivery_cond);
 }
 
 static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -2973,6 +2979,8 @@ static void spapr_machine_init(MachineState *machine)
 
         kvmppc_spapr_enable_inkernel_multitce();
     }
+
+    qemu_cond_init(&spapr->mc_delivery_cond);
 }
 
 static int spapr_kvm_type(MachineState *machine, const char *vm_type)
hw/ppc/spapr_events.c +37 -0
index e355e000d0..dfc0de840a 100644
@@ -40,6 +40,7 @@
 #include "hw/ppc/spapr_drc.h"
 #include "qemu/help_option.h"
 #include "qemu/bcd.h"
+#include "qemu/main-loop.h"
 #include "hw/ppc/spapr_ovec.h"
 #include <libfdt.h>
 
@@ -622,6 +623,42 @@ void spapr_hotplug_req_remove_by_count_indexed(SpaprDrcType drc_type,
                             RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
 }
 
+void spapr_mce_req_event(PowerPCCPU *cpu)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    CPUState *cs = CPU(cpu);
+
+    if (spapr->guest_machine_check_addr == -1) {
+        /*
+         * This implies that we have hit a machine check either when the
+         * guest has not registered FWNMI (i.e., "ibm,nmi-register" not
+         * called) or between system reset and "ibm,nmi-register".
+         * Fall back to the old machine check behavior in such cases.
+         */
+        cs->exception_index = POWERPC_EXCP_MCHECK;
+        ppc_cpu_do_interrupt(cs);
+        return;
+    }
+
+    while (spapr->mc_status != -1) {
+        /*
+         * Check whether the same CPU got machine check error
+         * while still handling the mc error (i.e., before
+         * that CPU called "ibm,nmi-interlock")
+         */
+        if (spapr->mc_status == cpu->vcpu_id) {
+            qemu_system_guest_panicked(NULL);
+            return;
+        }
+        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
+        /* Meanwhile if the system is reset, then just return */
+        if (spapr->guest_machine_check_addr == -1) {
+            return;
+        }
+    }
+    spapr->mc_status = cpu->vcpu_id;
+}
+
 static void check_exception(PowerPCCPU *cpu, SpaprMachineState *spapr,
                             uint32_t token, uint32_t nargs,
                             target_ulong args,
include/hw/ppc/spapr.h +10 -0
index 7bc5fc3a9e..909d3976f9 100644
@@ -191,6 +191,15 @@ struct SpaprMachineState {
      * occurs during the unplug process. */
     QTAILQ_HEAD(, SpaprDimmState) pending_dimm_unplugs;
 
+    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
+    target_ulong guest_machine_check_addr;
+    /*
+     * mc_status is set to -1 if mc is not in progress, else is set to the CPU
+     * handling the mc.
+     */
+    int mc_status;
+    QemuCond mc_delivery_cond;
+
     /*< public >*/
     char *kvm_type;
     char *host_model;
@@ -804,6 +813,7 @@ void spapr_clear_pending_events(SpaprMachineState *spapr);
 int spapr_max_server_number(SpaprMachineState *spapr);
 void spapr_store_hpte(PowerPCCPU *cpu, hwaddr ptex,
                       uint64_t pte0, uint64_t pte1);
+void spapr_mce_req_event(PowerPCCPU *cpu);
 
 /* DRC callbacks. */
 void spapr_core_release(DeviceState *dev);
target/ppc/kvm.c +14 -0
index 518de7e4b7..2aebb44e2f 100644
@@ -1708,6 +1708,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
         ret = 0;
         break;
 
+    case KVM_EXIT_NMI:
+        trace_kvm_handle_nmi_exception();
+        ret = kvm_handle_nmi(cpu, run);
+        break;
+
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
@@ -2803,6 +2808,15 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
     return data & 0xffff;
 }
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
+{
+    cpu_synchronize_state(CPU(cpu));
+
+    spapr_mce_req_event(cpu);
+
+    return 0;
+}
+
 int kvmppc_enable_hwrng(void)
 {
     if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
target/ppc/kvm_ppc.h +2 -0
index eedb15d48e..bd2a0edf43 100644
@@ -83,6 +83,8 @@ void kvm_check_mmu(PowerPCCPU *cpu, Error **errp);
 void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online);
 void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset);
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
+
 #else
 
 static inline uint32_t kvmppc_get_tbfreq(void)
target/ppc/trace-events +1 -0
index 3dc6740706..6d15aa90b4 100644
@@ -28,3 +28,4 @@ kvm_handle_papr_hcall(void) "handle PAPR hypercall"
 kvm_handle_epr(void) "handle epr"
 kvm_handle_watchdog_expiry(void) "handle watchdog expiry"
 kvm_handle_debug_exception(void) "handle debug exception"
+kvm_handle_nmi_exception(void) "handle NMI exception"
Signature · 1 line
2.17.2
#8 lore raw

Re: [PATCH v18 3/7] target/ppc: Handle NMI guest exit

Quoted 10 lines · On Thu, 2 Jan 2020 13:21:07 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Memory error such as bit flips that cannot be corrected by hardware are passed on to the kernel for handling. If the memory address in error belongs to guest then the guest kernel is responsible for taking suitable action. Patch [1] enhances KVM to exit guest with exit reason set to KVM_EXIT_NMI in such cases. This patch handles KVM_EXIT_NMI exit.

Quoted 7 lines

[1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
(e20bbd3d and related commits)

---

Missing S-o-b.

Quoted 171 lines
 hw/ppc/spapr.c          |  8 ++++++++
 hw/ppc/spapr_events.c   | 37 +++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h  | 10 ++++++++++
 target/ppc/kvm.c        | 14 ++++++++++++++
 target/ppc/kvm_ppc.h    |  2 ++
 target/ppc/trace-events |  1 +
 6 files changed, 72 insertions(+)
hw/ppc/spapr.c +8 -0
index c91e64aad0..975d7da734 100644
@@ -1679,6 +1679,12 @@ static void spapr_machine_reset(MachineState *machine)
     first_ppc_cpu->env.gpr[5] = 0;
 
     spapr->cas_reboot = false;
+
+    spapr->mc_status = -1;
+    spapr->guest_machine_check_addr = -1;
+
+    /* Signal all vCPUs waiting on this condition */
+    qemu_cond_broadcast(&spapr->mc_delivery_cond);
 }
 
 static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -2973,6 +2979,8 @@ static void spapr_machine_init(MachineState *machine)
 
         kvmppc_spapr_enable_inkernel_multitce();
     }
+
+    qemu_cond_init(&spapr->mc_delivery_cond);
 }
 
 static int spapr_kvm_type(MachineState *machine, const char *vm_type)
hw/ppc/spapr_events.c +37 -0
index e355e000d0..dfc0de840a 100644
@@ -40,6 +40,7 @@
 #include "hw/ppc/spapr_drc.h"
 #include "qemu/help_option.h"
 #include "qemu/bcd.h"
+#include "qemu/main-loop.h"
 #include "hw/ppc/spapr_ovec.h"
 #include <libfdt.h>
 
@@ -622,6 +623,42 @@ void spapr_hotplug_req_remove_by_count_indexed(SpaprDrcType drc_type,
                             RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
 }
 
+void spapr_mce_req_event(PowerPCCPU *cpu)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    CPUState *cs = CPU(cpu);
+
+    if (spapr->guest_machine_check_addr == -1) {
+        /*
+         * This implies that we have hit a machine check either when the
+         * guest has not registered FWNMI (i.e., "ibm,nmi-register" not
+         * called) or between system reset and "ibm,nmi-register".
+         * Fall back to the old machine check behavior in such cases.
+         */
+        cs->exception_index = POWERPC_EXCP_MCHECK;
+        ppc_cpu_do_interrupt(cs);
+        return;
+    }
+
+    while (spapr->mc_status != -1) {
+        /*
+         * Check whether the same CPU got machine check error
+         * while still handling the mc error (i.e., before
+         * that CPU called "ibm,nmi-interlock")
+         */
+        if (spapr->mc_status == cpu->vcpu_id) {
+            qemu_system_guest_panicked(NULL);
+            return;
+        }
+        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
+        /* Meanwhile if the system is reset, then just return */
+        if (spapr->guest_machine_check_addr == -1) {
+            return;
+        }
+    }
+    spapr->mc_status = cpu->vcpu_id;
+}
+
 static void check_exception(PowerPCCPU *cpu, SpaprMachineState *spapr,
                             uint32_t token, uint32_t nargs,
                             target_ulong args,
include/hw/ppc/spapr.h +10 -0
index 7bc5fc3a9e..909d3976f9 100644
@@ -191,6 +191,15 @@ struct SpaprMachineState {
      * occurs during the unplug process. */
     QTAILQ_HEAD(, SpaprDimmState) pending_dimm_unplugs;
 
+    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
+    target_ulong guest_machine_check_addr;
+    /*
+     * mc_status is set to -1 if mc is not in progress, else is set to the CPU
+     * handling the mc.
+     */
+    int mc_status;
+    QemuCond mc_delivery_cond;
+
     /*< public >*/
     char *kvm_type;
     char *host_model;
@@ -804,6 +813,7 @@ void spapr_clear_pending_events(SpaprMachineState *spapr);
 int spapr_max_server_number(SpaprMachineState *spapr);
 void spapr_store_hpte(PowerPCCPU *cpu, hwaddr ptex,
                       uint64_t pte0, uint64_t pte1);
+void spapr_mce_req_event(PowerPCCPU *cpu);
 
 /* DRC callbacks. */
 void spapr_core_release(DeviceState *dev);
target/ppc/kvm.c +14 -0
index 518de7e4b7..2aebb44e2f 100644
@@ -1708,6 +1708,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
         ret = 0;
         break;
 
+    case KVM_EXIT_NMI:
+        trace_kvm_handle_nmi_exception();
+        ret = kvm_handle_nmi(cpu, run);
+        break;
+
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
@@ -2803,6 +2808,15 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
     return data & 0xffff;
 }
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
+{
+    cpu_synchronize_state(CPU(cpu));
+
+    spapr_mce_req_event(cpu);
+
+    return 0;
+}
+
 int kvmppc_enable_hwrng(void)
 {
     if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
target/ppc/kvm_ppc.h +2 -0
index eedb15d48e..bd2a0edf43 100644
@@ -83,6 +83,8 @@ void kvm_check_mmu(PowerPCCPU *cpu, Error **errp);
 void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online);
 void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset);
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
+
 #else
 
 static inline uint32_t kvmppc_get_tbfreq(void)
target/ppc/trace-events +1 -0
index 3dc6740706..6d15aa90b4 100644
@@ -28,3 +28,4 @@ kvm_handle_papr_hcall(void) "handle PAPR hypercall"
 kvm_handle_epr(void) "handle epr"
 kvm_handle_watchdog_expiry(void) "handle watchdog expiry"
 kvm_handle_debug_exception(void) "handle debug exception"
+kvm_handle_nmi_exception(void) "handle NMI exception"
author
#9 lore raw

Re: [PATCH v18 3/7] target/ppc: Handle NMI guest exit

Quoted 13 lines · Greg Kurz
Quoted 10 lines · On Thu, 2 Jan 2020 13:21:07 +0530 Ganesh Goudar <[email protected]> wrote:

From: Aravinda Prasad <[email protected]>

Memory error such as bit flips that cannot be corrected by hardware are passed on to the kernel for handling. If the memory address in error belongs to guest then the guest kernel is responsible for taking suitable action. Patch [1] enhances KVM to exit guest with exit reason set to KVM_EXIT_NMI in such cases. This patch handles KVM_EXIT_NMI exit.

Quoted 8 lines
Quoted 7 lines

[1] https://www.spinics.net/lists/kvm-ppc/msg12637.html
(e20bbd3d and related commits)

---

Missing S-o-b.

will add, Thanks.

Quoted 172 lines
Quoted 172 lines
  hw/ppc/spapr.c          |  8 ++++++++
  hw/ppc/spapr_events.c   | 37 +++++++++++++++++++++++++++++++++++++
  include/hw/ppc/spapr.h  | 10 ++++++++++
  target/ppc/kvm.c        | 14 ++++++++++++++
  target/ppc/kvm_ppc.h    |  2 ++
  target/ppc/trace-events |  1 +
  6 files changed, 72 insertions(+)
hw/ppc/spapr.c +8 -0
index c91e64aad0..975d7da734 100644
@@ -1679,6 +1679,12 @@ static void spapr_machine_reset(MachineState *machine)
      first_ppc_cpu->env.gpr[5] = 0;
 
      spapr->cas_reboot = false;
+
+    spapr->mc_status = -1;
+    spapr->guest_machine_check_addr = -1;
+
+    /* Signal all vCPUs waiting on this condition */
+    qemu_cond_broadcast(&spapr->mc_delivery_cond);
  }
 
  static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -2973,6 +2979,8 @@ static void spapr_machine_init(MachineState *machine)
 
          kvmppc_spapr_enable_inkernel_multitce();
      }
+
+    qemu_cond_init(&spapr->mc_delivery_cond);
  }
 
  static int spapr_kvm_type(MachineState *machine, const char *vm_type)
hw/ppc/spapr_events.c +37 -0
index e355e000d0..dfc0de840a 100644
@@ -40,6 +40,7 @@
  #include "hw/ppc/spapr_drc.h"
  #include "qemu/help_option.h"
  #include "qemu/bcd.h"
+#include "qemu/main-loop.h"
  #include "hw/ppc/spapr_ovec.h"
  #include <libfdt.h>
 
@@ -622,6 +623,42 @@ void spapr_hotplug_req_remove_by_count_indexed(SpaprDrcType drc_type,
                              RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
  }
 
+void spapr_mce_req_event(PowerPCCPU *cpu)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    CPUState *cs = CPU(cpu);
+
+    if (spapr->guest_machine_check_addr == -1) {
+        /*
+         * This implies that we have hit a machine check either when the
+         * guest has not registered FWNMI (i.e., "ibm,nmi-register" not
+         * called) or between system reset and "ibm,nmi-register".
+         * Fall back to the old machine check behavior in such cases.
+         */
+        cs->exception_index = POWERPC_EXCP_MCHECK;
+        ppc_cpu_do_interrupt(cs);
+        return;
+    }
+
+    while (spapr->mc_status != -1) {
+        /*
+         * Check whether the same CPU got machine check error
+         * while still handling the mc error (i.e., before
+         * that CPU called "ibm,nmi-interlock")
+         */
+        if (spapr->mc_status == cpu->vcpu_id) {
+            qemu_system_guest_panicked(NULL);
+            return;
+        }
+        qemu_cond_wait_iothread(&spapr->mc_delivery_cond);
+        /* Meanwhile if the system is reset, then just return */
+        if (spapr->guest_machine_check_addr == -1) {
+            return;
+        }
+    }
+    spapr->mc_status = cpu->vcpu_id;
+}
+
  static void check_exception(PowerPCCPU *cpu, SpaprMachineState *spapr,
                              uint32_t token, uint32_t nargs,
                              target_ulong args,
include/hw/ppc/spapr.h +10 -0
index 7bc5fc3a9e..909d3976f9 100644
@@ -191,6 +191,15 @@ struct SpaprMachineState {
       * occurs during the unplug process. */
      QTAILQ_HEAD(, SpaprDimmState) pending_dimm_unplugs;
 
+    /* State related to "ibm,nmi-register" and "ibm,nmi-interlock" calls */
+    target_ulong guest_machine_check_addr;
+    /*
+     * mc_status is set to -1 if mc is not in progress, else is set to the CPU
+     * handling the mc.
+     */
+    int mc_status;
+    QemuCond mc_delivery_cond;
+
      /*< public >*/
      char *kvm_type;
      char *host_model;
@@ -804,6 +813,7 @@ void spapr_clear_pending_events(SpaprMachineState *spapr);
  int spapr_max_server_number(SpaprMachineState *spapr);
  void spapr_store_hpte(PowerPCCPU *cpu, hwaddr ptex,
                        uint64_t pte0, uint64_t pte1);
+void spapr_mce_req_event(PowerPCCPU *cpu);
 
  /* DRC callbacks. */
  void spapr_core_release(DeviceState *dev);
target/ppc/kvm.c +14 -0
index 518de7e4b7..2aebb44e2f 100644
@@ -1708,6 +1708,11 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
          ret = 0;
          break;
 
+    case KVM_EXIT_NMI:
+        trace_kvm_handle_nmi_exception();
+        ret = kvm_handle_nmi(cpu, run);
+        break;
+
      default:
          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
          ret = -1;
@@ -2803,6 +2808,15 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
      return data & 0xffff;
  }
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
+{
+    cpu_synchronize_state(CPU(cpu));
+
+    spapr_mce_req_event(cpu);
+
+    return 0;
+}
+
  int kvmppc_enable_hwrng(void)
  {
      if (!kvm_enabled() || !kvm_check_extension(kvm_state, KVM_CAP_PPC_HWRNG)) {
target/ppc/kvm_ppc.h +2 -0
index eedb15d48e..bd2a0edf43 100644
@@ -83,6 +83,8 @@ void kvm_check_mmu(PowerPCCPU *cpu, Error **errp);
  void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online);
  void kvmppc_set_reg_tb_offset(PowerPCCPU *cpu, int64_t tb_offset);
 
+int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run);
+
  #else
 
  static inline uint32_t kvmppc_get_tbfreq(void)
target/ppc/trace-events +1 -0
index 3dc6740706..6d15aa90b4 100644
@@ -28,3 +28,4 @@ kvm_handle_papr_hcall(void) "handle PAPR hypercall"
  kvm_handle_epr(void) "handle epr"
  kvm_handle_watchdog_expiry(void) "handle watchdog expiry"
  kvm_handle_debug_exception(void) "handle debug exception"
+kvm_handle_nmi_exception(void) "handle NMI exception"
#10 lore raw

PATCH v18 4/7 target/ppc: Build rtas error log upon an MCE

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

Upon a machine check exception (MCE) in a guest address space, KVM causes a guest exit to enable QEMU to build and pass the error to the guest in the PAPR defined rtas error log format.

This patch builds the rtas error log, copies it to the rtas_addr and then invokes the guest registered machine check handler. The handler in the guest takes suitable action(s) depending on the type and criticality of the error. For example, if an error is unrecoverable memory corruption in an application inside the guest, then the guest kernel sends a SIGBUS to the application. For recoverable errors, the guest performs recovery actions and logs the error.

[Assume SLOF has allocated enough room for rtas error log]
---
 hw/ppc/spapr_events.c  | 220 ++++++++++++++++++++++++++++++++++++++++-
 hw/ppc/spapr_rtas.c    |  26 +++++
 include/hw/ppc/spapr.h |   6 +-
 target/ppc/kvm.c       |   4 +-
 4 files changed, 253 insertions(+), 3 deletions(-)
hw/ppc/spapr_events.c +219 -1
index dfc0de840a..54eaf28a9e 100644
@@ -214,6 +214,104 @@ struct hp_extended_log {
     struct rtas_event_log_v6_hp hp;
 } QEMU_PACKED;
 
+struct rtas_event_log_v6_mc {
+#define RTAS_LOG_V6_SECTION_ID_MC                   0x4D43 /* MC */
+    struct rtas_event_log_v6_section_header hdr;
+    uint32_t fru_id;
+    uint32_t proc_id;
+    uint8_t error_type;
+#define RTAS_LOG_V6_MC_TYPE_UE                           0
+#define RTAS_LOG_V6_MC_TYPE_SLB                          1
+#define RTAS_LOG_V6_MC_TYPE_ERAT                         2
+#define RTAS_LOG_V6_MC_TYPE_TLB                          4
+#define RTAS_LOG_V6_MC_TYPE_D_CACHE                      5
+#define RTAS_LOG_V6_MC_TYPE_I_CACHE                      7
+    uint8_t sub_err_type;
+#define RTAS_LOG_V6_MC_UE_INDETERMINATE                  0
+#define RTAS_LOG_V6_MC_UE_IFETCH                         1
+#define RTAS_LOG_V6_MC_UE_PAGE_TABLE_WALK_IFETCH         2
+#define RTAS_LOG_V6_MC_UE_LOAD_STORE                     3
+#define RTAS_LOG_V6_MC_UE_PAGE_TABLE_WALK_LOAD_STORE     4
+#define RTAS_LOG_V6_MC_SLB_PARITY                        0
+#define RTAS_LOG_V6_MC_SLB_MULTIHIT                      1
+#define RTAS_LOG_V6_MC_SLB_INDETERMINATE                 2
+#define RTAS_LOG_V6_MC_ERAT_PARITY                       1
+#define RTAS_LOG_V6_MC_ERAT_MULTIHIT                     2
+#define RTAS_LOG_V6_MC_ERAT_INDETERMINATE                3
+#define RTAS_LOG_V6_MC_TLB_PARITY                        1
+#define RTAS_LOG_V6_MC_TLB_MULTIHIT                      2
+#define RTAS_LOG_V6_MC_TLB_INDETERMINATE                 3
+    uint8_t reserved_1[6];
+    uint64_t effective_address;
+    uint64_t logical_address;
+} QEMU_PACKED;
+
+struct mc_extended_log {
+    struct rtas_event_log_v6 v6hdr;
+    struct rtas_event_log_v6_mc mc;
+} QEMU_PACKED;
+
+struct MC_ierror_table {
+    unsigned long srr1_mask;
+    unsigned long srr1_value;
+    bool nip_valid; /* nip is a valid indicator of faulting address */
+    uint8_t error_type;
+    uint8_t error_subtype;
+    unsigned int initiator;
+    unsigned int severity;
+};
+
+static const struct MC_ierror_table mc_ierror_table[] = {
+{ 0x00000000081c0000, 0x0000000000040000, true,
+  RTAS_LOG_V6_MC_TYPE_UE, RTAS_LOG_V6_MC_UE_IFETCH,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000000081c0000, 0x0000000000080000, true,
+  RTAS_LOG_V6_MC_TYPE_SLB, RTAS_LOG_V6_MC_SLB_PARITY,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000000081c0000, 0x00000000000c0000, true,
+  RTAS_LOG_V6_MC_TYPE_SLB, RTAS_LOG_V6_MC_SLB_MULTIHIT,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000000081c0000, 0x0000000000100000, true,
+  RTAS_LOG_V6_MC_TYPE_ERAT, RTAS_LOG_V6_MC_ERAT_MULTIHIT,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000000081c0000, 0x0000000000140000, true,
+  RTAS_LOG_V6_MC_TYPE_TLB, RTAS_LOG_V6_MC_TLB_MULTIHIT,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000000081c0000, 0x0000000000180000, true,
+  RTAS_LOG_V6_MC_TYPE_UE, RTAS_LOG_V6_MC_UE_PAGE_TABLE_WALK_IFETCH,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, } };
+
+struct MC_derror_table {
+    unsigned long dsisr_value;
+    bool dar_valid; /* dar is a valid indicator of faulting address */
+    uint8_t error_type;
+    uint8_t error_subtype;
+    unsigned int initiator;
+    unsigned int severity;
+};
+
+static const struct MC_derror_table mc_derror_table[] = {
+{ 0x00008000, false,
+  RTAS_LOG_V6_MC_TYPE_UE, RTAS_LOG_V6_MC_UE_LOAD_STORE,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00004000, true,
+  RTAS_LOG_V6_MC_TYPE_UE, RTAS_LOG_V6_MC_UE_PAGE_TABLE_WALK_LOAD_STORE,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000800, true,
+  RTAS_LOG_V6_MC_TYPE_ERAT, RTAS_LOG_V6_MC_ERAT_MULTIHIT,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000400, true,
+  RTAS_LOG_V6_MC_TYPE_TLB, RTAS_LOG_V6_MC_TLB_MULTIHIT,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000080, true,
+  RTAS_LOG_V6_MC_TYPE_SLB, RTAS_LOG_V6_MC_SLB_MULTIHIT,  /* Before PARITY */
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, },
+{ 0x00000100, true,
+  RTAS_LOG_V6_MC_TYPE_SLB, RTAS_LOG_V6_MC_SLB_PARITY,
+  RTAS_LOG_INITIATOR_CPU, RTAS_LOG_SEVERITY_ERROR_SYNC, } };
+
+#define SRR1_MC_LOADSTORE(srr1) ((srr1) & PPC_BIT(42))
+
 typedef enum EventClass {
     EVENT_CLASS_INTERNAL_ERRORS     = 0,
     EVENT_CLASS_EPOW                = 1,
@@ -623,7 +721,125 @@ void spapr_hotplug_req_remove_by_count_indexed(SpaprDrcType drc_type,
                             RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
 }
 
-void spapr_mce_req_event(PowerPCCPU *cpu)
+static uint32_t spapr_mce_get_elog_type(PowerPCCPU *cpu, bool recovered,
+                                        struct mc_extended_log *ext_elog)
+{
+    int i;
+    CPUPPCState *env = &cpu->env;
+    uint32_t summary;
+    uint64_t dsisr = env->spr[SPR_DSISR];
+
+    summary = RTAS_LOG_VERSION_6 | RTAS_LOG_OPTIONAL_PART_PRESENT;
+    if (recovered) {
+        summary |= RTAS_LOG_DISPOSITION_FULLY_RECOVERED;
+    } else {
+        summary |= RTAS_LOG_DISPOSITION_NOT_RECOVERED;
+    }
+
+    if (SRR1_MC_LOADSTORE(env->spr[SPR_SRR1])) {
+        for (i = 0; i < ARRAY_SIZE(mc_derror_table); i++) {
+            if (!(dsisr & mc_derror_table[i].dsisr_value)) {
+                continue;
+            }
+
+            ext_elog->mc.error_type = mc_derror_table[i].error_type;
+            ext_elog->mc.sub_err_type = mc_derror_table[i].error_subtype;
+            if (mc_derror_table[i].dar_valid) {
+                ext_elog->mc.effective_address = cpu_to_be64(env->spr[SPR_DAR]);
+            }
+
+            summary |= mc_derror_table[i].initiator
+                        | mc_derror_table[i].severity;
+
+            return summary;
+        }
+    } else {
+        for (i = 0; i < ARRAY_SIZE(mc_ierror_table); i++) {
+            if ((env->spr[SPR_SRR1] & mc_ierror_table[i].srr1_mask) !=
+                    mc_ierror_table[i].srr1_value) {
+                continue;
+            }
+
+            ext_elog->mc.error_type = mc_ierror_table[i].error_type;
+            ext_elog->mc.sub_err_type = mc_ierror_table[i].error_subtype;
+            if (mc_ierror_table[i].nip_valid) {
+                ext_elog->mc.effective_address = cpu_to_be64(env->nip);
+            }
+
+            summary |= mc_ierror_table[i].initiator
+                        | mc_ierror_table[i].severity;
+
+            return summary;
+        }
+    }
+
+    summary |= RTAS_LOG_INITIATOR_CPU;
+    return summary;
+}
+
+static void spapr_mce_dispatch_elog(PowerPCCPU *cpu, bool recovered)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    CPUState *cs = CPU(cpu);
+    uint64_t rtas_addr;
+    CPUPPCState *env = &cpu->env;
+    PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
+    target_ulong msr = 0;
+    struct rtas_error_log log;
+    struct mc_extended_log *ext_elog;
+    uint32_t summary;
+
+    /*
+     * Properly set bits in MSR before we invoke the handler.
+     * SRR0/1, DAR and DSISR are properly set by KVM
+     */
+    if (!(*pcc->interrupts_big_endian)(cpu)) {
+        msr |= (1ULL << MSR_LE);
+    }
+
+    if (env->msr & (1ULL << MSR_SF)) {
+        msr |= (1ULL << MSR_SF);
+    }
+
+    msr |= (1ULL << MSR_ME);
+
+    ext_elog = g_malloc0(sizeof(*ext_elog));
+    summary = spapr_mce_get_elog_type(cpu, recovered, ext_elog);
+
+    log.summary = cpu_to_be32(summary);
+    log.extended_length = cpu_to_be32(sizeof(*ext_elog));
+
+    spapr_init_v6hdr(&ext_elog->v6hdr);
+    ext_elog->mc.hdr.section_id = cpu_to_be16(RTAS_LOG_V6_SECTION_ID_MC);
+    ext_elog->mc.hdr.section_length =
+                    cpu_to_be16(sizeof(struct rtas_event_log_v6_mc));
+    ext_elog->mc.hdr.section_version = 1;
+
+    /* get rtas addr from fdt */
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        /* Unable to fetch rtas_addr. Hence reset the guest */
+        ppc_cpu_do_system_reset(cs);
+        g_free(ext_elog);
+        return;
+    }
+
+    stq_be_phys(&address_space_memory, rtas_addr + RTAS_ERROR_LOG_OFFSET,
+                env->gpr[3]);
+    cpu_physical_memory_write(rtas_addr + RTAS_ERROR_LOG_OFFSET +
+                              sizeof(env->gpr[3]), &log, sizeof(log));
+    cpu_physical_memory_write(rtas_addr + RTAS_ERROR_LOG_OFFSET +
+                              sizeof(env->gpr[3]) + sizeof(log), ext_elog,
+                              sizeof(*ext_elog));
+
+    env->gpr[3] = rtas_addr + RTAS_ERROR_LOG_OFFSET;
+    env->msr = msr;
+    env->nip = spapr->guest_machine_check_addr;
+
+    g_free(ext_elog);
+}
+
+void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
 {
     SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
     CPUState *cs = CPU(cpu);
@@ -657,6 +873,8 @@ void spapr_mce_req_event(PowerPCCPU *cpu)
         }
     }
     spapr->mc_status = cpu->vcpu_id;
+
+    spapr_mce_dispatch_elog(cpu, recovered);
 }
 
 static void check_exception(PowerPCCPU *cpu, SpaprMachineState *spapr,
hw/ppc/spapr_rtas.c +26 -0
index 8d8d8cdfcb..2c066a372d 100644
@@ -477,6 +477,32 @@ void spapr_dt_rtas_tokens(void *fdt, int rtas)
     }
 }
 
+hwaddr spapr_get_rtas_addr(void)
+{
+    SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
+    int rtas_node;
+    const fdt32_t *rtas_data;
+    void *fdt = spapr->fdt_blob;
+
+    /* fetch rtas addr from fdt */
+    rtas_node = fdt_path_offset(fdt, "/rtas");
+    if (rtas_node < 0) {
+        return 0;
+    }
+
+    rtas_data = fdt_getprop(fdt, rtas_node, "linux,rtas-base", NULL);
+    if (!rtas_data) {
+        return 0;
+    }
+
+    /*
+     * We assume that the OS called RTAS instantiate-rtas, but some other
+     * OS might call RTAS instantiate-rtas-64 instead. This fine as of now
+     * as SLOF only supports 32-bit variant.
+     */
+    return (hwaddr)fdt32_to_cpu(*rtas_data);
+}
+
 static void core_rtas_register_types(void)
 {
     spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +5 -1
index 909d3976f9..652a5514e8 100644
@@ -727,6 +727,9 @@ void spapr_load_rtas(SpaprMachineState *spapr, void *fdt, hwaddr addr);
 
 #define RTAS_ERROR_LOG_MAX      2048
 
+/* Offset from rtas-base where error log is placed */
+#define RTAS_ERROR_LOG_OFFSET       0x30
+
 #define RTAS_EVENT_SCAN_RATE    1
 
 /* This helper should be used to encode interrupt specifiers when the related
@@ -813,7 +816,7 @@ void spapr_clear_pending_events(SpaprMachineState *spapr);
 int spapr_max_server_number(SpaprMachineState *spapr);
 void spapr_store_hpte(PowerPCCPU *cpu, hwaddr ptex,
                       uint64_t pte0, uint64_t pte1);
-void spapr_mce_req_event(PowerPCCPU *cpu);
+void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered);
 
 /* DRC callbacks. */
 void spapr_core_release(DeviceState *dev);
@@ -904,4 +907,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
 #define SPAPR_OV5_XIVE_BOTH     0x80 /* Only to advertise on the platform */
 
 void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
+hwaddr spapr_get_rtas_addr(void);
 #endif /* HW_SPAPR_H */
target/ppc/kvm.c +3 -1
index 2aebb44e2f..910b8f8d7f 100644
@@ -2810,9 +2810,11 @@ int kvm_arch_msi_data_to_gsi(uint32_t data)
 
 int kvm_handle_nmi(PowerPCCPU *cpu, struct kvm_run *run)
 {
+    bool recovered = run->flags & KVM_RUN_PPC_NMI_DISP_FULLY_RECOV;
+
     cpu_synchronize_state(CPU(cpu));
 
-    spapr_mce_req_event(cpu);
+    spapr_mce_req_event(cpu, recovered);
 
     return 0;
 }
Signature · 1 line
2.17.2
#11 lore raw

PATCH v18 5/7 ppc: spapr: Handle "ibm, nmi-register" and "ibm, nmi-interlock" RTAS calls

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
 hw/ppc/spapr_caps.c    |  6 +++++
 hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++-
 3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +6 -0
index 3001098601..e922419cfb 100644
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
     if (!val) {
         return; /* Disabled by default */
     }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();
+        spapr->fwnmi_calls_registered = true;
+    }
 }
 
 SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0

70 lines not on this page · raw message

include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
 
     unsigned gpu_numa_id;
     SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
 };
 
 #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
 #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
 #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
 
 /* RTAS ibm,get-system-parameter token values */
 #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
 
 void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
 hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
 #endif /* HW_SPAPR_H */
Signature · 1 line
2.17.2
#12 lore raw

Re: [PATCH v18 5/7] ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls

Quoted 32 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
 hw/ppc/spapr_caps.c    |  6 +++++
 hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
 include/hw/ppc/spapr.h |  7 ++++-
 3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +0 -0
index 3001098601..e922419cfb 100644

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 506

Quoted 5 lines
hw/ppc/spapr_caps.c +2 -0
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
     if (!val) {
         return; /* Disabled by default */
     }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {

So, we definitely need the kvmppc_set_fwnmi() call here. But in the case where we *do* have KVM, but the call fails, we should fail the .apply hook, rather than ignoring it silently.

As we've discussed although TCG behaviour with fwnmi isn't 100% correct, it's close enough to pass for most purposes - so it's reasonable to continue if the cap is selected. But if the cap is selected and we're running with KVM we *must* enable the capability in KVM or we're not providing the environment the user requested.

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 508

Quoted 2 lines
hw/ppc/spapr_caps.c +2 -0
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();

We discussed registering the hypercalls here, but I thought after that I suggested just always registering them, and having them bail out when called if the cap is not set. I see that you've implemented that bailout for register, though not for interlock. I think that's simpler than registering them here.

Quoted 112 lines
hw/ppc/spapr_caps.c +2 -0
+        spapr->fwnmi_calls_registered = true;
+    }
 }
 
 SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0
index 2c066a372d..54b142f35b 100644
@@ -400,6 +400,56 @@ static void rtas_get_power_level(PowerPCCPU *cpu, SpaprMachineState *spapr,
     rtas_st(rets, 1, 100);
 }
 
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  SpaprMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    hwaddr rtas_addr;
+
+    if (spapr_get_cap(spapr, SPAPR_CAP_FWNMI_MCE) == SPAPR_CAP_OFF) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   SpaprMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (spapr->guest_machine_check_addr == -1) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    if (spapr->mc_status != cpu->vcpu_id) {
+        /* The vCPU that hit the NMI should invoke "ibm,nmi-interlock" */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    /*
+     * vCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+     * hence unset mc_status.
+     */
+    spapr->mc_status = -1;
+    qemu_cond_signal(&spapr->mc_delivery_cond);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
 static struct rtas_call {
     const char *name;
     spapr_rtas_fn fn;
@@ -503,6 +553,14 @@ hwaddr spapr_get_rtas_addr(void)
     return (hwaddr)fdt32_to_cpu(*rtas_data);
 }
 
+void spapr_fwnmi_register(void)
+{
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
+}
+
 static void core_rtas_register_types(void)
 {
     spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
 
     unsigned gpu_numa_id;
     SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
 };
 
 #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
 #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
 #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
 #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
 
 /* RTAS ibm,get-system-parameter token values */
 #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
 
 void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
 hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
 #endif /* HW_SPAPR_H */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
author
#13 lore raw

Re: [PATCH v18 5/7] ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls

Quoted 48 lines · David Gibson
Quoted 32 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
  hw/ppc/spapr_caps.c    |  6 +++++
  hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
  include/hw/ppc/spapr.h |  7 ++++-
  3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +0 -0
index 3001098601..e922419cfb 100644

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 506

Quoted 5 lines
hw/ppc/spapr_caps.c +2 -0
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
      if (!val) {
          return; /* Disabled by default */
      }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {

So, we definitely need the kvmppc_set_fwnmi() call here. But in the case where we *do* have KVM, but the call fails, we should fail the .apply hook, rather than ignoring it silently.

As we've discussed although TCG behaviour with fwnmi isn't 100% correct, it's close enough to pass for most purposes - so it's reasonable to continue if the cap is selected. But if the cap is selected and we're running with KVM we *must* enable the capability in KVM or we're not providing the environment the user requested.

Quoted 7 lines

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 508

Quoted 2 lines
hw/ppc/spapr_caps.c +2 -0
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();

We discussed registering the hypercalls here, but I thought after that I suggested just always registering them, and having them bail out when called if the cap is not set. I see that you've implemented that bailout for register, though not for interlock. I think that's simpler than registering them here.

something like this?, with bailout in interlock as well.

{
    if (!val) {
        return; /* Disabled by default */
    }

    if (kvm_enabled()) {
        if (kvmppc_set_fwnmi() < 0) { error_report("Could not enable fwnmi capability"); exit(1)
    }

    if (!spapr->fwnmi_calls_registered) {
        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */ spapr_fwnmi_register(); spapr->fwnmi_calls_registered = true;
    }
}

Quoted 113 lines
Quoted 113 lines
hw/ppc/spapr_caps.c +2 -0
+        spapr->fwnmi_calls_registered = true;
+    }
  }
 
  SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0
index 2c066a372d..54b142f35b 100644
@@ -400,6 +400,56 @@ static void rtas_get_power_level(PowerPCCPU *cpu, SpaprMachineState *spapr,
      rtas_st(rets, 1, 100);
  }
 
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  SpaprMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    hwaddr rtas_addr;
+
+    if (spapr_get_cap(spapr, SPAPR_CAP_FWNMI_MCE) == SPAPR_CAP_OFF) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   SpaprMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (spapr->guest_machine_check_addr == -1) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    if (spapr->mc_status != cpu->vcpu_id) {
+        /* The vCPU that hit the NMI should invoke "ibm,nmi-interlock" */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    /*
+     * vCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+     * hence unset mc_status.
+     */
+    spapr->mc_status = -1;
+    qemu_cond_signal(&spapr->mc_delivery_cond);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
  static struct rtas_call {
      const char *name;
      spapr_rtas_fn fn;
@@ -503,6 +553,14 @@ hwaddr spapr_get_rtas_addr(void)
      return (hwaddr)fdt32_to_cpu(*rtas_data);
  }
 
+void spapr_fwnmi_register(void)
+{
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
+}
+
  static void core_rtas_register_types(void)
  {
      spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
 
      unsigned gpu_numa_id;
      SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
  };
 
  #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
  #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
  #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
  #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
 
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
 
  /* RTAS ibm,get-system-parameter token values */
  #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
 
  void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
  hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
  #endif /* HW_SPAPR_H */
#14 lore raw

Re: [PATCH v18 5/7] ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls

Quoted 71 lines · Ganesh
Quoted 48 lines · David Gibson
Quoted 32 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
  hw/ppc/spapr_caps.c    |  6 +++++
  hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
  include/hw/ppc/spapr.h |  7 ++++-
  3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +0 -0
index 3001098601..e922419cfb 100644

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 506

Quoted 5 lines
hw/ppc/spapr_caps.c +2 -0
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
      if (!val) {
          return; /* Disabled by default */
      }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {

So, we definitely need the kvmppc_set_fwnmi() call here. But in the case where we *do* have KVM, but the call fails, we should fail the .apply hook, rather than ignoring it silently.

As we've discussed although TCG behaviour with fwnmi isn't 100% correct, it's close enough to pass for most purposes - so it's reasonable to continue if the cap is selected. But if the cap is selected and we're running with KVM we *must* enable the capability in KVM or we're not providing the environment the user requested.

Quoted 7 lines

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 508

Quoted 2 lines
hw/ppc/spapr_caps.c +2 -0
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();

We discussed registering the hypercalls here, but I thought after that I suggested just always registering them, and having them bail out when called if the cap is not set. I see that you've implemented that bailout for register, though not for interlock. I think that's simpler than registering them here.

something like this?, with bailout in interlock as well.

{
    if (!val) {
        return; /* Disabled by default */
    }

    if (kvm_enabled()) {
        if (kvmppc_set_fwnmi() < 0) { error_report("Could not enable fwnmi capability"); exit(1)
    }

Quoted 6 lines

    if (!spapr->fwnmi_calls_registered) {
        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */ spapr_fwnmi_register(); spapr->fwnmi_calls_registered = true;
    }
}

Uh.. no.. not really. I was suggesting registering the calls unconditionally, but in each of the call implementations you have

    if(!fwnmi_enabed)
         return H_FUNCTION;

or maybe H_NOT_AVAILABLE.

Quoted 107 lines
Quoted 106 lines
Quoted 106 lines
hw/ppc/spapr_caps.c +2 -0
+        spapr->fwnmi_calls_registered = true;
+    }
  }
  SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0
index 2c066a372d..54b142f35b 100644
@@ -400,6 +400,56 @@ static void rtas_get_power_level(PowerPCCPU *cpu, SpaprMachineState *spapr,
      rtas_st(rets, 1, 100);
  }
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  SpaprMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    hwaddr rtas_addr;
+
+    if (spapr_get_cap(spapr, SPAPR_CAP_FWNMI_MCE) == SPAPR_CAP_OFF) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   SpaprMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (spapr->guest_machine_check_addr == -1) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    if (spapr->mc_status != cpu->vcpu_id) {
+        /* The vCPU that hit the NMI should invoke "ibm,nmi-interlock" */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    /*
+     * vCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+     * hence unset mc_status.
+     */
+    spapr->mc_status = -1;
+    qemu_cond_signal(&spapr->mc_delivery_cond);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
  static struct rtas_call {
      const char *name;
      spapr_rtas_fn fn;
@@ -503,6 +553,14 @@ hwaddr spapr_get_rtas_addr(void)
      return (hwaddr)fdt32_to_cpu(*rtas_data);
  }
+void spapr_fwnmi_register(void)
+{
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
+}
+
  static void core_rtas_register_types(void)
  {
      spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
      unsigned gpu_numa_id;
      SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
  };
  #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
  #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
  #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
  #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
  /* RTAS ibm,get-system-parameter token values */
  #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
  void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
  hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
  #endif /* HW_SPAPR_H */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
author
#15 lore raw

Re: [PATCH v18 5/7] ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls

Quoted 70 lines · David Gibson
Quoted 69 lines · Ganesh
Quoted 48 lines · David Gibson
Quoted 32 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
   hw/ppc/spapr_caps.c    |  6 +++++
   hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
   include/hw/ppc/spapr.h |  7 ++++-
   3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +0 -0
index 3001098601..e922419cfb 100644

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 506

Quoted 5 lines
hw/ppc/spapr_caps.c +2 -0
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
       if (!val) {
           return; /* Disabled by default */
       }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {

So, we definitely need the kvmppc_set_fwnmi() call here. But in the case where we *do* have KVM, but the call fails, we should fail the .apply hook, rather than ignoring it silently.

As we've discussed although TCG behaviour with fwnmi isn't 100% correct, it's close enough to pass for most purposes - so it's reasonable to continue if the cap is selected. But if the cap is selected and we're running with KVM we *must* enable the capability in KVM or we're not providing the environment the user requested.

Quoted 7 lines

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 508

Quoted 2 lines
hw/ppc/spapr_caps.c +2 -0
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();

We discussed registering the hypercalls here, but I thought after that I suggested just always registering them, and having them bail out when called if the cap is not set. I see that you've implemented that bailout for register, though not for interlock. I think that's simpler than registering them here.

something like this?, with bailout in interlock as well.

{
    if (!val) {
        return; /* Disabled by default */
    }

    if (kvm_enabled()) {
        if (kvmppc_set_fwnmi() < 0) { error_report("Could not enable fwnmi capability"); exit(1)
    }

Quoted 8 lines
Quoted 6 lines

    if (!spapr->fwnmi_calls_registered) {
        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */ spapr_fwnmi_register(); spapr->fwnmi_calls_registered = true;
    }
}

Uh.. no.. not really. I was suggesting registering the calls unconditionally, but in each of the call implementations you have

Yes I got your point, here I am just trying to avoid registering the calls more than once, otherwise we may hit an assert.

Quoted 6 lines

     if(!fwnmi_enabed)
          return H_FUNCTION;

or maybe H_NOT_AVAILABLE.

Sure, ill add this check in interlock call also.

Quoted 105 lines
Quoted 105 lines
Quoted 105 lines
Quoted 105 lines
hw/ppc/spapr_caps.c +2 -0
+        spapr->fwnmi_calls_registered = true;
+    }
   }
   SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0
index 2c066a372d..54b142f35b 100644
@@ -400,6 +400,56 @@ static void rtas_get_power_level(PowerPCCPU *cpu, SpaprMachineState *spapr,
       rtas_st(rets, 1, 100);
   }
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  SpaprMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    hwaddr rtas_addr;
+
+    if (spapr_get_cap(spapr, SPAPR_CAP_FWNMI_MCE) == SPAPR_CAP_OFF) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   SpaprMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (spapr->guest_machine_check_addr == -1) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    if (spapr->mc_status != cpu->vcpu_id) {
+        /* The vCPU that hit the NMI should invoke "ibm,nmi-interlock" */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    /*
+     * vCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+     * hence unset mc_status.
+     */
+    spapr->mc_status = -1;
+    qemu_cond_signal(&spapr->mc_delivery_cond);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
   static struct rtas_call {
       const char *name;
       spapr_rtas_fn fn;
@@ -503,6 +553,14 @@ hwaddr spapr_get_rtas_addr(void)
       return (hwaddr)fdt32_to_cpu(*rtas_data);
   }
+void spapr_fwnmi_register(void)
+{
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
+}
+
   static void core_rtas_register_types(void)
   {
       spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
       unsigned gpu_numa_id;
       SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
   };
   #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
   #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
   #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
   #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
   /* RTAS ibm,get-system-parameter token values */
   #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
   void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
   hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
   #endif /* HW_SPAPR_H */
#16 lore raw

Re: [PATCH v18 5/7] ppc: spapr: Handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls

Quoted 79 lines · Ganesh
Quoted 76 lines · David Gibson
Quoted 75 lines · Ganesh
Quoted 48 lines · David Gibson
Quoted 32 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch adds support in QEMU to handle "ibm,nmi-register" and "ibm,nmi-interlock" RTAS calls.

The machine check notification address is saved when the OS issues "ibm,nmi-register" RTAS call.

This patch also handles the case when multiple processors experience machine check at or about the same time by handling "ibm,nmi-interlock" call. In such cases, as per PAPR, subsequent processors serialize waiting for the first processor to issue the "ibm,nmi-interlock" call. The second processor that also received a machine check error waits till the first processor is done reading the error log. The first processor issues "ibm,nmi-interlock" call when the error log is consumed.

[Move fwnmi registration to .apply hook]
---
   hw/ppc/spapr_caps.c    |  6 +++++
   hw/ppc/spapr_rtas.c    | 58 ++++++++++++++++++++++++++++++++++++++++++
   include/hw/ppc/spapr.h |  7 ++++-
   3 files changed, 70 insertions(+), 1 deletion(-)
hw/ppc/spapr_caps.c +0 -0
index 3001098601..e922419cfb 100644

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 506

Quoted 5 lines
hw/ppc/spapr_caps.c +2 -0
@@ -502,6 +502,12 @@ static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val,
       if (!val) {
           return; /* Disabled by default */
       }
+
+    if (!spapr->fwnmi_calls_registered && !kvmppc_set_fwnmi()) {

So, we definitely need the kvmppc_set_fwnmi() call here. But in the case where we *do* have KVM, but the call fails, we should fail the .apply hook, rather than ignoring it silently.

As we've discussed although TCG behaviour with fwnmi isn't 100% correct, it's close enough to pass for most purposes - so it's reasonable to continue if the cap is selected. But if the cap is selected and we're running with KVM we *must* enable the capability in KVM or we're not providing the environment the user requested.

Quoted 7 lines

hw/ppc/spapr_caps.c static void cap_fwnmi_mce_apply(SpaprMachineState *spapr, uint8_t val, · line 508

Quoted 2 lines
hw/ppc/spapr_caps.c +2 -0
+        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */
+        spapr_fwnmi_register();

We discussed registering the hypercalls here, but I thought after that I suggested just always registering them, and having them bail out when called if the cap is not set. I see that you've implemented that bailout for register, though not for interlock. I think that's simpler than registering them here.

something like this?, with bailout in interlock as well.

{
    if (!val) {
        return; /* Disabled by default */
    }

    if (kvm_enabled()) {
        if (kvmppc_set_fwnmi() < 0) { error_report("Could not enable fwnmi capability"); exit(1)
    }

    if (!spapr->fwnmi_calls_registered) {
        /* Register ibm,nmi-register and ibm,nmi-interlock RTAS calls */ spapr_fwnmi_register(); spapr->fwnmi_calls_registered = true;
    }
}

Quoted 4 lines
Quoted 2 lines

Uh.. no.. not really. I was suggesting registering the calls unconditionally, but in each of the call implementations you have

Yes I got your point, here I am just trying to avoid registering the calls more than once, otherwise we may hit an assert.

Oh, right. My point is that an advantage of registering them unconditionally is you can do it early with the rest of the statically registered calls, and so the problem of multiple registering them no longer arises.

Quoted 113 lines
Quoted 6 lines

     if(!fwnmi_enabed)
          return H_FUNCTION;

or maybe H_NOT_AVAILABLE.

Sure, ill add this check in interlock call also.

Quoted 105 lines
Quoted 105 lines
Quoted 105 lines
Quoted 105 lines
hw/ppc/spapr_caps.c +2 -0
+        spapr->fwnmi_calls_registered = true;
+    }
   }
   SpaprCapabilityInfo capability_table[SPAPR_CAP_NUM] = {
hw/ppc/spapr_rtas.c +58 -0
index 2c066a372d..54b142f35b 100644
@@ -400,6 +400,56 @@ static void rtas_get_power_level(PowerPCCPU *cpu, SpaprMachineState *spapr,
       rtas_st(rets, 1, 100);
   }
+static void rtas_ibm_nmi_register(PowerPCCPU *cpu,
+                                  SpaprMachineState *spapr,
+                                  uint32_t token, uint32_t nargs,
+                                  target_ulong args,
+                                  uint32_t nret, target_ulong rets)
+{
+    hwaddr rtas_addr;
+
+    if (spapr_get_cap(spapr, SPAPR_CAP_FWNMI_MCE) == SPAPR_CAP_OFF) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    rtas_addr = spapr_get_rtas_addr();
+    if (!rtas_addr) {
+        rtas_st(rets, 0, RTAS_OUT_NOT_SUPPORTED);
+        return;
+    }
+
+    spapr->guest_machine_check_addr = rtas_ld(args, 1);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
+static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
+                                   SpaprMachineState *spapr,
+                                   uint32_t token, uint32_t nargs,
+                                   target_ulong args,
+                                   uint32_t nret, target_ulong rets)
+{
+    if (spapr->guest_machine_check_addr == -1) {
+        /* NMI register not called */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    if (spapr->mc_status != cpu->vcpu_id) {
+        /* The vCPU that hit the NMI should invoke "ibm,nmi-interlock" */
+        rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
+        return;
+    }
+
+    /*
+     * vCPU issuing "ibm,nmi-interlock" is done with NMI handling,
+     * hence unset mc_status.
+     */
+    spapr->mc_status = -1;
+    qemu_cond_signal(&spapr->mc_delivery_cond);
+    rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+}
+
   static struct rtas_call {
       const char *name;
       spapr_rtas_fn fn;
@@ -503,6 +553,14 @@ hwaddr spapr_get_rtas_addr(void)
       return (hwaddr)fdt32_to_cpu(*rtas_data);
   }
+void spapr_fwnmi_register(void)
+{
+    spapr_rtas_register(RTAS_IBM_NMI_REGISTER, "ibm,nmi-register",
+                        rtas_ibm_nmi_register);
+    spapr_rtas_register(RTAS_IBM_NMI_INTERLOCK, "ibm,nmi-interlock",
+                        rtas_ibm_nmi_interlock);
+}
+
   static void core_rtas_register_types(void)
   {
       spapr_rtas_register(RTAS_DISPLAY_CHARACTER, "display-character",
include/hw/ppc/spapr.h +6 -1
index 652a5514e8..a90e677cc3 100644
@@ -218,6 +218,8 @@ struct SpaprMachineState {
       unsigned gpu_numa_id;
       SpaprTpmProxy *tpm_proxy;
+
+    bool fwnmi_calls_registered;
   };
   #define H_SUCCESS         0
@@ -656,8 +658,10 @@ target_ulong spapr_hypercall(PowerPCCPU *cpu, target_ulong opcode,
   #define RTAS_IBM_REMOVE_PE_DMA_WINDOW           (RTAS_TOKEN_BASE + 0x28)
   #define RTAS_IBM_RESET_PE_DMA_WINDOW            (RTAS_TOKEN_BASE + 0x29)
   #define RTAS_IBM_SUSPEND_ME                     (RTAS_TOKEN_BASE + 0x2A)
+#define RTAS_IBM_NMI_REGISTER                   (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_IBM_NMI_INTERLOCK                  (RTAS_TOKEN_BASE + 0x2C)
-#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2B)
+#define RTAS_TOKEN_MAX                          (RTAS_TOKEN_BASE + 0x2D)
   /* RTAS ibm,get-system-parameter token values */
   #define RTAS_SYSPARM_SPLPAR_CHARACTERISTICS      20
@@ -908,4 +912,5 @@ void spapr_check_pagesize(SpaprMachineState *spapr, hwaddr pagesize,
   void spapr_set_all_lpcrs(target_ulong value, target_ulong mask);
   hwaddr spapr_get_rtas_addr(void);
+void spapr_fwnmi_register(void);
   #endif /* HW_SPAPR_H */
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#17 lore raw

PATCH v18 6/7 migration: Include migration support for machine check handling

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch includes migration support for machine check handling. Especially this patch blocks VM migration requests until the machine check error handling is complete as these errors are specific to the source hardware and is irrelevant on the target hardware.

[Do not set FWNMI cap in post_load, now its done in .apply hook]
---
 hw/ppc/spapr.c         | 41 +++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_events.c  | 20 +++++++++++++++++++-
 hw/ppc/spapr_rtas.c    |  4 ++++
 include/hw/ppc/spapr.h |  1 +
 4 files changed, 65 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +41 -0
index 975d7da734..4acdc30100 100644
@@ -46,6 +46,7 @@
 #include "migration/qemu-file-types.h"
 #include "migration/global_state.h"
 #include "migration/register.h"
+#include "migration/blocker.h"
 #include "mmu-hash64.h"
 #include "mmu-book3s-v3.h"
 #include "cpu-models.h"
@@ -1685,6 +1686,8 @@ static void spapr_machine_reset(MachineState *machine)
 
     /* Signal all vCPUs waiting on this condition */
     qemu_cond_broadcast(&spapr->mc_delivery_cond);
+
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
 }
 
 static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -1967,6 +1970,43 @@ static const VMStateDescription vmstate_spapr_dtb = {
     },
 };
 
+static bool spapr_fwnmi_needed(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    return spapr->fwnmi_calls_registered;
+}
+
+static int spapr_fwnmi_pre_save(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    /*
+     * Check if machine check handling is in progress and print a
+     * warning message.
+     */
+    if (spapr->mc_status != -1) {
+        warn_report("A machine check is being handled during migration. The"
+                "handler may run and log hardware error on the destination");
+    }
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_spapr_machine_check = {
+    .name = "spapr_machine_check",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = spapr_fwnmi_needed,
+    .pre_save = spapr_fwnmi_pre_save,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(guest_machine_check_addr, SpaprMachineState),
+        VMSTATE_INT32(mc_status, SpaprMachineState),
+        VMSTATE_BOOL(fwnmi_calls_registered, SpaprMachineState),
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 static const VMStateDescription vmstate_spapr = {
     .name = "spapr",
     .version_id = 3,
@@ -2001,6 +2041,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_cap_large_decr,
         &vmstate_spapr_cap_ccf_assist,
         &vmstate_spapr_cap_fwnmi,
+        &vmstate_spapr_machine_check,
         NULL
     }
 };
hw/ppc/spapr_events.c +19 -1
index 54eaf28a9e..7092687fa0 100644
@@ -43,6 +43,7 @@
 #include "qemu/main-loop.h"
 #include "hw/ppc/spapr_ovec.h"
 #include <libfdt.h>
+#include "migration/blocker.h"
 
 #define RTAS_LOG_VERSION_MASK                   0xff000000
 #define   RTAS_LOG_VERSION_6                    0x06000000
@@ -843,6 +844,8 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
 {
     SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
     CPUState *cs = CPU(cpu);
+    int ret;
+    Error *local_err = NULL;
 
     if (spapr->guest_machine_check_addr == -1) {
         /*
@@ -872,8 +875,23 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
             return;
         }
     }
-    spapr->mc_status = cpu->vcpu_id;
 
+    error_setg(&spapr->fwnmi_migration_blocker,
+               "Live migration not supported during machine check handling");
+    ret = migrate_add_blocker(spapr->fwnmi_migration_blocker, &local_err);
+    if (ret == -EBUSY) {
+        /*
+         * We don't want to abort so we let the migration to continue.
+         * In a rare case, the machine check handler will run on the target.
+         * Though this is not preferable, it is better than aborting
+         * the migration or killing the VM.
+         */
+        error_free(spapr->fwnmi_migration_blocker);
+        spapr->fwnmi_migration_blocker = NULL;
+        warn_report("Received a fwnmi while migration was in progress");
+    }
+
+    spapr->mc_status = cpu->vcpu_id;
     spapr_mce_dispatch_elog(cpu, recovered);
 }
 
hw/ppc/spapr_rtas.c +4 -0
index 54b142f35b..3409f6b896 100644
@@ -50,6 +50,7 @@
 #include "hw/ppc/fdt.h"
 #include "target/ppc/mmu-hash64.h"
 #include "target/ppc/mmu-book3s-v3.h"
+#include "migration/blocker.h"
 
 static void rtas_display_character(PowerPCCPU *cpu, SpaprMachineState *spapr,
                                    uint32_t token, uint32_t nargs,
@@ -448,6 +449,9 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
     spapr->mc_status = -1;
     qemu_cond_signal(&spapr->mc_delivery_cond);
     rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
+    error_free(spapr->fwnmi_migration_blocker);
+    spapr->fwnmi_migration_blocker = NULL;
 }
 
 static struct rtas_call {
include/hw/ppc/spapr.h +1 -0
index a90e677cc3..ac246c8be3 100644
@@ -220,6 +220,7 @@ struct SpaprMachineState {
     SpaprTpmProxy *tpm_proxy;
 
     bool fwnmi_calls_registered;
+    Error *fwnmi_migration_blocker;
 };
 
 #define H_SUCCESS         0
Signature · 1 line
2.17.2
#18 lore raw

Re: [PATCH v18 6/7] migration: Include migration support for machine check handling

Quoted 40 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch includes migration support for machine check handling. Especially this patch blocks VM migration requests until the machine check error handling is complete as these errors are specific to the source hardware and is irrelevant on the target hardware.

[Do not set FWNMI cap in post_load, now its done in .apply hook]
---
 hw/ppc/spapr.c         | 41 +++++++++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_events.c  | 20 +++++++++++++++++++-
 hw/ppc/spapr_rtas.c    |  4 ++++
 include/hw/ppc/spapr.h |  1 +
 4 files changed, 65 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +3 -0
index 975d7da734..4acdc30100 100644
@@ -46,6 +46,7 @@
 #include "migration/qemu-file-types.h"
 #include "migration/global_state.h"
 #include "migration/register.h"
+#include "migration/blocker.h"
 #include "mmu-hash64.h"
 #include "mmu-book3s-v3.h"
 #include "cpu-models.h"
@@ -1685,6 +1686,8 @@ static void spapr_machine_reset(MachineState *machine)
 
     /* Signal all vCPUs waiting on this condition */
     qemu_cond_broadcast(&spapr->mc_delivery_cond);
+
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
 }
 
 static void spapr_create_nvram(SpaprMachineState *spapr)

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 1977

Quoted 8 lines
hw/ppc/spapr.c +5 -0
@@ -1967,6 +1970,43 @@ static const VMStateDescription vmstate_spapr_dtb = {
     },
 };
 
+static bool spapr_fwnmi_needed(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    return spapr->fwnmi_calls_registered;

I think it would be better to base this directly on the cap, rather than a variable set later.

Quoted 18 lines
hw/ppc/spapr.c +18 -0
+}
+
+static int spapr_fwnmi_pre_save(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    /*
+     * Check if machine check handling is in progress and print a
+     * warning message.
+     */
+    if (spapr->mc_status != -1) {
+        warn_report("A machine check is being handled during migration. The"
+                "handler may run and log hardware error on the destination");
+    }
+
+    return 0;
+}
+

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 2005

Quoted 10 lines
hw/ppc/spapr.c +10 -0
+static const VMStateDescription vmstate_spapr_machine_check = {
+    .name = "spapr_machine_check",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .needed = spapr_fwnmi_needed,
+    .pre_save = spapr_fwnmi_pre_save,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(guest_machine_check_addr, SpaprMachineState),
+        VMSTATE_INT32(mc_status, SpaprMachineState),
+        VMSTATE_BOOL(fwnmi_calls_registered, SpaprMachineState),

This doesn't make sense to migrate - it will always have its final value by the time the guest is running in a migratable state.

Quoted 45 lines
hw/ppc/spapr.c +5 -0
+        VMSTATE_END_OF_LIST()
+    },
+};
+
 static const VMStateDescription vmstate_spapr = {
     .name = "spapr",
     .version_id = 3,
@@ -2001,6 +2041,7 @@ static const VMStateDescription vmstate_spapr = {
         &vmstate_spapr_cap_large_decr,
         &vmstate_spapr_cap_ccf_assist,
         &vmstate_spapr_cap_fwnmi,
+        &vmstate_spapr_machine_check,
         NULL
     }
 };
hw/ppc/spapr_events.c +6 -1
index 54eaf28a9e..7092687fa0 100644
@@ -43,6 +43,7 @@
 #include "qemu/main-loop.h"
 #include "hw/ppc/spapr_ovec.h"
 #include <libfdt.h>
+#include "migration/blocker.h"
 
 #define RTAS_LOG_VERSION_MASK                   0xff000000
 #define   RTAS_LOG_VERSION_6                    0x06000000
@@ -843,6 +844,8 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
 {
     SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
     CPUState *cs = CPU(cpu);
+    int ret;
+    Error *local_err = NULL;
 
     if (spapr->guest_machine_check_addr == -1) {
         /*
@@ -872,8 +875,23 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
             return;
         }
     }
-    spapr->mc_status = cpu->vcpu_id;
 
+    error_setg(&spapr->fwnmi_migration_blocker,
+               "Live migration not supported during machine check handling");
+    ret = migrate_add_blocker(spapr->fwnmi_migration_blocker, &local_err);

hw/ppc/spapr_events.c void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered) · line 891

Quoted 10 lines
hw/ppc/spapr_events.c +10 -0
+    if (ret == -EBUSY) {
+        /*
+         * We don't want to abort so we let the migration to continue.
+         * In a rare case, the machine check handler will run on the target.
+         * Though this is not preferable, it is better than aborting
+         * the migration or killing the VM.
+         */
+        error_free(spapr->fwnmi_migration_blocker);
+        spapr->fwnmi_migration_blocker = NULL;
+        warn_report("Received a fwnmi while migration was in progress");

Didn't we change from initializing the blocker Error at init time because there was a case where we could have two migration blockers registered at once? If that's so then we need entirely different instances of the blocker Error. Just dynamiically allocating them doesn't help us if there can still only be one at a time.

Quoted 40 lines
hw/ppc/spapr_events.c +3 -0
+    }
+
+    spapr->mc_status = cpu->vcpu_id;
     spapr_mce_dispatch_elog(cpu, recovered);
 }
 
hw/ppc/spapr_rtas.c +4 -0
index 54b142f35b..3409f6b896 100644
@@ -50,6 +50,7 @@
 #include "hw/ppc/fdt.h"
 #include "target/ppc/mmu-hash64.h"
 #include "target/ppc/mmu-book3s-v3.h"
+#include "migration/blocker.h"
 
 static void rtas_display_character(PowerPCCPU *cpu, SpaprMachineState *spapr,
                                    uint32_t token, uint32_t nargs,
@@ -448,6 +449,9 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
     spapr->mc_status = -1;
     qemu_cond_signal(&spapr->mc_delivery_cond);
     rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
+    error_free(spapr->fwnmi_migration_blocker);
+    spapr->fwnmi_migration_blocker = NULL;
 }
 
 static struct rtas_call {
include/hw/ppc/spapr.h +1 -0
index a90e677cc3..ac246c8be3 100644
@@ -220,6 +220,7 @@ struct SpaprMachineState {
     SpaprTpmProxy *tpm_proxy;
 
     bool fwnmi_calls_registered;
+    Error *fwnmi_migration_blocker;
 };
 
 #define H_SUCCESS         0
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
author
#19 lore raw

Re: [PATCH v18 6/7] migration: Include migration support for machine check handling

Quoted 44 lines · David Gibson
Quoted 43 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch includes migration support for machine check handling. Especially this patch blocks VM migration requests until the machine check error handling is complete as these errors are specific to the source hardware and is irrelevant on the target hardware.

[Do not set FWNMI cap in post_load, now its done in .apply hook]
---
  hw/ppc/spapr.c         | 41 +++++++++++++++++++++++++++++++++++++++++
  hw/ppc/spapr_events.c  | 20 +++++++++++++++++++-
  hw/ppc/spapr_rtas.c    |  4 ++++
  include/hw/ppc/spapr.h |  1 +
  4 files changed, 65 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +3 -0
index 975d7da734..4acdc30100 100644
@@ -46,6 +46,7 @@
  #include "migration/qemu-file-types.h"
  #include "migration/global_state.h"
  #include "migration/register.h"
+#include "migration/blocker.h"
  #include "mmu-hash64.h"
  #include "mmu-book3s-v3.h"
  #include "cpu-models.h"
@@ -1685,6 +1686,8 @@ static void spapr_machine_reset(MachineState *machine)
 
      /* Signal all vCPUs waiting on this condition */
      qemu_cond_broadcast(&spapr->mc_delivery_cond);
+
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
  }
 
  static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -1967,6 +1970,43 @@ static const VMStateDescription vmstate_spapr_dtb = {
      },
  };
 
Quoted 7 lines

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 1977

Quoted 5 lines
hw/ppc/spapr.c +5 -0
+static bool spapr_fwnmi_needed(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    return spapr->fwnmi_calls_registered;

I think it would be better to base this directly on the cap, rather than a variable set later.

Ok, ill revert to older way

Quoted 23 lines
Quoted 23 lines
hw/ppc/spapr.c +22 -0
+}
+
+static int spapr_fwnmi_pre_save(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    /*
+     * Check if machine check handling is in progress and print a
+     * warning message.
+     */
+    if (spapr->mc_status != -1) {
+        warn_report("A machine check is being handled during migration. The"
+                "handler may run and log hardware error on the destination");
+    }
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_spapr_machine_check = {
+    .name = "spapr_machine_check",
+    .version_id = 1,
+    .minimum_version_id = 1,
Quoted 8 lines

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 2005

Quoted 6 lines
hw/ppc/spapr.c +6 -0
+    .needed = spapr_fwnmi_needed,
+    .pre_save = spapr_fwnmi_pre_save,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(guest_machine_check_addr, SpaprMachineState),
+        VMSTATE_INT32(mc_status, SpaprMachineState),
+        VMSTATE_BOOL(fwnmi_calls_registered, SpaprMachineState),

This doesn't make sense to migrate - it will always have its final value by the time the guest is running in a migratable state.

Ok, ill remove it.

Quoted 53 lines
Quoted 53 lines
hw/ppc/spapr.c +5 -0
+        VMSTATE_END_OF_LIST()
+    },
+};
+
  static const VMStateDescription vmstate_spapr = {
      .name = "spapr",
      .version_id = 3,
@@ -2001,6 +2041,7 @@ static const VMStateDescription vmstate_spapr = {
          &vmstate_spapr_cap_large_decr,
          &vmstate_spapr_cap_ccf_assist,
          &vmstate_spapr_cap_fwnmi,
+        &vmstate_spapr_machine_check,
          NULL
      }
  };
hw/ppc/spapr_events.c +13 -1
index 54eaf28a9e..7092687fa0 100644
@@ -43,6 +43,7 @@
  #include "qemu/main-loop.h"
  #include "hw/ppc/spapr_ovec.h"
  #include <libfdt.h>
+#include "migration/blocker.h"
 
  #define RTAS_LOG_VERSION_MASK                   0xff000000
  #define   RTAS_LOG_VERSION_6                    0x06000000
@@ -843,6 +844,8 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
  {
      SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
      CPUState *cs = CPU(cpu);
+    int ret;
+    Error *local_err = NULL;
 
      if (spapr->guest_machine_check_addr == -1) {
          /*
@@ -872,8 +875,23 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
              return;
          }
      }
-    spapr->mc_status = cpu->vcpu_id;
 
+    error_setg(&spapr->fwnmi_migration_blocker,
+               "Live migration not supported during machine check handling");
+    ret = migrate_add_blocker(spapr->fwnmi_migration_blocker, &local_err);
+    if (ret == -EBUSY) {
+        /*
+         * We don't want to abort so we let the migration to continue.
+         * In a rare case, the machine check handler will run on the target.
+         * Though this is not preferable, it is better than aborting
+         * the migration or killing the VM.
+         */
Quoted 8 lines

hw/ppc/spapr_events.c void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered) · line 891

Quoted 3 lines
hw/ppc/spapr_events.c +3 -0
+        error_free(spapr->fwnmi_migration_blocker);
+        spapr->fwnmi_migration_blocker = NULL;
+        warn_report("Received a fwnmi while migration was in progress");

Didn't we change from initializing the blocker Error at init time because there was a case where we could have two migration blockers registered at once? If that's so then we need entirely different instances of the blocker Error. Just dynamiically allocating them doesn't help us if there can still only be one at a time.

I agree, but this how we were doing it before.

Are you suggesting to have per cpu blocker Error instance ?

I think initializing the blocker Error at init time and not freeing it, is much simpler

and cleaner. And if we receive multiple fwnmi events on different cpus almost

at the same time, Though we will be prepending same migration blocker instance

multiple times to the migration_blockers list, IIUC we will not be unblocking migration

till the migration_blockers list is empty. Please let me know if you are ok with initializing

blocker error at init time.

Quoted 41 lines
Quoted 41 lines
hw/ppc/spapr_events.c +3 -0
+    }
+
+    spapr->mc_status = cpu->vcpu_id;
      spapr_mce_dispatch_elog(cpu, recovered);
  }
 
hw/ppc/spapr_rtas.c +4 -0
index 54b142f35b..3409f6b896 100644
@@ -50,6 +50,7 @@
  #include "hw/ppc/fdt.h"
  #include "target/ppc/mmu-hash64.h"
  #include "target/ppc/mmu-book3s-v3.h"
+#include "migration/blocker.h"
 
  static void rtas_display_character(PowerPCCPU *cpu, SpaprMachineState *spapr,
                                     uint32_t token, uint32_t nargs,
@@ -448,6 +449,9 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
      spapr->mc_status = -1;
      qemu_cond_signal(&spapr->mc_delivery_cond);
      rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
+    error_free(spapr->fwnmi_migration_blocker);
+    spapr->fwnmi_migration_blocker = NULL;
  }
 
  static struct rtas_call {
include/hw/ppc/spapr.h +1 -0
index a90e677cc3..ac246c8be3 100644
@@ -220,6 +220,7 @@ struct SpaprMachineState {
      SpaprTpmProxy *tpm_proxy;
 
      bool fwnmi_calls_registered;
+    Error *fwnmi_migration_blocker;
  };
 
  #define H_SUCCESS         0
#20 lore raw

Re: [PATCH v18 6/7] migration: Include migration support for machine check handling

Quoted 142 lines · Ganesh
Quoted 40 lines · David Gibson
Quoted 39 lines · Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch includes migration support for machine check handling. Especially this patch blocks VM migration requests until the machine check error handling is complete as these errors are specific to the source hardware and is irrelevant on the target hardware.

[Do not set FWNMI cap in post_load, now its done in .apply hook]
---
  hw/ppc/spapr.c         | 41 +++++++++++++++++++++++++++++++++++++++++
  hw/ppc/spapr_events.c  | 20 +++++++++++++++++++-
  hw/ppc/spapr_rtas.c    |  4 ++++
  include/hw/ppc/spapr.h |  1 +
  4 files changed, 65 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +3 -0
index 975d7da734..4acdc30100 100644
@@ -46,6 +46,7 @@
  #include "migration/qemu-file-types.h"
  #include "migration/global_state.h"
  #include "migration/register.h"
+#include "migration/blocker.h"
  #include "mmu-hash64.h"
  #include "mmu-book3s-v3.h"
  #include "cpu-models.h"
@@ -1685,6 +1686,8 @@ static void spapr_machine_reset(MachineState *machine)
      /* Signal all vCPUs waiting on this condition */
      qemu_cond_broadcast(&spapr->mc_delivery_cond);
+
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
  }
  static void spapr_create_nvram(SpaprMachineState *spapr)
@@ -1967,6 +1970,43 @@ static const VMStateDescription vmstate_spapr_dtb = {
      },
Quoted 8 lines

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 1977

Quoted 6 lines
hw/ppc/spapr.c +5 -0
  };
+static bool spapr_fwnmi_needed(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    return spapr->fwnmi_calls_registered;

I think it would be better to base this directly on the cap, rather than a variable set later.

Ok, ill revert to older way

Quoted 23 lines
Quoted 23 lines
hw/ppc/spapr.c +22 -0
+}
+
+static int spapr_fwnmi_pre_save(void *opaque)
+{
+    SpaprMachineState *spapr = (SpaprMachineState *)opaque;
+
+    /*
+     * Check if machine check handling is in progress and print a
+     * warning message.
+     */
+    if (spapr->mc_status != -1) {
+        warn_report("A machine check is being handled during migration. The"
+                "handler may run and log hardware error on the destination");
+    }
+
+    return 0;
+}
+
+static const VMStateDescription vmstate_spapr_machine_check = {
+    .name = "spapr_machine_check",
+    .version_id = 1,
+    .minimum_version_id = 1,
Quoted 8 lines

hw/ppc/spapr.c static const VMStateDescription vmstate_spapr_dtb = { · line 2005

Quoted 6 lines
hw/ppc/spapr.c +6 -0
+    .needed = spapr_fwnmi_needed,
+    .pre_save = spapr_fwnmi_pre_save,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT64(guest_machine_check_addr, SpaprMachineState),
+        VMSTATE_INT32(mc_status, SpaprMachineState),
+        VMSTATE_BOOL(fwnmi_calls_registered, SpaprMachineState),

This doesn't make sense to migrate - it will always have its final value by the time the guest is running in a migratable state.

Ok, ill remove it.

Quoted 50 lines
Quoted 50 lines
hw/ppc/spapr.c +5 -0
+        VMSTATE_END_OF_LIST()
+    },
+};
+
  static const VMStateDescription vmstate_spapr = {
      .name = "spapr",
      .version_id = 3,
@@ -2001,6 +2041,7 @@ static const VMStateDescription vmstate_spapr = {
          &vmstate_spapr_cap_large_decr,
          &vmstate_spapr_cap_ccf_assist,
          &vmstate_spapr_cap_fwnmi,
+        &vmstate_spapr_machine_check,
          NULL
      }
  };
hw/ppc/spapr_events.c +13 -1
index 54eaf28a9e..7092687fa0 100644
@@ -43,6 +43,7 @@
  #include "qemu/main-loop.h"
  #include "hw/ppc/spapr_ovec.h"
  #include <libfdt.h>
+#include "migration/blocker.h"
  #define RTAS_LOG_VERSION_MASK                   0xff000000
  #define   RTAS_LOG_VERSION_6                    0x06000000
@@ -843,6 +844,8 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
  {
      SpaprMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
      CPUState *cs = CPU(cpu);
+    int ret;
+    Error *local_err = NULL;
      if (spapr->guest_machine_check_addr == -1) {
          /*
@@ -872,8 +875,23 @@ void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered)
              return;
          }
      }
-    spapr->mc_status = cpu->vcpu_id;
+    error_setg(&spapr->fwnmi_migration_blocker,
+               "Live migration not supported during machine check handling");
+    ret = migrate_add_blocker(spapr->fwnmi_migration_blocker, &local_err);
+    if (ret == -EBUSY) {
+        /*
+         * We don't want to abort so we let the migration to continue.
+         * In a rare case, the machine check handler will run on the target.
+         * Though this is not preferable, it is better than aborting
+         * the migration or killing the VM.
+         */
Quoted 8 lines

hw/ppc/spapr_events.c void spapr_mce_req_event(PowerPCCPU *cpu, bool recovered) · line 891

Quoted 3 lines
hw/ppc/spapr_events.c +3 -0
+        error_free(spapr->fwnmi_migration_blocker);
+        spapr->fwnmi_migration_blocker = NULL;
+        warn_report("Received a fwnmi while migration was in progress");

Didn't we change from initializing the blocker Error at init time because there was a case where we could have two migration blockers registered at once? If that's so then we need entirely different instances of the blocker Error. Just dynamiically allocating them doesn't help us if there can still only be one at a time.

Quoted 3 lines

I agree, but this how we were doing it before.

Are you suggesting to have per cpu blocker Error instance ?

I was, but..

Quoted 9 lines

I think initializing the blocker Error at init time and not freeing it, is much simpler

and cleaner. And if we receive multiple fwnmi events on different cpus almost

at the same time, Though we will be prepending same migration blocker instance

Quoted 7 lines

multiple times to the migration_blockers list, IIUC we will not be unblocking migration

till the migration_blockers list is empty. Please let me know if you are ok with initializing

blocker error at init time.

.. I realized I was mistaken. It wasn't premature unblocking I was concerned about, but corrupting the actual list structure. I thought we were using an intrusive linked list like the QLIST_*() stuff for this, which can't tolerate double adding an element. In fact we're using g_slist_*() which has it's own wrapper nodes around the pointers given here, so we're ok after all.

So creating the blocker error at init time is the way to go after all.

Quoted 36 lines
Quoted 35 lines
Quoted 35 lines
hw/ppc/spapr_events.c +3 -0
+    }
+
+    spapr->mc_status = cpu->vcpu_id;
      spapr_mce_dispatch_elog(cpu, recovered);
  }
hw/ppc/spapr_rtas.c +4 -0
index 54b142f35b..3409f6b896 100644
@@ -50,6 +50,7 @@
  #include "hw/ppc/fdt.h"
  #include "target/ppc/mmu-hash64.h"
  #include "target/ppc/mmu-book3s-v3.h"
+#include "migration/blocker.h"
  static void rtas_display_character(PowerPCCPU *cpu, SpaprMachineState *spapr,
                                     uint32_t token, uint32_t nargs,
@@ -448,6 +449,9 @@ static void rtas_ibm_nmi_interlock(PowerPCCPU *cpu,
      spapr->mc_status = -1;
      qemu_cond_signal(&spapr->mc_delivery_cond);
      rtas_st(rets, 0, RTAS_OUT_SUCCESS);
+    migrate_del_blocker(spapr->fwnmi_migration_blocker);
+    error_free(spapr->fwnmi_migration_blocker);
+    spapr->fwnmi_migration_blocker = NULL;
  }
  static struct rtas_call {
include/hw/ppc/spapr.h +1 -0
index a90e677cc3..ac246c8be3 100644
@@ -220,6 +220,7 @@ struct SpaprMachineState {
      SpaprTpmProxy *tpm_proxy;
      bool fwnmi_calls_registered;
+    Error *fwnmi_migration_blocker;
  };
  #define H_SUCCESS         0
Signature · 4 lines
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
#21 lore raw

PATCH v18 7/7 ppc: spapr: Activate the FWNMI functionality

In reply to Ganesh Goudar

From: Aravinda Prasad <[email protected]>

This patch sets the default value of SPAPR_CAP_FWNMI_MCE to SPAPR_CAP_ON for machine type 4.2.

---
 hw/ppc/spapr.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/ppc/spapr.c +2 -1
index 4acdc30100..e705b0f9e9 100644
@@ -4450,7 +4450,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF;
     smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON;
     smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF;
-    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
+    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_ON;
     spapr_caps_add_properties(smc, &error_abort);
     smc->irq = &spapr_irq_dual;
     smc->dr_phb_enabled = true;
@@ -4540,6 +4540,7 @@ static void spapr_machine_4_1_class_options(MachineClass *mc)
     smc->smp_threads_vsmt = false;
     compat_props_add(mc->compat_props, hw_compat_4_1, hw_compat_4_1_len);
     compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat));
+    smc->default_caps.caps[SPAPR_CAP_FWNMI_MCE] = SPAPR_CAP_OFF;
 }
 
 DEFINE_SPAPR_MACHINE(4_1, "4.1", false);
Signature · 1 line
2.17.2