Skip to contentSkip to conversation

ppc/pnv: Better big-core model, lpar-per-core, PC unit

PATCH Superseded Nicholas Piggin 2 participants 29 messages Jul 11, 2024

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover ppc/pnv: Better big-core model, lpar-per-core, PC unit

Primary motivation for this series is to improve big-core support.

This also fixes POWER8 SMT running Linux with the msgsnd fix and setting lpar-per-core mode (which is always true on POWER8).

I think I accounted for all feedback from Cedric and Harsh from the last RFC (except a couple of style suggestions from Harsh).

Since rfc:
- Fixed POWER8 SMT so it doesn't have to be disabled.
- Fixed inadvertent spapr SMT bug.
- Renamed PnvCPUState.core pointer to pnv_core. (Harsh)
- Moved where it is initialised (clg)
- Avoided most qdev_get_machine() calls by adding a PnvMachineState pointer from PnvChip, new patch 3 (clg).
- Rename TB state to use camel case (Harsh and clg)
- Add comment to explain SPRC/SPRD is only accessed with powernv.
- Use mc->desc for error messages and avoid splitting machine init handlers (Harsh).
- Add max_smt_threads class attribute to avoid duplicating checks (clg)
- Rename processor_id() class method to get_pir_tir (Harsh and clg)
- Add a comment for get_pir_tir() (clg)
- Allow get_pir_tir() to be passed NULL pointers to avoid dummy pir/tir variables (Harsh)
- Move the PPC_CPU_HAS_CORE_SIBLINGS macros to inline functions (clg)
- Invert them (test for single-thread rather than for siblings) because the callers read a little better that way (Harsh).
- Propagate lpar and big-core options down to chip and core levels rather than having to test machine (clg)
- Significantly split the big-core patch (clg).
- Rework big-core device-tree handling to simplify it (clg).
- Make new has_smt_siblings property bool (Harsh)
- Make the big-core timebase tod quirk a machine class property rather than machine state (Harsh).

Thanks,
Nick

Nicholas Piggin (18):
target/ppc: Fix msgsnd for POWER8
ppc/pnv: Add pointer from PnvCPUState to PnvCore ppc/pnv: Add a pointer from PnvChip to PnvMachineState ppc/pnv: Move timebase state into PnvCore
target/ppc: Move SPR indirect registers into PnvCore ppc/pnv: specialise init for powernv8/9/10 machines ppc/pnv: Extend chip_pir class method to TIR as well
ppc: Add a core_index to CPUPPCState for SMT vCPUs target/ppc: Add helpers to check for SMT sibling threads
ppc: Add has_smt_siblings property to CPUPPCState ppc/pnv: Add a big-core mode that joins two regular cores ppc/pnv: Add allow for big-core differences in DT generation ppc/pnv: Implement big-core PVR for Power9/10
ppc/pnv: Implement Power9 CPU core thread state indirect register ppc/pnv: Add POWER10 ChipTOD quirk for big-core ppc/pnv: Add big-core machine property
ppc/pnv: Implement POWER10 PC xscom registers for direct controls ppc/pnv: Add an LPAR per core machine option

 include/hw/core/cpu.h        |   8 +
 include/hw/ppc/pnv.h         |   7 +
 include/hw/ppc/pnv_chip.h    |   6 +-
 include/hw/ppc/pnv_core.h    |  31 ++++
 target/ppc/cpu.h             |  41 ++---
 hw/ppc/pnv.c                 | 298 ++++++++++++++++++++++++++++-------
 hw/ppc/pnv_chiptod.c         |   7 +-
 hw/ppc/pnv_core.c            | 130 +++++++++++++--
 hw/ppc/spapr_cpu_core.c      |  16 +-
 system/cpus.c                |  10 ++
 target/ppc/cpu_init.c        |  26 +--
 target/ppc/excp_helper.c     |  69 ++++----
 target/ppc/misc_helper.c     | 104 ++++++------
 target/ppc/timebase_helper.c |  82 +++++-----
 14 files changed, 601 insertions(+), 234 deletions(-)
Signature · 1 line
2.45.1
#1 lore raw

PATCH 1/18 target/ppc: Fix msgsnd for POWER8

POWER8 (ISA v2.07S) introduced the doorbell facility, the msgsnd instruction behaved mostly like msgsndp, it was addressed by TIR and could only send interrupts between threads on the core.

ISA v3.0 changed msgsnd to be addressed by PIR and can interrupt any thread in the system.

msgsnd only implements the v3.0 semantics, which can make multi-threaded POWER8 hang when booting Linux (due to IPIs failing). This change adds v2.07 semantics.

---
 target/ppc/excp_helper.c | 74 ++++++++++++++++++++++++----------------
 1 file changed, 44 insertions(+), 30 deletions(-)
target/ppc/excp_helper.c +44 -30
index 0cd542675f..c0120c8a88 100644
@@ -2998,6 +2998,41 @@ static inline bool dbell_bcast_subproc(target_ulong rb)
     return (rb & DBELL_BRDCAST_MASK) == DBELL_BRDCAST_SUBPROC;
 }
 
+/*
+ * Send an interrupt to a thread in the same core as env).
+ */
+static void msgsnd_core_tir(CPUPPCState *env, uint32_t target_tir, int irq)
+{
+    PowerPCCPU *cpu = env_archcpu(env);
+    CPUState *cs = env_cpu(env);
+    uint32_t nr_threads = cs->nr_threads;
+
+    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
+    }
+
+    if (target_tir >= nr_threads) {
+        return;
+    }
+
+    if (nr_threads == 1) {
+        ppc_set_irq(cpu, irq, 1);
+    } else {
+        CPUState *ccs;
+
+        /* Does iothread need to be locked for walking CPU list? */
+        bql_lock();
+        THREAD_SIBLING_FOREACH(cs, ccs) {
+            PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+            if (target_tir == ppc_cpu_tir(ccpu)) {
+                ppc_set_irq(ccpu, irq, 1);
+                break;
+            }
+        }
+        bql_unlock();
+    }
+}
+
 void helper_book3s_msgclr(CPUPPCState *env, target_ulong rb)
 {
     if (!dbell_type_server(rb)) {
@@ -3018,6 +3053,13 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
         return;
     }
 
+    /* POWER8 msgsnd is like msgsndp (targets a thread within core) */
+    if (!(env->insns_flags2 & PPC2_ISA300)) {
+        msgsnd_core_tir(env, rb & PPC_BITMASK(57, 63), PPC_INTERRUPT_HDOORBELL);
+        return;
+    }
+
+    /* POWER9 and later msgsnd is a global (targets any thread) */
     cpu = ppc_get_vcpu_by_pir(pir);
     if (!cpu) {
         return;
@@ -3064,41 +3106,13 @@ void helper_book3s_msgclrp(CPUPPCState *env, target_ulong rb)
  */
 void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
 {
-    CPUState *cs = env_cpu(env);
-    PowerPCCPU *cpu = env_archcpu(env);
-    CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
-    int ttir = rb & PPC_BITMASK(57, 63);
-
     helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
-    }
-
-    if (!dbell_type_server(rb) || ttir >= nr_threads) {
-        return;
-    }
-
-    if (nr_threads == 1) {
-        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
+    if (!dbell_type_server(rb)) {
         return;
     }
 
-    /* Does iothread need to be locked for walking CPU list? */
-    bql_lock();
-    THREAD_SIBLING_FOREACH(cs, ccs) {
-        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
-        uint32_t thread_id = ppc_cpu_tir(ccpu);
-
-        if (ttir == thread_id) {
-            ppc_set_irq(ccpu, PPC_INTERRUPT_DOORBELL, 1);
-            bql_unlock();
-            return;
-        }
-    }
-
-    g_assert_not_reached();
+    msgsnd_core_tir(env, rb & PPC_BITMASK(57, 63), PPC_INTERRUPT_DOORBELL);
 }
 #endif /* TARGET_PPC64 */
 
Signature · 1 line
2.45.1
#2 lore raw

Re: [PATCH 01/18] target/ppc: Fix msgsnd for POWER8

Quoted 10 lines · Nicholas Piggin

POWER8 (ISA v2.07S) introduced the doorbell facility, the msgsnd instruction behaved mostly like msgsndp, it was addressed by TIR and could only send interrupts between threads on the core.

ISA v3.0 changed msgsnd to be addressed by PIR and can interrupt any thread in the system.

msgsnd only implements the v3.0 semantics, which can make multi-threaded POWER8 hang when booting Linux (due to IPIs failing). This change adds v2.07 semantics.

S-o-b is missing.

Thanks,

C.

Quoted 108 lines
---
  target/ppc/excp_helper.c | 74 ++++++++++++++++++++++++----------------
  1 file changed, 44 insertions(+), 30 deletions(-)
target/ppc/excp_helper.c +44 -30
index 0cd542675f..c0120c8a88 100644
@@ -2998,6 +2998,41 @@ static inline bool dbell_bcast_subproc(target_ulong rb)
      return (rb & DBELL_BRDCAST_MASK) == DBELL_BRDCAST_SUBPROC;
  }
 
+/*
+ * Send an interrupt to a thread in the same core as env).
+ */
+static void msgsnd_core_tir(CPUPPCState *env, uint32_t target_tir, int irq)
+{
+    PowerPCCPU *cpu = env_archcpu(env);
+    CPUState *cs = env_cpu(env);
+    uint32_t nr_threads = cs->nr_threads;
+
+    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
+    }
+
+    if (target_tir >= nr_threads) {
+        return;
+    }
+
+    if (nr_threads == 1) {
+        ppc_set_irq(cpu, irq, 1);
+    } else {
+        CPUState *ccs;
+
+        /* Does iothread need to be locked for walking CPU list? */
+        bql_lock();
+        THREAD_SIBLING_FOREACH(cs, ccs) {
+            PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+            if (target_tir == ppc_cpu_tir(ccpu)) {
+                ppc_set_irq(ccpu, irq, 1);
+                break;
+            }
+        }
+        bql_unlock();
+    }
+}
+
  void helper_book3s_msgclr(CPUPPCState *env, target_ulong rb)
  {
      if (!dbell_type_server(rb)) {
@@ -3018,6 +3053,13 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
          return;
      }
 
+    /* POWER8 msgsnd is like msgsndp (targets a thread within core) */
+    if (!(env->insns_flags2 & PPC2_ISA300)) {
+        msgsnd_core_tir(env, rb & PPC_BITMASK(57, 63), PPC_INTERRUPT_HDOORBELL);
+        return;
+    }
+
+    /* POWER9 and later msgsnd is a global (targets any thread) */
      cpu = ppc_get_vcpu_by_pir(pir);
      if (!cpu) {
          return;
@@ -3064,41 +3106,13 @@ void helper_book3s_msgclrp(CPUPPCState *env, target_ulong rb)
   */
  void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
  {
-    CPUState *cs = env_cpu(env);
-    PowerPCCPU *cpu = env_archcpu(env);
-    CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
-    int ttir = rb & PPC_BITMASK(57, 63);
-
      helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
-    }
-
-    if (!dbell_type_server(rb) || ttir >= nr_threads) {
-        return;
-    }
-
-    if (nr_threads == 1) {
-        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
+    if (!dbell_type_server(rb)) {
          return;
      }
 
-    /* Does iothread need to be locked for walking CPU list? */
-    bql_lock();
-    THREAD_SIBLING_FOREACH(cs, ccs) {
-        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
-        uint32_t thread_id = ppc_cpu_tir(ccpu);
-
-        if (ttir == thread_id) {
-            ppc_set_irq(ccpu, PPC_INTERRUPT_DOORBELL, 1);
-            bql_unlock();
-            return;
-        }
-    }
-
-    g_assert_not_reached();
+    msgsnd_core_tir(env, rb & PPC_BITMASK(57, 63), PPC_INTERRUPT_DOORBELL);
  }
  #endif /* TARGET_PPC64 */
 
#3 lore raw

PATCH 2/18 ppc/pnv: Add pointer from PnvCPUState to PnvCore

In reply to Nicholas Piggin

This helps move core state from CPU to core structures.

---
 include/hw/ppc/pnv_core.h | 1 +
 hw/ppc/pnv_core.c         | 3 +++
 2 files changed, 4 insertions(+)
include/hw/ppc/pnv_core.h +1 -0
index c6d62fd145..29cab9dfd9 100644
@@ -54,6 +54,7 @@ struct PnvCoreClass {
 #define PNV_CORE_TYPE_NAME(cpu_model) cpu_model PNV_CORE_TYPE_SUFFIX
 
 typedef struct PnvCPUState {
+    PnvCore *pnv_core;
     Object *intc;
 } PnvCPUState;
 
hw/ppc/pnv_core.c +3 -0
index f40ab721d6..2da271ffb6 100644
@@ -278,6 +278,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
         PowerPCCPU *cpu;
+        PnvCPUState *pnv_cpu;
 
         obj = object_new(typename);
         cpu = POWERPC_CPU(obj);
@@ -288,6 +289,8 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
         object_property_add_child(OBJECT(pc), name, obj);
 
         cpu->machine_data = g_new0(PnvCPUState, 1);
+        pnv_cpu = pnv_cpu_state(cpu);
+        pnv_cpu->pnv_core = pc;
 
         object_unref(obj);
     }
Signature · 1 line
2.45.1
#4 lore raw

PATCH 3/18 ppc/pnv: Add a pointer from PnvChip to PnvMachineState

In reply to Nicholas Piggin

This helps avoid qdev_get_machine() calls.

---
 include/hw/ppc/pnv_chip.h | 2 ++
 hw/ppc/pnv.c              | 3 ++-
 2 files changed, 4 insertions(+), 1 deletion(-)
include/hw/ppc/pnv_chip.h +2 -0
index 4eaa7d3999..decfbc0ff7 100644
@@ -23,6 +23,8 @@ struct PnvChip {
     SysBusDevice parent_obj;
 
     /*< public >*/
+    PnvMachineState *pnv_machine;
+
     uint32_t     chip_id;
     uint64_t     ram_start;
     uint64_t     ram_size;
hw/ppc/pnv.c +2 -1
index 7878fed43c..3bcf11984c 100644
@@ -2202,6 +2202,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
         return;
     }
 
+    chip->pnv_machine = pnv;
     chip->cores = g_new0(PnvCore *, chip->nr_cores);
 
     for (i = 0, core_hwid = 0; (core_hwid < sizeof(chip->cores_mask) * 8)
@@ -2614,7 +2615,7 @@ static void pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
 
 static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
 {
-    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+    PnvMachineState *pnv = PNV_MACHINE(n);
     int i;
 
     for (i = 0; i < pnv->num_chips; i++) {
Signature · 1 line
2.45.1
#5 lore raw

Re: [PATCH 03/18] ppc/pnv: Add a pointer from PnvChip to PnvMachineState

Quoted 1 line · Nicholas Piggin

This helps avoid qdev_get_machine() calls.

I see two uses of this pointer in pnv_core_realize() :

    ...
    PnvMachineState *pnv = pc->chip->pnv_machine;
    ....
    pc->big_core = pnv->big_core;
    pc->lpar_per_core = pc->chip->pnv_machine->lpar_per_core;

Instead of this (ugly) machine pointer, why not add "lpar-per-core" and "big-core" properties to PnvCore and simply set them in pnv_chip_core_realize() ? I would prefer this solution.

Thanks,

C.

Quoted 40 lines
---
  include/hw/ppc/pnv_chip.h | 2 ++ 
  hw/ppc/pnv.c              | 3 ++-
  2 files changed, 4 insertions(+), 1 deletion(-)
include/hw/ppc/pnv_chip.h +2 -0
index 4eaa7d3999..decfbc0ff7 100644
@@ -23,6 +23,8 @@ struct PnvChip {
      SysBusDevice parent_obj;
 
      /*< public >*/
+    PnvMachineState *pnv_machine;
+
      uint32_t     chip_id;
      uint64_t     ram_start;
      uint64_t     ram_size;
hw/ppc/pnv.c +2 -1
index 7878fed43c..3bcf11984c 100644
@@ -2202,6 +2202,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
          return;
      }
 
+    chip->pnv_machine = pnv;
      chip->cores = g_new0(PnvCore *, chip->nr_cores);
 
      for (i = 0, core_hwid = 0; (core_hwid < sizeof(chip->cores_mask) * 8)
@@ -2614,7 +2615,7 @@ static void pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
 
  static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
  {
-    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+    PnvMachineState *pnv = PNV_MACHINE(n);
      int i;
 
      for (i = 0; i < pnv->num_chips; i++) {
#6 lore raw

PATCH 4/18 ppc/pnv: Move timebase state into PnvCore

In reply to Nicholas Piggin

The timebase state machine is per per-core state and can be driven by any thread in the core. It is currently implemented as a hack where the state is in a CPU structure and only thread 0's state is accessed by the chiptod, which limits programming the timebase side of the state machine to thread 0 of a core.

Move the state out into PnvCore and share it among all threads.

---
 include/hw/ppc/pnv_core.h    | 17 ++++++++++++
 target/ppc/cpu.h             | 21 --------------
 hw/ppc/pnv_chiptod.c         |  7 ++---
 target/ppc/timebase_helper.c | 53 ++++++++++++++++++++----------------
 4 files changed, 49 insertions(+), 49 deletions(-)
include/hw/ppc/pnv_core.h +17 -0
index 29cab9dfd9..ffec8516ae 100644
@@ -25,6 +25,20 @@
 #include "hw/ppc/pnv.h"
 #include "qom/object.h"
 
+/* Per-core ChipTOD / TimeBase state */
+typedef struct PnvCoreTODState {
+    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
+    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
+
+    /*
+     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
+     * is polled for such events. These are just used to ensure firmware
+     * performs the polling at least a few times.
+     */
+    int tb_state_timer;
+    int tb_sync_pulse_timer;
+} PnvCoreTODState;
+
 #define TYPE_PNV_CORE "powernv-cpu-core"
 OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
                     PNV_CORE)
@@ -38,6 +52,9 @@ struct PnvCore {
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
+
+    PnvCoreTODState tod_state;
+
     PnvChip *chip;
 
     MemoryRegion xscom_regs;
target/ppc/cpu.h +0 -21
index 2015e603d4..c78d6ca91a 100644
@@ -1196,21 +1196,6 @@ DEXCR_ASPECT(SRAPD, 4)
 DEXCR_ASPECT(NPHIE, 5)
 DEXCR_ASPECT(PHIE, 6)
 
-/*****************************************************************************/
-/* PowerNV ChipTOD and TimeBase State Machine */
-struct pnv_tod_tbst {
-    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
-    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
-
-    /*
-     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
-     * is polled for such events. These are just used to ensure firmware
-     * performs the polling at least a few times.
-     */
-    int tb_state_timer;
-    int tb_sync_pulse_timer;
-};
-
 /*****************************************************************************/
 /* The whole PowerPC CPU context */
 
@@ -1291,12 +1276,6 @@ struct CPUArchState {
     uint32_t tlb_need_flush; /* Delayed flush needed */
 #define TLB_NEED_LOCAL_FLUSH   0x1
 #define TLB_NEED_GLOBAL_FLUSH  0x2
-
-#if defined(TARGET_PPC64)
-    /* PowerNV chiptod / timebase facility state. */
-    /* Would be nice to put these into PnvCore */
-    struct pnv_tod_tbst pnv_tod_tbst;
-#endif
 #endif
 
     /* Other registers */
hw/ppc/pnv_chiptod.c +3 -4
index 3831a72101..1e41fe557a 100644
@@ -364,8 +364,7 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
             qemu_log_mask(LOG_GUEST_ERROR, "pnv_chiptod: xscom write reg"
                           " TOD_MOVE_TOD_TO_TB_REG with no slave target\n");
         } else {
-            PowerPCCPU *cpu = chiptod->slave_pc_target->threads[0];
-            CPUPPCState *env = &cpu->env;
+            PnvCore *pc = chiptod->slave_pc_target;
 
             /*
              * Moving TOD to TB will set the TB of all threads in a
@@ -377,8 +376,8 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
              * thread 0.
              */
 
-            if (env->pnv_tod_tbst.tb_ready_for_tod) {
-                env->pnv_tod_tbst.tod_sent_to_tb = 1;
+            if (pc->tod_state.tb_ready_for_tod) {
+                pc->tod_state.tod_sent_to_tb = 1;
             } else {
                 qemu_log_mask(LOG_GUEST_ERROR, "pnv_chiptod: xscom write reg"
                               " TOD_MOVE_TOD_TO_TB_REG with TB not ready to"
target/ppc/timebase_helper.c +29 -24

132 lines not on this page · raw message

Signature · 1 line
2.45.1
#7 lore raw

PATCH 5/18 target/ppc: Move SPR indirect registers into PnvCore

In reply to Nicholas Piggin

SPRC/SPRD were recently added to all BookS CPUs supported, but they are only tested on POWER9 and POWER10, so restrict them to those CPUs.

SPR indirect scratch registers presently replicated per-CPU like SMT SPRs, but the PnvCore is a better place for them since they are restricted to P9/P10.

Also add SPR indirect read access to core thread state for POWER9 since skiboot accesses that when booting to check for big-core mode.

---
 include/hw/ppc/pnv_core.h |  1 +
 target/ppc/cpu.h          |  3 --
 target/ppc/cpu_init.c     | 21 +++++++-------
 target/ppc/misc_helper.c  | 60 ++++++++++++++++++---------------------
 4 files changed, 39 insertions(+), 46 deletions(-)
include/hw/ppc/pnv_core.h +1 -0
index ffec8516ae..693acb189b 100644
@@ -53,6 +53,7 @@ struct PnvCore {
     uint32_t hwid;
     uint64_t hrmor;
 
+    target_ulong scratch[8]; /* SPRC/SPRD indirect SCRATCH registers */
     PnvCoreTODState tod_state;
 
     PnvChip *chip;
target/ppc/cpu.h +0 -3
index c78d6ca91a..95ba9e7590 100644
@@ -1253,9 +1253,6 @@ struct CPUArchState {
     ppc_slb_t slb[MAX_SLB_ENTRIES]; /* PowerPC 64 SLB area */
     struct CPUBreakpoint *ciabr_breakpoint;
     struct CPUWatchpoint *dawr0_watchpoint;
-
-    /* POWER CPU regs/state */
-    target_ulong scratch[8]; /* SCRATCH registers (shared across core) */
 #endif
     target_ulong sr[32];   /* segment registers */
     uint32_t nb_BATs;      /* number of BATs */
target/ppc/cpu_init.c +11 -10
index 01e358a4a5..ae483e20c4 100644
@@ -5759,16 +5759,6 @@ static void register_power_common_book4_sprs(CPUPPCState *env)
                  SPR_NOACCESS, SPR_NOACCESS,
                  &spr_read_generic, &spr_core_write_generic,
                  0x00000000);
-    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 &spr_read_generic, &spr_write_sprc,
-                 0x00000000);
-    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 &spr_read_sprd, &spr_write_sprd,
-                 0x00000000);
 #endif
 }
 
@@ -5781,6 +5771,17 @@ static void register_power9_book4_sprs(CPUPPCState *env)
                      SPR_NOACCESS, SPR_NOACCESS,
                      &spr_read_generic, &spr_write_generic,
                      KVM_REG_PPC_WORT, 0);
+    /* SPRC/SPRD exist in earlier CPUs but only tested on POWER9/10 */
+    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 &spr_read_generic, &spr_write_sprc,
+                 0x00000000);
+    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 &spr_read_sprd, &spr_write_sprd,
+                 0x00000000);
 #endif
 }
 
target/ppc/misc_helper.c +27 -33

85 lines not on this page · raw message

Signature · 1 line
2.45.1
#8 lore raw

PATCH 6/18 ppc/pnv: specialise init for powernv8/9/10 machines

In reply to Nicholas Piggin

This will allow different settings and checks for different machine types with later changes.

---
 include/hw/ppc/pnv.h |  1 +
 hw/ppc/pnv.c         | 12 +++++++++---
 2 files changed, 10 insertions(+), 3 deletions(-)
include/hw/ppc/pnv.h +1 -0
index 476b136146..1993dededf 100644
@@ -76,6 +76,7 @@ struct PnvMachineClass {
     /*< public >*/
     const char *compat;
     int compat_size;
+    int max_smt_threads;
 
     void (*dt_power_mgt)(PnvMachineState *pnv, void *fdt);
     void (*i2c_init)(PnvMachineState *pnv);
hw/ppc/pnv.c +9 -3
index 3bcf11984c..4252bcd28d 100644
@@ -899,6 +899,7 @@ static void pnv_init(MachineState *machine)
     PnvMachineState *pnv = PNV_MACHINE(machine);
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     PnvMachineClass *pmc = PNV_MACHINE_GET_CLASS(machine);
+    int max_smt_threads = pmc->max_smt_threads;
     char *fw_filename;
     long fw_size;
     uint64_t chip_ram_start = 0;
@@ -997,17 +998,19 @@ static void pnv_init(MachineState *machine)
     pnv->num_chips =
         machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
-    if (machine->smp.threads > 8) {
-        error_report("Cannot support more than 8 threads/core "
-                     "on a powernv machine");
+    if (machine->smp.threads > max_smt_threads) {
+        error_report("Cannot support more than %d threads/core "
+                     "on %s machine", max_smt_threads, mc->desc);
         exit(1);
     }
+
     if (!is_power_of_2(machine->smp.threads)) {
         error_report("Cannot support %d threads/core on a powernv"
                      "machine because it must be a power of 2",
                      machine->smp.threads);
         exit(1);
     }
+
     /*
      * TODO: should we decide on how many chips we can create based
      * on #cores and Venice vs. Murano vs. Naples chip type etc...,
@@ -2491,6 +2494,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 8;
 
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
 }
@@ -2515,6 +2519,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
 
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 4;
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
@@ -2539,6 +2544,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
 
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 4;
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     xfc->match_nvt = pnv10_xive_match_nvt;
Signature · 1 line
2.45.1
Reviewed-by on v1 6/18
#9 lore raw

Re: [PATCH 06/18] ppc/pnv: specialise init for powernv8/9/10 machines

Quoted 5 lines · Nicholas Piggin

This will allow different settings and checks for different machine types with later changes.

Thanks,

C.

Quoted 76 lines
---
  include/hw/ppc/pnv.h |  1 +
  hw/ppc/pnv.c         | 12 +++++++++---
  2 files changed, 10 insertions(+), 3 deletions(-)
include/hw/ppc/pnv.h +1 -0
index 476b136146..1993dededf 100644
@@ -76,6 +76,7 @@ struct PnvMachineClass {
      /*< public >*/
      const char *compat;
      int compat_size;
+    int max_smt_threads;
 
      void (*dt_power_mgt)(PnvMachineState *pnv, void *fdt);
      void (*i2c_init)(PnvMachineState *pnv);
hw/ppc/pnv.c +9 -3
index 3bcf11984c..4252bcd28d 100644
@@ -899,6 +899,7 @@ static void pnv_init(MachineState *machine)
      PnvMachineState *pnv = PNV_MACHINE(machine);
      MachineClass *mc = MACHINE_GET_CLASS(machine);
      PnvMachineClass *pmc = PNV_MACHINE_GET_CLASS(machine);
+    int max_smt_threads = pmc->max_smt_threads;
      char *fw_filename;
      long fw_size;
      uint64_t chip_ram_start = 0;
@@ -997,17 +998,19 @@ static void pnv_init(MachineState *machine)
      pnv->num_chips =
          machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
-    if (machine->smp.threads > 8) {
-        error_report("Cannot support more than 8 threads/core "
-                     "on a powernv machine");
+    if (machine->smp.threads > max_smt_threads) {
+        error_report("Cannot support more than %d threads/core "
+                     "on %s machine", max_smt_threads, mc->desc);
          exit(1);
      }
+
      if (!is_power_of_2(machine->smp.threads)) {
          error_report("Cannot support %d threads/core on a powernv"
                       "machine because it must be a power of 2",
                       machine->smp.threads);
          exit(1);
      }
+
      /*
       * TODO: should we decide on how many chips we can create based
       * on #cores and Venice vs. Murano vs. Naples chip type etc...,
@@ -2491,6 +2494,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 
      pmc->compat = compat;
      pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 8;
 
      machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
  }
@@ -2515,6 +2519,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
 
      pmc->compat = compat;
      pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 4;
      pmc->dt_power_mgt = pnv_dt_power_mgt;
 
      machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
@@ -2539,6 +2544,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
 
      pmc->compat = compat;
      pmc->compat_size = sizeof(compat);
+    pmc->max_smt_threads = 4;
      pmc->dt_power_mgt = pnv_dt_power_mgt;
 
      xfc->match_nvt = pnv10_xive_match_nvt;
#10 lore raw

PATCH 7/18 ppc/pnv: Extend chip_pir class method to TIR as well

In reply to Nicholas Piggin

The chip_pir chip class method allows the platform to set the PIR processor identification register. Extend this to a more general ID function which also allows the TIR to be set. This is in preparation for "big core", which is a more complicated topology of cores and threads.

---
 include/hw/ppc/pnv_chip.h |  4 +-
 hw/ppc/pnv.c              | 85 +++++++++++++++++++++++++--------------
 hw/ppc/pnv_core.c         | 10 +++--
 3 files changed, 64 insertions(+), 35 deletions(-)
include/hw/ppc/pnv_chip.h +3 -1
index decfbc0ff7..1fca540eb1 100644
@@ -152,7 +152,9 @@ struct PnvChipClass {
 
     DeviceRealize parent_realize;
 
-    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
+    /* Get PIR and TIR values for a CPU thread identified by core/thread id */
+    void (*get_pir_tir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
+                         uint32_t *pir, uint32_t *tir);
     void (*intc_create)(PnvChip *chip, PowerPCCPU *cpu, Error **errp);
     void (*intc_reset)(PnvChip *chip, PowerPCCPU *cpu);
     void (*intc_destroy)(PnvChip *chip, PowerPCCPU *cpu);
hw/ppc/pnv.c +55 -30

164 lines not on this page · raw message

hw/ppc/pnv_core.c +6 -4
index 2da271ffb6..28ca61926d 100644
@@ -227,8 +227,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
 {
     CPUPPCState *env = &cpu->env;
     int core_hwid;
-    ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
-    ppc_spr_t *tir = &env->spr_cb[SPR_TIR];
+    ppc_spr_t *pir_spr = &env->spr_cb[SPR_PIR];
+    ppc_spr_t *tir_spr = &env->spr_cb[SPR_TIR];
+    uint32_t pir, tir;
     Error *local_err = NULL;
     PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
 
@@ -244,8 +245,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
 
     core_hwid = object_property_get_uint(OBJECT(pc), "hwid", &error_abort);
 
-    tir->default_value = thread_index;
-    pir->default_value = pcc->chip_pir(pc->chip, core_hwid, thread_index);
+    pcc->get_pir_tir(pc->chip, core_hwid, thread_index, &pir, &tir);
+    pir_spr->default_value = pir;
+    tir_spr->default_value = tir;
 
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
Signature · 1 line
2.45.1
Reviewed-by on v1 7/18
#11 lore raw

Re: [PATCH 07/18] ppc/pnv: Extend chip_pir class method to TIR as well

Quoted 7 lines · Nicholas Piggin

The chip_pir chip class method allows the platform to set the PIR processor identification register. Extend this to a more general ID function which also allows the TIR to be set. This is in preparation for "big core", which is a more complicated topology of cores and threads.

Thanks,

C.

Quoted 229 lines
---
  include/hw/ppc/pnv_chip.h |  4 +-
  hw/ppc/pnv.c              | 85 +++++++++++++++++++++++++--------------
  hw/ppc/pnv_core.c         | 10 +++--
  3 files changed, 64 insertions(+), 35 deletions(-)
include/hw/ppc/pnv_chip.h +3 -1
index decfbc0ff7..1fca540eb1 100644
@@ -152,7 +152,9 @@ struct PnvChipClass {
 
      DeviceRealize parent_realize;
 
-    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
+    /* Get PIR and TIR values for a CPU thread identified by core/thread id */
+    void (*get_pir_tir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
+                         uint32_t *pir, uint32_t *tir);
      void (*intc_create)(PnvChip *chip, PowerPCCPU *cpu, Error **errp);
      void (*intc_reset)(PnvChip *chip, PowerPCCPU *cpu);
      void (*intc_destroy)(PnvChip *chip, PowerPCCPU *cpu);
hw/ppc/pnv.c +55 -30
index 4252bcd28d..b593a41f7c 100644
@@ -154,7 +154,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
      char *nodename;
      int cpus_offset = get_cpus_node(fdt);
 
-    pir = pnv_cc->chip_pir(chip, pc->hwid, 0);
+    pnv_cc->get_pir_tir(chip, pc->hwid, 0, &pir, NULL);
 
      nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
      offset = fdt_add_subnode(fdt, cpus_offset, nodename);
@@ -236,7 +236,8 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
 
      /* Build interrupt servers properties */
      for (i = 0; i < smt_threads; i++) {
-        servers_prop[i] = cpu_to_be32(pnv_cc->chip_pir(chip, pc->hwid, i));
+        pnv_cc->get_pir_tir(chip, pc->hwid, i, &pir, NULL);
+        servers_prop[i] = cpu_to_be32(pir);
      }
      _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
                         servers_prop, sizeof(*servers_prop) * smt_threads)));
@@ -248,14 +249,17 @@ static void pnv_dt_icp(PnvChip *chip, void *fdt, uint32_t hwid,
                         uint32_t nr_threads)
  {
      PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
-    uint32_t pir = pcc->chip_pir(chip, hwid, 0);
-    uint64_t addr = PNV_ICP_BASE(chip) | (pir << 12);
+    uint32_t pir;
+    uint64_t addr;
      char *name;
      const char compat[] = "IBM,power8-icp\0IBM,ppc-xicp";
      uint32_t irange[2], i, rsize;
      uint64_t *reg;
      int offset;
 
+    pcc->get_pir_tir(chip, hwid, 0, &pir, NULL);
+    addr = PNV_ICP_BASE(chip) | (pir << 12);
+
      irange[0] = cpu_to_be32(pir);
      irange[1] = cpu_to_be32(nr_threads);
 
@@ -1106,10 +1110,16 @@ static void pnv_init(MachineState *machine)
   *   25:28  Core number
   *   29:31  Thread ID
   */
-static uint32_t pnv_chip_pir_p8(PnvChip *chip, uint32_t core_id,
-                                uint32_t thread_id)
+static void pnv_get_pir_tir_p8(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
  {
-    return (chip->chip_id << 7) | (core_id << 3) | thread_id;
+    if (pir) {
+        *pir = (chip->chip_id << 7) | (core_id << 3) | thread_id;
+    }
+    if (tir) {
+        *tir = thread_id;
+    }
  }
 
  static void pnv_chip_power8_intc_create(PnvChip *chip, PowerPCCPU *cpu,
@@ -1161,14 +1171,20 @@ static void pnv_chip_power8_intc_print_info(PnvChip *chip, PowerPCCPU *cpu,
   *
   * We only care about the lower bits. uint32_t is fine for the moment.
   */
-static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
-                                uint32_t thread_id)
-{
-    if (chip->nr_threads == 8) {
-        return (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
-               (thread_id >> 1);
-    } else {
-        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
+static void pnv_get_pir_tir_p9(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
+{
+    if (pir) {
+        if (chip->nr_threads == 8) {
+            *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) |
+                   (core_id << 3) | (thread_id >> 1);
+        } else {
+            *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
+        }
+    }
+    if (tir) {
+        *tir = thread_id;
      }
  }
 
@@ -1183,14 +1199,20 @@ static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
   *
   * We only care about the lower bits. uint32_t is fine for the moment.
   */
-static uint32_t pnv_chip_pir_p10(PnvChip *chip, uint32_t core_id,
-                                 uint32_t thread_id)
-{
-    if (chip->nr_threads == 8) {
-        return (chip->chip_id << 8) | ((core_id / 4) << 4) |
-               ((core_id % 2) << 3) | thread_id;
-    } else {
-        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
+static void pnv_get_pir_tir_p10(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
+{
+    if (pir) {
+        if (chip->nr_threads == 8) {
+            *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
+                    ((core_id % 2) << 3) | thread_id;
+        } else {
+            *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
+        }
+    }
+    if (tir) {
+        *tir = thread_id;
      }
  }
 
@@ -1370,8 +1392,11 @@ static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp)
          int core_hwid = CPU_CORE(pnv_core)->core_id;
 
          for (j = 0; j < CPU_CORE(pnv_core)->nr_threads; j++) {
-            uint32_t pir = pcc->chip_pir(chip, core_hwid, j);
-            PnvICPState *icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
+            uint32_t pir;
+            PnvICPState *icp;
+
+            pcc->get_pir_tir(chip, core_hwid, j, &pir, NULL);
+            icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
 
              memory_region_add_subregion(&chip8->icp_mmio, pir << 12,
                                          &icp->mmio);
@@ -1483,7 +1508,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data)
      k->chip_cfam_id = 0x221ef04980000000ull;  /* P8 Murano DD2.1 */
      k->cores_mask = POWER8E_CORE_MASK;
      k->num_phbs = 3;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->get_pir_tir = pnv_get_pir_tir_p8;
      k->intc_create = pnv_chip_power8_intc_create;
      k->intc_reset = pnv_chip_power8_intc_reset;
      k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1507,7 +1532,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data)
      k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */
      k->cores_mask = POWER8_CORE_MASK;
      k->num_phbs = 3;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->get_pir_tir = pnv_get_pir_tir_p8;
      k->intc_create = pnv_chip_power8_intc_create;
      k->intc_reset = pnv_chip_power8_intc_reset;
      k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1531,7 +1556,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data)
      k->chip_cfam_id = 0x120d304980000000ull;  /* P8 Naples DD1.0 */
      k->cores_mask = POWER8_CORE_MASK;
      k->num_phbs = 4;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->get_pir_tir = pnv_get_pir_tir_p8;
      k->intc_create = pnv_chip_power8_intc_create;
      k->intc_reset = pnv_chip_power8_intc_reset;
      k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1814,7 +1839,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data)
 
      k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */
      k->cores_mask = POWER9_CORE_MASK;
-    k->chip_pir = pnv_chip_pir_p9;
+    k->get_pir_tir = pnv_get_pir_tir_p9;
      k->intc_create = pnv_chip_power9_intc_create;
      k->intc_reset = pnv_chip_power9_intc_reset;
      k->intc_destroy = pnv_chip_power9_intc_destroy;
@@ -2136,7 +2161,7 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
 
      k->chip_cfam_id = 0x120da04900008000ull; /* P10 DD1.0 (with NX) */
      k->cores_mask = POWER10_CORE_MASK;
-    k->chip_pir = pnv_chip_pir_p10;
+    k->get_pir_tir = pnv_get_pir_tir_p10;
      k->intc_create = pnv_chip_power10_intc_create;
      k->intc_reset = pnv_chip_power10_intc_reset;
      k->intc_destroy = pnv_chip_power10_intc_destroy;
hw/ppc/pnv_core.c +6 -4
index 2da271ffb6..28ca61926d 100644
@@ -227,8 +227,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
  {
      CPUPPCState *env = &cpu->env;
      int core_hwid;
-    ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
-    ppc_spr_t *tir = &env->spr_cb[SPR_TIR];
+    ppc_spr_t *pir_spr = &env->spr_cb[SPR_PIR];
+    ppc_spr_t *tir_spr = &env->spr_cb[SPR_TIR];
+    uint32_t pir, tir;
      Error *local_err = NULL;
      PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
 
@@ -244,8 +245,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
 
      core_hwid = object_property_get_uint(OBJECT(pc), "hwid", &error_abort);
 
-    tir->default_value = thread_index;
-    pir->default_value = pcc->chip_pir(pc->chip, core_hwid, thread_index);
+    pcc->get_pir_tir(pc->chip, core_hwid, thread_index, &pir, &tir);
+    pir_spr->default_value = pir;
+    tir_spr->default_value = tir;
 
      /* Set time-base frequency to 512 MHz */
      cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
#12 lore raw

PATCH 8/18 ppc: Add a core_index to CPUPPCState for SMT vCPUs

In reply to Nicholas Piggin

The way SMT thread siblings are matched is clunky, using hard-coded logic that checks the PIR SPR.

Change that to use a new core_index variable in the CPUPPCState, where all siblings have the same core_index. CPU realize routines have flexibility in setting core/sibling topology.

---
 target/ppc/cpu.h        | 5 ++++-
 hw/ppc/pnv_core.c       | 2 ++
 hw/ppc/spapr_cpu_core.c | 3 +++
 3 files changed, 9 insertions(+), 1 deletion(-)
target/ppc/cpu.h +4 -1
index 95ba9e7590..c3a33d2965 100644
@@ -1247,6 +1247,9 @@ struct CPUArchState {
     /* when a memory exception occurs, the access type is stored here */
     int access_type;
 
+    /* For SMT processors */
+    int core_index;
+
 #if !defined(CONFIG_USER_ONLY)
     /* MMU context, only relevant for full system emulation */
 #if defined(TARGET_PPC64)
@@ -1403,7 +1406,7 @@ struct CPUArchState {
 };
 
 #define _CORE_ID(cs)                                            \
-    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
+    (POWERPC_CPU(cs)->env.core_index)
 
 #define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
     CPU_FOREACH(cs_sibling)                                     \
hw/ppc/pnv_core.c +2 -0
index 28ca61926d..7bda29b9c7 100644
@@ -249,6 +249,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     pir_spr->default_value = pir;
     tir_spr->default_value = tir;
 
+    env->core_index = core_hwid;
+
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
 }
hw/ppc/spapr_cpu_core.c +3 -0
index 21920ea054..d9116c8409 100644
@@ -302,16 +302,19 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
     g_autofree char *id = NULL;
     CPUState *cs;
     PowerPCCPU *cpu;
+    CPUPPCState *env;
 
     obj = object_new(scc->cpu_type);
 
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    env = &cpu->env;
     /*
      * All CPUs start halted. CPU0 is unhalted from the machine level reset code
      * and the rest are explicitly started up by the guest using an RTAS call.
      */
     qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
+    env->core_index = cc->core_id;
     cs->cpu_index = cc->core_id + i;
     if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
         return NULL;
Signature · 1 line
2.45.1
Reviewed-by on v1 8/18
#13 lore raw

Re: [PATCH 08/18] ppc: Add a core_index to CPUPPCState for SMT vCPUs

Quoted 29 lines · Nicholas Piggin

The way SMT thread siblings are matched is clunky, using hard-coded logic that checks the PIR SPR.

Change that to use a new core_index variable in the CPUPPCState, where all siblings have the same core_index. CPU realize routines have flexibility in setting core/sibling topology.

---
  target/ppc/cpu.h        | 5 ++++-
  hw/ppc/pnv_core.c       | 2 ++
  hw/ppc/spapr_cpu_core.c | 3 +++
  3 files changed, 9 insertions(+), 1 deletion(-)
target/ppc/cpu.h +3 -0
index 95ba9e7590..c3a33d2965 100644
@@ -1247,6 +1247,9 @@ struct CPUArchState {
      /* when a memory exception occurs, the access type is stored here */
      int access_type;
 
+    /* For SMT processors */
+    int core_index;
+
  #if !defined(CONFIG_USER_ONLY)
      /* MMU context, only relevant for full system emulation */
  #if defined(TARGET_PPC64)

target/ppc/cpu.h struct CPUArchState { · line 1408

Quoted 3 lines
target/ppc/cpu.h +0 -0
@@ -1403,7 +1406,7 @@ struct CPUArchState {
  };
 
  #define _CORE_ID(cs)                                            \

That's a very short name for a macro in an header file. May be add a prefix.

Anyhow,

Thanks,

C.

Quoted 42 lines
target/ppc/cpu.h +1 -1
-    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
+    (POWERPC_CPU(cs)->env.core_index)
 
  #define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
      CPU_FOREACH(cs_sibling)                                     \
hw/ppc/pnv_core.c +2 -0
index 28ca61926d..7bda29b9c7 100644
@@ -249,6 +249,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
      pir_spr->default_value = pir;
      tir_spr->default_value = tir;
 
+    env->core_index = core_hwid;
+
      /* Set time-base frequency to 512 MHz */
      cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
  }
hw/ppc/spapr_cpu_core.c +3 -0
index 21920ea054..d9116c8409 100644
@@ -302,16 +302,19 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
      g_autofree char *id = NULL;
      CPUState *cs;
      PowerPCCPU *cpu;
+    CPUPPCState *env;
 
      obj = object_new(scc->cpu_type);
 
      cs = CPU(obj);
      cpu = POWERPC_CPU(obj);
+    env = &cpu->env;
      /*
       * All CPUs start halted. CPU0 is unhalted from the machine level reset code
       * and the rest are explicitly started up by the guest using an RTAS call.
       */
      qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
+    env->core_index = cc->core_id;
      cs->cpu_index = cc->core_id + i;
      if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
          return NULL;
#14 lore raw

PATCH 9/18 target/ppc: Add helpers to check for SMT sibling threads

In reply to Nicholas Piggin

Add helpers for TCG code to determine if there are SMT siblings sharing per-core and per-lpar registers. This simplifies the callers and makes SMT register topology simpler to modify with later changes.

---
 target/ppc/cpu.h             | 11 +++++++++++
 target/ppc/cpu_init.c        |  2 +-
 target/ppc/excp_helper.c     | 17 +++++------------
 target/ppc/misc_helper.c     | 27 ++++++---------------------
 target/ppc/timebase_helper.c | 20 +++++++-------------
 5 files changed, 30 insertions(+), 47 deletions(-)
target/ppc/cpu.h +11 -0
index c3a33d2965..a74b753c99 100644
@@ -1514,6 +1514,17 @@ struct PowerPCCPUClass {
     int  (*check_attn)(CPUPPCState *env);
 };
 
+static inline bool ppc_cpu_core_single_threaded(CPUState *cs)
+{
+    return cs->nr_threads == 1;
+}
+
+static inline bool ppc_cpu_lpar_single_threaded(CPUState *cs)
+{
+    return !(POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) ||
+           ppc_cpu_core_single_threaded(cs);
+}
+
 ObjectClass *ppc_cpu_class_by_name(const char *name);
 PowerPCCPUClass *ppc_cpu_class_by_pvr(uint32_t pvr);
 PowerPCCPUClass *ppc_cpu_class_by_pvr_mask(uint32_t pvr);
target/ppc/cpu_init.c +1 -1
index ae483e20c4..9349001b76 100644
@@ -6975,7 +6975,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
 
     pcc->parent_realize(dev, errp);
 
-    if (env_cpu(env)->nr_threads > 1) {
+    if (!ppc_cpu_core_single_threaded(cs)) {
         env->flags |= POWERPC_FLAG_SMT;
     }
 
target/ppc/excp_helper.c +5 -12
index c0120c8a88..f33fc36db2 100644
@@ -3005,18 +3005,11 @@ static void msgsnd_core_tir(CPUPPCState *env, uint32_t target_tir, int irq)
 {
     PowerPCCPU *cpu = env_archcpu(env);
     CPUState *cs = env_cpu(env);
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
-    }
-
-    if (target_tir >= nr_threads) {
-        return;
-    }
-
-    if (nr_threads == 1) {
-        ppc_set_irq(cpu, irq, 1);
+    if (ppc_cpu_lpar_single_threaded(cs)) {
+        if (target_tir == 0) {
+            ppc_set_irq(cpu, irq, 1);
+        }
     } else {
         CPUState *ccs;
 
@@ -3071,7 +3064,7 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
         brdcast = true;
     }
 
-    if (cs->nr_threads == 1 || !brdcast) {
+    if (ppc_cpu_core_single_threaded(cs) || !brdcast) {
         ppc_set_irq(cpu, PPC_INTERRUPT_HDOORBELL, 1);
         return;
     }
target/ppc/misc_helper.c +6 -21
index da7764c008..2032f3f7c1 100644
@@ -49,9 +49,8 @@ void helper_spr_core_write_generic(CPUPPCState *env, uint32_t sprn,
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1) {
+    if (ppc_cpu_core_single_threaded(cs)) {
         env->spr[sprn] = val;
         return;
     }
@@ -196,7 +195,7 @@ void helper_store_ptcr(CPUPPCState *env, target_ulong val)
             return;
         }
 
-        if (cs->nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+        if (ppc_cpu_lpar_single_threaded(cs)) {
             env->spr[SPR_PTCR] = val;
             tlb_flush(cs);
         } else {
@@ -243,16 +242,12 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
     target_ulong dpdes = 0;
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
-    }
-
-    if (nr_threads == 1) {
+    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
             dpdes = 1;
         }
@@ -279,21 +274,11 @@ void helper_store_dpdes(CPUPPCState *env, target_ulong val)
     PowerPCCPU *cpu = env_archcpu(env);
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
-    }
-
-    if (val & ~(nr_threads - 1)) {
-        qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
-                      TARGET_FMT_lx"\n", val);
-        val &= (nr_threads - 1); /* Ignore the invalid bits */
-    }
-
-    if (nr_threads == 1) {
+    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
         return;
     }
target/ppc/timebase_helper.c +7 -13
index 52f9e6669c..44cacf065e 100644
@@ -63,9 +63,8 @@ void helper_store_purr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_purr(env, val);
         return;
     }
@@ -82,9 +81,8 @@ void helper_store_tbl(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_tbl(env, val);
         return;
     }
@@ -99,9 +97,8 @@ void helper_store_tbu(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_tbu(env, val);
         return;
     }
@@ -141,9 +138,8 @@ void helper_store_hdecr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_hdecr(env, val);
         return;
     }
@@ -158,9 +154,8 @@ void helper_store_vtb(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_vtb(env, val);
         return;
     }
@@ -175,9 +170,8 @@ void helper_store_tbu40(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (ppc_cpu_lpar_single_threaded(cs)) {
         cpu_ppc_store_tbu40(env, val);
         return;
     }
@@ -288,7 +282,7 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
 
-    if (cs->nr_threads == 1) {
+    if (ppc_cpu_core_single_threaded(cs)) {
         env->spr[SPR_TFMR] = val;
     } else {
         CPUState *ccs;
Signature · 1 line
2.45.1
#15 lore raw

PATCH 10/18 ppc: Add has_smt_siblings property to CPUPPCState

In reply to Nicholas Piggin

The decision to branch out to a slower SMT path in instruction emulation will become a bit more complicated with the way that "big-core" topology that will be implemented in subsequent changes. Hide these details from the wider CPU emulation code with a bool has_smt_siblings flag that can be set by machine initialisation.

---
 target/ppc/cpu.h        |  3 ++-
 hw/ppc/pnv_core.c       |  3 +++
 hw/ppc/spapr_cpu_core.c | 15 +++++++++++----
 3 files changed, 16 insertions(+), 5 deletions(-)
target/ppc/cpu.h +2 -1
index a74b753c99..35a1cb65cb 100644
@@ -1248,6 +1248,7 @@ struct CPUArchState {
     int access_type;
 
     /* For SMT processors */
+    bool has_smt_siblings;
     int core_index;
 
 #if !defined(CONFIG_USER_ONLY)
@@ -1516,7 +1517,7 @@ struct PowerPCCPUClass {
 
 static inline bool ppc_cpu_core_single_threaded(CPUState *cs)
 {
-    return cs->nr_threads == 1;
+    return !POWERPC_CPU(cs)->env.has_smt_siblings;
 }
 
 static inline bool ppc_cpu_lpar_single_threaded(CPUState *cs)
hw/ppc/pnv_core.c +3 -0
index 7bda29b9c7..8cfa94fbfa 100644
@@ -288,6 +288,9 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
         cpu = POWERPC_CPU(obj);
 
         pc->threads[i] = POWERPC_CPU(obj);
+        if (cc->nr_threads > 1) {
+            cpu->env.has_smt_siblings = true;
+        }
 
         snprintf(name, sizeof(name), "thread[%d]", i);
         object_property_add_child(OBJECT(pc), name, obj);
hw/ppc/spapr_cpu_core.c +11 -4
index d9116c8409..2c6eeb41a4 100644
@@ -314,12 +314,13 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
      * and the rest are explicitly started up by the guest using an RTAS call.
      */
     qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
-    env->core_index = cc->core_id;
     cs->cpu_index = cc->core_id + i;
     if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
         return NULL;
     }
 
+    env->core_index = cc->core_id;
+
     cpu->node_id = sc->node_id;
 
     id = g_strdup_printf("thread[%d]", i);
@@ -350,9 +351,15 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
     qemu_register_reset(spapr_cpu_core_reset_handler, sc);
     sc->threads = g_new0(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
-        sc->threads[i] = spapr_create_vcpu(sc, i, errp);
-        if (!sc->threads[i] ||
-            !spapr_realize_vcpu(sc->threads[i], spapr, sc, i, errp)) {
+        PowerPCCPU *cpu;
+
+        cpu = spapr_create_vcpu(sc, i, errp);
+        sc->threads[i] = cpu;
+        if (cpu && cc->nr_threads > 1) {
+            cpu->env.has_smt_siblings = true;
+        }
+
+        if (!cpu || !spapr_realize_vcpu(cpu, spapr, sc, i, errp)) {
             spapr_cpu_core_unrealize(dev);
             return;
         }
Signature · 1 line
2.45.1
Reviewed-by on v1 10/18
#16 lore raw

Re: [PATCH 10/18] ppc: Add has_smt_siblings property to CPUPPCState

Quoted 7 lines · Nicholas Piggin

The decision to branch out to a slower SMT path in instruction emulation will become a bit more complicated with the way that "big-core" topology that will be implemented in subsequent changes. Hide these details from the wider CPU emulation code with a bool has_smt_siblings flag that can be set by machine initialisation.

The distinction between what's done in pnv_core_realize() and pnv_core_cpu_realize() is not clear. Not this patch fault though.

Thanks,

C.

Quoted 79 lines
---
  target/ppc/cpu.h        |  3 ++-
  hw/ppc/pnv_core.c       |  3 +++
  hw/ppc/spapr_cpu_core.c | 15 +++++++++++----
  3 files changed, 16 insertions(+), 5 deletions(-)
target/ppc/cpu.h +2 -1
index a74b753c99..35a1cb65cb 100644
@@ -1248,6 +1248,7 @@ struct CPUArchState {
      int access_type;
 
      /* For SMT processors */
+    bool has_smt_siblings;
      int core_index;
 
  #if !defined(CONFIG_USER_ONLY)
@@ -1516,7 +1517,7 @@ struct PowerPCCPUClass {
 
  static inline bool ppc_cpu_core_single_threaded(CPUState *cs)
  {
-    return cs->nr_threads == 1;
+    return !POWERPC_CPU(cs)->env.has_smt_siblings;
  }
 
  static inline bool ppc_cpu_lpar_single_threaded(CPUState *cs)
hw/ppc/pnv_core.c +3 -0
index 7bda29b9c7..8cfa94fbfa 100644
@@ -288,6 +288,9 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
          cpu = POWERPC_CPU(obj);
 
          pc->threads[i] = POWERPC_CPU(obj);
+        if (cc->nr_threads > 1) {
+            cpu->env.has_smt_siblings = true;
+        }
 
          snprintf(name, sizeof(name), "thread[%d]", i);
          object_property_add_child(OBJECT(pc), name, obj);
hw/ppc/spapr_cpu_core.c +11 -4
index d9116c8409..2c6eeb41a4 100644
@@ -314,12 +314,13 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
       * and the rest are explicitly started up by the guest using an RTAS call.
       */
      qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
-    env->core_index = cc->core_id;
      cs->cpu_index = cc->core_id + i;
      if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
          return NULL;
      }
 
+    env->core_index = cc->core_id;
+
      cpu->node_id = sc->node_id;
 
      id = g_strdup_printf("thread[%d]", i);
@@ -350,9 +351,15 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
      qemu_register_reset(spapr_cpu_core_reset_handler, sc);
      sc->threads = g_new0(PowerPCCPU *, cc->nr_threads);
      for (i = 0; i < cc->nr_threads; i++) {
-        sc->threads[i] = spapr_create_vcpu(sc, i, errp);
-        if (!sc->threads[i] ||
-            !spapr_realize_vcpu(sc->threads[i], spapr, sc, i, errp)) {
+        PowerPCCPU *cpu;
+
+        cpu = spapr_create_vcpu(sc, i, errp);
+        sc->threads[i] = cpu;
+        if (cpu && cc->nr_threads > 1) {
+            cpu->env.has_smt_siblings = true;
+        }
+
+        if (!cpu || !spapr_realize_vcpu(cpu, spapr, sc, i, errp)) {
              spapr_cpu_core_unrealize(dev);
              return;
          }
#17 lore raw

PATCH 11/18 ppc/pnv: Add a big-core mode that joins two regular cores

In reply to Nicholas Piggin

POWER9 and POWER10 machines come in two variants, big-core and small-core. Big-core machines are SMT8 from software's point of view, but the low level platform topology ("xscom registers and pervasive addressing"), these look more like a pair of small cores ganged together.

Presently the way this is modelled is to create one SMT8 PnvCore and add special cases to xscom and pervasive for big-core mode that tries to split this into two small cores, but this is becoming too complicated to manage.

A better approach is to create 2 core structures and ganging them together to look like an SMT8 core in TCG. Then the xscom and pervasive models mostly do not need to differentiate big and small core modes.

This change adds initial mode bits and QEMU topology handling to split SMT8 cores into 2xSMT4 cores.

---
 include/hw/ppc/pnv.h      |  2 +
 include/hw/ppc/pnv_core.h |  1 +
 hw/ppc/pnv.c              | 79 ++++++++++++++++++++++++++++++++-------
 hw/ppc/pnv_core.c         |  7 +++-
 4 files changed, 75 insertions(+), 14 deletions(-)
include/hw/ppc/pnv.h +2 -0
index 1993dededf..283ddd50e7 100644
@@ -101,6 +101,8 @@ struct PnvMachineState {
     PnvPnor      *pnor;
 
     hwaddr       fw_load_addr;
+
+    bool         big_core;
 };
 
 PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
include/hw/ppc/pnv_core.h +1 -0
index 693acb189b..50164e9e1f 100644
@@ -49,6 +49,7 @@ struct PnvCore {
 
     /*< public >*/
     PowerPCCPU **threads;
+    bool big_core;
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
hw/ppc/pnv.c +66 -13

122 lines not on this page · raw message

hw/ppc/pnv_core.c +6 -1
index 8cfa94fbfa..16d40392db 100644
@@ -249,7 +249,12 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     pir_spr->default_value = pir;
     tir_spr->default_value = tir;
 
-    env->core_index = core_hwid;
+    if (pc->big_core) {
+        /* 2 "small cores" get the same core index for SMT operations */
+        env->core_index = core_hwid >> 1;
+    } else {
+        env->core_index = core_hwid;
+    }
 
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
Signature · 1 line
2.45.1
#18 lore raw

Re: [PATCH 11/18] ppc/pnv: Add a big-core mode that joins two regular cores

Quoted 97 lines · Nicholas Piggin

POWER9 and POWER10 machines come in two variants, big-core and small-core. Big-core machines are SMT8 from software's point of view, but the low level platform topology ("xscom registers and pervasive addressing"), these look more like a pair of small cores ganged together.

Presently the way this is modelled is to create one SMT8 PnvCore and add special cases to xscom and pervasive for big-core mode that tries to split this into two small cores, but this is becoming too complicated to manage.

A better approach is to create 2 core structures and ganging them together to look like an SMT8 core in TCG. Then the xscom and pervasive models mostly do not need to differentiate big and small core modes.

This change adds initial mode bits and QEMU topology handling to split SMT8 cores into 2xSMT4 cores.

---
  include/hw/ppc/pnv.h      |  2 +
  include/hw/ppc/pnv_core.h |  1 +
  hw/ppc/pnv.c              | 79 ++++++++++++++++++++++++++++++++-------
  hw/ppc/pnv_core.c         |  7 +++-
  4 files changed, 75 insertions(+), 14 deletions(-)
include/hw/ppc/pnv.h +2 -0
index 1993dededf..283ddd50e7 100644
@@ -101,6 +101,8 @@ struct PnvMachineState {
      PnvPnor      *pnor;
 
      hwaddr       fw_load_addr;
+
+    bool         big_core;
  };
 
  PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
include/hw/ppc/pnv_core.h +1 -0
index 693acb189b..50164e9e1f 100644
@@ -49,6 +49,7 @@ struct PnvCore {
 
      /*< public >*/
      PowerPCCPU **threads;
+    bool big_core;
      uint32_t pir;
      uint32_t hwid;
      uint64_t hrmor;
hw/ppc/pnv.c +26 -1
index b593a41f7c..d2ddc49142 100644
@@ -1002,14 +1002,39 @@ static void pnv_init(MachineState *machine)
      pnv->num_chips =
          machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
+    if (pnv->big_core) {
+        if (machine->smp.threads % 2 == 1) {
+            error_report("Cannot support %d threads with big-core option "
+                         "because it must be an even number",
+                         machine->smp.threads);
+            exit(1);
+        }
+        max_smt_threads *= 2;
+    }
+
      if (machine->smp.threads > max_smt_threads) {
          error_report("Cannot support more than %d threads/core "
                       "on %s machine", max_smt_threads, mc->desc);
+        if (pmc->max_smt_threads == 4) {
+            error_report("(use big-core=on for 8 threads per core)");
+        }
          exit(1);
      }
 
+    if (pnv->big_core) {
+        /*
+         * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvCore
+         * per core, so adjust topology here. pnv_dt_core() processor
+         * device-tree and TCG SMT code make the 2 cores appear as one big core
+         * from software point of view. pnv pervasive models and xscoms tend to
+         * see the big core as 2 small core halves.
+         */
+        machine->smp.cores *= 2;
+        machine->smp.threads /= 2;
+    }
+
      if (!is_power_of_2(machine->smp.threads)) {
-        error_report("Cannot support %d threads/core on a powernv"
+        error_report("Cannot support %d threads/core on a powernv "
                       "machine because it must be a power of 2",
                       machine->smp.threads);
          exit(1);

hw/ppc/pnv.c static void pnv_get_pir_tir_p9(PnvChip *chip, · line 1203

Quoted 9 lines
hw/ppc/pnv.c +1 -5
@@ -1175,11 +1200,19 @@ static void pnv_get_pir_tir_p9(PnvChip *chip,
                                  uint32_t core_id, uint32_t thread_id,
                                  uint32_t *pir, uint32_t *tir)
  {
-    if (pir) {
-        if (chip->nr_threads == 8) {
-            *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) |
-                   (core_id << 3) | (thread_id >> 1);
-        } else {
+    PnvMachineState *pnv = PNV_MACHINE(chip->pnv_machine);

The chip also needs a "big-core" property IMO.

Thanks,

C.

Quoted 94 lines
hw/ppc/pnv.c +38 -7
+    if (pnv->big_core) {
+        /* Big-core interleaves thread ID between small-cores */
+        thread_id <<= 1;
+        thread_id |= core_id & 1;
+        core_id >>= 1;
+
+        if (pir) {
+            *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
+        }
+    } else {
+        if (pir) {
              *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
          }
      }
@@ -1203,11 +1236,19 @@ static void pnv_get_pir_tir_p10(PnvChip *chip,
                                  uint32_t core_id, uint32_t thread_id,
                                  uint32_t *pir, uint32_t *tir)
  {
-    if (pir) {
-        if (chip->nr_threads == 8) {
-            *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
-                    ((core_id % 2) << 3) | thread_id;
-        } else {
+    PnvMachineState *pnv = PNV_MACHINE(chip->pnv_machine);
+
+    if (pnv->big_core) {
+        /* Big-core interleaves thread ID between small-cores */
+        thread_id <<= 1;
+        thread_id |= core_id & 1;
+        core_id >>= 1;
+
+        if (pir) {
+            *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
+        }
+    } else {
+        if (pir) {
              *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
          }
      }
@@ -2180,7 +2221,8 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
                                      &k->parent_realize);
  }
 
-static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
+static void pnv_chip_core_sanitize(PnvMachineState *pnv, PnvChip *chip,
+                                   Error **errp)
  {
      PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
      int cores_max;
@@ -2201,6 +2243,17 @@ static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
      }
      chip->cores_mask &= pcc->cores_mask;
 
+    /* Ensure small-cores a paired up in big-core mode */
+    if (pnv->big_core) {
+        uint64_t even_cores = chip->cores_mask & 0x5555555555555555ULL;
+        uint64_t odd_cores = chip->cores_mask & 0xaaaaaaaaaaaaaaaaULL;
+
+        if (even_cores ^ (odd_cores >> 1)) {
+            error_setg(errp, "warning: unpaired cores in big-core mode !");
+            return;
+        }
+    }
+
      /* now that we have a sane layout, let check the number of cores */
      cores_max = ctpop64(chip->cores_mask);
      if (chip->nr_cores > cores_max) {
@@ -2224,7 +2277,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
      }
 
      /* Cores */
-    pnv_chip_core_sanitize(chip, &error);
+    pnv_chip_core_sanitize(pnv, chip, &error);
      if (error) {
          error_propagate(errp, error);
          return;
hw/ppc/pnv_core.c +6 -1
index 8cfa94fbfa..16d40392db 100644
@@ -249,7 +249,12 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
      pir_spr->default_value = pir;
      tir_spr->default_value = tir;
 
-    env->core_index = core_hwid;
+    if (pc->big_core) {
+        /* 2 "small cores" get the same core index for SMT operations */
+        env->core_index = core_hwid >> 1;
+    } else {
+        env->core_index = core_hwid;
+    }
 
      /* Set time-base frequency to 512 MHz */
      cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
#19 lore raw

PATCH 12/18 ppc/pnv: Add allow for big-core differences in DT generation

In reply to Nicholas Piggin

device-tree building needs to account for big-core mode, because it is driven by qemu cores (small cores). Every second core should be skipped, and every core should describe threads for both small-cores that make up the big core.

---
 hw/ppc/pnv.c | 43 +++++++++++++++++++++++++++++++++++--------
 1 file changed, 35 insertions(+), 8 deletions(-)
hw/ppc/pnv.c +35 -8
index d2ddc49142..d7488be74c 100644
@@ -141,9 +141,9 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     CPUPPCState *env = &cpu->env;
     PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cs);
     PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
-    g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
+    uint32_t *servers_prop;
     int i;
-    uint32_t pir;
+    uint32_t pir, tir;
     uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
                        0xffffffff, 0xffffffff};
     uint32_t tbfreq = PNV_TIMEBASE_FREQ;
@@ -154,7 +154,10 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     char *nodename;
     int cpus_offset = get_cpus_node(fdt);
 
-    pnv_cc->get_pir_tir(chip, pc->hwid, 0, &pir, NULL);
+    pnv_cc->get_pir_tir(chip, pc->hwid, 0, &pir, &tir);
+
+    /* Only one DT node per (big) core */
+    g_assert(tir == 0);
 
     nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
     offset = fdt_add_subnode(fdt, cpus_offset, nodename);
@@ -235,12 +238,28 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     }
 
     /* Build interrupt servers properties */
-    for (i = 0; i < smt_threads; i++) {
-        pnv_cc->get_pir_tir(chip, pc->hwid, i, &pir, NULL);
-        servers_prop[i] = cpu_to_be32(pir);
+    if (pc->big_core) {
+        servers_prop = g_new(uint32_t, smt_threads * 2);
+        for (i = 0; i < smt_threads; i++) {
+            pnv_cc->get_pir_tir(chip, pc->hwid, i, &pir, NULL);
+            servers_prop[i * 2] = cpu_to_be32(pir);
+
+            pnv_cc->get_pir_tir(chip, pc->hwid + 1, i, &pir, NULL);
+            servers_prop[i * 2 + 1] = cpu_to_be32(pir);
+        }
+        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
+                          servers_prop, sizeof(*servers_prop) * smt_threads
+                                        * 2)));
+    } else {
+        servers_prop = g_new(uint32_t, smt_threads);
+        for (i = 0; i < smt_threads; i++) {
+            pnv_cc->get_pir_tir(chip, pc->hwid, i, &pir, NULL);
+            servers_prop[i] = cpu_to_be32(pir);
+        }
+        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
+                          servers_prop, sizeof(*servers_prop) * smt_threads)));
     }
-    _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
-                       servers_prop, sizeof(*servers_prop) * smt_threads)));
+    g_free(servers_prop);
 
     return offset;
 }
@@ -389,6 +408,10 @@ static void pnv_chip_power9_dt_populate(PnvChip *chip, void *fdt)
 
         _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
                            pa_features_300, sizeof(pa_features_300))));
+
+        if (pnv_core->big_core) {
+            i++; /* Big-core groups two QEMU cores */
+        }
     }
 
     if (chip->ram_size) {
@@ -450,6 +473,10 @@ static void pnv_chip_power10_dt_populate(PnvChip *chip, void *fdt)
 
         _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
                            pa_features_31, sizeof(pa_features_31))));
+
+        if (pnv_core->big_core) {
+            i++; /* Big-core groups two QEMU cores */
+        }
     }
 
     if (chip->ram_size) {
Signature · 1 line
2.45.1
#20 lore raw

PATCH 13/18 ppc/pnv: Implement big-core PVR for Power9/10

In reply to Nicholas Piggin

Power9/10 CPUs have PVR[51] set in small-core mode and clear in big-core mode. This is used by skiboot firmware.

PVR is not hypervisor-privileged but it is not so important that spapr to implement this because it's generally masked out of PVR matching code in kernels, and only used by firmware.

---
 hw/ppc/pnv_core.c | 4 ++++
 1 file changed, 4 insertions(+)
hw/ppc/pnv_core.c +4 -0
index 16d40392db..a96ec4e2b9 100644
@@ -58,6 +58,10 @@ static void pnv_core_cpu_reset(PnvCore *pc, PowerPCCPU *cpu)
     env->nip = 0x10;
     env->msr |= MSR_HVB; /* Hypervisor mode */
     env->spr[SPR_HRMOR] = pc->hrmor;
+    if (pc->big_core) {
+        /* Clear "small core" bit on Power9/10 (this is set in default PVR) */
+        env->spr[SPR_PVR] &= ~PPC_BIT(51);
+    }
     hreg_compute_hflags(env);
     ppc_maybe_interrupt(env);
 
Signature · 1 line
2.45.1
#21 lore raw

PATCH 14/18 ppc/pnv: Implement Power9 CPU core thread state indirect register

In reply to Nicholas Piggin

Power9 CPUs have a core thread state register accessible via SPRC/SPRD indirect registers. This register includes a bit for big-core mode, which skiboot requires.

---
 target/ppc/misc_helper.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
target/ppc/misc_helper.c +17 -0
index 2032f3f7c1..aa255233b4 100644
@@ -319,6 +319,23 @@ target_ulong helper_load_sprd(CPUPPCState *env)
     case 0: /* SCRATCH0-3 */
     case 1: /* SCRATCH4-7 */
         return pc->scratch[(sprc >> 3) & 0x7];
+
+    case 0x1e0: /* core thread state */
+        if (env->excp_model == POWERPC_EXCP_POWER9) {
+            /*
+             * Only implement for POWER9 because skiboot uses it to check
+             * big-core mode. Other bits are unimplemented so we would
+             * prefer to get unimplemented message on POWER10 if it were
+             * used anywhere.
+             */
+            if (pc->big_core) {
+                return PPC_BIT(63);
+            } else {
+                return 0;
+            }
+        }
+        /* fallthru */
+
     default:
         qemu_log_mask(LOG_UNIMP, "mfSPRD: Unimplemented SPRC:0x"
                                   TARGET_FMT_lx"\n", sprc);
Signature · 1 line
2.45.1
#22 lore raw

PATCH 15/18 ppc/pnv: Add POWER10 ChipTOD quirk for big-core

In reply to Nicholas Piggin

POWER10 has a quirk in its ChipTOD addressing that requires the even small-core to be selected even when programming the odd small-core. This allows skiboot chiptod init to run in big-core mode.

---
 include/hw/ppc/pnv.h         | 1 +
 include/hw/ppc/pnv_core.h    | 7 +++++++
 hw/ppc/pnv.c                 | 1 +
 hw/ppc/pnv_core.c            | 3 +++
 target/ppc/timebase_helper.c | 9 +++++++++
 5 files changed, 21 insertions(+)
include/hw/ppc/pnv.h +1 -0
index 283ddd50e7..c56d152889 100644
@@ -77,6 +77,7 @@ struct PnvMachineClass {
     const char *compat;
     int compat_size;
     int max_smt_threads;
+    bool quirk_tb_big_core;
 
     void (*dt_power_mgt)(PnvMachineState *pnv, void *fdt);
     void (*i2c_init)(PnvMachineState *pnv);
include/hw/ppc/pnv_core.h +7 -0
index 50164e9e1f..c8784777a4 100644
@@ -27,6 +27,13 @@
 
 /* Per-core ChipTOD / TimeBase state */
 typedef struct PnvCoreTODState {
+    /*
+     * POWER10 DD2.0 - big core TFMR drives the state machine on the even
+     * small core. Skiboot has a workaround that targets the even small core
+     * for CHIPTOD_TO_TB ops.
+     */
+    bool big_core_quirk;
+
     int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
     int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
 
hw/ppc/pnv.c +1 -0
index d7488be74c..efc9cf2cc3 100644
@@ -2650,6 +2650,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
     pmc->compat = compat;
     pmc->compat_size = sizeof(compat);
     pmc->max_smt_threads = 4;
+    pmc->quirk_tb_big_core = true;
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     xfc->match_nvt = pnv10_xive_match_nvt;
hw/ppc/pnv_core.c +3 -0
index a96ec4e2b9..68cc5914c6 100644
@@ -280,6 +280,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
     PnvCore *pc = PNV_CORE(OBJECT(dev));
     PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
     CPUCore *cc = CPU_CORE(OBJECT(dev));
+    PnvMachineClass *pmc = PNV_MACHINE_GET_CLASS(pc->chip->pnv_machine);
     const char *typename = pnv_core_cpu_typename(pc);
     Error *local_err = NULL;
     void *obj;
@@ -288,6 +289,8 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
     assert(pc->chip);
 
+    pc->tod_state.big_core_quirk = pmc->quirk_tb_big_core;
+
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
         PowerPCCPU *cpu;
target/ppc/timebase_helper.c +9 -0
index 44cacf065e..019b8ee41f 100644
@@ -20,6 +20,7 @@
 #include "cpu.h"
 #include "hw/ppc/ppc.h"
 #include "hw/ppc/pnv_core.h"
+#include "hw/ppc/pnv_chip.h"
 #include "exec/helper-proto.h"
 #include "exec/exec-all.h"
 #include "qemu/log.h"
@@ -297,6 +298,14 @@ static PnvCoreTODState *cpu_get_tbst(PowerPCCPU *cpu)
 {
     PnvCore *pc = pnv_cpu_state(cpu)->pnv_core;
 
+    if (pc->big_core && pc->tod_state.big_core_quirk) {
+        /* Must operate on the even small core */
+        int core_id = CPU_CORE(pc)->core_id;
+        if (core_id & 1) {
+            pc = pc->chip->cores[core_id & ~1];
+        }
+    }
+
     return &pc->tod_state;
 }
 
Signature · 1 line
2.45.1
#23 lore raw

PATCH 16/18 ppc/pnv: Add big-core machine property

In reply to Nicholas Piggin

Big-core implementation is complete, so expose it as a machine property that may be set with big-core=on option on powernv9 and powernv10 machines.

---
 hw/ppc/pnv.c      | 61 ++++++++++++++++++++++++++++++++++-------------
 hw/ppc/pnv_core.c |  6 +++--
 2 files changed, 49 insertions(+), 18 deletions(-)
hw/ppc/pnv.c +45 -16
index efc9cf2cc3..575f18958d 100644
@@ -2577,6 +2577,34 @@ static int pnv10_xive_match_nvt(XiveFabric *xfb, uint8_t format,
     return total_count;
 }
 
+static bool pnv_machine_get_big_core(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    return pnv->big_core;
+}
+
+static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    pnv->big_core = value;
+}
+
+static bool pnv_machine_get_hb(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+
+    return !!pnv->fw_load_addr;
+}
+
+static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+
+    if (value) {
+        pnv->fw_load_addr = 0x8000000;
+    }
+}
+
 static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -2628,6 +2656,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
+
+    object_class_property_add_bool(oc, "big-core",
+                                   pnv_machine_get_big_core,
+                                   pnv_machine_set_big_core);
+    object_class_property_set_description(oc, "big-core",
+                              "Use big-core (aka fused-core) mode");
 }
 
 static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
@@ -2664,6 +2698,17 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
 
     pnv_machine_p10_common_class_init(oc, data);
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
+
+    /*
+     * This is the parent of POWER10 Rainier class, so properies go here
+     * rather than common init (which would add them to both parent and
+     * child which is invalid).
+     */
+    object_class_property_add_bool(oc, "big-core",
+                                   pnv_machine_get_big_core,
+                                   pnv_machine_set_big_core);
+    object_class_property_set_description(oc, "big-core",
+                              "Use big-core (aka fused-core) mode");
 }
 
 static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
@@ -2676,22 +2721,6 @@ static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
     pmc->i2c_init = pnv_rainier_i2c_init;
 }
 
-static bool pnv_machine_get_hb(Object *obj, Error **errp)
-{
-    PnvMachineState *pnv = PNV_MACHINE(obj);
-
-    return !!pnv->fw_load_addr;
-}
-
-static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
-{
-    PnvMachineState *pnv = PNV_MACHINE(obj);
-
-    if (value) {
-        pnv->fw_load_addr = 0x8000000;
-    }
-}
-
 static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
 {
     CPUPPCState *env = cpu_env(cs);
hw/ppc/pnv_core.c +4 -2
index 68cc5914c6..e03ac5441e 100644
@@ -280,7 +280,8 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
     PnvCore *pc = PNV_CORE(OBJECT(dev));
     PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
     CPUCore *cc = CPU_CORE(OBJECT(dev));
-    PnvMachineClass *pmc = PNV_MACHINE_GET_CLASS(pc->chip->pnv_machine);
+    PnvMachineState *pnv = pc->chip->pnv_machine;
+    PnvMachineClass *pmc = PNV_MACHINE_GET_CLASS(pnv);
     const char *typename = pnv_core_cpu_typename(pc);
     Error *local_err = NULL;
     void *obj;
@@ -289,6 +290,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
     assert(pc->chip);
 
+    pc->big_core = pnv->big_core;
     pc->tod_state.big_core_quirk = pmc->quirk_tb_big_core;
 
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
@@ -299,7 +301,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
         obj = object_new(typename);
         cpu = POWERPC_CPU(obj);
 
-        pc->threads[i] = POWERPC_CPU(obj);
+        pc->threads[i] = cpu;
         if (cc->nr_threads > 1) {
             cpu->env.has_smt_siblings = true;
         }
Signature · 1 line
2.45.1
#24 lore raw

PATCH 17/18 ppc/pnv: Implement POWER10 PC xscom registers for direct controls

In reply to Nicholas Piggin

The PC unit in the processor core contains xscom registers that provide low level status and control of the CPU.

This implements "direct controls" sufficient for OPAL (skiboot) firmware use, which is to stop threads and send them non-maskable IPIs in the form of SRESET interrupts.

POWER10 is sufficiently different (particularly QME and special wakeup) from POWER9 that it is not trivial to implement by reusing the code.

---
 include/hw/core/cpu.h     |  8 ++++
 include/hw/ppc/pnv.h      |  2 +
 include/hw/ppc/pnv_core.h |  3 ++
 hw/ppc/pnv.c              | 19 +++++++--
 hw/ppc/pnv_core.c         | 88 ++++++++++++++++++++++++++++++++++++---
 system/cpus.c             | 10 +++++
 6 files changed, 122 insertions(+), 8 deletions(-)
include/hw/core/cpu.h +8 -0
index a2c8536943..410e7d6f03 100644
@@ -984,6 +984,14 @@ void cpu_reset_interrupt(CPUState *cpu, int mask);
  */
 void cpu_exit(CPUState *cpu);
 
+/**
+ * cpu_pause:
+ * @cpu: The CPU to pause.
+ *
+ * Resumes CPU, i.e. puts CPU into stopped state.
+ */
+void cpu_pause(CPUState *cpu);
+
 /**
  * cpu_resume:
  * @cpu: The CPU to resume.
include/hw/ppc/pnv.h +2 -0
index c56d152889..b7858d310d 100644
@@ -112,6 +112,8 @@ PnvChip *pnv_chip_add_phb(PnvChip *chip, PnvPHB *phb);
 #define PNV_FDT_ADDR          0x01000000
 #define PNV_TIMEBASE_FREQ     512000000ULL
 
+void pnv_cpu_do_nmi_resume(CPUState *cs);
+
 /*
  * BMC helpers
  */
include/hw/ppc/pnv_core.h +3 -0
index c8784777a4..1de79a818e 100644
@@ -109,6 +109,9 @@ OBJECT_DECLARE_TYPE(PnvQuad, PnvQuadClass, PNV_QUAD)
 struct PnvQuad {
     DeviceState parent_obj;
 
+    bool special_wakeup_done;
+    bool special_wakeup[4];
+
     uint32_t quad_id;
     MemoryRegion xscom_regs;
     MemoryRegion xscom_qme_regs;
hw/ppc/pnv.c +16 -3
index 575f18958d..71b2b3806c 100644
@@ -2747,11 +2747,24 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
          */
         env->spr[SPR_SRR1] |= SRR1_WAKESCOM;
     }
+    if (arg.host_int) {
+        cpu_resume(cs);
+    }
+}
+
+static void pnv_cpu_do_nmi(CPUState *cs, int resume)
+{
+    async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_HOST_INT(resume));
+}
+
+void pnv_cpu_do_nmi_resume(CPUState *cs)
+{
+    pnv_cpu_do_nmi(cs, 1);
 }
 
-static void pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
+static void do_pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
 {
-    async_run_on_cpu(CPU(cpu), pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
+    pnv_cpu_do_nmi(CPU(cpu), 0);
 }
 
 static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
@@ -2760,7 +2773,7 @@ static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
     int i;
 
     for (i = 0; i < pnv->num_chips; i++) {
-        pnv_chip_foreach_cpu(pnv->chips[i], pnv_cpu_do_nmi, NULL);
+        pnv_chip_foreach_cpu(pnv->chips[i], do_pnv_cpu_do_nmi, NULL);
     }
 }
 
hw/ppc/pnv_core.c +83 -5

133 lines not on this page · raw message

system/cpus.c +10 -0
index d3640c9503..083abbc393 100644
@@ -613,6 +613,16 @@ void pause_all_vcpus(void)
     bql_lock();
 }
 
+void cpu_pause(CPUState *cpu)
+{
+    if (qemu_cpu_is_self(cpu)) {
+        qemu_cpu_stop(cpu, true);
+    } else {
+        cpu->stop = true;
+        qemu_cpu_kick(cpu);
+    }
+}
+
 void cpu_resume(CPUState *cpu)
 {
     cpu->stop = false;
Signature · 1 line
2.45.1
#25 lore raw

Re: [PATCH 17/18] ppc/pnv: Implement POWER10 PC xscom registers for direct controls

Quoted 26 lines · Nicholas Piggin

The PC unit in the processor core contains xscom registers that provide low level status and control of the CPU.

This implements "direct controls" sufficient for OPAL (skiboot) firmware use, which is to stop threads and send them non-maskable IPIs in the form of SRESET interrupts.

POWER10 is sufficiently different (particularly QME and special wakeup) from POWER9 that it is not trivial to implement by reusing the code.

---
  include/hw/core/cpu.h     |  8 ++++
  include/hw/ppc/pnv.h      |  2 +
  include/hw/ppc/pnv_core.h |  3 ++
  hw/ppc/pnv.c              | 19 +++++++--
  hw/ppc/pnv_core.c         | 88 ++++++++++++++++++++++++++++++++++++---
  system/cpus.c             | 10 +++++
  6 files changed, 122 insertions(+), 8 deletions(-)
include/hw/core/cpu.h +0 -0
index a2c8536943..410e7d6f03 100644
@@ -984,6 +984,14 @@ void cpu_reset_interrupt(CPUState *cpu, int mask);
   */

include/hw/core/cpu.h void cpu_reset_interrupt(CPUState *cpu, int mask); · line 994

Quoted 10 lines
include/hw/core/cpu.h +8 -0
  void cpu_exit(CPUState *cpu);
 
+/**
+ * cpu_pause:
+ * @cpu: The CPU to pause.
+ *
+ * Resumes CPU, i.e. puts CPU into stopped state.
+ */
+void cpu_pause(CPUState *cpu);
+

I think cpu_pause() deserves to be in a separate patch.

Thanks,

C.

Quoted 233 lines
include/hw/core/cpu.h +0 -0
  /**
   * cpu_resume:
   * @cpu: The CPU to resume.
include/hw/ppc/pnv.h +2 -0
index c56d152889..b7858d310d 100644
@@ -112,6 +112,8 @@ PnvChip *pnv_chip_add_phb(PnvChip *chip, PnvPHB *phb);
  #define PNV_FDT_ADDR          0x01000000
  #define PNV_TIMEBASE_FREQ     512000000ULL
 
+void pnv_cpu_do_nmi_resume(CPUState *cs);
+
  /*
   * BMC helpers
   */
include/hw/ppc/pnv_core.h +3 -0
index c8784777a4..1de79a818e 100644
@@ -109,6 +109,9 @@ OBJECT_DECLARE_TYPE(PnvQuad, PnvQuadClass, PNV_QUAD)
  struct PnvQuad {
      DeviceState parent_obj;
 
+    bool special_wakeup_done;
+    bool special_wakeup[4];
+
      uint32_t quad_id;
      MemoryRegion xscom_regs;
      MemoryRegion xscom_qme_regs;
hw/ppc/pnv.c +16 -3
index 575f18958d..71b2b3806c 100644
@@ -2747,11 +2747,24 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
           */
          env->spr[SPR_SRR1] |= SRR1_WAKESCOM;
      }
+    if (arg.host_int) {
+        cpu_resume(cs);
+    }
+}
+
+static void pnv_cpu_do_nmi(CPUState *cs, int resume)
+{
+    async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_HOST_INT(resume));
+}
+
+void pnv_cpu_do_nmi_resume(CPUState *cs)
+{
+    pnv_cpu_do_nmi(cs, 1);
  }
 
-static void pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
+static void do_pnv_cpu_do_nmi(PnvChip *chip, PowerPCCPU *cpu, void *opaque)
  {
-    async_run_on_cpu(CPU(cpu), pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
+    pnv_cpu_do_nmi(CPU(cpu), 0);
  }
 
  static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
@@ -2760,7 +2773,7 @@ static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
      int i;
 
      for (i = 0; i < pnv->num_chips; i++) {
-        pnv_chip_foreach_cpu(pnv->chips[i], pnv_cpu_do_nmi, NULL);
+        pnv_chip_foreach_cpu(pnv->chips[i], do_pnv_cpu_do_nmi, NULL);
      }
  }
 
hw/ppc/pnv_core.c +83 -5
index e03ac5441e..a685a5dc1b 100644
@@ -185,16 +185,40 @@ static const MemoryRegionOps pnv_core_power9_xscom_ops = {
   */
 
  #define PNV10_XSCOM_EC_CORE_THREAD_STATE    0x412
+#define PNV10_XSCOM_EC_CORE_THREAD_INFO     0x413
+#define PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS 0x449
+#define PNV10_XSCOM_EC_CORE_RAS_STATUS      0x454
 
  static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
                                             unsigned int width)
  {
+    PnvCore *pc = PNV_CORE(opaque);
+    int nr_threads = CPU_CORE(pc)->nr_threads;
+    int i;
      uint32_t offset = addr >> 3;
      uint64_t val = 0;
 
      switch (offset) {
      case PNV10_XSCOM_EC_CORE_THREAD_STATE:
-        val = 0;
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+
+            if (cs->halted) {
+                val |= PPC_BIT(56 + i);
+            }
+        }
+        break;
+    case PNV10_XSCOM_EC_CORE_THREAD_INFO:
+        break;
+    case PNV10_XSCOM_EC_CORE_RAS_STATUS:
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+            if (cs->stopped) {
+                val |= PPC_BIT(0 + 8*i) | PPC_BIT(1 + 8*i);
+            }
+        }
          break;
      default:
          qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
@@ -207,9 +231,45 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
  static void pnv_core_power10_xscom_write(void *opaque, hwaddr addr,
                                           uint64_t val, unsigned int width)
  {
+    PnvCore *pc = PNV_CORE(opaque);
+    int nr_threads = CPU_CORE(pc)->nr_threads;
+    int i;
      uint32_t offset = addr >> 3;
 
      switch (offset) {
+    case PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS:
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+
+            if (val & PPC_BIT(7 + 8*i)) { /* stop */
+                val &= ~PPC_BIT(7 + 8*i);
+                cpu_pause(cs);
+            }
+            if (val & PPC_BIT(6 + 8*i)) { /* start */
+                val &= ~PPC_BIT(6 + 8*i);
+                cpu_resume(cs);
+            }
+            if (val & PPC_BIT(4 + 8*i)) { /* sreset */
+                val &= ~PPC_BIT(4 + 8*i);
+                pnv_cpu_do_nmi_resume(cs);
+            }
+            if (val & PPC_BIT(3 + 8*i)) { /* clear maint */
+                /*
+                 * Hardware has very particular cases for where clear maint
+                 * must be used and where start must be used to resume a
+                 * thread. These are not modelled exactly, just treat
+                 * this and start the same.
+                 */
+                val &= ~PPC_BIT(3 + 8*i);
+                cpu_resume(cs);
+            }
+        }
+        if (val) {
+            qemu_log_mask(LOG_UNIMP, "%s: unimp bits in DIRECT_CONTROLS 0x%016lx\n", __func__, val);
+        }
+        break;
+
      default:
          qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
                        offset);
@@ -528,6 +588,7 @@ static const MemoryRegionOps pnv_quad_power10_xscom_ops = {
  static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
                                              unsigned int width)
  {
+    PnvQuad *eq = PNV_QUAD(opaque);
      uint32_t offset = addr >> 3;
      uint64_t val = -1;
 
@@ -535,10 +596,14 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
       * Forth nibble selects the core within a quad, mask it to process read
       * for any core.
       */
-    switch (offset & ~0xf000) {
-    case P10_QME_SPWU_HYP:
+    switch (offset & ~PPC_BITMASK32(16, 19)) {
      case P10_QME_SSH_HYP:
-        return 0;
+        val = 0;
+        if (eq->special_wakeup_done) {
+            val |= PPC_BIT(1); /* SPWU DONE */
+            val |= PPC_BIT(4); /* SSH SPWU DONE */
+        }
+        break;
      default:
          qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
                        offset);
@@ -550,9 +615,22 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
  static void pnv_qme_power10_xscom_write(void *opaque, hwaddr addr,
                                           uint64_t val, unsigned int width)
  {
+    PnvQuad *eq = PNV_QUAD(opaque);
      uint32_t offset = addr >> 3;
+    bool set;
+    int i;
 
-    switch (offset) {
+    switch (offset & ~PPC_BITMASK32(16, 19)) {
+    case P10_QME_SPWU_HYP:
+        set = !!(val & PPC_BIT(0));
+        eq->special_wakeup_done = set;
+        for (i = 0; i < 4; i++) {
+            /* These bits select cores in the quad */
+            if (offset & PPC_BIT32(16 + i)) {
+                eq->special_wakeup[i] = set;
+            }
+        }
+        break;
      default:
          qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
                        offset);
system/cpus.c +10 -0
index d3640c9503..083abbc393 100644
@@ -613,6 +613,16 @@ void pause_all_vcpus(void)
      bql_lock();
  }
 
+void cpu_pause(CPUState *cpu)
+{
+    if (qemu_cpu_is_self(cpu)) {
+        qemu_cpu_stop(cpu, true);
+    } else {
+        cpu->stop = true;
+        qemu_cpu_kick(cpu);
+    }
+}
+
  void cpu_resume(CPUState *cpu)
  {
      cpu->stop = false;
#26 lore raw

PATCH 18/18 ppc/pnv: Add an LPAR per core machine option

In reply to Nicholas Piggin

Recent POWER CPUs can operate in "LPAR per core" or "LPAR per thread" modes. In per-core mode, some SPRs and IPI doorbells are shared between threads in a core. In per-thread mode, supervisor and user state is not shared between threads.

OpenPOWER systems after POWER8 use LPAR per thread mode, and it is required for KVM. Enterprise systems use LPAR per core mode, as they partition the machine by core.

Implement a lpar-per-core machine option for powernv machines. This is fixed true for POWER8 machines, and defaults off for P9 and P10.

With this change, powernv8 SMT now works sufficiently to run Linux, including KVM.

---
 include/hw/ppc/pnv.h      |  1 +
 include/hw/ppc/pnv_core.h |  1 +
 hw/ppc/pnv.c              | 29 +++++++++++++++++++++++++++++
 hw/ppc/pnv_core.c         |  8 ++++++++
 target/ppc/cpu_init.c     |  3 ++-
 5 files changed, 41 insertions(+), 1 deletion(-)
include/hw/ppc/pnv.h +1 -0
index b7858d310d..73f0d72f55 100644
@@ -104,6 +104,7 @@ struct PnvMachineState {
     hwaddr       fw_load_addr;
 
     bool         big_core;
+    bool         lpar_per_core;
 };
 
 PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
include/hw/ppc/pnv_core.h +1 -0
index 1de79a818e..d8afb4f95f 100644
@@ -57,6 +57,7 @@ struct PnvCore {
     /*< public >*/
     PowerPCCPU **threads;
     bool big_core;
+    bool lpar_per_core;
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
hw/ppc/pnv.c +29 -0
index 71b2b3806c..f9a05fa0ff 100644
@@ -1026,6 +1026,11 @@ static void pnv_init(MachineState *machine)
         exit(1);
     }
 
+    if (!object_property_find(OBJECT(pnv), "lpar-per-core")) {
+        /* POWER8 is always in lpar-per-core mode */
+        pnv->lpar_per_core = true;
+    }
+
     pnv->num_chips =
         machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
@@ -2589,6 +2594,18 @@ static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
     pnv->big_core = value;
 }
 
+static bool pnv_machine_get_1lpar(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    return pnv->lpar_per_core;
+}
+
+static void pnv_machine_set_1lpar(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    pnv->lpar_per_core = value;
+}
+
 static bool pnv_machine_get_hb(Object *obj, Error **errp)
 {
     PnvMachineState *pnv = PNV_MACHINE(obj);
@@ -2662,6 +2679,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
                                    pnv_machine_set_big_core);
     object_class_property_set_description(oc, "big-core",
                               "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
 }
 
 static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
@@ -2709,6 +2732,12 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
                                    pnv_machine_set_big_core);
     object_class_property_set_description(oc, "big-core",
                               "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
 }
 
 static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
hw/ppc/pnv_core.c +8 -0
index a685a5dc1b..5a6fcb6edc 100644
@@ -208,6 +208,9 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
                 val |= PPC_BIT(56 + i);
             }
         }
+        if (pc->lpar_per_core) {
+            val |= PPC_BIT(62);
+        }
         break;
     case PNV10_XSCOM_EC_CORE_THREAD_INFO:
         break;
@@ -320,6 +323,10 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
         env->core_index = core_hwid;
     }
 
+    if (pc->lpar_per_core) {
+        cpu_ppc_set_1lpar(cpu);
+    }
+
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
 }
@@ -352,6 +359,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
     pc->big_core = pnv->big_core;
     pc->tod_state.big_core_quirk = pmc->quirk_tb_big_core;
+    pc->lpar_per_core = pc->chip->pnv_machine->lpar_per_core;
 
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
target/ppc/cpu_init.c +2 -1
index 9349001b76..0fc83d5cc2 100644
@@ -6785,7 +6785,8 @@ void cpu_ppc_set_1lpar(PowerPCCPU *cpu)
 
     /*
      * pseries SMT means "LPAR per core" mode, e.g., msgsndp is usable
-     * between threads.
+     * between threads. powernv be in either mode, and it mostly affects
+     * supervisor visible registers and instructions.
      */
     if (env->flags & POWERPC_FLAG_SMT) {
         env->flags |= POWERPC_FLAG_SMT_1LPAR;
Signature · 1 line
2.45.1
#27 lore raw

Re: [PATCH 18/18] ppc/pnv: Add an LPAR per core machine option

Quoted 53 lines · Nicholas Piggin

Recent POWER CPUs can operate in "LPAR per core" or "LPAR per thread" modes. In per-core mode, some SPRs and IPI doorbells are shared between threads in a core. In per-thread mode, supervisor and user state is not shared between threads.

OpenPOWER systems after POWER8 use LPAR per thread mode, and it is required for KVM. Enterprise systems use LPAR per core mode, as they partition the machine by core.

Implement a lpar-per-core machine option for powernv machines. This is fixed true for POWER8 machines, and defaults off for P9 and P10.

With this change, powernv8 SMT now works sufficiently to run Linux, including KVM.

---
  include/hw/ppc/pnv.h      |  1 +
  include/hw/ppc/pnv_core.h |  1 +
  hw/ppc/pnv.c              | 29 +++++++++++++++++++++++++++++
  hw/ppc/pnv_core.c         |  8 ++++++++
  target/ppc/cpu_init.c     |  3 ++-
  5 files changed, 41 insertions(+), 1 deletion(-)
include/hw/ppc/pnv.h +1 -0
index b7858d310d..73f0d72f55 100644
@@ -104,6 +104,7 @@ struct PnvMachineState {
      hwaddr       fw_load_addr;
 
      bool         big_core;
+    bool         lpar_per_core;
  };
 
  PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
include/hw/ppc/pnv_core.h +1 -0
index 1de79a818e..d8afb4f95f 100644
@@ -57,6 +57,7 @@ struct PnvCore {
      /*< public >*/
      PowerPCCPU **threads;
      bool big_core;
+    bool lpar_per_core;
      uint32_t pir;
      uint32_t hwid;
      uint64_t hrmor;
hw/ppc/pnv.c +0 -0
index 71b2b3806c..f9a05fa0ff 100644

hw/ppc/pnv.c static void pnv_init(MachineState *machine) · line 1032

Quoted 7 lines
hw/ppc/pnv.c +4 -0
@@ -1026,6 +1026,11 @@ static void pnv_init(MachineState *machine)
          exit(1);
      }
 
+    if (!object_property_find(OBJECT(pnv), "lpar-per-core")) {
+        /* POWER8 is always in lpar-per-core mode */
+        pnv->lpar_per_core = true;
+    }

yep.

A better alternative would be to introduce a PnvChipClass::lpar_per_core attribute (set to true for POWER8) and OR PnvChipClass::lpar_per_core with pnv->lpar_per_core to set the "lpar-per-core" property of PnvCore in pnv_chip_core_realize().

Thanks,

C.

Quoted 95 lines
hw/ppc/pnv.c +24 -0
      pnv->num_chips =
          machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
@@ -2589,6 +2594,18 @@ static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
      pnv->big_core = value;
  }
 
+static bool pnv_machine_get_1lpar(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    return pnv->lpar_per_core;
+}
+
+static void pnv_machine_set_1lpar(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    pnv->lpar_per_core = value;
+}
+
  static bool pnv_machine_get_hb(Object *obj, Error **errp)
  {
      PnvMachineState *pnv = PNV_MACHINE(obj);
@@ -2662,6 +2679,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
                                     pnv_machine_set_big_core);
      object_class_property_set_description(oc, "big-core",
                                "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
  }
 
  static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
@@ -2709,6 +2732,12 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
                                     pnv_machine_set_big_core);
      object_class_property_set_description(oc, "big-core",
                                "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
  }
 
  static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
hw/ppc/pnv_core.c +8 -0
index a685a5dc1b..5a6fcb6edc 100644
@@ -208,6 +208,9 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
                  val |= PPC_BIT(56 + i);
              }
          }
+        if (pc->lpar_per_core) {
+            val |= PPC_BIT(62);
+        }
          break;
      case PNV10_XSCOM_EC_CORE_THREAD_INFO:
          break;
@@ -320,6 +323,10 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
          env->core_index = core_hwid;
      }
 
+    if (pc->lpar_per_core) {
+        cpu_ppc_set_1lpar(cpu);
+    }
+
      /* Set time-base frequency to 512 MHz */
      cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
  }
@@ -352,6 +359,7 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
      pc->big_core = pnv->big_core;
      pc->tod_state.big_core_quirk = pmc->quirk_tb_big_core;
+    pc->lpar_per_core = pc->chip->pnv_machine->lpar_per_core;
 
      pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
      for (i = 0; i < cc->nr_threads; i++) {
target/ppc/cpu_init.c +2 -1
index 9349001b76..0fc83d5cc2 100644
@@ -6785,7 +6785,8 @@ void cpu_ppc_set_1lpar(PowerPCCPU *cpu)
 
      /*
       * pseries SMT means "LPAR per core" mode, e.g., msgsndp is usable
-     * between threads.
+     * between threads. powernv be in either mode, and it mostly affects
+     * supervisor visible registers and instructions.
       */
      if (env->flags & POWERPC_FLAG_SMT) {
          env->flags |= POWERPC_FLAG_SMT_1LPAR;
Tested-by on the v1 series
#28 lore raw
In reply to Nicholas Piggin

Hello Nick,

Quoted 4 lines · Nicholas Piggin

Primary motivation for this series is to improve big-core support.

This also fixes POWER8 SMT running Linux with the msgsnd fix and setting lpar-per-core mode (which is always true on POWER8).

It does. I gave a try to the powernv8 machine with :

-smp 8,sockets=1,cores=1,threads=8
-smp 8,sockets=1,cores=2,threads=4
-smp 8,sockets=1,cores=4,threads=2
-smp 8,sockets=1,cores=8,threads=1

and

-smp 16,sockets=1,cores=2,threads=8
-smp 16,sockets=1,cores=4,threads=4
-smp 16,sockets=2,cores=8,threads=1

So,

However there is an issue with smt on multisocket. CPU hang. I guess POWER8 is not a priority.

It would be good to improve slightly the models with properties. slightly, because I don't think adding the "big-core" and "lpar-per-core" properties to the chip and the core are much work. I am (nearly) sure it will remove a few lines of code ! Anyhow, this is not a blocker and I hope we can merge these improvements for 9.1 [*]

One thing we could do also is deprecate the POWER8NVL and POWER8E chips in the 9.1 cycle.

Thanks,

C.

[*] the SPI model and ADU, LPC PSI serirq also. I haven't seen any
updates for PHB and XIVE,

Quoted 70 lines

I think I accounted for all feedback from Cedric and Harsh from the last RFC (except a couple of style suggestions from Harsh).

Since rfc:
- Fixed POWER8 SMT so it doesn't have to be disabled.
- Fixed inadvertent spapr SMT bug.
- Renamed PnvCPUState.core pointer to pnv_core. (Harsh)
- Moved where it is initialised (clg)
- Avoided most qdev_get_machine() calls by adding a PnvMachineState pointer from PnvChip, new patch 3 (clg).
- Rename TB state to use camel case (Harsh and clg)
- Add comment to explain SPRC/SPRD is only accessed with powernv.
- Use mc->desc for error messages and avoid splitting machine init handlers (Harsh).
- Add max_smt_threads class attribute to avoid duplicating checks (clg)
- Rename processor_id() class method to get_pir_tir (Harsh and clg)
- Add a comment for get_pir_tir() (clg)
- Allow get_pir_tir() to be passed NULL pointers to avoid dummy pir/tir variables (Harsh)
- Move the PPC_CPU_HAS_CORE_SIBLINGS macros to inline functions (clg)
- Invert them (test for single-thread rather than for siblings) because the callers read a little better that way (Harsh).
- Propagate lpar and big-core options down to chip and core levels rather than having to test machine (clg)
- Significantly split the big-core patch (clg).
- Rework big-core device-tree handling to simplify it (clg).
- Make new has_smt_siblings property bool (Harsh)
- Make the big-core timebase tod quirk a machine class property rather than machine state (Harsh).

Thanks,
Nick

Nicholas Piggin (18):
target/ppc: Fix msgsnd for POWER8
ppc/pnv: Add pointer from PnvCPUState to PnvCore ppc/pnv: Add a pointer from PnvChip to PnvMachineState ppc/pnv: Move timebase state into PnvCore
target/ppc: Move SPR indirect registers into PnvCore ppc/pnv: specialise init for powernv8/9/10 machines ppc/pnv: Extend chip_pir class method to TIR as well
ppc: Add a core_index to CPUPPCState for SMT vCPUs target/ppc: Add helpers to check for SMT sibling threads
ppc: Add has_smt_siblings property to CPUPPCState ppc/pnv: Add a big-core mode that joins two regular cores ppc/pnv: Add allow for big-core differences in DT generation ppc/pnv: Implement big-core PVR for Power9/10
ppc/pnv: Implement Power9 CPU core thread state indirect register ppc/pnv: Add POWER10 ChipTOD quirk for big-core ppc/pnv: Add big-core machine property
ppc/pnv: Implement POWER10 PC xscom registers for direct controls ppc/pnv: Add an LPAR per core machine option

  include/hw/core/cpu.h        |   8 +
  include/hw/ppc/pnv.h         |   7 +
  include/hw/ppc/pnv_chip.h    |   6 +-
  include/hw/ppc/pnv_core.h    |  31 ++++
  target/ppc/cpu.h             |  41 ++---
  hw/ppc/pnv.c                 | 298 ++++++++++++++++++++++++++++-------
  hw/ppc/pnv_chiptod.c         |   7 +-
  hw/ppc/pnv_core.c            | 130 +++++++++++++--
  hw/ppc/spapr_cpu_core.c      |  16 +-
  system/cpus.c                |  10 ++
  target/ppc/cpu_init.c        |  26 +--
  target/ppc/excp_helper.c     |  69 ++++----
  target/ppc/misc_helper.c     | 104 ++++++------
  target/ppc/timebase_helper.c |  82 +++++-----
  14 files changed, 601 insertions(+), 234 deletions(-)