Skip to contentSkip to conversation

target/ppc: TCG SMT support for spapr machine

PATCH Applied Nicholas Piggin 2 participants 17 messages Jun 22, 2023 – Jun 23, 2023 PowerPC

lore
Share & tools
lore

Nicholas Piggin

author reviewer
#0 lore raw

v2 cover target/ppc: TCG SMT support for spapr machine

This series is based on some previously posted TCG fixes, in particular the CTRL register fix is required.

Also added the Philippe's patch in the series to prevent conflict.

Since v1, main changes are just some tidying of comments and changelogs, and addition of avocado tests to boot Linux on SMT machine and make sure the CPUs come up, as suggested by Cedric.

Thanks,
Nick

Nicholas Piggin (6):
target/ppc: Add initial flags and helpers for SMT support target/ppc: Add support for SMT CTRL register target/ppc: Add msgsnd/p and DPDES SMT support
spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs tests/avocado: boot ppc64 pseries to Linux VFS mount tests/avocado: Add ppc64 pseries multiprocessor boot tests

Philippe Mathieu-Daudé (1):
hw/ppc/spapr: Test whether TCG is enabled with tcg_enabled()

 hw/ppc/ppc.c                 |  6 ++++
 hw/ppc/spapr.c               | 16 ++++++---
 hw/ppc/spapr_caps.c          | 14 ++++++++
 hw/ppc/spapr_cpu_core.c      |  7 ++--
 include/hw/ppc/ppc.h         |  1 +
 target/ppc/cpu.h             |  9 +++++
 target/ppc/cpu_init.c        |  5 +++
 target/ppc/excp_helper.c     | 30 +++++++++++++---
 target/ppc/helper.h          |  2 ++
 target/ppc/misc_helper.c     | 69 ++++++++++++++++++++++++++++++++----
 target/ppc/translate.c       | 46 +++++++++++++++++++++++-
 tests/avocado/ppc_pseries.py | 62 ++++++++++++++++++++++++++++----
 12 files changed, 243 insertions(+), 24 deletions(-)
Signature · 1 line
2.40.1
author reviewer
#1 lore raw

PATCH v2 1/7 target/ppc: Add initial flags and helpers for SMT support

TGC SMT emulation needs to know whether it is running with SMT siblings, to be able to iterate over siblings in a core, and to serialise threads to access per-core shared SPRs. Add infrastructure to do these things.

For now the sibling iteration and serialisation are implemented in a simple but inefficient way. SMT shared state and sibling access is not too common, and SMT configurations are mainly useful to test system code, so performance is not to critical.

---
 target/ppc/cpu.h       |  9 +++++++++
 target/ppc/cpu_init.c  |  5 +++++
 target/ppc/translate.c | 20 ++++++++++++++++++++
 3 files changed, 34 insertions(+)
target/ppc/cpu.h +9 -0
index bfa1777289..0087ce66e2 100644
@@ -672,6 +672,8 @@ enum {
     POWERPC_FLAG_TM       = 0x00100000,
     /* Has SCV (ISA 3.00)                                                    */
     POWERPC_FLAG_SCV      = 0x00200000,
+    /* Has >1 thread per core                                                */
+    POWERPC_FLAG_SMT      = 0x00400000,
 };
 
 /*
@@ -1270,6 +1272,13 @@ struct CPUArchState {
     uint64_t pmu_base_time;
 };
 
+#define _CORE_ID(cs)                                            \
+    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
+
+#define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
+    CPU_FOREACH(cs_sibling)                                     \
+        if (_CORE_ID(cs) == _CORE_ID(cs_sibling))
+
 #define SET_FIT_PERIOD(a_, b_, c_, d_)          \
 do {                                            \
     env->fit_period[0] = (a_);                  \
target/ppc/cpu_init.c +5 -0
index dccc064053..aeff71d063 100644
@@ -6755,6 +6755,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
 {
     CPUState *cs = CPU(dev);
     PowerPCCPU *cpu = POWERPC_CPU(dev);
+    CPUPPCState *env = &cpu->env;
     PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
     Error *local_err = NULL;
 
@@ -6786,6 +6787,10 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
 
     pcc->parent_realize(dev, errp);
 
+    if (env_cpu(env)->nr_threads > 1) {
+        env->flags |= POWERPC_FLAG_SMT;
+    }
+
     return;
 
 unrealize:
target/ppc/translate.c +20 -0
index b62b624682..5d585393c5 100644
@@ -236,6 +236,26 @@ struct opc_handler_t {
     void (*handler)(DisasContext *ctx);
 };
 
+static inline bool gen_serialize(DisasContext *ctx)
+{
+    if (tb_cflags(ctx->base.tb) & CF_PARALLEL) {
+        /* Restart with exclusive lock.  */
+        gen_helper_exit_atomic(cpu_env);
+        ctx->base.is_jmp = DISAS_NORETURN;
+        return false;
+    }
+    return true;
+}
+
+static inline bool gen_serialize_core(DisasContext *ctx)
+{
+    if (ctx->flags & POWERPC_FLAG_SMT) {
+        return gen_serialize(ctx);
+    }
+
+    return true;
+}
+
 /* SPR load/store helpers */
 static inline void gen_load_spr(TCGv t, int reg)
 {
Signature · 1 line
2.40.1
Reviewed-by on v2 1/7
#2 lore raw

Re: [PATCH v2 1/7] target/ppc: Add initial flags and helpers for SMT support

Quoted 11 lines · Nicholas Piggin

TGC SMT emulation needs to know whether it is running with SMT siblings, to be able to iterate over siblings in a core, and to serialise threads to access per-core shared SPRs. Add infrastructure to do these things.

For now the sibling iteration and serialisation are implemented in a simple but inefficient way. SMT shared state and sibling access is not too common, and SMT configurations are mainly useful to test system code, so performance is not to critical.

Thanks,

C.

Quoted 87 lines
---
  target/ppc/cpu.h       |  9 +++++++++
  target/ppc/cpu_init.c  |  5 +++++
  target/ppc/translate.c | 20 ++++++++++++++++++++
  3 files changed, 34 insertions(+)
target/ppc/cpu.h +9 -0
index bfa1777289..0087ce66e2 100644
@@ -672,6 +672,8 @@ enum {
      POWERPC_FLAG_TM       = 0x00100000,
      /* Has SCV (ISA 3.00)                                                    */
      POWERPC_FLAG_SCV      = 0x00200000,
+    /* Has >1 thread per core                                                */
+    POWERPC_FLAG_SMT      = 0x00400000,
  };
 
  /*
@@ -1270,6 +1272,13 @@ struct CPUArchState {
      uint64_t pmu_base_time;
  };
 
+#define _CORE_ID(cs)                                            \
+    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
+
+#define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
+    CPU_FOREACH(cs_sibling)                                     \
+        if (_CORE_ID(cs) == _CORE_ID(cs_sibling))
+
  #define SET_FIT_PERIOD(a_, b_, c_, d_)          \
  do {                                            \
      env->fit_period[0] = (a_);                  \
target/ppc/cpu_init.c +5 -0
index dccc064053..aeff71d063 100644
@@ -6755,6 +6755,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
  {
      CPUState *cs = CPU(dev);
      PowerPCCPU *cpu = POWERPC_CPU(dev);
+    CPUPPCState *env = &cpu->env;
      PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cpu);
      Error *local_err = NULL;
 
@@ -6786,6 +6787,10 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
 
      pcc->parent_realize(dev, errp);
 
+    if (env_cpu(env)->nr_threads > 1) {
+        env->flags |= POWERPC_FLAG_SMT;
+    }
+
      return;
 
  unrealize:
target/ppc/translate.c +20 -0
index b62b624682..5d585393c5 100644
@@ -236,6 +236,26 @@ struct opc_handler_t {
      void (*handler)(DisasContext *ctx);
  };
 
+static inline bool gen_serialize(DisasContext *ctx)
+{
+    if (tb_cflags(ctx->base.tb) & CF_PARALLEL) {
+        /* Restart with exclusive lock.  */
+        gen_helper_exit_atomic(cpu_env);
+        ctx->base.is_jmp = DISAS_NORETURN;
+        return false;
+    }
+    return true;
+}
+
+static inline bool gen_serialize_core(DisasContext *ctx)
+{
+    if (ctx->flags & POWERPC_FLAG_SMT) {
+        return gen_serialize(ctx);
+    }
+
+    return true;
+}
+
  /* SPR load/store helpers */
  static inline void gen_load_spr(TCGv t, int reg)
  {
author reviewer
#3 lore raw

PATCH v2 2/7 target/ppc: Add support for SMT CTRL register

In reply to Nicholas Piggin

A relatively simple case to begin with, CTRL is a SMT shared register where reads and writes need to synchronise against state changes by other threads in the core.

Atomic serialisation operations are used to achieve this.

---
 target/ppc/helper.h      |  2 ++
 target/ppc/misc_helper.c | 25 +++++++++++++++++++++++++
 target/ppc/translate.c   | 18 +++++++++++++++++-
 3 files changed, 44 insertions(+), 1 deletion(-)
target/ppc/helper.h +2 -0
index 38efbc351c..fda40b8a60 100644
@@ -704,6 +704,8 @@ DEF_HELPER_3(store_dcr, void, env, tl, tl)
 
 DEF_HELPER_2(load_dump_spr, void, env, i32)
 DEF_HELPER_2(store_dump_spr, void, env, i32)
+DEF_HELPER_3(spr_write_CTRL, void, env, i32, tl)
+
 DEF_HELPER_4(fscr_facility_check, void, env, i32, i32, i32)
 DEF_HELPER_4(msr_facility_check, void, env, i32, i32, i32)
 DEF_HELPER_FLAGS_1(load_tbl, TCG_CALL_NO_RWG, tl, env)
target/ppc/misc_helper.c +25 -0
index 40ddc5c08c..a058eb24cd 100644
@@ -43,6 +43,31 @@ void helper_store_dump_spr(CPUPPCState *env, uint32_t sprn)
              env->spr[sprn]);
 }
 
+void helper_spr_write_CTRL(CPUPPCState *env, uint32_t sprn,
+                           target_ulong val)
+{
+    CPUState *cs = env_cpu(env);
+    CPUState *ccs;
+    uint32_t run = val & 1;
+    uint32_t ts, ts_mask;
+
+    assert(sprn == SPR_CTRL);
+
+    env->spr[sprn] &= ~1U;
+    env->spr[sprn] |= run;
+
+    ts_mask = ~(1U << (8 + env->spr[SPR_TIR]));
+    ts = run << (8 + env->spr[SPR_TIR]);
+
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        CPUPPCState *cenv = &POWERPC_CPU(ccs)->env;
+
+        cenv->spr[sprn] &= ts_mask;
+        cenv->spr[sprn] |= ts;
+    }
+}
+
+
 #ifdef TARGET_PPC64
 static void raise_hv_fu_exception(CPUPPCState *env, uint32_t bit,
                                   const char *caller, uint32_t cause,
target/ppc/translate.c +17 -1
index 5d585393c5..41a8b800bd 100644
@@ -438,7 +438,7 @@ void spr_write_generic32(DisasContext *ctx, int sprn, int gprn)
 #endif
 }
 
-void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
+static void spr_write_CTRL_ST(DisasContext *ctx, int sprn, int gprn)
 {
     /* This does not implement >1 thread */
     TCGv t0 = tcg_temp_new();
@@ -447,6 +447,22 @@ void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
     tcg_gen_shli_tl(t1, t0, 8); /* Duplicate the bit in TS */
     tcg_gen_or_tl(t1, t1, t0);
     gen_store_spr(sprn, t1);
+}
+
+void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
+{
+    if (!(ctx->flags & POWERPC_FLAG_SMT)) {
+        spr_write_CTRL_ST(ctx, sprn, gprn);
+        goto out;
+    }
+
+    if (!gen_serialize(ctx)) {
+        return;
+    }
+
+    gen_helper_spr_write_CTRL(cpu_env, tcg_constant_i32(sprn),
+                              cpu_gpr[gprn]);
+out:
     spr_store_dump_spr(sprn);
 
     /*
Signature · 1 line
2.40.1
Reviewed-by on v2 2/7
#4 lore raw

Re: [PATCH v2 2/7] target/ppc: Add support for SMT CTRL register

Quoted 7 lines · Nicholas Piggin

A relatively simple case to begin with, CTRL is a SMT shared register where reads and writes need to synchronise against state changes by other threads in the core.

Atomic serialisation operations are used to achieve this.

Thanks,

C.

Quoted 91 lines
---
  target/ppc/helper.h      |  2 ++
  target/ppc/misc_helper.c | 25 +++++++++++++++++++++++++
  target/ppc/translate.c   | 18 +++++++++++++++++-
  3 files changed, 44 insertions(+), 1 deletion(-)
target/ppc/helper.h +2 -0
index 38efbc351c..fda40b8a60 100644
@@ -704,6 +704,8 @@ DEF_HELPER_3(store_dcr, void, env, tl, tl)
 
  DEF_HELPER_2(load_dump_spr, void, env, i32)
  DEF_HELPER_2(store_dump_spr, void, env, i32)
+DEF_HELPER_3(spr_write_CTRL, void, env, i32, tl)
+
  DEF_HELPER_4(fscr_facility_check, void, env, i32, i32, i32)
  DEF_HELPER_4(msr_facility_check, void, env, i32, i32, i32)
  DEF_HELPER_FLAGS_1(load_tbl, TCG_CALL_NO_RWG, tl, env)
target/ppc/misc_helper.c +25 -0
index 40ddc5c08c..a058eb24cd 100644
@@ -43,6 +43,31 @@ void helper_store_dump_spr(CPUPPCState *env, uint32_t sprn)
               env->spr[sprn]);
  }
 
+void helper_spr_write_CTRL(CPUPPCState *env, uint32_t sprn,
+                           target_ulong val)
+{
+    CPUState *cs = env_cpu(env);
+    CPUState *ccs;
+    uint32_t run = val & 1;
+    uint32_t ts, ts_mask;
+
+    assert(sprn == SPR_CTRL);
+
+    env->spr[sprn] &= ~1U;
+    env->spr[sprn] |= run;
+
+    ts_mask = ~(1U << (8 + env->spr[SPR_TIR]));
+    ts = run << (8 + env->spr[SPR_TIR]);
+
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        CPUPPCState *cenv = &POWERPC_CPU(ccs)->env;
+
+        cenv->spr[sprn] &= ts_mask;
+        cenv->spr[sprn] |= ts;
+    }
+}
+
+
  #ifdef TARGET_PPC64
  static void raise_hv_fu_exception(CPUPPCState *env, uint32_t bit,
                                    const char *caller, uint32_t cause,
target/ppc/translate.c +17 -1
index 5d585393c5..41a8b800bd 100644
@@ -438,7 +438,7 @@ void spr_write_generic32(DisasContext *ctx, int sprn, int gprn)
  #endif
  }
 
-void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
+static void spr_write_CTRL_ST(DisasContext *ctx, int sprn, int gprn)
  {
      /* This does not implement >1 thread */
      TCGv t0 = tcg_temp_new();
@@ -447,6 +447,22 @@ void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
      tcg_gen_shli_tl(t1, t0, 8); /* Duplicate the bit in TS */
      tcg_gen_or_tl(t1, t1, t0);
      gen_store_spr(sprn, t1);
+}
+
+void spr_write_CTRL(DisasContext *ctx, int sprn, int gprn)
+{
+    if (!(ctx->flags & POWERPC_FLAG_SMT)) {
+        spr_write_CTRL_ST(ctx, sprn, gprn);
+        goto out;
+    }
+
+    if (!gen_serialize(ctx)) {
+        return;
+    }
+
+    gen_helper_spr_write_CTRL(cpu_env, tcg_constant_i32(sprn),
+                              cpu_gpr[gprn]);
+out:
      spr_store_dump_spr(sprn);
 
      /*
author reviewer
#5 lore raw

PATCH v2 3/7 target/ppc: Add msgsnd/p and DPDES SMT support

In reply to Nicholas Piggin

Doorbells in SMT need to coordinate msgsnd/msgclr and DPDES access from multiple threads that affect the same state.

---
 hw/ppc/ppc.c             |  6 ++++++
 include/hw/ppc/ppc.h     |  1 +
 target/ppc/excp_helper.c | 30 ++++++++++++++++++++++-----
 target/ppc/misc_helper.c | 44 ++++++++++++++++++++++++++++++++++------
 target/ppc/translate.c   |  8 ++++++++
 5 files changed, 78 insertions(+), 11 deletions(-)
hw/ppc/ppc.c +6 -0
index 1b1220c423..82e4408c5c 100644
@@ -1436,6 +1436,12 @@ int ppc_cpu_pir(PowerPCCPU *cpu)
     return env->spr_cb[SPR_PIR].default_value;
 }
 
+int ppc_cpu_tir(PowerPCCPU *cpu)
+{
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_TIR].default_value;
+}
+
 PowerPCCPU *ppc_get_vcpu_by_pir(int pir)
 {
     CPUState *cs;
include/hw/ppc/ppc.h +1 -0
index 02af03ada2..e095c002dc 100644
@@ -6,6 +6,7 @@
 void ppc_set_irq(PowerPCCPU *cpu, int n_IRQ, int level);
 PowerPCCPU *ppc_get_vcpu_by_pir(int pir);
 int ppc_cpu_pir(PowerPCCPU *cpu);
+int ppc_cpu_tir(PowerPCCPU *cpu);
 
 /* PowerPC hardware exceptions management helpers */
 typedef void (*clk_setup_cb)(void *opaque, uint32_t freq);
target/ppc/excp_helper.c +25 -5
index 7d45035447..d40eecb4c7 100644
@@ -3187,22 +3187,42 @@ void helper_book3s_msgclrp(CPUPPCState *env, target_ulong rb)
 }
 
 /*
- * sends a message to other threads that are on the same
+ * sends a message to another thread  on the same
  * multi-threaded processor
  */
 void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
 {
-    int pir = env->spr_cb[SPR_PIR].default_value;
+    CPUState *cs = env_cpu(env);
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+    CPUState *ccs;
+    uint32_t nr_threads = cs->nr_threads;
+    int ttir = rb & PPC_BITMASK(57, 63);
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
 
-    if (!dbell_type_server(rb)) {
+    if (!dbell_type_server(rb) || ttir >= nr_threads) {
+        return;
+    }
+
+    if (nr_threads == 1) {
+        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
         return;
     }
 
-    /* TODO: TCG supports only one thread */
+    /* Does iothread need to be locked for walking CPU list? */
+    qemu_mutex_lock_iothread();
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+        uint32_t thread_id = ppc_cpu_tir(ccpu);
+
+        if (ttir == thread_id) {
+            ppc_set_irq(ccpu, PPC_INTERRUPT_DOORBELL, 1);
+            qemu_mutex_unlock_iothread();
+            return;
+        }
+    }
 
-    book3s_msgsnd_common(pir, PPC_INTERRUPT_DOORBELL);
+    g_assert_not_reached();
 }
 #endif /* TARGET_PPC64 */
 
target/ppc/misc_helper.c +38 -6

69 lines not on this page · raw message

target/ppc/translate.c +8 -0
index 41a8b800bd..eb278c2683 100644
@@ -815,11 +815,19 @@ void spr_write_pcr(DisasContext *ctx, int sprn, int gprn)
 /* DPDES */
 void spr_read_dpdes(DisasContext *ctx, int gprn, int sprn)
 {
+    if (!gen_serialize_core(ctx)) {
+        return;
+    }
+
     gen_helper_load_dpdes(cpu_gpr[gprn], cpu_env);
 }
 
 void spr_write_dpdes(DisasContext *ctx, int sprn, int gprn)
 {
+    if (!gen_serialize_core(ctx)) {
+        return;
+    }
+
     gen_helper_store_dpdes(cpu_env, cpu_gpr[gprn]);
 }
 #endif
Signature · 1 line
2.40.1
Reviewed-by on v2 3/7
#6 lore raw

Re: [PATCH v2 3/7] target/ppc: Add msgsnd/p and DPDES SMT support

Quoted 4 lines · Nicholas Piggin

Doorbells in SMT need to coordinate msgsnd/msgclr and DPDES access from multiple threads that affect the same state.

Thanks,

C.

Quoted 188 lines
---
  hw/ppc/ppc.c             |  6 ++++++
  include/hw/ppc/ppc.h     |  1 +
  target/ppc/excp_helper.c | 30 ++++++++++++++++++++++-----
  target/ppc/misc_helper.c | 44 ++++++++++++++++++++++++++++++++++------
  target/ppc/translate.c   |  8 ++++++++
  5 files changed, 78 insertions(+), 11 deletions(-)
hw/ppc/ppc.c +6 -0
index 1b1220c423..82e4408c5c 100644
@@ -1436,6 +1436,12 @@ int ppc_cpu_pir(PowerPCCPU *cpu)
      return env->spr_cb[SPR_PIR].default_value;
  }
 
+int ppc_cpu_tir(PowerPCCPU *cpu)
+{
+    CPUPPCState *env = &cpu->env;
+    return env->spr_cb[SPR_TIR].default_value;
+}
+
  PowerPCCPU *ppc_get_vcpu_by_pir(int pir)
  {
      CPUState *cs;
include/hw/ppc/ppc.h +1 -0
index 02af03ada2..e095c002dc 100644
@@ -6,6 +6,7 @@
  void ppc_set_irq(PowerPCCPU *cpu, int n_IRQ, int level);
  PowerPCCPU *ppc_get_vcpu_by_pir(int pir);
  int ppc_cpu_pir(PowerPCCPU *cpu);
+int ppc_cpu_tir(PowerPCCPU *cpu);
 
  /* PowerPC hardware exceptions management helpers */
  typedef void (*clk_setup_cb)(void *opaque, uint32_t freq);
target/ppc/excp_helper.c +25 -5
index 7d45035447..d40eecb4c7 100644
@@ -3187,22 +3187,42 @@ void helper_book3s_msgclrp(CPUPPCState *env, target_ulong rb)
  }
 
  /*
- * sends a message to other threads that are on the same
+ * sends a message to another thread  on the same
   * multi-threaded processor
   */
  void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
  {
-    int pir = env->spr_cb[SPR_PIR].default_value;
+    CPUState *cs = env_cpu(env);
+    PowerPCCPU *cpu = POWERPC_CPU(cs);
+    CPUState *ccs;
+    uint32_t nr_threads = cs->nr_threads;
+    int ttir = rb & PPC_BITMASK(57, 63);
 
      helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
 
-    if (!dbell_type_server(rb)) {
+    if (!dbell_type_server(rb) || ttir >= nr_threads) {
+        return;
+    }
+
+    if (nr_threads == 1) {
+        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
          return;
      }
 
-    /* TODO: TCG supports only one thread */
+    /* Does iothread need to be locked for walking CPU list? */
+    qemu_mutex_lock_iothread();
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+        uint32_t thread_id = ppc_cpu_tir(ccpu);
+
+        if (ttir == thread_id) {
+            ppc_set_irq(ccpu, PPC_INTERRUPT_DOORBELL, 1);
+            qemu_mutex_unlock_iothread();
+            return;
+        }
+    }
 
-    book3s_msgsnd_common(pir, PPC_INTERRUPT_DOORBELL);
+    g_assert_not_reached();
  }
  #endif /* TARGET_PPC64 */
 
target/ppc/misc_helper.c +38 -6
index a058eb24cd..1f1af21f33 100644
@@ -184,14 +184,31 @@ void helper_store_pcr(CPUPPCState *env, target_ulong value)
   */
  target_ulong helper_load_dpdes(CPUPPCState *env)
  {
+    CPUState *cs = env_cpu(env);
+    CPUState *ccs;
+    uint32_t nr_threads = cs->nr_threads;
      target_ulong dpdes = 0;
 
      helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
 
-    /* TODO: TCG supports only one thread */
-    if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
-        dpdes = 1;
+    if (nr_threads == 1) {
+        if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
+            dpdes = 1;
+        }
+        return dpdes;
+    }
+
+    qemu_mutex_lock_iothread();
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+        CPUPPCState *cenv = &ccpu->env;
+        uint32_t thread_id = ppc_cpu_tir(ccpu);
+
+        if (cenv->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
+            dpdes |= (0x1 << thread_id);
+        }
      }
+    qemu_mutex_unlock_iothread();
 
      return dpdes;
  }
@@ -199,17 +216,32 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
  void helper_store_dpdes(CPUPPCState *env, target_ulong val)
  {
      PowerPCCPU *cpu = env_archcpu(env);
+    CPUState *cs = env_cpu(env);
+    CPUState *ccs;
+    uint32_t nr_threads = cs->nr_threads;
 
      helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
 
-    /* TODO: TCG supports only one thread */
-    if (val & ~0x1) {
+    if (val & ~(nr_threads - 1)) {
          qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
                        TARGET_FMT_lx"\n", val);
+        val &= (nr_threads - 1); /* Ignore the invalid bits */
+    }
+
+    if (nr_threads == 1) {
+        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
          return;
      }
 
-    ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
+    /* Does iothread need to be locked for walking CPU list? */
+    qemu_mutex_lock_iothread();
+    THREAD_SIBLING_FOREACH(cs, ccs) {
+        PowerPCCPU *ccpu = POWERPC_CPU(ccs);
+        uint32_t thread_id = ppc_cpu_tir(ccpu);
+
+        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & (0x1 << thread_id));
+    }
+    qemu_mutex_unlock_iothread();
  }
  #endif /* defined(TARGET_PPC64) */
 
target/ppc/translate.c +8 -0
index 41a8b800bd..eb278c2683 100644
@@ -815,11 +815,19 @@ void spr_write_pcr(DisasContext *ctx, int sprn, int gprn)
  /* DPDES */
  void spr_read_dpdes(DisasContext *ctx, int gprn, int sprn)
  {
+    if (!gen_serialize_core(ctx)) {
+        return;
+    }
+
      gen_helper_load_dpdes(cpu_gpr[gprn], cpu_env);
  }
 
  void spr_write_dpdes(DisasContext *ctx, int sprn, int gprn)
  {
+    if (!gen_serialize_core(ctx)) {
+        return;
+    }
+
      gen_helper_store_dpdes(cpu_env, cpu_gpr[gprn]);
  }
  #endif
author reviewer
#7 lore raw

PATCH v2 4/7 hw/ppc/spapr: Test whether TCG is enabled with tcg_enabled()

In reply to Nicholas Piggin

From: Philippe Mathieu-Daudé <[email protected]>

Although the PPC target only supports the TCG and KVM accelerators, QEMU supports more. We can not assume that '!kvm == tcg', so test for the correct accelerator. This also eases code review, because here we don't care about KVM, we really want to test for TCG.

---
 hw/ppc/spapr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
hw/ppc/spapr.c +1 -1
index e55905a1f0..8e7d497f25 100644
@@ -2525,7 +2525,7 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
     int ret;
     unsigned int smp_threads = ms->smp.threads;
 
-    if (!kvm_enabled() && (smp_threads > 1)) {
+    if (tcg_enabled() && (smp_threads > 1)) {
         error_setg(errp, "TCG cannot support more than 1 thread/core "
                    "on a pseries machine");
         return;
Signature · 1 line
2.40.1
author reviewer
#8 lore raw

PATCH v2 5/7 spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs

In reply to Nicholas Piggin

PPC TCG supports SMT CPU configurations for non-hypervisor state, so permit POWER8-10 pseries machines to enable SMT.

This requires PIR and TIR be set, because that's how sibling thread matching is done by TCG.

spapr's nested-HV capability does not currently coexist with SMT, so that combination is prohibited (interestingly somewhat analogous to LPAR-per-core mode on real hardware which also does not support KVM).

---
 hw/ppc/spapr.c          | 16 ++++++++++++----
 hw/ppc/spapr_caps.c     | 14 ++++++++++++++
 hw/ppc/spapr_cpu_core.c |  7 +++++--
 3 files changed, 31 insertions(+), 6 deletions(-)
hw/ppc/spapr.c +12 -4
index 8e7d497f25..677b5eef9d 100644
@@ -2525,10 +2525,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
     int ret;
     unsigned int smp_threads = ms->smp.threads;
 
-    if (tcg_enabled() && (smp_threads > 1)) {
-        error_setg(errp, "TCG cannot support more than 1 thread/core "
-                   "on a pseries machine");
-        return;
+    if (tcg_enabled()) {
+        if (!ppc_type_check_compat(ms->cpu_type, CPU_POWERPC_LOGICAL_2_07, 0,
+                                   spapr->max_compat_pvr)) {
+            error_setg(errp, "TCG only supports SMT on POWER8 or newer CPUs");
+            return;
+        }
+
+        if (smp_threads > 8) {
+            error_setg(errp, "TCG cannot support more than 8 threads/core "
+                       "on a pseries machine");
+            return;
+        }
     }
     if (!is_power_of_2(smp_threads)) {
         error_setg(errp, "Cannot support %d threads/core on a pseries "
hw/ppc/spapr_caps.c +14 -0
index 3fd45a6dec..5a0755d34f 100644
@@ -473,6 +473,20 @@ static void cap_nested_kvm_hv_apply(SpaprMachineState *spapr,
                 error_append_hint(errp,
                                   "Try appending -machine cap-nested-hv=off\n");
         }
+    } else if (tcg_enabled()) {
+        MachineState *ms = MACHINE(spapr);
+        unsigned int smp_threads = ms->smp.threads;
+
+        /*
+         * Nested-HV vCPU env state to L2, so SMT-shared SPR updates, for
+         * example, do not necessarily update the correct SPR value on sibling
+         * threads that are in a different guest/host context.
+         */
+        if (smp_threads > 1) {
+            error_setg(errp, "TCG does not support nested-HV with SMT");
+            error_append_hint(errp, "Try appending -machine cap-nested-hv=off "
+                                    "or use threads=1 with -smp\n");
+        }
     }
 }
 
hw/ppc/spapr_cpu_core.c +5 -2
index 9b88dd549a..a4e3c2fadd 100644
@@ -255,7 +255,7 @@ static void spapr_cpu_core_unrealize(DeviceState *dev)
 }
 
 static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
-                               SpaprCpuCore *sc, Error **errp)
+                               SpaprCpuCore *sc, int thread_index, Error **errp)
 {
     CPUPPCState *env = &cpu->env;
     CPUState *cs = CPU(cpu);
@@ -267,6 +267,9 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
     cpu_ppc_set_vhyp(cpu, PPC_VIRTUAL_HYPERVISOR(spapr));
     kvmppc_set_papr(cpu);
 
+    env->spr_cb[SPR_PIR].default_value = cs->cpu_index;
+    env->spr_cb[SPR_TIR].default_value = thread_index;
+
     /* Set time-base frequency to 512 MHz. vhyp must be set first. */
     cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ);
 
@@ -337,7 +340,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
     for (i = 0; i < cc->nr_threads; i++) {
         sc->threads[i] = spapr_create_vcpu(sc, i, errp);
         if (!sc->threads[i] ||
-            !spapr_realize_vcpu(sc->threads[i], spapr, sc, errp)) {
+            !spapr_realize_vcpu(sc->threads[i], spapr, sc, i, errp)) {
             spapr_cpu_core_unrealize(dev);
             return;
         }
Signature · 1 line
2.40.1
#9 lore raw

Re: [PATCH v2 5/7] spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs

Quoted 25 lines · Nicholas Piggin

PPC TCG supports SMT CPU configurations for non-hypervisor state, so permit POWER8-10 pseries machines to enable SMT.

This requires PIR and TIR be set, because that's how sibling thread matching is done by TCG.

spapr's nested-HV capability does not currently coexist with SMT, so that combination is prohibited (interestingly somewhat analogous to LPAR-per-core mode on real hardware which also does not support KVM).

---
  hw/ppc/spapr.c          | 16 ++++++++++++----
  hw/ppc/spapr_caps.c     | 14 ++++++++++++++
  hw/ppc/spapr_cpu_core.c |  7 +++++--
  3 files changed, 31 insertions(+), 6 deletions(-)
hw/ppc/spapr.c +0 -0
index 8e7d497f25..677b5eef9d 100644
@@ -2525,10 +2525,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
      int ret;
      unsigned int smp_threads = ms->smp.threads;
 

hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2533

Quoted 10 lines
hw/ppc/spapr.c +6 -4
-    if (tcg_enabled() && (smp_threads > 1)) {
-        error_setg(errp, "TCG cannot support more than 1 thread/core "
-                   "on a pseries machine");
-        return;
+    if (tcg_enabled()) {
+        if (!ppc_type_check_compat(ms->cpu_type, CPU_POWERPC_LOGICAL_2_07, 0,
+                                   spapr->max_compat_pvr)) {
+            error_setg(errp, "TCG only supports SMT on POWER8 or newer CPUs");
+            return;
+        }

So if TCG is enabled and the CPU < P8, QEMU bails out ? You should run qemu-ppc-boot :)

Thanks,

C.

Quoted 66 lines
hw/ppc/spapr.c +6 -0
+
+        if (smp_threads > 8) {
+            error_setg(errp, "TCG cannot support more than 8 threads/core "
+                       "on a pseries machine");
+            return;
+        }
      }
      if (!is_power_of_2(smp_threads)) {
          error_setg(errp, "Cannot support %d threads/core on a pseries "
hw/ppc/spapr_caps.c +14 -0
index 3fd45a6dec..5a0755d34f 100644
@@ -473,6 +473,20 @@ static void cap_nested_kvm_hv_apply(SpaprMachineState *spapr,
                  error_append_hint(errp,
                                    "Try appending -machine cap-nested-hv=off\n");
          }
+    } else if (tcg_enabled()) {
+        MachineState *ms = MACHINE(spapr);
+        unsigned int smp_threads = ms->smp.threads;
+
+        /*
+         * Nested-HV vCPU env state to L2, so SMT-shared SPR updates, for
+         * example, do not necessarily update the correct SPR value on sibling
+         * threads that are in a different guest/host context.
+         */
+        if (smp_threads > 1) {
+            error_setg(errp, "TCG does not support nested-HV with SMT");
+            error_append_hint(errp, "Try appending -machine cap-nested-hv=off "
+                                    "or use threads=1 with -smp\n");
+        }
      }
  }
 
hw/ppc/spapr_cpu_core.c +5 -2
index 9b88dd549a..a4e3c2fadd 100644
@@ -255,7 +255,7 @@ static void spapr_cpu_core_unrealize(DeviceState *dev)
  }
 
  static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
-                               SpaprCpuCore *sc, Error **errp)
+                               SpaprCpuCore *sc, int thread_index, Error **errp)
  {
      CPUPPCState *env = &cpu->env;
      CPUState *cs = CPU(cpu);
@@ -267,6 +267,9 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
      cpu_ppc_set_vhyp(cpu, PPC_VIRTUAL_HYPERVISOR(spapr));
      kvmppc_set_papr(cpu);
 
+    env->spr_cb[SPR_PIR].default_value = cs->cpu_index;
+    env->spr_cb[SPR_TIR].default_value = thread_index;
+
      /* Set time-base frequency to 512 MHz. vhyp must be set first. */
      cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ);
 
@@ -337,7 +340,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
      for (i = 0; i < cc->nr_threads; i++) {
          sc->threads[i] = spapr_create_vcpu(sc, i, errp);
          if (!sc->threads[i] ||
-            !spapr_realize_vcpu(sc->threads[i], spapr, sc, errp)) {
+            !spapr_realize_vcpu(sc->threads[i], spapr, sc, i, errp)) {
              spapr_cpu_core_unrealize(dev);
              return;
          }
#10 lore raw

Re: [PATCH v2 5/7] spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs

Quoted 18 lines · Cédric Le Goater
Quoted 17 lines · Nicholas Piggin

PPC TCG supports SMT CPU configurations for non-hypervisor state, so permit POWER8-10 pseries machines to enable SMT.

This requires PIR and TIR be set, because that's how sibling thread matching is done by TCG.

spapr's nested-HV capability does not currently coexist with SMT, so that combination is prohibited (interestingly somewhat analogous to LPAR-per-core mode on real hardware which also does not support KVM).

---
  hw/ppc/spapr.c          | 16 ++++++++++++----
  hw/ppc/spapr_caps.c     | 14 ++++++++++++++
  hw/ppc/spapr_cpu_core.c |  7 +++++--

  3 files changed, 31 insertions(+), 6 deletions(-)
Quoted 12 lines

hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2527

Quoted 12 lines
hw/ppc/spapr.c +1 -4
index 8e7d497f25..677b5eef9d 100644
@@ -2525,10 +2525,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
       int ret;
       unsigned int smp_threads = ms->smp.threads;
-    if (tcg_enabled() && (smp_threads > 1)) {
-        error_setg(errp, "TCG cannot support more than 1 thread/core "
-                   "on a pseries machine");
-        return;
+    if (tcg_enabled()) {

I will add :

            if (smp_threads > 1 &&

No need to resend for that.

C.

Quoted 76 lines

hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2533

Quoted 5 lines
hw/ppc/spapr.c +5 -0
+        if (!ppc_type_check_compat(ms->cpu_type, CPU_POWERPC_LOGICAL_2_07, 0,
+                                   spapr->max_compat_pvr)) {
+            error_setg(errp, "TCG only supports SMT on POWER8 or newer CPUs");
+            return;
+        }

So if TCG is enabled and the CPU < P8, QEMU bails out ? You should run qemu-ppc-boot :)

Thanks,

C.

Quoted 62 lines
hw/ppc/spapr.c +6 -0
+
+        if (smp_threads > 8) {
+            error_setg(errp, "TCG cannot support more than 8 threads/core "
+                       "on a pseries machine");
+            return;
+        }
       }
       if (!is_power_of_2(smp_threads)) {

          error_setg(errp, "Cannot support %d threads/core on a pseries "

hw/ppc/spapr_caps.c +14 -0
index 3fd45a6dec..5a0755d34f 100644
@@ -473,6 +473,20 @@ static void cap_nested_kvm_hv_apply(SpaprMachineState *spapr,
                   error_append_hint(errp,
                                     "Try appending -machine cap-nested-hv=off\n");
           }
+    } else if (tcg_enabled()) {
+        MachineState *ms = MACHINE(spapr);
+        unsigned int smp_threads = ms->smp.threads;
+
+        /*
+         * Nested-HV vCPU env state to L2, so SMT-shared SPR updates, for
+         * example, do not necessarily update the correct SPR value on sibling
+         * threads that are in a different guest/host context.
+         */
+        if (smp_threads > 1) {
+            error_setg(errp, "TCG does not support nested-HV with SMT");
+            error_append_hint(errp, "Try appending -machine cap-nested-hv=off "
+                                    "or use threads=1 with -smp\n");
+        }
       }
   }
hw/ppc/spapr_cpu_core.c +4 -1
index 9b88dd549a..a4e3c2fadd 100644
@@ -255,7 +255,7 @@ static void spapr_cpu_core_unrealize(DeviceState *dev)
   }
   static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
-                               SpaprCpuCore *sc, Error **errp)
+                               SpaprCpuCore *sc, int thread_index, Error **errp)
   {
       CPUPPCState *env = &cpu->env;
       CPUState *cs = CPU(cpu);
@@ -267,6 +267,9 @@ static bool spapr_realize_vcpu(PowerPCCPU *cpu, SpaprMachineState *spapr,
       cpu_ppc_set_vhyp(cpu, PPC_VIRTUAL_HYPERVISOR(spapr));
       kvmppc_set_papr(cpu);
+    env->spr_cb[SPR_PIR].default_value = cs->cpu_index;
+    env->spr_cb[SPR_TIR].default_value = thread_index;
+
       /* Set time-base frequency to 512 MHz. vhyp must be set first. */
       cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ);
@@ -337,7 +340,7 @@ static void spapr_cpu_core_realize(DeviceState *dev, Error **errp)
       for (i = 0; i < cc->nr_threads; i++) {
           sc->threads[i] = spapr_create_vcpu(sc, i, errp);

          if (!sc->threads[i] ||
-            !spapr_realize_vcpu(sc->threads[i], spapr, sc, errp)) { +            !spapr_realize_vcpu(sc->threads[i], spapr, sc, i, errp)) { spapr_cpu_core_unrealize(dev);
              return;
          }

Reviewed-by
#11 lore raw

Re: [PATCH v2 5/7] spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs

Quoted 34 lines · Cédric Le Goater
Quoted 18 lines · Cédric Le Goater
Quoted 17 lines · Nicholas Piggin

PPC TCG supports SMT CPU configurations for non-hypervisor state, so permit POWER8-10 pseries machines to enable SMT.

This requires PIR and TIR be set, because that's how sibling thread matching is done by TCG.

spapr's nested-HV capability does not currently coexist with SMT, so that combination is prohibited (interestingly somewhat analogous to LPAR-per-core mode on real hardware which also does not support KVM).

---
  hw/ppc/spapr.c          | 16 ++++++++++++----
  hw/ppc/spapr_caps.c     | 14 ++++++++++++++
  hw/ppc/spapr_cpu_core.c |  7 +++++--

  3 files changed, 31 insertions(+), 6 deletions(-)
Quoted 12 lines

hw/ppc/spapr.c static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp) · line 2527

Quoted 12 lines
hw/ppc/spapr.c +1 -4
index 8e7d497f25..677b5eef9d 100644
@@ -2525,10 +2525,18 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
       int ret;
       unsigned int smp_threads = ms->smp.threads;
-    if (tcg_enabled() && (smp_threads > 1)) {
-        error_setg(errp, "TCG cannot support more than 1 thread/core "
-                   "on a pseries machine");
-        return;
+    if (tcg_enabled()) {

I will add :

Quoted 3 lines

            if (smp_threads > 1 &&

No need to resend for that.

and

Thanks,

C.

author reviewer
#12 lore raw

PATCH v2 6/7 tests/avocado: boot ppc64 pseries to Linux VFS mount

In reply to Nicholas Piggin

This machine can boot Linux to VFS mount, so don't stop in early boot.

---
 tests/avocado/ppc_pseries.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/avocado/ppc_pseries.py +1 -1
index d8b04dc3ea..a152cf222e 100644
@@ -31,5 +31,5 @@ def test_ppc64_pseries(self):
         self.vm.add_args('-kernel', kernel_path,
                          '-append', kernel_command_line)
         self.vm.launch()
-        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        console_pattern = 'VFS: Cannot open root device'
         wait_for_console_pattern(self, console_pattern, self.panic_message)
Signature · 1 line
2.40.1
Reviewed-by on v2 6/7
#13 lore raw

Re: [PATCH v2 6/7] tests/avocado: boot ppc64 pseries to Linux VFS mount

Quoted 3 lines · Nicholas Piggin

This machine can boot Linux to VFS mount, so don't stop in early boot.

Thanks,

C.

Quoted 15 lines
---
  tests/avocado/ppc_pseries.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
tests/avocado/ppc_pseries.py +1 -1
index d8b04dc3ea..a152cf222e 100644
@@ -31,5 +31,5 @@ def test_ppc64_pseries(self):
          self.vm.add_args('-kernel', kernel_path,
                           '-append', kernel_command_line)
          self.vm.launch()
-        console_pattern = 'Kernel command line: %s' % kernel_command_line
+        console_pattern = 'VFS: Cannot open root device'
          wait_for_console_pattern(self, console_pattern, self.panic_message)
author reviewer
#14 lore raw

PATCH v2 7/7 tests/avocado: Add ppc64 pseries multiprocessor boot tests

In reply to Nicholas Piggin

Add mult-thread/core/socket Linux boot tests that ensure the right topology comes up. Of particular note is a SMT test, which is a new capability for TCG.

---
 tests/avocado/ppc_pseries.py | 60 +++++++++++++++++++++++++++++++++---
 1 file changed, 55 insertions(+), 5 deletions(-)
tests/avocado/ppc_pseries.py +55 -5
index a152cf222e..ff42c770f2 100644
@@ -14,12 +14,9 @@ class pseriesMachine(QemuSystemTest):
     timeout = 90
     KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
     panic_message = 'Kernel panic - not syncing'
+    good_message = 'VFS: Cannot open root device'
 
-    def test_ppc64_pseries(self):
-        """
-        :avocado: tags=arch:ppc64
-        :avocado: tags=machine:pseries
-        """
+    def do_test_ppc64_linux_boot(self):
         kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                       '/fedora-secondary/releases/29/Everything/ppc64le/os'
                       '/ppc/ppc64/vmlinuz')
@@ -31,5 +28,58 @@ def test_ppc64_pseries(self):
         self.vm.add_args('-kernel', kernel_path,
                          '-append', kernel_command_line)
         self.vm.launch()
+
+    def test_ppc64_linux_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.do_test_ppc64_linux_boot()
         console_pattern = 'VFS: Cannot open root device'
         wait_for_console_pattern(self, console_pattern, self.panic_message)
+
+    def test_ppc64_linux_smp_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '4')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'smp: Brought up 1 node, 4 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+    def test_ppc64_linux_smt_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '4,threads=4')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'CPU maps initialized for 4 threads per core'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        console_pattern = 'smp: Brought up 1 node, 4 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+    def test_ppc64_linux_big_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
+        self.vm.add_args('-m', '512M',
+                         '-object', 'memory-backend-ram,size=256M,id=m0',
+                         '-object', 'memory-backend-ram,size=256M,id=m1')
+        self.vm.add_args('-numa', 'node,nodeid=0,memdev=m0')
+        self.vm.add_args('-numa', 'node,nodeid=1,memdev=m1')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'CPU maps initialized for 4 threads per core'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
Signature · 1 line
2.40.1
Reviewed-by on v2 7/7
#15 lore raw

Re: [PATCH v2 7/7] tests/avocado: Add ppc64 pseries multiprocessor boot tests

Quoted 5 lines · Nicholas Piggin

Add mult-thread/core/socket Linux boot tests that ensure the right topology comes up. Of particular note is a SMT test, which is a new capability for TCG.

Thanks,

C.

Quoted 82 lines
---
  tests/avocado/ppc_pseries.py | 60 +++++++++++++++++++++++++++++++++---
  1 file changed, 55 insertions(+), 5 deletions(-)
tests/avocado/ppc_pseries.py +55 -5
index a152cf222e..ff42c770f2 100644
@@ -14,12 +14,9 @@ class pseriesMachine(QemuSystemTest):
      timeout = 90
      KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
      panic_message = 'Kernel panic - not syncing'
+    good_message = 'VFS: Cannot open root device'
 
-    def test_ppc64_pseries(self):
-        """
-        :avocado: tags=arch:ppc64
-        :avocado: tags=machine:pseries
-        """
+    def do_test_ppc64_linux_boot(self):
          kernel_url = ('https://archives.fedoraproject.org/pub/archive'
                        '/fedora-secondary/releases/29/Everything/ppc64le/os'
                        '/ppc/ppc64/vmlinuz')
@@ -31,5 +28,58 @@ def test_ppc64_pseries(self):
          self.vm.add_args('-kernel', kernel_path,
                           '-append', kernel_command_line)
          self.vm.launch()
+
+    def test_ppc64_linux_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.do_test_ppc64_linux_boot()
          console_pattern = 'VFS: Cannot open root device'
          wait_for_console_pattern(self, console_pattern, self.panic_message)
+
+    def test_ppc64_linux_smp_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '4')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'smp: Brought up 1 node, 4 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+    def test_ppc64_linux_smt_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '4,threads=4')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'CPU maps initialized for 4 threads per core'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        console_pattern = 'smp: Brought up 1 node, 4 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
+
+    def test_ppc64_linux_big_boot(self):
+        """
+        :avocado: tags=arch:ppc64
+        :avocado: tags=machine:pseries
+        """
+
+        self.vm.add_args('-smp', '16,threads=4,cores=2,sockets=2')
+        self.vm.add_args('-m', '512M',
+                         '-object', 'memory-backend-ram,size=256M,id=m0',
+                         '-object', 'memory-backend-ram,size=256M,id=m1')
+        self.vm.add_args('-numa', 'node,nodeid=0,memdev=m0')
+        self.vm.add_args('-numa', 'node,nodeid=1,memdev=m1')
+        self.do_test_ppc64_linux_boot()
+        console_pattern = 'CPU maps initialized for 4 threads per core'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        console_pattern = 'smp: Brought up 2 nodes, 16 CPUs'
+        wait_for_console_pattern(self, console_pattern, self.panic_message)
+        wait_for_console_pattern(self, self.good_message, self.panic_message)
#16 lore raw
In reply to Nicholas Piggin
Quoted 23 lines · Nicholas Piggin

This series is based on some previously posted TCG fixes, in particular the CTRL register fix is required.

Also added the Philippe's patch in the series to prevent conflict.

Since v1, main changes are just some tidying of comments and changelogs, and addition of avocado tests to boot Linux on SMT machine and make sure the CPUs come up, as suggested by Cedric.

Thanks,
Nick

Nicholas Piggin (6):
target/ppc: Add initial flags and helpers for SMT support target/ppc: Add support for SMT CTRL register target/ppc: Add msgsnd/p and DPDES SMT support
spapr: TCG allow up to 8-thread SMT on POWER8 and newer CPUs tests/avocado: boot ppc64 pseries to Linux VFS mount tests/avocado: Add ppc64 pseries multiprocessor boot tests

Philippe Mathieu-Daudé (1):
hw/ppc/spapr: Test whether TCG is enabled with tcg_enabled()

Quoted 14 lines
  hw/ppc/ppc.c                 |  6 ++++
  hw/ppc/spapr.c               | 16 ++++++---
  hw/ppc/spapr_caps.c          | 14 ++++++++
  hw/ppc/spapr_cpu_core.c      |  7 ++--
  include/hw/ppc/ppc.h         |  1 +
  target/ppc/cpu.h             |  9 +++++
  target/ppc/cpu_init.c        |  5 +++
  target/ppc/excp_helper.c     | 30 +++++++++++++---
  target/ppc/helper.h          |  2 ++
  target/ppc/misc_helper.c     | 69 ++++++++++++++++++++++++++++++++----
  target/ppc/translate.c       | 46 +++++++++++++++++++++++-
  tests/avocado/ppc_pseries.py | 62 ++++++++++++++++++++++++++++----
  12 files changed, 243 insertions(+), 24 deletions(-)

Applied to ppc-next.

Thanks,

C.